Phase 1 first chunk: inventory schema + DO droplet sync

Models:
- cloud_projects: arcadia-cloud's mirror of DO Projects, indexed by
  (provider, provider_id); tenant_id + purpose classify each project.
- cloud_resources: single unified resource table; kind-specific bits in
  attrs JSONB; first_seen_at / last_seen_at / stale_strike_count drive
  three-strike deletion.
- cloud_resource_events: append-only audit (discovered, updated, deleted,
  drift_detected, tagged, restored).

ArcadiaCloud.Cloud context owns the single upsert chokepoint that:
- inserts new with `discovered` event
- updates existing only when meaningful fields change
- restores tombstoned rows seen again
- bumps last_seen_at and resets strike count
mark_stale/3 implements the three-strike rule.

ArcadiaCloud.DigitalOcean.Client is a Req wrapper with auto-pagination.
Per-purpose token resolution via .Tokens (phase 1: env DO_API_TOKEN;
phase 2: vault). Per project_arcadia_cloud memory the long-term shape
is one PAT per queue purpose for rate-limit isolation.

ArcadiaCloud.Sync.Bootstrap ensures the skyai-internal DO Project exists
on first sync, idempotent thereafter. ArcadiaCloud.Sync.DropletsWorker
runs full droplet sync on the cloud_sync_full Oban queue.

InventoryController wired to real data: platform_admin sees all,
tenants see only their scope.

Live smoke test against real DO: 5 droplets synced; skyai-internal
project auto-created; events written; endpoint returns scoped results.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-19 22:07:29 +10:00
parent a66dde6618
commit c1cbd434ac
10 changed files with 658 additions and 4 deletions

View File

@@ -0,0 +1,106 @@
defmodule ArcadiaCloud.DigitalOcean.Client do
@moduledoc """
Thin Req wrapper over the DigitalOcean v2 API.
Token resolution: per-purpose, looked up via `ArcadiaCloud.DigitalOcean.Tokens`.
Phase 0/1: env var `DO_API_TOKEN`. Phase 2: from the secrets vault.
Paginated list endpoints stream all pages by default.
"""
alias ArcadiaCloud.DigitalOcean.Tokens
@base "https://api.digitalocean.com/v2"
@page_size 100
# ---- public ---------------------------------------------------------------
def list_droplets(opts \\ []), do: list_paginated("/droplets", "droplets", opts)
def list_projects(opts \\ []), do: list_paginated("/projects", "projects", opts)
def create_project(name, purpose, description \\ "", opts \\ []) do
body = %{
name: name,
purpose: purpose,
description: description,
environment: "Development"
}
request(:post, "/projects", body: body, purpose: opts[:purpose] || "provisioning")
|> case do
{:ok, %{"project" => project}} -> {:ok, project}
other -> other
end
end
def list_project_resources(project_id, opts \\ []) do
list_paginated("/projects/#{project_id}/resources", "resources", opts)
end
def assign_to_project(project_id, urns, opts \\ []) when is_list(urns) do
request(:post, "/projects/#{project_id}/resources",
body: %{resources: urns},
purpose: opts[:purpose] || "provisioning"
)
end
# ---- core -----------------------------------------------------------------
defp list_paginated(path, root_key, opts) do
purpose = opts[:purpose] || "sync_full"
do_paginate(path, root_key, purpose, [], 1)
end
defp do_paginate(path, root_key, purpose, acc, page) do
params = [page: page, per_page: @page_size]
case request(:get, path, params: params, purpose: purpose) do
{:ok, %{} = body} ->
items = Map.get(body, root_key, [])
new_acc = acc ++ items
if has_next?(body) do
do_paginate(path, root_key, purpose, new_acc, page + 1)
else
{:ok, new_acc}
end
err ->
err
end
end
defp has_next?(%{"links" => %{"pages" => %{"next" => _}}}), do: true
defp has_next?(_), do: false
defp request(method, path, opts) do
purpose = Keyword.fetch!(opts, :purpose)
with {:ok, token} <- Tokens.fetch(purpose) do
req_opts =
[
method: method,
url: @base <> path,
headers: [{"authorization", "Bearer " <> token}],
retry: :transient,
max_retries: 3
]
|> maybe_put(:params, opts[:params])
|> maybe_put(:json, opts[:body])
case Req.request(req_opts) do
{:ok, %Req.Response{status: status, body: body}} when status in 200..299 ->
{:ok, body}
{:ok, %Req.Response{status: status, body: body}} ->
{:error, {:http, status, body}}
{:error, exception} ->
{:error, {:transport, exception}}
end
end
end
defp maybe_put(opts, _key, nil), do: opts
defp maybe_put(opts, key, value), do: Keyword.put(opts, key, value)
end

View File

@@ -0,0 +1,24 @@
defmodule ArcadiaCloud.DigitalOcean.Tokens do
@moduledoc """
DO API token resolver. Per-purpose so worker queues use separate tokens
(rate limit isolation + blast radius — see project_arcadia_cloud memory).
Phase 0/1 implementation: all purposes fall back to the single
`DO_API_TOKEN` env var (or `:default_token` app env).
Phase 2: read per-purpose bundles from the secrets vault.
"""
@env_var "DO_API_TOKEN"
def fetch(purpose) when is_binary(purpose) do
case resolve(purpose) do
nil -> {:error, :no_token_configured}
token -> {:ok, token}
end
end
defp resolve(_purpose) do
Application.get_env(:arcadia_cloud, :do_api_token) ||
System.get_env(@env_var)
end
end