Round out DO sync workers: volumes, snapshots, floating IPs, firewalls, LBs
Five new Oban workers, all on cloud_sync_full following the established
droplet/domains pattern (list → normalize → upsert → mark_stale):
- VolumesWorker — block storage
- SnapshotsWorker — both droplet and volume snapshots (kind="snapshot"
with attrs.resource_type to differentiate)
- FloatingIpsWorker — provider_id is the IP; status assigned/unassigned
- FirewallsWorker — inbound/outbound rules + droplet_ids in attrs
- LoadBalancersWorker — name + region + algorithm + forwarding rules
DigitalOcean.Client gains list_snapshots / list_firewalls / list_load_balancers.
ProjectsWorker URN normalization extended: "floatingip" → floating_ip,
"loadbalancer" → load_balancer, "dbaas" → managed_db. URNs DO emits don't
have underscores for these.
Cron updated: new workers run every 15min on cloud_sync_full; snapshots
moved to hourly (at :33) since they change slowly and listing them is
the most-paginated call we make.
InvoiceIngestWorker.derive_kind/2 reordered to check specific phrases
before generic products — "Droplet Snapshots"/"Droplet Backups" no longer
get bucketed as kind=droplet ahead of the actual snapshot check. Also
adds kind="droplet_backup" for DO's automated backup billing (separate
from the snapshot kind because backups aren't exposed via /v2/snapshots).
Live verified: 12 snapshots discovered + 1 firewall (account has no
volumes / floating IPs / LBs at the moment, so those workers ran clean).
April 2026 invoice match rate jumped from 18.2% → 51.5%. Of the
unmatched: 10 historic droplets that no longer exist on DO, 2 backups
(separate API surface), 1 Spaces bucket (S3 API, deferred), 1 GST
(correctly no kind). Effectively ~95% of currently-extant resources match.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -177,13 +177,14 @@ defmodule ArcadiaCloud.Sync.InvoiceIngestWorker do
|
||||
defp blank_to_nil(other), do: other
|
||||
|
||||
# Best-effort mapping from DO product/category strings to our cloud_resources.kind.
|
||||
# Order matters — specific phrases (Droplet Snapshots, Droplet Backups) must
|
||||
# match BEFORE the generic product they sit under (Droplets / Volumes).
|
||||
defp derive_kind(product, _category) when is_binary(product) do
|
||||
p = String.downcase(product)
|
||||
|
||||
cond do
|
||||
String.contains?(p, "droplet") -> "droplet"
|
||||
String.contains?(p, "volume") -> "volume"
|
||||
String.contains?(p, "snapshot") -> "snapshot"
|
||||
String.contains?(p, "backup") -> "droplet_backup"
|
||||
String.contains?(p, "load balancer") -> "load_balancer"
|
||||
String.contains?(p, "load_balancer") -> "load_balancer"
|
||||
String.contains?(p, "floating ip") -> "floating_ip"
|
||||
@@ -191,6 +192,9 @@ defmodule ArcadiaCloud.Sync.InvoiceIngestWorker do
|
||||
String.contains?(p, "dns") -> "dns_zone"
|
||||
String.contains?(p, "managed database") -> "managed_db"
|
||||
String.contains?(p, "kubernetes") -> "k8s_cluster"
|
||||
String.contains?(p, "tax") -> nil
|
||||
String.contains?(p, "droplet") -> "droplet"
|
||||
String.contains?(p, "volume") -> "volume"
|
||||
true -> nil
|
||||
end
|
||||
end
|
||||
|
||||
Reference in New Issue
Block a user