Giuliano Silvestro
8bdf500214
Five new Oban workers, all on cloud_sync_full following the established
droplet/domains pattern (list → normalize → upsert → mark_stale):
- VolumesWorker — block storage
- SnapshotsWorker — both droplet and volume snapshots (kind="snapshot"
with attrs.resource_type to differentiate)
- FloatingIpsWorker — provider_id is the IP; status assigned/unassigned
- FirewallsWorker — inbound/outbound rules + droplet_ids in attrs
- LoadBalancersWorker — name + region + algorithm + forwarding rules
DigitalOcean.Client gains list_snapshots / list_firewalls / list_load_balancers.
ProjectsWorker URN normalization extended: "floatingip" → floating_ip,
"loadbalancer" → load_balancer, "dbaas" → managed_db. URNs DO emits don't
have underscores for these.
Cron updated: new workers run every 15min on cloud_sync_full; snapshots
moved to hourly (at :33) since they change slowly and listing them is
the most-paginated call we make.
InvoiceIngestWorker.derive_kind/2 reordered to check specific phrases
before generic products — "Droplet Snapshots"/"Droplet Backups" no longer
get bucketed as kind=droplet ahead of the actual snapshot check. Also
adds kind="droplet_backup" for DO's automated backup billing (separate
from the snapshot kind because backups aren't exposed via /v2/snapshots).
Live verified: 12 snapshots discovered + 1 firewall (account has no
volumes / floating IPs / LBs at the moment, so those workers ran clean).
April 2026 invoice match rate jumped from 18.2% → 51.5%. Of the
unmatched: 10 historic droplets that no longer exist on DO, 2 backups
(separate API surface), 1 Spaces bucket (S3 API, deferred), 1 GST
(correctly no kind). Effectively ~95% of currently-extant resources match.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
74 lines
2.5 KiB
Elixir
74 lines
2.5 KiB
Elixir
# This file is responsible for configuring your application
|
|
# and its dependencies with the aid of the Config module.
|
|
#
|
|
# This configuration file is loaded before any dependency and
|
|
# is restricted to this project.
|
|
|
|
# General application configuration
|
|
import Config
|
|
|
|
config :arcadia_cloud,
|
|
ecto_repos: [ArcadiaCloud.Repo],
|
|
generators: [timestamp_type: :utc_datetime, binary_id: true]
|
|
|
|
# Configure the endpoint
|
|
config :arcadia_cloud, ArcadiaCloudWeb.Endpoint,
|
|
url: [host: "localhost"],
|
|
adapter: Bandit.PhoenixAdapter,
|
|
render_errors: [
|
|
formats: [json: ArcadiaCloudWeb.ErrorJSON],
|
|
layout: false
|
|
],
|
|
pubsub_server: ArcadiaCloud.PubSub,
|
|
live_view: [signing_salt: "4W6q5pDB"]
|
|
|
|
# Configure Elixir's Logger
|
|
config :logger, :default_formatter,
|
|
format: "$time $metadata[$level] $message\n",
|
|
metadata: [:request_id]
|
|
|
|
# Use Jason for JSON parsing in Phoenix
|
|
config :phoenix, :json_library, Jason
|
|
|
|
# Guardian — JWTs are issued by arcadia-app. arcadia-cloud only verifies them.
|
|
# Issuer and secret_key MUST match arcadia-app's Arcadia.Guardian config.
|
|
config :arcadia_cloud, ArcadiaCloud.Guardian,
|
|
issuer: "arcadia",
|
|
verify_issuer: true
|
|
|
|
# Oban — provisioning, sync, billing, gateway-event consumption queues
|
|
config :arcadia_cloud, Oban,
|
|
engine: Oban.Engines.Basic,
|
|
queues: [
|
|
provisioning: 5,
|
|
cloud_sync_fast: 5,
|
|
cloud_sync_full: 3,
|
|
cloud_sync_slow: 1,
|
|
cloud_billing: 1,
|
|
metering: 2,
|
|
default: 5
|
|
],
|
|
plugins: [
|
|
{Oban.Plugins.Cron,
|
|
crontab: [
|
|
# ProjectsWorker first so attribution is fresh before resource syncs
|
|
{"*/15 * * * *", ArcadiaCloud.Sync.ProjectsWorker},
|
|
{"*/15 * * * *", ArcadiaCloud.Sync.DropletsWorker},
|
|
{"*/15 * * * *", ArcadiaCloud.Sync.DomainsWorker},
|
|
{"*/15 * * * *", ArcadiaCloud.Sync.VolumesWorker},
|
|
{"*/15 * * * *", ArcadiaCloud.Sync.FloatingIpsWorker},
|
|
{"*/15 * * * *", ArcadiaCloud.Sync.FirewallsWorker},
|
|
{"*/15 * * * *", ArcadiaCloud.Sync.LoadBalancersWorker},
|
|
# Snapshots change slowly; hourly is enough and reduces API churn
|
|
{"33 * * * *", ArcadiaCloud.Sync.SnapshotsWorker},
|
|
# Billing: hourly balance, daily invoice discovery
|
|
{"7 * * * *", ArcadiaCloud.Sync.BalanceWorker},
|
|
{"23 2 * * *", ArcadiaCloud.Sync.BillingHistoryWorker}
|
|
]}
|
|
],
|
|
repo: ArcadiaCloud.Repo
|
|
|
|
# Import environment specific config. This must remain at the bottom
|
|
# of this file so it overrides the configuration defined above.
|
|
import_config "#{config_env()}.exs"
|