Phase 2: drift detection

Compares cloud_provisioned.spec (what we asked DO for) against the live
cloud_resources row (what DO actually has). Any divergence becomes an
operator-resolvable drift record.

cloud_drift table: one row per drifted field. status open/accepted/
reverted/stale. Partial unique index keeps at most one OPEN drift per
(resource, field); resolved rows are retained as history.

ArcadiaCloud.Drift context:
- detect_all/0 — sweeps every provisioned resource. Per field in spec,
  resolves the actual value (top-level schema field first, then attrs),
  compares with loose equality (stringified scalars; lists as sets so
  JSON round-trips don't false-positive). Mismatches upsert a cloud_drift
  row + emit a drift_detected event in the resource event log.
- close_stale_drift — an open drift whose field no longer mismatches
  (fixed elsewhere) closes as "stale" on the next sweep.
- accept_drift/2 — the live value becomes the new desired-state: parent
  cloud_provisioned.spec is updated, spec_version bumped, drift closed
  "accepted". Revert (mutating live infra back to spec) is intentionally
  NOT here — it needs a saga and lands with the droplet-resize work.

DriftDetectionWorker — Oban cron at :20 past the hour, offset past the
:15 resource syncs so it compares against fresh inventory.

Provisioning.record_provisioned/3 — populates cloud_provisioned desired-
state (upsert on resource_id, bumps spec_version). Future provisioning
sagas call this; for now it's how drift gets something to detect.

API (platform_admin only):
- GET  /api/v1/drift            — open drift inbox
- POST /api/v1/drift/:id/accept — adopt live value as desired-state

Smoke verified: recorded a droplet's desired spec with a deliberately
wrong size_slug + a correct region; detect_all flagged only size_slug,
wrote the drift_detected event; accept updated the spec to the live
value and closed the drift; re-detect found zero drift.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-05-20 11:08:27 +10:00
parent b1a124f044
commit 445b7b60d4
8 changed files with 436 additions and 0 deletions

View File

@@ -0,0 +1,25 @@
defmodule ArcadiaCloud.Sync.DriftDetectionWorker do
@moduledoc """
Periodic drift sweep — compares every provisioned resource's desired
spec against its live cloud_resources row. Runs after the inventory
sync workers so it compares against fresh data.
"""
use Oban.Worker, queue: :cloud_sync_full, max_attempts: 3
require Logger
alias ArcadiaCloud.Drift
@impl Oban.Worker
def perform(_job) do
summary = Drift.detect_all()
Logger.info(
"[drift] checked=#{summary.checked} drifted_fields=#{summary.drifted_fields} " <>
"new=#{summary.new_drift} stale_closed=#{summary.resolved_stale}"
)
:ok
end
end