ADR 0017: A generic, declarative http adapter

Architecture decision record 0017. Status: Proposed

Date: 2026-10-11

Status

Proposed

Context

Sponson ships first-class adapters for a few providers, each a TypeScript file with its own sim routes, scenarios and docs. What ships beside the code of a typical web product is much wider: feature flags, webhook endpoints, auth redirect and CORS allow-lists, per-environment config vars, DNS records, sending domains. A coverage survey of Western SaaS found dozens of such rows; most are plain CRUD on one object, or one entry in a list held by a parent object. Writing an adapter per provider can never cover that long tail, and every row left uncovered is a console step someone has to remember for every pull request.

The engine’s guarantees must not weaken on the way: intents before creates (ADR 0005), idempotent apply, drift by hash (ADR 0006), destroy only what Sponson created, secrets as references that never reach output (ADR 0014), and markers that never reach apply. The plan stays YAML (ADR 0001): no expressions, no scripts.

Two things in the adapter contract stood in the way of a generic adapter:

• Identity. A ledger entry’s identity is adapter + the whole providers.<adapter> block + key. A generic adapter needs one configuration per API, and if they all lived in one block, adding a second API would re-identify every resource of the first.
• Outputs. OpSpec.outputs is static, and from: references are checked against it before any provider call. A generic object’s outputs (a webhook’s signing secret, a flag’s key) are named by the plan line.

Decision

A built-in adapter http with two ops, configured entirely in the plan, built with the stable authoring API (clientFor, requireEnv, paramError, desiredSide, assertNoPending, listAll, isProviderError).

The provider block: one per API

providers:
  http:
    statsig:
      base_url: https://statsigapi.net/console/v1
      auth: { header: STATSIG-API-KEY, value_env: STATSIG_CONSOLE_KEY }
    stripe:
      base_url: https://api.stripe.com/v1
      auth: { bearer_env: STRIPE_SECRET_KEY }
      encoding: form

• auth is { bearer_env }, { header, value_env } or { basic: { user_env, password_env } }: always variable names, never values. The credential’s name must end like the names the engine already masks (TOKEN, SECRET, _KEY, …), so its value is masked in every output whatever the adapter does; the adapter masks it (and a Basic credential’s encoded form) in its own error text too.
• Optional base_url_env (an override, as every built-in adapter has), static headers, encoding: json | form.

Two engine hooks, both optional on OpSpec

• providerFor(block, params): the part of providers.<adapter> a line uses. The engine uses it for actx.provider and for ledger identity, so providers.http.<api> is each API’s own identity, and destroy finds its base URL and credential variable in the ledger as for any adapter. An unknown API is PLAN_INVALID before any request.
• outputsFor(params): the outputs one line declares. The engine specialises the op per line in prepare, so a from: naming an undeclared output is still REF_OUTPUT_UNKNOWN before any request.

Adapters that do not define them behave exactly as before.

Op resource: one object per item

- id: flag
  adapter: http
  op: resource
  api: statsig
  find: { path: /gates, list_path: /data, match: { name: "checkout-${ctx.scope}" } }
  read: { path: "/gates/{id}" }
  create: { path: /gates }
  update: { method: PATCH }
  item_path: /data
  fields: { description: New checkout for this preview, isEnabled: true }

• Located by find (a list, the array in it, the fields that identify the object) or by a read path that names it (a client-chosen id). find runs in read, before every create, so a create whose answer was lost is found and claimed instead of duplicated; create.idempotency_key: true adds an Idempotency-Key (derived from the resource key, the commit and the body) and makes the create safe to retry within the run.
• Key: <find.path>[<field>=<value>,…], or the read path: every dimension that tells two objects apart, the base URL being in the provider block. Record id: the request that removes it (DELETE /gates/<id>), so destroy needs nothing but the ledger, and a re-created object shows as replaced.
• Hash: sha256(canonicalJson) of the declared fields only, compared in the same form by diff. Undeclared fields are not Sponson’s. { keep: true } takes the live value. A difference with no update declared is PARAM_INVALID at plan time.
• exists_status / gone_status add a provider’s own statuses for “already exists” and “gone”; destroy: keep leaves the object in place.
• Amended: create and update take per-request headers (values may be references) and send_fields: false (the declared fields are compared but not put in the body, for an API that takes them in headers, as QStash takes a schedule’s cron in Upstash-Cron); a path placeholder written {+name} keeps / and : (RFC 6570 reserved expansion), for an API that takes a URL in the path. The sim’s REST styles gained header_fields and suffix_field to stand in for such APIs in the recipe tests.

Op list_item: one entry in a collection on a parent

- id: origin
  adapter: http
  op: list_item
  api: auth0
  parent: { path: "/clients/{client}" }
  vars: { client: abc123 }
  list_path: /web_origins
  item: { from: web.preview_url }

• The collection is a JSON array (shape: array, items are values or maps identified by key_field), a delimited string (shape: delimited, a comma-separated allow-list), or a keyed map (shape: map, item is the name and value the value; written as a merge patch where null deletes, as config-var APIs do).
• Read-modify-write: read the parent, change only this item, write the collection (alone, or with send: parent the whole object back), then read again; the write counts only once the item is seen as planned, and is retried up to three times if a concurrent writer dropped it. Writes are sent as idempotent (PUT, or PATCH/POST with idempotent: true), since they set state, so a dropped connection is retried.
• Amended: parent.read_method: POST reads the parent with a POST without a body, for an API whose read is an action that changes nothing (Azure Resource Manager’s …/listAppSettings, which has no GET). The read is sent as idempotent, like a GET. The method joins the locator (read_method) only when it is POST, so existing record ids are unchanged; the REST sim lists such paths in its seed’s reads and keeps them out of the write log (R8), for which ProviderSim.isWrite now also receives the provider’s state.
• Key: <parent.path>#<list_path>=<item>. The record id is the item’s locator (paths, method, shape, item), so destroy removes exactly that item and nothing else. Every other item is left as it is and reported as unmanaged (listScope).

Invariants, and where each is kept

Invariant	How
Intents before creates	actx.intend([key]) right before a create request or before adding an item; created lists only keys this call created.
Idempotent apply	apply compares the declared state with the live one first and returns without a request when they agree.
Drift by hash	One hash per resource over the declared state in canonical JSON; diff uses the same form.
Destroy only what Sponson created	Destroy works from ledger records (the engine skips adopted ones); a list item’s destroy removes that item only; 404 (or gone_status) is success.
Secrets as references	Credentials are variable names; secret values travel in params only during apply; hashes, never values, reach records; sensitive outputs are declared so.
Markers never reach apply	assertNoPending at the top of apply; read returns null and diff shows pending while a key depends on a pending value.

What it deliberately does not do

• No preconditions (ETag / If-Match, version fields) for list writes. The re-read narrows the window and reports what it sees; it cannot close it. Amended: http.list_item now declares its parent object with lockOn (ADR 0019), so two scopes editing one parent no longer lose a write; only a writer outside Sponson still can.
• No polling: an object that becomes ready asynchronously, or whose output appears later, needs a first-class adapter.
• Create-only outputs (a Stripe webhook signing secret) are returned by the run that creates the object and reach dependent lines in that run, but a later run cannot read them again. There is no declaration that makes the engine refuse such a later run. Amended (ADR 0020, Amendment 2): outputs.<name>: { path, sensitive: true, once: true } declares one.
• No OAuth2 client-credentials exchange, no command-produced tokens, no request signing (SigV4). Added by the amendment below, with HTTP Digest.
• No expressions or scripts, no JSON Patch generation, no pagination by page number or Link header (only a cursor read from the body), no listScope for resource (a collection holds much that is not this release’s) and no adopt. (GraphQL requests were added by the amendment.)

Alternatives considered

• JavaScript plugins only (SPONSON_PLUGINS). Already possible, and still the answer for anything with a real lifecycle. But it asks every team to write, test and version TypeScript for what is one request each way, and an agent editing a plan cannot add a row without leaving the plan. The declarative adapter covers the CRUD rows; the plugin path stays for the rest.
• OpenAPI-driven adapters (point at a provider’s spec, pick an operation). Many providers publish no spec or an incomplete one; specs do not say which fields identify an object, which are write-only, or what “already exists” looks like, so the plan would need the same declarations anyway, plus a spec loader and its failure modes.
• An exec/script adapter (run a command per step). It would make any side effect possible and none of the invariants checkable: no idempotency, no hash the engine understands, secrets in process arguments and logs, and a dependency on whatever is installed on the runner.
• One providers.http block for all APIs without providerFor. Adding or editing one API would re-identify every resource of every other API; rejected for the identity reason above.

Consequences

• A plan can manage a webhook, a flag or an allow-list entry of any JSON REST API without code, with the same receipts, drift and destroy as the built-in adapters.
• OpSpec gained two optional hooks (providerFor, outputsFor). Third-party adapters may use them; nothing requires them.
• Editing an API’s block (base URL, credential variable, headers, encoding) re-identifies its resources, as moving a line to another project would; destroy the scope first, or expect the old entries to become orphans.
• The record id encodes how an object is removed; changing delete (or a list item’s parent request) for an applied line looks like a replacement and needs --reconcile, which then treats the resource as adopted.
• The sim gained a generic REST provider (packages/sim/src/routes/rest.ts): collections, parents with lists, form bodies, merge patch, header credentials; its assumptions describe a conventional REST API, not one provider. The plan line remains the only description of a real API, so a wrong line is found by sponson plan against the real provider, not by the sim.
• destroy: keep is an explicit, per-line exception to “after a destroy nothing Sponson created survives”.

Amendment: archive deletes and refused credentials (2026-10-11)

Create-only outputs are declared since ADR 0020, Amendment 2 (outputs.<name>: { path, sensitive: true, once: true }), so “Create-only outputs … there is no declaration” under What it deliberately does not do no longer holds. Two more gaps stood between the adapter and Stripe’s webhook endpoints and catalog objects (ADR 0020, Amendment 3). Each is closed with the smallest declaration that says what the provider does, and neither changes an existing line’s identity:

• delete.body with method: POST: the archive request for objects a provider never deletes (a Stripe price is only ever active: false). The body (literals only) is appended to the record id as body=<canonical JSON>, so destroy still needs nothing but the ledger; record ids without a body are unchanged.
• refuse_credential: { prefixes, unless_env } on an API block: a credential starting with one of the prefixes is PROVIDER_AUTH before any request, in plan, apply and destroy alike, unless the named variable is 1 or true. Stripe’s sk_live_/rk_live_ keys must never create objects for a preview. It is part of the block, hence of ledger identity, like headers; a recipe’s refusal cannot be overridden by the plan (the variable is the opt-out), so it changes only if the recipe does, behind the identity lock.

Amendment: run-time credentials, XML and GraphQL

Date: 2026-10-11

Context

The coverage survey left about twenty rows the generic adapter could express as requests but not authenticate or encode: Auth0, Kinde and Okta take an OAuth2 client-credentials token; Google Cloud (Firebase authorized domains, GCS CORS) and Azure take a token their CLI prints; AWS (S3 bucket CORS, Amplify branch variables) signs every request with SigV4, and S3 speaks XML; MongoDB Atlas API keys use HTTP Digest; Railway and Fly.io manage variables and secrets through GraphQL. Each would otherwise need a first-class adapter for one credential flow, or a plugin.

Decision

Four more auth forms, each still a set of environment variable NAMES (or a literal command), so the API block (the ledger identity) holds nothing secret:

Form	At run time
oauth2_client_credentials: { token_url, client_id_env, client_secret_env, audience?, scope?, extra?, auth_method?, token_url_env? }	RFC 6749 §4.4 against the token endpoint (client_secret_post or client_secret_basic); the access token sent as a bearer token.
command: [argv…]	The CLI run without a shell through the same injectable Exec as the CLI-backed secret sources; its trimmed stdout is the bearer token. An argument that is (or holds) a reference is refused: nothing is expanded.
aws_sigv4: { region, service, access_key_id_env, secret_access_key_env, session_token_env? }	Every request signed (header form, payload hash signed); S3 bodies also get Content-MD5. No AWS SDK: a signer of ~150 lines checked against AWS’s published test vectors.
digest: { user_env, password_env }	RFC 7616, MD5 or SHA-256 (and -sess), qop=auth; the 401’s challenge is answered, then reused with the next nonce count.

Two static forms grew too: { header, value_env, prefix } sends literal text before the value (Authorization: Api-Key <key>, Token <key>), and { headers: [{ header, value_env, prefix? }, …] } sends several credential headers together (Datadog’s API and application keys). The prefix and the header names are part of the block, so of the identity; the values never are.

The HTTP client (apiClient, stable authoring API) gained an optional RequestSigner: asked for each request’s headers (every attempt, so a signature or token is fresh), and told about a 401, after which the request is sent once more. A 401 means nothing happened, so resending is safe for any method.

Tokens are secrets from the moment they exist. AdapterContext gained an optional secret(value), which the engine implements by registering the value with the run’s Redactor; the adapter calls it before the token is first sent, and masks every token it obtained in its own error text as well (for engines without the hook). Tokens live in memory only, one per run and API block (a cache keyed by the run’s ctx, the block and a hash of the credential’s inputs), are refreshed shortly before they expire (OAuth2 expires_in; five minutes for a command) and once after a 401.

encoding: xml, with a minimal codec (xml.ts): elements to maps, repeated elements to lists, text to strings, attributes as @name. No DTDs or entities beyond the predefined ones (no XXE, no expansion). Where list_item expects a list in an XML document, a lone element is a list of one. Two list_item parent options make S3 CORS expressible, and are not S3-specific: if_missing: empty (a 404 parent is an empty collection) and when_empty: delete (an API that refuses an empty collection has it removed instead). send: collection, with parent.item_path (from main), covers APIs that replace a map they are sent, read nested and written flat (Amplify UpdateBranch). A versioned JSON media type (media_type) covers Atlas.

GraphQL as a request kind of http.resource: any of find, read, create, update, delete may be { graphql: { query, variables, data_path, idempotent } }, a POST to the API’s base_url; errors[] are classified into the same PROVIDER_* codes as HTTP statuses. The key of a GraphQL-located object is its root field, its variables and its data path (not the query text); the record id is the delete mutation with its variables, so destroy still needs only the ledger. { secret } is refused in the variables of the requests that become the key or the record id. A scalar answer is read as { value }, so Railway’s variable map can be compared field by field.

Invariants

Invariant	How it is kept
Credentials only from environment variable NAMES	The forms take names (*_env, ending like the names the engine masks); a command takes literal arguments only.
No credential or token in output, receipts or errors (I1)	Engine-masked variables; obtained tokens registered through actx.secret before first use and masked locally too; GraphQL error names never include mutation variables; scenarios assert I1 with every issued token listed.
Identity holds no secret	The block is names and URLs; token caches are keyed by a hash, in memory.
plan writes nothing (I4)	Token requests and GraphQL queries are POSTs that change nothing: the sims leave them out of the write log (ProviderSim.isWrite), and a scenario per form checks it.
Idempotent apply, drift by hash, destroy only what Sponson created	Unchanged: the new forms change how a request is authenticated or encoded, not what is sent or compared.

Testing

The sims check each scheme for real, server side, with code written from the specifications independently of the client (the sim depends on nothing in the repo): the REST sim’s auth seed adds an OAuth2 token endpoint (expiry, revocation), fixed bearer tokens (a command’s), Digest challenges and SigV4 verification; an aws sim verifies SigV4 for S3 CORS (XML, Content-MD5, NoSuchCORSConfiguration, MalformedXML) and Amplify branches; railway and fly sims answer the operations those providers’ official clients send. Both the client signer and the sim’s verifier are pinned to AWS’s published vectors, and the Digest code to RFC 7616’s worked example, so a wrong signer fails against the sim instead of being mirrored by it.

Not done

• private_key_jwt client authentication (Okta’s own management API requires it), the OAuth2 device or authorization-code flows, AWS profiles, instance roles and SSO (only keys from variables), SigV4a, presigned URLs, chunked uploads, Digest auth-int.
• GraphQL subscriptions, pagination of a GraphQL find, and filling mutation variables from fields (values are written in both places).
• A secret whose value the API never returns (Fly.io) is not compared, so a changed value is not detected.
• The examples for Auth0, Kinde, Okta, Google, Azure, AWS, Atlas, Railway and Fly.io are checked against sims built from those providers’ documentation and official clients, not against the live APIs (docs/api-verification.md).

Consequences

• ApiClientOptions.signer / accept and encoding: "xml", the RequestSigner and SignableRequest types, and AdapterContext.secret are new optional parts of the authoring API; nothing existing changes meaning.
• ProviderSim.authorized receives the request (for signatures) and the provider’s state, and may answer with a Reply (a challenge); ProviderSim.isWrite lets a sim keep non-mutating POSTs out of the write log.

Amendment: line credentials, deletes that name the object, XML lists

Date: 2026-10-11

Context

Three coverage rows stopped at the request level (docs/coverage.md; ADR 0020, Amendment 2 listed them as out of reach):

• A credential that is another line’s output. Postmark’s Server API authenticates with the token of one server, and the plan creates that server per environment (postmark.server, whose server_token is a sensitive output shown on every read). An API block names environment variables only, and must: it is the ledger identity of the API’s resources, and it is checked before any line has run.
• A delete that names the object in its body. Datadog deletes Synthetic tests with POST /synthetics/tests/delete and { public_ids: [...] }; delete.body (the archive amendment above) took literals only.
• One-element lists in XML. S3 answers a CORS rule with one AllowedMethod as that element alone, so a rule that declares AllowedMethod: [GET] showed an update on every plan.

Decision

Line credentials. A line of http.resource or http.list_item (hand-written or a recipe line) may say credential: { from: <line>.<output> } or credential: { secret: <reference> }. Its value replaces the value of the API block’s variable for that line only; the block’s header name and prefix stay. Rules:

Rule	Why	Where it is checked
Only a block whose auth is a single static credential (bearer_env, or header with value_env)	the other forms derive what they send from several values or exchanges	providerFor, before any request (PARAM_INVALID)
A reference, never a literal	a literal would be a credential in the plan file	the engine, in prepare (PARAM_INVALID)
A from names an output declared sensitive	a non-sensitive output is written to receipts, so the credential would be too	the engine, in prepare, from the producing line’s output declarations
A from does not name a once output	after the creating run, ADR 0018 resolves it to { keep: true }: a dependent’s fingerprint proves it holds the value, not that a request can be sent with it, so every later read would fail	the engine, in prepare
The line says destroy: keep	destroy (and rollback) work from the ledger alone, which never holds a credential; what the line made goes away with the object the credential belongs to (a Postmark server takes its webhooks), or stays when that object is kept	credentialInputs, in prepare (PARAM_INVALID)
Never part of identity	the provider block, key and record id are those of the same line without it, so rotating the producer’s token (--recreate) re-identifies nothing	by construction
Masked from the moment it exists	sensitive outputs and secrets are registered with the redactor by the engine; the adapter calls actx.secret before the value is first sent, and refuse_credential checks it like the variable it replaces	engine and adapter

The engine gains one optional, provider-neutral hook, OpSpec.credentialInputs(params): the names of the params that carry a credential. prepare checks each as above. While the producing line is still to be created, the reference is pending: read returns null (nothing can exist under a credential that does not exist yet), diff shows a create, and the engine applies the line after its producer, as for any from:.

delete.body naming the object. A string in a POST delete’s body may hold {id}; it is filled with the object’s id when the record id is made ( body=<canonical JSON>, filled), so destroy still needs only the ledger. Bodies without {id} archive as before; record ids without a body are unchanged.

XML lists. In http.list_item over encoding: xml, a field the map item declares as a list reads a lone value as that list of one, as the collection itself already did. JSON APIs are unaffected: there a scalar where a list is declared is real drift.

Alternatives considered

• The credential’s reference (not value) in the provider block or the key. It would let destroy find the producing line in the ledger, but the producer is destroyed in the same run (dependents first, then producers), and its output is not in the ledger either (sensitive outputs never are); it would also make a recipe line and a hand-written line for the same object two resources.
• Writing the credential to the ledger, encrypted or not. Breaks invariant 1’s spirit (receipts hold no secret, ADR 0009) for a convenience; and the scope key that would encrypt it is in the same repository.
• The engine reading the producer during destroy to hand the dependent its credential. Destroy would then depend on the plan (a line removed from it could no longer be destroyed) and on another line’s live state; ADR 0005’s “destroy needs only the ledger” is worth more than the webhook a server deletion removes anyway.
• A first-class Postmark adapter. It would hard-code one provider’s chaining for a shape (a sub-API keyed by a parent’s token) that also fits other providers’ per-project keys.

Consequences

• The Postmark server-and-webhook row is covered by two recipe lines in one plan (scenarios/adapters/http-line-credential.yaml, against a REST sim that accepts the server API only with a token it generated for a server, assumption R10); Datadog’s Synthetic tests are a recipe (R11: the sim’s bulk delete), and S3’s one-method CORS rules compare equal.
• A line with credential leaves what it made when the scope is destroyed and its credential’s object is kept: an explicit, per-line exception to invariant 8, like destroy: keep.
• One plan must not manage the same key through two different credentials (two servers’ webhooks with one URL): their identity would be the same. Other scopes’ resources with the same key are refused as foreign, never deleted.
• OpSpec.credentialInputs is a new optional part of the adapter contract; ops that do not define it behave as before.

Amendment: a third op, http.action (2026-10-11)

One-off side effects with nothing to read back or delete (an Inngest sync, a cache purge, a deploy notification) are neither a resource nor a list_item. ADR 0022 adds http.action: one request (or GraphQL request) sent when its declared trigger changes, its state owned by the engine through a new optional hook, OpSpec.action. It reuses this record’s API blocks, credentials and GraphQL requests, and adds a request form this record did not have: an absolute url (the app’s own endpoint), sent without any credential or API header, by a line that names no api.