Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Manifest reference

A tungsten project has two manifests next to each other: tungsten.yml (the shape and naming of the SDKs) and agent.yml (the agent semantics). Both are YAML with JSON Schemas shipped in the binary: tungsten schema tungsten and tungsten schema agent print them, and both are validated before compilation (schema errors are TG06xx diagnostics). For any agent-facing property the precedence is agent.yml, then a spec x-agent-* extension, then the inferred default.

tungsten.yml

Shape and naming of the SDK, independent of agent semantics.

tungsten: 1
api:
  name: acme
  title: Acme
  docs_url: https://docs.example.test
inputs:                                   # several documents, each a namespace
  - namespace: public
    spec: ../acme/protocol/v1/openapi/public-v1.json
    include: { extension: x-implemented, equals: true }
    planned_from: { extension: x-implemented, equals: false }
  - namespace: sealed
    spec: ../acme/protocol/v1/openapi/sealed-v1.json
  - namespace: workflow
    spec: ../acme/protocol/v1/openapi/workflow-tools-v1.json
    rpc_unflatten:
      path: /v1/workflow/tools
      method: post
      discriminator: method
      params: params
      name_strip_prefix: "workflow."
overlays: []                                # OpenAPI Overlay 1.0 files, applied in order
servers:
  default: https://api.example.test     # placeholder
  allow_override: true
resources:                                  # optional overrides of the inferred tree
  public:
    webhooks:
      path: /v1/webhooks
      children:
        deliveries: { path: "/v1/webhooks/{endpoint_id}/deliveries" }
    devices: { path: /v1/enrollment/devices }
    owner_messages: { path: /v1/owner/messages }
    alpha_messages: { path: /v1/alpha/messages }
naming:
  operations:
    public.listWebhookEndpoints: list
    public.createWebhookEndpoint: create
    public.replayWebhookDelivery: replay
  fields: preserve_wire_names_in_ts: false
auth_profiles:
  ownerSession:                             # Composite scheme
    composite:
      - cookie: __Host-acme_session
      - cookie: __Host-acme_csrf
      - header: { name: x-acme-csrf, equals_cookie: __Host-acme_csrf }
      - header: { name: Origin, from_config: origin, when: mutation }
    satisfies: [ownerSession, ownerCsrf]    # the spec's requirement pair
  bearerAuth: { bearer: { env: ACME_API_KEY } }
  workflowCredential: { bearer: { env: ACME_WORKFLOW_TOKEN, prefix: "ztw_" } }
pagination:                                 # explicit overrides; otherwise inferred
  public.listOwnerMessages: { cursor: { param: before, field: next_cursor, items: messages } }
targets:
  typescript: { package: "@acme/sdk", out: generated/typescript, runtime: "^0.1" }
  python:     { package: acme_sdk, out: generated/python, models: pydantic }
  rust:       { crate: acme-sdk, out: generated/rust, cli: { bin: acme } }
  mcp:        { package: "@acme/mcp", out: generated/mcp }
  docs:       { out: generated/docs }
  mock:       { out: generated/mock }

Resource tree inference

When resources is absent: strip the first path segment if it is a version (/v1), split remaining segments, drop {param} segments, and nest. Tags override segment names when a tag covers all operations under a prefix. Method names: list for GET collection, get for GET item, create for POST collection, update for PATCH/PUT item, delete for DELETE, and the last path verb segment for POST actions (enable, rotate, replay, cancel). The operationId camel-cased is the fallback.

:

  • Placement. A configured resource of the namespace whose path is the longest segment prefix of the operation's path wins ({a} matches any {b}; ties go to the first declared). Otherwise the tree is inferred: a leading version segment (v1, v2, v1beta, v2alpha1) is dropped, a trailing action is set aside, and the remaining literal segments nest (/v1/webhooks/{id}/deliveries → webhooks › deliveries). A literal segment that only leads to children becomes a resource with no operations (/v1/owner/messages → owner › messages). An operation with no literal segment goes to a resource named root. A segment that mixes text and placeholders ({date}.csv) is a Template segment in the IR and counts as an item, like {param}; it never names a resource.
  • Custom methods. A :verb suffix on the last segment (/v1/users/{id}:archive, /v1/users:search, any HTTP method) is an action: the operation is placed by the path without the suffix and named after it (users.archive, users.search).
  • Action. The last segment of a POST path is an action when it is a literal right after a {param}, every operation on that exact path is a POST, no path continues below it, and it reads as a verb: one of a fixed verb list (enable, rotate, replay, cancel, revoke, ...) or a last word not ending in s. So /{id}/enable is the action enable, while /{id}/sealed-events is a sub-resource whose POST is create.
  • Method names. naming.operations wins; then, relative to the resource's path: list for a GET of a collection, get for any other GET of the resource's own path (a singleton: GET /v1/health, /users/me, /users/{id}/settings), create (POST of the resource's path), update (PATCH, PUT) and delete on a singleton path, get / update (PATCH, PUT) / delete on one item, the action for an action; anything else keeps its operation id. A GET is a collection when it is paginated, its success JSON body is an array or an object with exactly one array property, or (no JSON body says otherwise) an item path <path>/{id} exists or its last segment is a plural (deliveries). PUT, PATCH and DELETE on a collection path keep their ids. Collisions inside a resource, and between sibling resources, are renamed with a numeric suffix and TG0401 (list, list2 for /v1/pets and /v2/pets, which merge into one pets resource).
  • rpc methods (rpc_unflatten). A method goes under the configured resource of its HTTP path (or the top level), then one level per dotted part of its name except the last, and is named by the last part: workflow.action.send with name_strip_prefix: "workflow." → tools › action › send. A method without dots is placed like an HTTP operation. Child resources declared without a path are matched by name.
  • path_prefix is the longest common segment prefix of the resource's operations and the path that names it (configured path, inferred path up to its literal segment, or the HTTP path of rpc methods), without trailing parameters; a resource with neither uses its descendants.
  • Order. Resources and children appear in the order of their first operation in the document; operations keep document order. Planned operations get method names but stay out of the tree.
  • A configured path that matches no path of the input is TG0604; a naming.operations or pagination key that names no operation is TG0603.

Inputs, filters and gates

  • include and planned_from compare one operation extension with a value. Without include everything is included, and an operation that matches planned_from is planned. With include, an operation that fails it is planned when it matches planned_from and dropped otherwise. Every excluded operation is a TG0504 info naming the reason; planned ones keep that reason in OperationStatus::Planned and live in Namespace.planned, never in the resource tree.

  • rpc_unflatten needs the named path and method to exist and its JSON request body to be a oneOf whose every member has a string const (or one-value enum) under discriminator, with distinct values that leave a dotted name after name_strip_prefix. Any failure is a TG0506 error and the envelope stays one ordinary operation.

  • A single OpenAPI file checked without a manifest compiles with a default one: namespace and api.name from the file stem, no filters, no profiles. Acme's documents compiled that way show every route as callable and infer their paginations (TG0501).

  • pagination.<op> declares one style (cursor, offset, page, link_header) or none: true, which only switches inference off. Declared entries apply to planned operations too.

  • types.prune_unreferenced (default true) drops the named types no operation reaches, after the agent manifest has been applied: roots are the parameters, request and response bodies and headers of every callable and planned operation and the error envelopes (including the one agent.yml names); everything they reach through arrays, maps, records, unions (discriminator mappings included) and intersections stays. The warnings and notes about the schemas of dropped types go with them (errors never do), and one TG0760 note counts the dropped types and names the first eight. A document without operations is a library of schemas and keeps every type. false generates every schema.

Auth profiles

  • A composite profile becomes one AuthScheme::Composite with the profile's name, its parts in order (cookie, header with equals_cookie / from_config / when: mutation → mutation_only, bearer with env / prefix) and satisfies resolved to IR scheme names. A satisfies entry naming no scheme of any namespace is TG0503; a spec name that was namespace-qualified expands to every qualified variant.
  • The profile may share its name with a spec scheme it satisfies. Both stay in Ir.auth, the composite sorted after the spec scheme; operation requirements keep naming the spec schemes.
  • Cookie and header names supplied by a composite give matching parameters the Auth role, except Origin, which keeps Origin.
  • config entries of other profiles are runtime configuration and do not change the IR.
  • a non-composite profile with bearer configures the HTTP bearer scheme of the same name (spec or IR name): HttpBearer.prefix and HttpBearer.env are additive IR fields. The spec's bearerFormat stays format and is only documentation; the prefix is enforced by the runtime before sending.

MCP embedding index (targets.mcp.search.embeddings)

targets:
  mcp:
    package: "@acme/mcp"
    out: generated/mcp
    search:
      embeddings:
        provider: command           # command | http
        command: "node tools/embed.mjs"   # provider: command (words split on spaces; quotes group)
        # url: http://127.0.0.1:8080/v1/embeddings   # provider: http (plain http at generation)
        model: my-embedding-model   # required: recorded in the header, keys the cache
        dimensions: 384             # optional: checked on every answer, sent to http endpoints
        # api_key_env: EMBED_API_KEY  # http: Bearer token read from this variable, never written
        required: false             # true: a provider failure fails generation
        batch_size: 32              # documents per call, 1 to 512
        timeout_ms: 60000           # per call, 1 to 3600000
KeyMeaningDefault
providercommand (JSON on standard input and output) or http (the common /embeddings request shape)required
commandthe command line of a command provider, run from the directory of tungsten.ymlrequired for command
urlthe endpoint of an http providerrequired for http
modelmodel name sent to the provider and recorded in index.embeddings.jsonrequired
dimensionsvector length; a different answer is TG0572taken from the first answer
api_key_envenvironment variable with the endpoint's API key (http only)none
requireda provider failure is an error instead of a warning and BM25 onlyfalse
batch_sizedocuments per provider call32
timeout_mswall-clock limit of one provider call60000

Command protocol (version 1), request on standard input and answer on standard output (exit status 0; standard error is free text):

{ "protocol": 1, "kind": "document", "model": "m", "dimensions": 384, "inputs": ["text", "..."] }
{ "embeddings": [[0.12, -0.03, "..."], ["..."]] }

agent.yml

Agent semantics. Every key compiles into AgentModel / OperationAgentMeta in the IR.

agent: 1
defaults:
  safety:
    get: read_only
    post: mutating
    put: mutating
    patch: mutating
    delete: destructive
  idempotency: none
  preview: local
  unknown_outcome:
    on_timeout: unknown            # never "retry"
    on_connection_reset: unknown
    on_ambiguous_status: unknown   # statuses marked ambiguous in `errors`
  retries:
    read_only: { max: 3, backoff: { base_ms: 200, max_ms: 5000, jitter: full } }
    mutating:  { max: 0 }          # retries only with an idempotency key, see idempotency
    honor_retry_after: true
  disclosure:
    mode: auto                     # auto | discrete | progressive
    threshold: 24
    description_budget_tokens: 60
    schema_budget_tokens: 600

tools:
  - operation: public.submitAlphaMessage
    safety: irreversible
    idempotency: { policy: caller_owned, header: Idempotency-Key, format: uuid_v4, persist: required }
    confirmation:
      summary_fields: [recipient_e164, device_id, test_case_id, expires_at_ms]
      message: "This queues an SMS to {recipient_e164} from device {device_id}. Carrier charges may apply."
    verify:
      operation: public.getAlphaMessageStatus
      args: { message_id: $response.message_id }
      expect: { state: { in: [accepted, queued, claimed, submitting, submitted, delivered] } }
      terminal: { state: { in: [delivered, delivery_unknown, unknown, failed, cancelled, expired] } }
      poll: { interval_ms: 1000, budget_ms: 60000 }
    remediation:
      queue_full: { text: "The device queue is full. Wait and retry with the SAME Idempotency-Key.", retryable: same_key_only }
      recipient_suppressed: { text: "Recipient opted out. Do not retry, do not pick another number. Stop and report.", retryable: never }
      billing_pending: { text: "Account billing is not active. Stop; the owner must act.", retryable: never }
      payment_hold: { text: "Payment hold on the account. Stop; the owner must act.", retryable: never }
  - operation: public.revokeOwnerDevice
    safety: destructive
    confirmation: { summary_fields: [device_id] }
  - operation: public.rotateWebhookEndpoint
    safety: destructive
    response: { sensitive_fields: [secret], shown_once: true }
    remediation_note: "Rotation disables the endpoint and retires pending deliveries. Re-enable after storing the new secret."
  - operation: public.createWebhookEndpoint
    response: { sensitive_fields: [secret], shown_once: true }
  - operation: public.disableWebhookEndpoint
    safety: destructive
  - operation: public.replayWebhookDelivery
    idempotency: { policy: caller_owned, header: Idempotency-Key, format: uuid_v4, persist: required }
    remediation:
      replay_not_eligible: { text: "Only exhausted failed deliveries on an enabled endpoint can be replayed. Check the delivery's terminal_reason and the endpoint's enabled flag; do not retry.", retryable: never }
      replay_limit: { text: "All three replay generations were used. Stop.", retryable: never }
  - operation: sealed.submitSealedOutbound
    safety: irreversible
    idempotency: { policy: content_identity, note: "Retries must resend the identical envelope bytes; the unsigned digest is the identity. Never attach an Idempotency-Key header." }

errors:
  envelope: { schema: Error, code_field: code }
  ambiguous_statuses: [408, 503]                  # on mutations → OUTCOME_UNKNOWN
  non_json:
    - { status: 413, media: text/plain, category: REQUEST_TOO_LARGE, retryable: never }
    - { status: 400, media: text/plain, category: MALFORMED_REQUEST, retryable: never, text: "A path or query parameter failed framework parsing before authentication. Fix the parameter format (UUIDs must be canonical)." }
  codes:
    unauthorized: { category: AUTH_FAILED, retryable: never, text: "Credential missing, expired or revoked. Do not retry; obtain a new credential." }
    forbidden:    { category: AUTH_FAILED, retryable: never, text: "CSRF or Origin check failed, or scope insufficient." }
    rate_limited: { category: RATE_LIMITED, retryable: after_delay }
    unavailable:  { category: UPSTREAM_UNAVAILABLE, retryable: after_delay }
    not_found:    { category: NOT_FOUND, retryable: never }
    conflict:     { category: CONFLICT, retryable: never }
    invalid_request: { category: VALIDATION_FAILED, retryable: never }

gates:                                              # x-runtime-gate rendering
  SEALED_WEBHOOK_DELIVERY_ENABLED:
    text: "This deployment returns 404 for sealed-event selection because SEALED_WEBHOOK_DELIVERY_ENABLED is off. This is a deployment setting, not a missing resource."

macros:
  - name: public.submitAlphaMessageAndAwait
    summary: "Submit a synthetic-alpha message and wait for a terminal delivery state."
    safety: irreversible                             # inherits the strictest step
    steps:
      - call: public.submitAlphaMessage
        args: $input
        as: accepted
      - poll: public.getAlphaMessageStatus
        args: { message_id: $accepted.message_id }
        until: { state: { in: [delivered, delivery_unknown, unknown, failed, cancelled, expired] } }
        interval_ms: 1000
        budget_ms: $input.wait_budget_ms
        as: status
    output: { message_id: $accepted.message_id, created: $accepted.created, state: $status.state, requires_reconciliation: { expr: "$status.state in ['unknown','delivery_unknown']" } }
  - name: public.bootstrapWebhookEndpoint
    summary: "Create a webhook endpoint, enable it, return the one-time secret."
    steps:
      - call: public.createWebhookEndpoint
        args: $input
        as: created
      - call: public.enableWebhookEndpoint
        args: { endpoint_id: $created.endpoint_id }
    output: { endpoint_id: $created.endpoint_id, secret: $created.secret }
    response: { sensitive_fields: [secret], shown_once: true }
  - name: public.listAllOwnerDevices
    summary: "All enrolled devices across pages."
    steps:
      - paginate: public.listOwnerDevices
        max_pages: 20
        as: devices
    output: $devices

disclosure:
  clusters:                                          # semantic grouping for search_tools
    - { name: messaging, operations: [public.alpha_messages.*, public.owner_messages.*, sealed.sealed_messages.*] }
    - { name: webhooks,  operations: [public.webhooks.*, sealed.sealed_webhooks.*] }
    - { name: devices,   operations: [public.devices.*, sealed.sealed_devices.*] }
    - { name: workflow,  operations: [workflow.*] }
  prune:
    descriptions: { max_sentences: 2, drop_phrases: ["Implemented today (", "crates/server/src/"] }
    drop_fields: []                                  # fields hidden from agent schemas
    keep_examples: false

Safety tiers

TierMeaningSDKCLIMCP
read_onlyNo side effectsretries allowedrunsreadOnlyHint: true
mutatingSide effect, reversible or idempotentno blind retry; preview availablerunsdestructiveHint: false, idempotentHint from policy
destructiveDeletes or disables something; recoverable only by re-creationrequires {confirm: true} option or preview firstrequires --yes or TTY confirmationdestructiveHint: true; needs confirmation_token from a prior preview call
irreversibleCannot be undone (money, messages sent to people, permanent deletion)requires {confirm: {token}} obtained from preview()requires --yes --i-understand and prints the summarydestructiveHint: true; needs confirmation_token; the MCP server refuses unless the client supplies the token returned by preview in the same session

Idempotency policies

PolicyWho generates the keyPersistenceRetry rule
nonenobody–mutation never auto-retried
autoruntime, UUIDv4 per logical callruntime IdempotencyStore (memory by default; file or caller-provided)retried with the same key within retries.mutating
caller_ownedthe caller, validated against formatcaller must persist; persist: required makes the runtime refuse a call without a keyretried with the same key only
content_hashruntime, hash of the canonical bodystore keyed by hashsame
content_identitythe body itself is the identity (binary envelopes)n/aresend identical bytes only

The blueprint's "auto-generate a key per call" is wrong for APIs like Acme where a key must survive the agent's own crash. persist: required exists for exactly that case and the generated docs say why.

Preview modes

ModeBehavior
localValidate arguments, resolve the auth profile, render the request (method, URL, headers with secrets redacted, body) and the manifest's side-effect summary. No network. Returns a confirmation_token bound to the argument digest.
headerAs local, then send with preview.header (for example X-Dry-Run: true) and return the server's response as the preview.
endpointCall preview.operation with mapped args.
none.preview() is not emitted.

Remediation table resolution

Runtime lookup order for an HTTP error: operation-specific tools[].remediation[code] → global errors.codes[code] → errors.non_json match on status and media type → generic by status class. The matched entry supplies category, remediation, retryable and, when present, a next_action referencing a verification hook or macro.

Server rules: what the server enforces and the spec leaves out

tools[].server_rules declares rules of the real server that the API description does not state, so that tungsten mock answers a request that breaks one the way the server does (the Acme live pilot found two: reserved callback hosts and an expiry window, pilots/acme/REPORT.md findings 1 and 2). The key is additive: a manifest without it, and the IR of an operation without rules, are unchanged. It is data, never read by the SDKs, the MCP server or the docs.

tools:
  - operation: public.createWebhookEndpoint
    server_rules:
      - { field: callback_url, reject_host_suffixes: [".test", ".example", ".invalid", ".local", ".localhost"], status: 400, code: invalid_webhook_endpoint }
  - operation: public.submitAlphaMessage
    server_rules:
      - { field: expires_at_ms, max_ahead_minutes: 15, status: 400, code: invalid_request }
KeyMeaning
fieldThe request member read: a dotted path into the JSON body (options.expires_at_ms), else a query parameter of that name. A member that is absent, or whose value is not of the kind the check reads, is not this rule's business.
reject_host_suffixesThe value is a URL; the rule is broken when its host (without userinfo, port, case or a trailing dot) is a suffix or the bare name of one (.test matches a.test and test, not latest.com).
max_ahead_minutesThe value is epoch milliseconds (a number or numeric text); the rule is broken when it is later than now plus that many minutes. A time in the past passes.
statusThe refusal's status, 400 to 599; default 400.
codeThe API error code written at the error envelope's code field (text/plain APIs lead the text with it).
messageThe reason (X-Tungsten-Reason); default names the field and the rule.

Exactly one of reject_host_suffixes and max_ahead_minutes per rule (TG0602 otherwise). Compiled form: OperationAgentMeta.server_rules: Vec<ServerRule> (field, check tagged host_suffixes { suffixes } or max_ahead_minutes { minutes }, status, code?, message?), written only when non-empty. The mock checks them after the description's own checks (authentication, parameters, body schema) and before idempotent replay, so a refused request is never stored; rules apply to their operation only, the first broken one answers. Tests: unit_mock_rules, unit_agent_*, corpus agent-server-rules, contract scenarios mock-refuses-reserved-callback-hosts and mock-refuses-an-expiry-beyond-the-window.

Vendor extensions accepted from the spec

x-agent-safety, x-agent-idempotency, x-agent-preview, x-agent-confirmation, x-agent-remediation, x-agent-verify, x-agent-sensitive (on schema fields), x-agent-cluster. Same shapes as the manifest entries. Unknown x-agent-* keys raise TG0610.

Details

tungsten-agent loads agent.yml (next to tungsten.yml; schema specs/agent-manifest.schema.json, printed by tungsten schema agent) and applies it to the IR after the build. Refinements of the plan above:

  • Precedence: agent.yml > x-agent-* > inferred (a required Idempotency-Key header makes a side-effecting operation caller_owned with persist_required) > the manifest's defaults > the method defaults. A rule that cannot apply is dropped with a warning (TG0603–TG0611) and the rest of the manifest still applies.
  • AgentModel is flat: macros, clusters, gates, error_codes, ambiguous_statuses, non_json, retries, unknown_outcome, disclosure. OperationAgentMeta also has remediation_note and hidden. A gates entry may declare disabled_status.
  • compact_doc is the summary plus the description, sentences with a drop_phrases entry removed, the first max_sentences kept, cut at a word boundary to description_budget_tokens (a token counted as four characters).
  • Clusters: each operation and macro belongs to at most one cluster, the one its tools entry or x-agent-cluster names, else the first declared cluster whose list (operation references, macro names, <ns>.* and <ns>.<resource path>.* globs) covers it.
  • Macros are checked against the compiled API and normalized to the canonical form on tungsten_ir::Macro (steps {kind, operation, args, as, until, interval_ms, budget_ms, max_pages}, input: {extends, add}); a macro carries cluster, sensitive_response_fields and shown_once. Its safety is the strictest step's (a poll counts as read_only); a weaker declared tier is raised (TG0609).
  • Macro confirmation is macro-level. A macro whose tier is destructive or irreversible (or that has such a step) needs one confirmation for the whole run: confirm: true for destructive, otherwise a token from the macro's own preview, bound to the macro name and its exact input (after defaults). That confirmation authorizes every step; steps never ask for tokens of their own, since their arguments can depend on earlier results. The macro preview validates and renders the first step and lists every step's effects (arguments only known later read <from an earlier step>), without sending anything.
  • At run time an added input field's JSON Schema default is applied when the input omits it, and a poll step whose budget runs out binds its as name to null (so {expr: "$status == null"} reports the timeout; the timed_out output). The first failing step's envelope is returned with the completed steps named in its remediation.
  • disclosure.prune.drop_fields and keep_examples are accepted but not applied in this release: agent tool schemas list exactly the SDK's arguments, and a non-empty drop_fields (or keep_examples: true) is a TG0613 warning instead of a silent no-op. Their semantics (which fields may be hidden, how they are named, what the MCP server does with a hidden field) are decided with the MCP emitter in this release.
  • tungsten.yml targets must have distinct, non-nested out directories (TG0612): each target's .tungsten/manifest.json removes the files it no longer generates, so targets sharing a directory deleted each other's output on every run.

Overlays

overlays: in tungsten.yml lists OpenAPI Overlay 1.0 files, applied in order before the document is compiled. Use them to repair a third-party description without editing it. tungsten overlay suggest writes an overlay whose actions fix a missing or repeated operationId:

tungsten overlay suggest . --write fixes.overlay.yaml