Operations

Process model

Every CX process — a one-shot program, an HTTP service, the store daemon, a XAP host — runs in the foreground and owns its lifecycle. There is no built-in daemonizer and no bespoke supervision machinery, by explicit project decision:

• The port is the only mutex. A serving process that cannot bind fails fatally and loudly — it never half-runs. At most one instance per port, arbitrated by the kernel; replacing a running instance is an explicit stop-then-start, never a silent takeover.
• A crash is a bug to root-cause, not respawn over. Development and attended operation run unsupervised. Unattended operation delegates restarts to the OS supervisor — systemd on Linux, launchd on macOS — which is also where restart policy, resource limits, and journaling belong.
• Clean shutdown on SIGTERM. Long-running processes stop accepting, drain in-flight work, flush, and exit 0. kill -TERM is the normal stop verb everywhere.
• Bounded serving. HTTP serving separates reactors (socket owners) from a bounded executor pool running handlers; overflow answers 503 rather than freezing the accept plane. Pool sizes are deploy-time environment config (subsection on environment variables below).

The two long-running processes the platform ships are the store daemon (ops-store-daemon) and the XAP host (ops-xap-host); both follow every rule above.

The store service daemon

Two daemons ship with the platform. The first is the CX Store service: cx store-serve runs a multi-threaded content-addressed store server over the XSP store profile — the store wire, reached by clients as cx-store:// (TLS) or cx-store+xsp:// (cleartext dev) — with an optional gRPC edge listener; plain HTTP serves only the bootstrap endpoints (health, ready, metrics, capabilities). Its sibling, cx fabric-serve, runs the single-node fabric eventing tier: journal-backed fabrics served to XSP-AUTH clients over raw XSP frames, configured the same way (--config fabric.service.cx; the fabric spec is spec/03-approved/platform/fabric.md). The rest of this subsection covers the store daemon. It is configured by a CX document and gated by the ordinary capability flags — serving binds a socket, so it needs --allow-net:

      $ cx store-serve --config /etc/cxstore/cxstore.service.cx --allow-net
    

The config is itself CX data — this is a complete minimal config serving one in-memory store on loopback:

      [cxstore-service
  [bind addr="127.0.0.1:18761"]
  [stores [store name="docs" url="mem://docs"]]]
    

On startup the daemon states exactly what it is doing — including an honest warning when you have not configured auth:

      cx store-serve: listening on tcp://127.0.0.1:18761 — 1 store(s): docs (mem://docs), 4 workers, auth OPEN — no [xsp [grants …]] configured, anonymous full access (configure grants for production)
    

Liveness and readiness are plain HTTP endpoints, and the binary doubles as its own probe (cx store-health exits 0 iff the daemon reports accepting — this is what the Docker HEALTHCHECK and load balancers call):

      $ curl -s http://127.0.0.1:18761/cx-store/v1/health
[health [status "ok"]]
$ curl -s http://127.0.0.1:18761/cx-store/v1/ready
[ready [accepting true] [draining false]]
$ cx store-health --url http://127.0.0.1:18761/cx-store/v1/ready && echo ready
ready
    

Lifecycle, in signals:

Two listeners. [bind addr=…] is required and is a bootstrap-only HTTP surface: health, ready and capabilities under /cx-store/v1/, plus a daemon-level /metrics. It carries no store operation — any other path answers 404 CXER1709 naming the profile as the wire. The data wire is the XSP store profile on its own listener ([xsp enabled=true addr=…]), with the optional gRPC edge ([grpc enabled=true addr=…]) beside it; the three addresses must differ, and [xsp enabled=true] requires an [identity did= seed-env=] — attach is XSP-AUTH and there is no anonymous responder. capabilities is the discovery advert: wire-version, server-impl, config-generation, the encodings, the admin-ops this daemon routes, the request and response size caps, the rate limit and — when the profile listener is up — [xsp [addr …] [did …]], so a management client learns the store wire's address and responder DID from the bootstrap port instead of being configured with them twice.

Two operator verbs round out the surface. cx store-rotate-kek re-wraps an encrypted store under a new key-encryption key, resumable and fail-closed (store-security). cx store-mint-principal mints an XSP-AUTH principal offline — the next subsection walks a deployment from a clean state with it. There is no token-minting verb, because authority is not a bearer secret: clients hold DIDs, and the daemon's [xsp [grants …]] table decides every operation (ops-authority). Metrics, tracing, logs and backpressure are ops-observability; the gRPC edge is store-transports.

Clean-state bootstrap — mint, grant, serve, present

Store credentials are XSP-AUTH principals: an Ed25519 did:key whose seed the holder proves possession of per attach. [xsp [grants …]] is the only grant table, and a non-empty table means deny-by-default — the daemon admits exactly the DIDs written into its config and refuses every other principal. That leaves one question a fresh deployment has to answer: where does the first principal come from? cx store-mint-principal answers it offline — it generates the seed and derives the DID locally, touching no network and no store. Minting grants nothing; the config is still the sole authority.

1 — Mint the principal. The seed lands in a 0600 file and is never printed. --caps is required — the authority a grant carries is stated at mint time, never inherited from a default:

      $ cx store-mint-principal --id ops --seed-file ./ops.seed --caps "read write admin\"
    

stdout is one canonical CX element — machine-readable and copy-paste-able — carrying the DID, the config stanza to splice into the daemon's [xsp …] section, and the open-opts a client presents:

      [xsp-principal id=ops for=grant did=did:key:z6Mk… seed-file='./ops.seed' seed-env=CX_XSP_SEED_OPS
  [config-stanza [xsp [grants [grant did=did:key:z6Mk… caps='read write admin']]]]
  [client-opts [map xsp-did=did:key:z6Mk… xsp-seed-env=CX_XSP_SEED_OPS]]]
    

Operator guidance goes to stderr. Back the seed file up — it is the only proof of that DID, and a lost seed means re-minting and re-granting. Re-running the verb against an existing --seed-file is refused; --force replaces it and invalidates the old DID. --id must be the canonical spelling of the environment variable it derives — lowercase, - for word breaks: CX_XSP_SEED_<NAME> upper-cases and folds - to _, so fleet_ops and Fleet-Ops would derive the same variable as fleet-ops, and two principals minted under two spellings would silently share one seed. The verb refuses the non-canonical spellings, naming the id to use and the variable they would have shared.

2 — Mint the daemon's own identity. Attach is XSP-AUTH in both directions, so the daemon needs a responder identity of its own. Same verb, --for identity, and no --caps: the daemon's identity is the root of its authority, not a grantee.

      $ cx store-mint-principal --id host --seed-file ./host.seed --for identity
    

3 — Assemble the config. Both [config-stanza] bodies splice into the same [xsp …] section verbatim — the identity row from step 2, the grant row from step 1 (the fabric daemon takes the same [identity did= seed-env=] row at its own top level):

      [cxstore-service
  [bind addr="127.0.0.1:18761"]
  [stores [store name="main" url="file:///var/lib/cxstore/main"]]
  [xsp enabled=true addr="127.0.0.1:18762"
    [identity did="did:key:z6MkHost…" seed-env="CX_XSP_SEED_HOST"]
    [grants [grant did="did:key:z6Mk…" caps="read write admin"]]]]
    

4 — Serve. The daemon resolves seed-env while parsing the config, so the host seed is exported before it starts. With grants present the daemon is deny-by-default from the first attach; no separate switch turns enforcement on:

      $ export CX_XSP_SEED_HOST="$(cat ./host.seed)"
$ cx store-serve --config cxstore.service.cx \\
    --allow-read --allow-write --allow-env \\
    --allow-net=127.0.0.1:18761 --allow-net=127.0.0.1:18762
    

The startup line says which posture it came up in — auth ENFORCED (XSP-AUTH, N grant(s)) with a grant table, auth OPEN — no [xsp [grants …]] configured, anonymous full access without one. Read it on every deploy; it is the daemon's own answer, not an inference from the config file you think you shipped.

5 — Present the identity. The client loads the seed from the environment — never from the URL, never from an opts literal — and passes the [client-opts] map verbatim:

      $ export CX_XSP_SEED_OPS="$(cat ./ops.seed)\"
    

      [?lib 'cx-platform/store' as=store]
[?let [= $c [$store:open-opts 'cx-store+xsp://127.0.0.1:18762/main/'
              [map xsp-did='did:key:z6Mk…' xsp-seed-env='CX_XSP_SEED_OPS']]]
  [$store:put-doc $c [note [body 'first write']]]]
    

A principal that was never granted is an equally well-formed identity and is still refused: the daemon rejects the op and the client raises CXER1131 (E_STORE_AUTH_FAILED) naming the rejected wire op. Deny-by-default is a property of the config, not of the credential's cryptographic quality.

Authority — one grant table

There is exactly one authority model, shared by the profile listener and the gRPC edge with no second implementation. [xsp [grants …]] is the only grant table; the retired store-HTTP-era [auth …] section — static tokens, JWT, OIDC, role bundles, the Bearer base — is a hard config error that names the replacement, never a silently ignored stanza. Each [grant did="…" caps="…" over=?] row compiles to an ordinary delegation in the session's authority basis at attach, so one decision function governs every verb — attenuation, revocation and explain come for free rather than as a parallel role engine:

        [?lib 'cx-stdlib/authz']
[?lib 'cx-platform/authz-store']
[?let [= $az [$authz-store:store {tenant: 'docs'}]]
  [= $g [$authz-store:delegate $az
    [delegation grant-1 [tenant docs]
      [from [principal 'did:key:z6Mkgxj2R3HLtQRpPnvfvpuKEceSqf3tZHBjdmZ3fFz3JHGG']]
      [to [agent 'did:key:z6MkneMkZqwqRiU5mJzSG3kDwzt9P8C59N4NGTfBLfSGE7c7']]
      [capabilities [read] [write]] [over '/orders'] [assurance :t1] [signature sig-daemon]]]]
  [$authz:check $az [authz-request
    [actor [agent 'did:key:z6MkneMkZqwqRiU5mJzSG3kDwzt9P8C59N4NGTfBLfSGE7c7']]
    [capability read] [slice '/orders/9'] [tenant docs]]]]
      

        $ cx prog.cx
[permit rooted-principal=did:key:z6Mkgxj2R3HLtQRpPnvfvpuKEceSqf3tZHBjdmZ3fFz3JHGG [delegation grant-1] [via grant-1] [tier :t1] [capability read]]
      

• The capability grammar is read, write, delete, admin, peer. An unrecognised class is a startup error listing the accepted set. over="/path" slices a grant to a subtree; [grant floor=true caps="read"] is the anonymous floor, paired with [xsp [policy mode=floor floor=NAME]]. admin carries status, gc, mounts and config-reload; peer is deliberately narrower than read — a peer daemon receives revocations without holding read on any data plane.
• Posture follows the table's presence. Grants configured means deny-by-default, every verb checked at the policy enforcement point and the PEP's [deny …] value crossing the wire verbatim. No grants means the open dev posture: data verbs open, the daemon-level admin verbs still requiring a DID-proven principal. Absent is open, present is enforced; there is no third switch.
• Delegation and presentation. A client may attenuate its authority further by presenting a credential chain ([vp …] on attach, or a later phase=present) whose terminal subject is the session principal byte for byte. Escalation on any axis refuses; a root issuer this deployment does not recognise compiles to nothing rather than conjuring authority; a credential from a revoked chain stops verifying at its next use.
• Revoking a principal is a [xsp [grants …]] edit plus SIGHUP, and it is live: the reload re-folds the grant table into every session already attached, so the revoked principal is refused at its very next call, not at the next restart. Revocation reaches delegated authority too — a credential that principal presented only ever conveyed authority through the config grant it was rooted in. The gRPC edge compiles its basis per call and picks the change up the same way. A live session's authority only ever narrows on reload: remove every grant and the attached sessions keep enforcing until they re-attach.
• Client identity never rides the wire URL or a literal. xsp-did names the DID, xsp-seed-env names the environment variable holding the seed.

The grammar and the refusal codes are the XSP store profile's listener-model section (spec/03-approved/xap/xsp_store_profile.md) and the gRPC edge's spec/03-approved/misc/cxstore-grpc.md §4, which share the one calculus; the gRPC edge authenticates per call — the authorization metadata carries a CxCall credential signed by the presenting DID over the exact method path, the exact request bytes, a timestamp inside a freshness window and a single-use nonce — against the same table and the same PEP.

Observability and backpressure

• GET /metrics — Prometheus exposition on the bootstrap port, an unauthenticated operator plane: the bind address is the operator's control, and a standard scraper cannot sign a request. It is not exempt from admission control — the global pre-auth bucket covers it, so a scrape flood cannot force unbounded work. Cardinality is bounded by construction (labels: endpoint, status, store — never per hash): request counters, histograms and bytes, an in-flight gauge, per-store document, object and dedup gauges. No fabricated zeros — a store-internal series attaches only when the backend instruments it.
• OpenTelemetry tracing — off by default; W3C trace-context is propagated either way so logs correlate; OTLP/HTTP export when enabled.
• Structured logs — one record per bootstrap-surface request (endpoint, store, status, latency, bytes in and out, trace id), CX or JSON, with the health and ready probes suppressed as noise. The record carries no credential material, and the startup banner redacts the userinfo of any mount URL that has one (an ftp://user:pass@… substrate) rather than printing it.
• Backpressure, on by default. The bootstrap surface and the gRPC edge run a token-bucket rate limit plus a concurrency cap keyed on the resolved store name — tenant-level backpressure — behind a global pre-auth bucket covering unauthenticated traffic. Every admission rejection is 429 CXER1706 with Retry-After; draining or not ready is 503 CXER1708. The profile listener carries its own flow control — [xsp [limits pending-window= liveness-ms= handshake-ms= max-frame-bytes=]] — and [xsp [limits [pushdown steps= memory-mb=]]] budgets daemon-side evaluation for the compute-class verbs: an exhausted budget is a loud typed refusal, never a daemon takedown.
• Reload, hot and restart-required. SIGHUP (or the admin-gated config-reload op) is validate-then-swap, all-or-nothing, fail-closed. Hot: [tls] (the cert and key contents, so rotation needs no restart), [timeouts], [limits], [observability] and [xsp [grants …]]. Restart-required: [bind], [grpc], [stores], [workers] and every part of [xsp] other than [grants] — a change there refuses the whole reload with CXER1712 naming the offenders. Env-injected secrets are rotation-blind; use file-based sources to rotate.

The service-tier contract is spec/03-approved/misc/cxstore_service_tier_phase2.md (config appendix, tenancy, observability); the tenant boundary it fixes — store-per-tenant, one dedup pool per mount — is taught in store-security. Intra-store tenancy, quotas and multi-node are the demand-gated next phase and are not implemented.

The XAP host

A XAP is authored as three data documents — a feature per grammar, the deployment document, a surface per medium (dist-authoring) — and served by one call: [$xap:host XAP-DOC OPTS] boots a complete XAP from its deployment document. It acquires the pinned packages, verifies them, composes the grammar (the compose gate at load), translates governance into runtime dials, loads each feature's contract module, and serves the standard surface — GET /grammar, /features, /surface/<f>, POST /intent, GET /stream. A XAP with no custom transport is zero server code; deployment-specific pieces (extra routes, ingest workers, a simulation cadence) register as adapters through OPTS and never fork the host. The reference deployment and its client are the XAP repository's reference/shop and reference/shop-web-client (their page: dist-reference-surfaces); a project built on them runs with three make targets:

      $ make run     # the XAP on :9001, foreground (its ingest adapter by default)
$ make dev     # the XAP, the web client and a browser; Ctrl-C stops both
$ make check   # the drift gate — specs against the implementation, fails on drift
    

The host obeys the process model above to the letter. The deployment entry uses [$xap:host …]! — the postfix that turns an error value into a fatal exit — so a XAP that cannot serve never half-runs its ingest workers; and the serve verb refuses a URL with no port, and refuses to serve an errored runtime at all, propagating the error rather than relabelling it:

        [?lib 'cx-platform/xap' as=xap]
[?let [= $rt [$xap:run {tenant: 'acme'}]]
  [$xap:serve 'https://acme.example' {runtime: $rt tenant: 'acme' shell: '/nonexistent' block: false}]]
      

        $ cx prog.cx
[err code=cx-err:CXER4852 message='E_XAP: serve URL needs an explicit port: https://acme.example' at='3:3']
      

        [?lib 'cx-platform/xap' as=xap]
[?let [= $g [$xap:compose [not-a-feature]]]
      [= $rt [$xap:run {tenant: 'acme' grammar: $g}]]
  [$xap:serve 'https://acme.example' {runtime: $rt block: false}]]
      

        $ cx prog.cx
[err code=cx-err:CXER4873 message='E_XAP_FEATURE_INVALID: argument 1: [not-a-feature …] is not a [feature …] document']
      

Two layers of authority, both deny-by-default, and never blurred: host capabilities — cx grants read, write, net, env per invocation, and an ungranted effect raises CXER0271 naming the flag (capabilities) — and XAP authority (the trust-model part of the XAP spec): authority originates only from principals, every intent passes the one policy enforcement point, grants are per verb, scoped and conditioned, and the dial moves who drives, never what is permitted. Identity is DID-first — did:key, offline, right for a disconnected deployment — and a delegation is a verifiable credential.

A session is a (principal, tenant) attachment: the server owns session state, clients attach, detach and mirror-attach, and the session survives the death of a client. Authoritative state is the journal fold, never the session. What runs today is the single in-process runtime a long-lived deployment uses; the multi-tenant gateway and worker topology is design-frozen and implemented incrementally (the session module spec and the XAP architecture notes). The attachment, replayed:

        [?lib 'cx-platform/session']
[?lib 'cx-stdlib/crypto']
[?lib 'cx-stdlib/time']
[?let [= $s [$session:attach [request scheme="https" [headers [header name="Authorization" value="Bearer eyJhbGciOiJSUzI1NiIsImtpZCI6InJzYS0xIiwidHlwIjoiSldUIn0.eyJpc3MiOiJodHRwczovL2lkcC5leGFtcGxlIiwic3ViIjoidXNlci0xIiwiYXVkIjoibXktYXBpIiwiZXhwIjoxNzAwMDAzNjAwLCJuYmYiOjE2OTk5OTk5OTAsImlhdCI6MTY5OTk5OTk5MH0.e4bn1A_a5Q0_4IZxKWipmmrUCCdmNkiTUs071mBMtMVXb68RGPtV2FTR1nRWvM3Zw2XTMuBsk_Y1HPinat_2JQkm3s90lltrErpYOky6Nwm6ha57BxQ0Sg5kWxuQF5KhWUqHzePIVDTfkr0WeN591nDTDn6VuajLaHn2pjXBsaJ0reT6mh6v40UQcBW-yy-XOfrsMeGVj_qZd1UmlE81XN957rjPENcPpH2E1SHfVhu3fQP3QZapxtdMADgWmryw_z5O2lDIIWb7miT0siXRCDjVEhDzKH6Hsf1aUPcYx9x44o1B5OBpX6z1uM1SP0qaOZu6x26H0AbHj84nIISE1A"]]] {jwks: [$crypto:jwks-parse "{\"keys\":[{\"kid\":\"rsa-1\",\"kty\":\"RSA\",\"alg\":\"RS256\",\"n\":\"jdwwBcaMZQLoSNYGNEm3l03HIQqpRIv0eqLUNUCkyv7ysVw4i6vZgdYxcdU0D3kSvUCIjH-icqk4PCDx5AwkeNp55Nqt6wKXDv9TH5pr-Wc3BWmZ1sEOEwyN8QlI8_5EpY3i1w5tysDdeFuiR7BOjpkD49RZzF0YajmocsB4_jXoZcldVNCOChXAbEfOw-BtjFJRN0N3EksymF4azkey8Q3rX07sXkRHavRfAOnowH119RL1V-Xmk_DUX8wSR-2zdlp8FxitJWPgbMmnszsTgEQrThTwsuMrndqzJ_nQ7HTnuK2LXEFmFwXw9NADspq2ZnQmZiN7CXnEYG8WCApzxw\",\"e\":\"AQAB\"}]}"] tenant-claim: "aud" now: [$time:datetime 2023 11 14 22 13 20]}]] [$session:principal $s]]
      

        $ cx prog.cx
[principal id=user-1 [tenant id=my-api]]
      

        [?lib 'cx-platform/session']
[?lib 'cx-stdlib/crypto']
[?lib 'cx-stdlib/time']
[?let [= $s1 [$session:attach-token "eyJhbGciOiJSUzI1NiIsImtpZCI6InJzYS0xIiwidHlwIjoiSldUIn0.eyJpc3MiOiJodHRwczovL2lkcC5leGFtcGxlIiwic3ViIjoidXNlci0xIiwiYXVkIjoibXktYXBpIiwiZXhwIjoxNzAwMDAzNjAwLCJuYmYiOjE2OTk5OTk5OTAsImlhdCI6MTY5OTk5OTk5MH0.e4bn1A_a5Q0_4IZxKWipmmrUCCdmNkiTUs071mBMtMVXb68RGPtV2FTR1nRWvM3Zw2XTMuBsk_Y1HPinat_2JQkm3s90lltrErpYOky6Nwm6ha57BxQ0Sg5kWxuQF5KhWUqHzePIVDTfkr0WeN591nDTDn6VuajLaHn2pjXBsaJ0reT6mh6v40UQcBW-yy-XOfrsMeGVj_qZd1UmlE81XN957rjPENcPpH2E1SHfVhu3fQP3QZapxtdMADgWmryw_z5O2lDIIWb7miT0siXRCDjVEhDzKH6Hsf1aUPcYx9x44o1B5OBpX6z1uM1SP0qaOZu6x26H0AbHj84nIISE1A" {jwks: [$crypto:jwks-parse "{\"keys\":[{\"kid\":\"rsa-1\",\"kty\":\"RSA\",\"alg\":\"RS256\",\"n\":\"jdwwBcaMZQLoSNYGNEm3l03HIQqpRIv0eqLUNUCkyv7ysVw4i6vZgdYxcdU0D3kSvUCIjH-icqk4PCDx5AwkeNp55Nqt6wKXDv9TH5pr-Wc3BWmZ1sEOEwyN8QlI8_5EpY3i1w5tysDdeFuiR7BOjpkD49RZzF0YajmocsB4_jXoZcldVNCOChXAbEfOw-BtjFJRN0N3EksymF4azkey8Q3rX07sXkRHavRfAOnowH119RL1V-Xmk_DUX8wSR-2zdlp8FxitJWPgbMmnszsTgEQrThTwsuMrndqzJ_nQ7HTnuK2LXEFmFwXw9NADspq2ZnQmZiN7CXnEYG8WCApzxw\",\"e\":\"AQAB\"}]}"] tenant-claim: "aud" now: [$time:datetime 2023 11 14 22 13 20]} {channel: "cx" tls: true}]] [= $s2 [$session:attach-token "eyJhbGciOiJSUzI1NiIsImtpZCI6InJzYS0xIiwidHlwIjoiSldUIn0.eyJpc3MiOiJodHRwczovL2lkcC5leGFtcGxlIiwic3ViIjoidXNlci0xIiwiYXVkIjoibXktYXBpIiwiZXhwIjoxNzAwMDAzNjAwLCJuYmYiOjE2OTk5OTk5OTAsImlhdCI6MTY5OTk5OTk5MH0.e4bn1A_a5Q0_4IZxKWipmmrUCCdmNkiTUs071mBMtMVXb68RGPtV2FTR1nRWvM3Zw2XTMuBsk_Y1HPinat_2JQkm3s90lltrErpYOky6Nwm6ha57BxQ0Sg5kWxuQF5KhWUqHzePIVDTfkr0WeN591nDTDn6VuajLaHn2pjXBsaJ0reT6mh6v40UQcBW-yy-XOfrsMeGVj_qZd1UmlE81XN957rjPENcPpH2E1SHfVhu3fQP3QZapxtdMADgWmryw_z5O2lDIIWb7miT0siXRCDjVEhDzKH6Hsf1aUPcYx9x44o1B5OBpX6z1uM1SP0qaOZu6x26H0AbHj84nIISE1A" {jwks: [$crypto:jwks-parse "{\"keys\":[{\"kid\":\"rsa-1\",\"kty\":\"RSA\",\"alg\":\"RS256\",\"n\":\"jdwwBcaMZQLoSNYGNEm3l03HIQqpRIv0eqLUNUCkyv7ysVw4i6vZgdYxcdU0D3kSvUCIjH-icqk4PCDx5AwkeNp55Nqt6wKXDv9TH5pr-Wc3BWmZ1sEOEwyN8QlI8_5EpY3i1w5tysDdeFuiR7BOjpkD49RZzF0YajmocsB4_jXoZcldVNCOChXAbEfOw-BtjFJRN0N3EksymF4azkey8Q3rX07sXkRHavRfAOnowH119RL1V-Xmk_DUX8wSR-2zdlp8FxitJWPgbMmnszsTgEQrThTwsuMrndqzJ_nQ7HTnuK2LXEFmFwXw9NADspq2ZnQmZiN7CXnEYG8WCApzxw\",\"e\":\"AQAB\"}]}"] tenant-claim: "aud" now: [$time:datetime 2023 11 14 22 13 20]} {channel: "http" tls: true}]] [$count [$session:clients $s2]]]
      

        $ cx prog.cx
2
      

        [?lib 'cx-platform/session']
[?lib 'cx-stdlib/crypto']
[?lib 'cx-stdlib/time']
[?let [= $s [$session:attach-token "eyJhbGciOiJSUzI1NiIsImtpZCI6InJzYS0xIiwidHlwIjoiSldUIn0.eyJpc3MiOiJodHRwczovL2lkcC5leGFtcGxlIiwic3ViIjoidXNlci0xIiwiYXVkIjoibXktYXBpIiwiZXhwIjoxNzAwMDAzNjAwLCJuYmYiOjE2OTk5OTk5OTAsImlhdCI6MTY5OTk5OTk5MH0.e4bn1A_a5Q0_4IZxKWipmmrUCCdmNkiTUs071mBMtMVXb68RGPtV2FTR1nRWvM3Zw2XTMuBsk_Y1HPinat_2JQkm3s90lltrErpYOky6Nwm6ha57BxQ0Sg5kWxuQF5KhWUqHzePIVDTfkr0WeN591nDTDn6VuajLaHn2pjXBsaJ0reT6mh6v40UQcBW-yy-XOfrsMeGVj_qZd1UmlE81XN957rjPENcPpH2E1SHfVhu3fQP3QZapxtdMADgWmryw_z5O2lDIIWb7miT0siXRCDjVEhDzKH6Hsf1aUPcYx9x44o1B5OBpX6z1uM1SP0qaOZu6x26H0AbHj84nIISE1A" {jwks: [$crypto:jwks-parse "{\"keys\":[{\"kid\":\"rsa-1\",\"kty\":\"RSA\",\"alg\":\"RS256\",\"n\":\"jdwwBcaMZQLoSNYGNEm3l03HIQqpRIv0eqLUNUCkyv7ysVw4i6vZgdYxcdU0D3kSvUCIjH-icqk4PCDx5AwkeNp55Nqt6wKXDv9TH5pr-Wc3BWmZ1sEOEwyN8QlI8_5EpY3i1w5tysDdeFuiR7BOjpkD49RZzF0YajmocsB4_jXoZcldVNCOChXAbEfOw-BtjFJRN0N3EksymF4azkey8Q3rX07sXkRHavRfAOnowH119RL1V-Xmk_DUX8wSR-2zdlp8FxitJWPgbMmnszsTgEQrThTwsuMrndqzJ_nQ7HTnuK2LXEFmFwXw9NADspq2ZnQmZiN7CXnEYG8WCApzxw\",\"e\":\"AQAB\"}]}"] tenant-claim: "aud" now: [$time:datetime 2023 11 14 22 13 20]} {id: "c-x" tls: true}]] [= $s2 [$session:detach-client $s "c-x"]] [$session:valid $s2]]
      

        $ cx prog.cx
true
      

Deployment artifacts

Reference deployment artifacts live in tooling/cxstore/ in the repository — use them as the template rather than hand-rolling:

• systemd unit (cxstore.service) — Type=notify: the daemon signals READY=1 only after the store is open and the listener is bound, so ordering and readiness are real, and it answers WatchdogSec pings. Install the unit, place the config at /etc/cxstore/cxstore.service.cx, then systemctl enable --now cxstore.
• Dockerfile — a minimal debian:stable-slim image: copy in the static cx binary and the config; the image HEALTHCHECK runs cx store-health against /cx-store/v1/ready. One binary, one config file — the whole deployment surface.
• Sample config (cxstore.service.cx) and a README with a copy-paste loopback smoke test.

Building the artifact itself: make build-vcx produces the optimized binary at vcx/target/cx, and make install-cli installs it to the prefix. There is nothing else to ship — a CX deployment is the binary, your .cx files, and the capability flags in the unit file. The binary self-describes: cx --version reports version, build profile, commit, build date, the compiled collector, the V-fork pin, the compiled storage engines, and the feature set.

Environment variables

The honest current set of deploy-relevant knobs, from the engine source. All are optional; the defaults are the recommendation unless you have measured otherwise:

VGC_GCTRACE=1 output against an allocation-heavy program, verified live:

      [gc 0] marked=31MB goal=62MB headroom=16384KB pause=5660us arenas=1 spans=1477 pool=0KB trimmed=0KB threads=1
[gc 1] marked=62MB goal=125MB headroom=32768KB pause=25188us arenas=2 spans=5509 pool=4680KB trimmed=0KB threads=1
[gc 2] marked=123MB goal=246MB headroom=65536KB pause=64069us arenas=3 spans=13070 pool=0KB trimmed=0KB threads=1
    

Garbage collection at a glance

The collector is a build-time property of the binary, and the binary tells you which one it carries:

      $ cx --version
cx vX.Y.Z             [# a RELEASE build — see below #]
  profile  platform
  commit   3ed9ade5
  built    2026-08-20T05:52:49Z
  gc       e — Perceus RC front line + precise STW vgc backstop
  V fork   b3d0da670
  engines  sqlite redis
  features (none)
    

The headline is a provenance claim, not an echo of a version file. A binary reports the bare cx vX.Y.Z only when it was built from a clean checkout of the annotated release tag; every other build — yours, a CI build, a build from an edited tree — reports a pre-release of that version, with the build commit as metadata:

      $ cx --version
cx vX.Y.Z-dev+3ed9ade5
  profile  platform
  commit   3ed9ade5
  ...
    

-dev+<commit> orders before the release it will become, so a version comparison against a shipped release answers correctly, and a work-in-progress binary can never be mistaken for the release whose number it carries. An edited tree marks both lines: cx vX.Y.Z-dev+3ed9ade5-dirty. What decides is git state at build time — the release version itself still comes from the one source of truth, the repo's VERSION file.

Mode e — the shipped default — is a two-layer design: precise reference counting (Perceus) reclaims most values immediately as the front line, and a precise stop-the-world tracing collector (vgc) backstops cycles and shared structures. Operationally that means: allocation-heavy programs pace themselves adaptively (collections start small and early, growing toward the soft limit), long-running services hold a steady RSS rather than ratcheting, and an out-of-memory at the hard arena ceiling is loud — a diagnostic dump at the wall, not a silent OOM-kill later. The three knobs that matter in production are VGC_MEMLIMIT_MB (pacing target), VGC_MAX_ARENAS (hard wall), and VGC_GCTRACE (observability) — see the table above. There is nothing to tune on day one.

Logging and journaling

Structured logging is a bundled module, not an external dependency. Leveled emits write to the active sink (stderr by default) and return null — bound and discarded here, so the nulls do not echo on stdout; [$log:configure] sets the minimum level, sink, and format process-wide:

      [?lib 'cx-stdlib/log']
[?let [= $c [$log:configure {level: :warn}]]
      [= $i [$log:info 'not shown']]
      [= $w [$log:warn 'disk nearly full']]
  'logged']
    

      warn disk nearly full
'logged'
    

Sinks include stderr, stdout, file (with size/time rotation), and syslog; formats include text, JSON-lines, and logfmt, with multi-sink fan-out — the full configuration surface is on the log module page in the Standard library section of this guide and in the module spec. Under systemd, logging to stderr and letting journald own retention is the intended default posture.

For audit-grade history there is journal — an append-only, hash-chained, tenant-partitioned event log over a store backend, with deterministic fold-to-state, replay, dry-run, verification, and signed snapshots. It is the persistence spine of the XAP runtime and usable standalone; see the journal module page in the Standard library section. The store daemon additionally exposes service-side observability (request metrics, optional OTel tracing, JSON access logs) — configured in the service config (ops-observability).

The operator's reading order for the whole platform runs through this guide: the store's embedded tier and porcelain (store), this page for the daemon and the host, the registries and the distribution machinery (xap-distribution), and the three contracts you operate under (bitemporal, consistency, event-evolution). Where a capability is designed but not yet shipped, the page says so by name; this section only teaches what the shipped binary was observed to do.

Identity-provider compatibility

The question an enterprise evaluation actually asks is whether CX will just work against the identity provider that organization already runs, and the only honest answer is a table saying what was run, against what, and when. This is that table. It is kept honest by two steps, and the split between them is the reason the table can be trusted: make test-sso-interop runs on every commit against an identity provider that ships with the sso module itself. Since the module was extracted into its own repository that provider reaches a cx build as the pinned checkout deps.cxd names (deps/cx-platform-sso/scripts/sso_interop/idp.cx, after make deps-sync), and the step still runs against the binary this tree built. tools/sso_interop_real.sh PROVIDER, in the same repository, runs against a real tenant at release time. A cell that names neither is an unverified cell, and it says so. A PARTIAL verdict names exactly which rows it rests on — in the OIDC and client-auth columns, not folded into the verdict word alone — so a partial run can never be read as more than what it drove.

What the per-commit step grades: discovery and its issuer mix-up refusal, the authorization request as the library forms it, all four client-authentication methods at the token endpoint, PKCE verification failure, ID-token signature selection by kid across a two-key JWKS, refresh-token rotation, the machine grant, and the token endpoint's own OAuth error body surfacing to the caller. What it CANNOT grade: the interactive authorize leg. A real login puts a human at a sign-in page, so no unattended step can perform one — the step reads the Location the identity provider answers with and redeems the code from it, which is exactly what a callback handler does, but the consent screen in between is not exercised by anything here.

Entra groups OVERAGE, recorded rather than left silent (RULED: 1406-b). Above its threshold Entra stops emitting the groups claim and sends _claim_names / _claim_sources naming a Microsoft Graph endpoint instead. session:map-claims reads that and records [groups-overage source=...] with NO groups, and an authz decision over such a binding refuses with :groups-indeterminate -- never an empty allow. FETCHING the withheld set over Graph is the deployment's step and is DEFERRED: the trigger for building it is the first Entra tenant in this matrix over the overage threshold. Until then the fail-closed refusal is the whole behavior.

A -- is "not measured", never "not supported". The five named products have no row content because no tenant has been driven yet, and tools/release-verify.sh refuses a release while any of Entra, Okta, Ping or ADFS still reads NOT YET VERIFIED -- which is the point of writing the unverified cells down rather than leaving the table off the page.

Scheduled, and deliberately absent above rather than quietly implied: client_secret_jwt as a fifth client-authentication method (RULED 1409-a) and the opt-in acceptance of SHA-1 signatures that some long-lived ADFS deployments still emit (RULED 1410-a). The ACS POST binding and RelayState are owed to #1394 and #1402b. Until each lands, the matrix has no column for it.

Mail-provider compatibility

An agent that reads and sends mail reads it from the provider the organization already runs, so the question is the one 17.11 answers for identity: what was run, against which provider, and when. Our client -- cx-platform/imap and cx-platform/smtp -- authenticates to a real provider with an OAuth 2 bearer token (OAUTHBEARER, RFC 7628, or XOAUTH2 where the provider still asks for it); a browserless agent holds that token through the device-code grant (RFC 8628) that cx-stdlib/oidc carries. IMAP with OAuth 2 is also the path that survives Microsoft's EWS retirement.

tools/mail_interop_real.sh PROVIDER, in the cx-platform-mail repository (deps/cx-platform-mail/tools/ after make deps-sync), drives six rows against an account: connect (greeting, CAPABILITY, AUTHENTICATE, ENABLE), list, select INBOX, a baseline sync, noop, and one submission to the account's own address. VERIFIED means all six passed; a row the driver could not reach counts as failed, never as passed. What it does not drive: IDLE, flag changes and DSN against a live mailbox, which change or wait on the account's own state.

A -- is "not measured", never "not supported". tools/release-verify.sh refuses a release while any of Gmail, Microsoft 365 or Fastmail still reads NOT YET VERIFIED (deps/cx-platform-mail/tools/mail_matrix_check.sh). Serving a mailbox from a CX process -- the in-tree IMAP server core -- is outside v0.18 (owner letter (b), 2026-09-13) and has no row here; it is graded again when a deployment needs CX to host mailboxes.