Surfaces and projections

CX has one canonical surface (the bracket form) and projections for JSON, YAML, TOML, XML, Markdown, CSV/TSV/PSV, and JSONL. Every projection is defined as a transformation of canonical CX. The normative rules live in [`spec/canonical.md`](../../spec/canonical.md) and [`spec/conversions.md`](../../spec/conversions.md); this section is the developer-facing working summary.

CX — the canonical dialect

CX has **two** canonical forms — one for source-of-truth formatting (preserves comments, anchors, presentation), one for content-addressing (strips presentation, expands aliases, normalizes everything that doesn't change data). Per [`spec/canonical.md §1`](../../spec/canonical.md).

Lossless canonical (cx fmt)

Lossless canonical (`cx fmt`) preserves every node: comments, anchors, aliases, merges, CX directives, BlockContent, RawText, processing instructions. It normalizes only **presentation**: whitespace, indent, quoting, number formatting, attribute ordering within their declared category.

  • **Idempotent:** `fmt(fmt(x)) == fmt(x)` for any valid `x`.
  • **Use cases:** opinionated source-file formatter; diff stability across edits that don't change data; code review where reviewers see only meaningful changes.
              [# Source — author-friendly form (loose layout, mixed quoting) #]
           [user
             email="ada@example.com"
             id=42
             active=true]

           [# cx fmt output — presentation normalized, comments preserved #]
           [user email='ada@example.com' id=42 active=true]
            

Strict canonical (cx canonical / cx hash)

Strict canonical (`cx canonical`) reduces input to **data-equivalence form**. Strips comments, expands anchors and aliases, resolves merges, removes presentation-only directives, normalizes date offsets to UTC. Two CX documents have identical strict canonical bytes if and only if they encode the same data.

  • **Use cases:** `cx hash` (content-addressable hashing — SHA-256 of strict canonical bytes); signed configuration (sign the strict canonical bytes; verify after canonicalizing); deduplication keyed on data, not file bytes.
  • **Strict canonical is what the identity story rests on.** Two documents that "encode the same data" but differ on comments, whitespace, attribute order, or anchor names still hash to the same bytes.
              [# Source #]
           [user
             [# regrettably noisy comment #]
             email='ada@example.com'
             id=42
             active=true]

           [# cx canonical output — comments stripped, attributes sorted #]
           [user active=true email='ada@example.com' id=42]
            

The strict-canonical bytes are what the SHA-256 hash is taken over (see hash) and what the round-trip guarantee is checked against (see roundtrip).

JSON projection

JSON is the most common interchange surface. CX → JSON mapping is straightforward; JSON → CX is well-defined for the JSON subset that maps cleanly to CX's data model. Per [`spec/canonical.md §5`](../../spec/canonical.md) and [`spec/conversions.md §4`](../../spec/conversions.md).

          [user id=42 email='ada@example.com' active=true
              [roles ('admin', 'editor')]]
        
          {
           "user": {
             "id": 42,
             "email": "ada@example.com",
             "active": true,
             "roles": ["admin", "editor"]
           }
         }
        

**Basic mapping:** Attributes become object keys; element bodies of pure-element content become nested objects keyed by child element name; sequences become arrays; scalars map directly.

Mixed content and the lossless $tag envelope

Mixed content — text interleaved with inline elements — has no native JSON form. The **default** projection is lossy: the text runs concatenate under a reserved `"_"` key and the inline elements become sibling keys, so run order is not recoverable. Under `--lossless`, an element instead emits as the reserved `$tag` **envelope** ([`spec/conversions.md §2.2.1`](../../spec/conversions.md)): `$children` is an ordered array of runs, so mixed content survives exactly.

              [para 'The function ' [code 'parse'] ' returns a value.']
            
              { "$tag": "para", "$children": ["The function ", {"$tag": "code", "$children": ["parse"]}, " returns a value."] }
            

JSON → CX reverses this unconditionally: a `$tag` envelope reconstructs the element with its body in order. The round-trip is byte-identical against `cx canonical` output (`cx eq` accepts the recovered document); the envelope also carries attributes (`$attrs` + `$attr-types`), element metadata (`$anchor` / `$merge` / `$id` / `$type`), and `[table]` payloads (`$cols` / `$rows`).

Namespace handling

CX namespace declarations (`xmlns:dc=URI`) project as JSON object keys with the `xmlns:` prefix preserved. Qualified names like `dc:title` project as JSON keys with the colon preserved (JSON keys are arbitrary strings, so this is legal). Round-trip is lossless when consumers preserve the `xmlns:` attributes.

Number precision and BigInt

JSON numbers are IEEE-754 doubles per RFC 8259, with ambiguous integer support beyond `2^53-1`. CX exports int64 integers as JSON numbers; `bigint` scalars export as JSON strings, with the type recovered in lossless mode via the `cx:type` sidecar (map positions) or the per-item `{"cx:bigint": "…"}` carrier (array positions) — see [`spec/conversions.md §0.2`](../../spec/conversions.md). Sized variants (`::i32`, `::u64`) keep their native number image; the width name rides the envelope's `$type` / `$attr-types` in lossless mode.

Floats with `nan` / `inf` / `-inf` are not reachable from pure CX arithmetic (CX floats are finite-only); at the FFI boundary the JSON module's `nan-handling` option controls `null` / string / error behavior.

null vs missing

JSON has no way to distinguish "key absent" from "key present with value null." CX preserves the distinction: `[user]` (no email attribute) projects to `{"user": {}}`; `[user email=null]` projects to `{"user": {"email": null}}`. JSON → CX reads absent keys as absent attributes (NOT as null).

YAML projection

YAML follows the JSON projection but uses YAML's mapping and sequence syntax — including the lossless `$tag` envelope, which is the same encoding in both lanes ([`spec/conversions.md §2.3.1`](../../spec/conversions.md)): CX anchors and merges ride the envelope's `$anchor` / `$merge` keys in lossless mode.

          [defaults &server-base host='localhost' tls=true]
         [service-a *server-base port=8001]
        
          $doc:
           - $tag: defaults
             $anchor: server-base
             $attrs:
               host: localhost
               tls: true
           - $tag: service-a
             $merge: server-base
             $attrs:
               port: 8001
        

YAML's loose schema (untyped scalars, multiple ways to spell the same value) is normalized on input: `yes`/`true`/`True` all map to canonical `true`, numeric strings stay strings, etc. The conversion is documented in [`spec/conversions.md §5`](../../spec/conversions.md).

YAML tags (!!str, !!int, !!binary)

YAML tags survive round-trip: `!!str 42` parses to CX `:string "42"`; `!!int "42"` parses to CX `:int 42`; `!!binary base64bytes` parses to CX `:bytes`. The CX emitter writes tags only when type inference would be ambiguous (e.g. emitting a numeric-looking string requires `!!str` to avoid being read back as a number).

Multiline scalars (| and >)

Literal block style (`|`) preserves newlines verbatim; folded block style (`>`) collapses single newlines to spaces but preserves blank lines. CX triple-quoted strings round-trip through `|` by default (preserves the as-authored shape). The `>` form is used on emit when the string content has no internal blank lines and the line length would exceed 80 columns.

Anchors and aliases

YAML-native anchors (`&name`) and aliases (`*name`) are resolved (expanded) on YAML → CX parse — the data content is preserved, the anchor/alias structure is not ([`spec/conversions.md §5.1`](../../spec/conversions.md)). In the other direction, CX anchors and merges are dropped by the default YAML emit and carried by the lossless envelope's `$anchor` / `$merge` keys, which the importer reconstructs — so a `--lossless` round-trip preserves them.

TOML projection

TOML's table syntax (`[server]`) projects to CX elements; nested tables (`[server.tls]`) project to nested elements; arrays of tables (`[[users]]`) project to repeated child elements. Per [`spec/conversions.md §6`](../../spec/conversions.md).

          [server]
         host = "localhost"
         tls = true

         [server.tls]
         cert = "site.pem"

         [[users]]
         id = 1
         email = "ada@example.com"

         [[users]]
         id = 2
         email = "grace@example.com" 
        
          [config
           [server host='localhost' tls=true
             [tls cert='site.pem']]
           [users
             [user id=1 email='ada@example.com']
             [user id=2 email='grace@example.com']]]
        

Inline tables vs dotted keys

TOML allows two shapes for nested scalars: inline tables (`server = { host = "x", tls = true }`) and dotted keys (`server.host = "x"`, `server.tls = true`). Both project to the same CX nested-element shape. Canonical CX → TOML output uses the inline form for tables with ≤ 4 entries and the section-header form otherwise.

Array of tables

`[[users]]` repeated produces an array. CX preserves order and emits as repeated child elements with the same name. CX → TOML emits `[[name]]` for any sibling group of N≥2 identically-named child elements; one-of-a-kind children use the dotted-key form.

XML projection

XML is the closest projection — both CX and XML are element-with-attributes-and-body languages. The mapping is essentially one-to-one for element / attribute / text content. Per [`spec/canonical.md §8`](../../spec/canonical.md) and [`spec/conversions.md §3`](../../spec/conversions.md).

          [user id=42 email='ada@example.com'
           [profile
             [bio """Computing pioneer."""]]]
        
          
           
             Computing pioneer.
           
         
        

Differences from XML: CX has no DOCTYPE / DTD layer (schemas are first-class CX, see schema); namespace declarations are CX attributes with the `xmlns:` prefix preserved; CDATA sections project to triple-quoted strings.

Processing instructions and comments

XML processing instructions (`<?xml-stylesheet …?>`) project to CX directive elements (`[?xml-stylesheet …]`). XML comments (`<!-- … -->`) project to CX comments (`[# … #]`). Both round-trip losslessly in the lossless canonical form; the strict canonicalizer strips comments per canonical-strict.

xml:space round-trip

`xml:space=preserve` survives XML round-trip identically. Same for `cx:lang` (round-trips as `xml:lang` if the input used that alias; preserves the chosen form). See xml-space and locale.

DOCTYPE and DTD

CX does not have a DTD layer. XML input with a DOCTYPE declaration is accepted; the DOCTYPE name is captured as a `[?cx doctype=…]` directive (see doctype). DTD content (`<!ELEMENT>`, `<!ATTLIST>`, `<!ENTITY>`) is parsed but not used for validation — schemas are first-class CX (`.cxs` files, see schema). External entity references are gated per include-capabilities.

Markdown projection

Markdown projects to CX as structured prose. Headings become `[h1] [h2] …`; paragraphs become `[p]`; lists become `[ul]` / `[ol]` / `[li]`; inline code becomes `[code]`; links become `[link]`. Round-trip is the design center — `cx convert README.md --to cx --to md` produces byte-identical output for documents within the supported MD subset. Per [`spec/canonical.md §10`](../../spec/canonical.md) and [`spec/conversions.md §7`](../../spec/conversions.md).

          # Hello

         A paragraph with *emphasis* and `code`.

         - item 1
         - item 2
        
          [h1 'Hello']
         [p 'A paragraph with ' [em 'emphasis'] ' and ' [code 'code'] '.']
         [ul
           [li 'item 1']
           [li 'item 2']]
        

GFM tables

GitHub-Flavored Markdown tables project to CX `[table]` elements with `[tr]` / `[th]` / `[td]` children. Column alignment markers (`:---`, `:---:`, `---:`) project to `align=left`/`center`/`right` attributes on the `[th]` cells.

              | Name | Age |
           |:-----|----:|
           | Ada  |  35 |
           | Linus | 24 |
            
              [table
           [tr [th align=left 'Name'] [th align=right 'Age']]
           [tr [td 'Ada']   [td 35]]
           [tr [td 'Linus'] [td 24]]]
            

Fenced code with language

Fenced code blocks (` ```lang ... ``` `) project to `[pre [code lang=lang ...]]`. The language identifier is preserved; canonical emit uses single-quoted attribute form for the language tag.

Frontmatter

YAML frontmatter (between `---` markers at top of file) projects to a `[frontmatter]` element with attributes from the YAML mapping. Round-trip preserves the frontmatter shape verbatim.

              ---
           title: My Post
           date: 2026-05-22
           ---

           # Body
            
              [frontmatter title='My Post' date=2026-05-22]
           [h1 'Body']
            

Footnotes

GFM footnotes (`text[^1]` … `[^1]: ref`) project to inline `[fn-ref id=1]` elements with corresponding `[fn-def id=1 ...]` at the document end. Round-trip preserves the labels.

Math blocks (KaTeX/MathJax)

`$inline$` and `$$display$$` math project to `[math]` elements with `display=true|false`. The body is the LaTeX source preserved verbatim (no LaTeX parsing). Renderers (KaTeX, MathJax, native renderers) consume the `[math]` element directly.

Delimited projection (CSV / TSV / PSV)

CSV (comma-separated), TSV (tab-separated), and PSV (pipe-separated) all project via the same delimited-data pipeline. Per [`spec/canonical.md §9`](../../spec/canonical.md). Delimited round-trip is well-defined but **lossy** (column types are not preserved on emit, only on the in-memory CX side).

          id,name,email,active
         1,Ada,ada@example.com,true
         2,Grace,grace@example.com,false
         3,Linus,linus@example.com,true
        
          [table :columns (id :int, name :string, email :string, active :bool)
         (1, 'Ada',   'ada@example.com',   true)
         (2, 'Grace', 'grace@example.com', false)
         (3, 'Linus', 'linus@example.com', true)]
        

Header detection

The reader inspects the first row: if all values are non-numeric, non-boolean, non-date strings AND the second row contains at least one typeable value, the first row is the header. The default heuristic can be overridden: `cx convert in.csv --header=auto|present|absent --to cx`.

When `header=absent`, columns are named `col1`, `col2`, …; an explicit schema (`--schema cols.cxs`) can name them.

Type inference

Per-column type inference scans the first 1000 rows (or all rows if fewer): all values match `:int` → `:int`; all match `:int` or `:float` → `:float`; all bool → `:bool`; all date or datetime → `:date` / `:datetime`; otherwise `:string`. An explicit schema overrides inference.

Quoting and escaping

RFC 4180 rules: fields containing the delimiter, double quote, or newline are wrapped in double quotes; internal double quotes are escaped as `""`. Output canonicalizes to RFC 4180 regardless of input quoting style. Lone CR / LF outside quotes are treated as row terminators; embedded CRLF inside a quoted field is preserved verbatim.

CSV → CX → CSV is byte-identical for RFC-4180-conformant input. CSV with non-standard quoting (single quotes, no escape) is accepted on input via `--delimiter-profile=relaxed` and emits as RFC 4180.

JSONL / NDJSON projection

JSON Lines (`.jsonl`, also called NDJSON) is one JSON value per line, separated by `\\n`. Projects to CX as a sequence of top-level documents — equivalent to CX's multi-document file format (`--- ` separator).

          {"id": 1, "msg": "first"}
         {"id": 2, "msg": "second"}
         {"id": 3, "msg": "third"}
        
          [record id=1 msg='first']
         ---
         [record id=2 msg='second']
         ---
         [record id=3 msg='third']
        

The CX → JSONL emitter is record-per-line, no trailing newline. Suitable for streaming log lines, event streams, and large-result-set delivery. The streaming reader emits CX records one at a time without materializing the whole file in memory.

JSON Pointer (RFC 6901) — the `path/within/json` form — projects to CXPath via the equivalent step sequence. `/data/0/name` → `/data[1]/name` (CXPath uses 1-based positional predicates; JSON Pointer uses 0-based array indexes).

Bytes scalar across surfaces

The `:bytes` scalar kind (see bytes) has a different wire form per surface — no surface has a single native binary form that all the others share. The CX↔surface bridge applies a deterministic encoding per surface:

surface wire form notes
CX (canonical) 0x<hex> hex digits, even count; the only literal form
data-bin length-prefixed raw no encoding overhead
ast-bin length-prefixed raw same as data-bin
JSON string (base64) lossless adds the cx:type sidecar; JSON has no native bytes
YAML string with !!binary tag YAML 1.2 binary tag
XML text; cx:type="bytes" when lossless hex text content
TOML string (base64) TOML has no binary type
Markdown text (base64) plain text form
CSV / TSV base64 string in the cell deserializer needs schema to know it is bytes
CXCol / Parquet binary column type native; zero-copy through the Arrow bridge

The table states the conversions contract, and the shipped lanes match it: JSON / TOML / Markdown emit base64, YAML emits `!!binary`, and the lossless JSON lane recovers the bytes type via the `cx:type` sidecar / `{"cx:bytes": …}` carrier (#458 closed the lane wiring; #475 added the array-position carrier).

Round-trip guarantees and lossiness matrix

The round-trip contract is the single most important property of CX's surface design. Stated formally:

For any CX document **D** in canonical form, and any supported projection format **F**, the following identity holds: `to_cx(to_F(D)) == D` (as canonical bytes) For any document **F** in projection format **F** that was produced by `to_F(D)` for some canonical CX **D**: `to_F(to_cx(F)) == F` (as bytes, modulo format-specific whitespace normalization rules in [`spec/conversions.md`](../../spec/conversions.md)).

Where this **does not** hold — the lossy projections — the loss is documented and bounded. The conformance suite includes round-trip fixtures for every supported projection; the runner shells out to `cx convert` in both directions and asserts byte equality (or documented-divergence) per format.

**Lossiness matrix** (per [`spec/conversions.md §9`](../../spec/conversions.md)):

from \ to CX XML JSON YAML TOML MD CSV JSONL CXCol/Parquet
CX lossless lossless‡ lossless▣ lossless▣ lossy◊ lossy lossy§ per-record table-only
XML lossless lossless lossless▣ lossless▣ lossy◊ lossy lossy§ per-record table-only
JSON adds struct adds struct lossless¶ lossless¶ lossless¶ lossy lossy§ lossless if 1-per-line table-only
YAML lossy* lossy* lossless¶ lossless¶ lossless¶ lossy lossy§ per-doc table-only
TOML lossless** lossless** lossless¶ lossless¶ lossless¶ lossy lossy§ 1-doc table-only
MD lossy lossy lossy lossy lossy lossless lossy§ 1-doc lossy
CSV / TSV lossy§ lossy§ lossy§ lossy§ lossy§ lossy§ lossless lossless via :table lossless
JSONL 1-doc-per-rec 1-doc-per-rec lossy (concat) lossless lossless if homogeneous lossy lossless if flat lossless lossless if flat
CXCol / Parquet :table form :table form per-row per-row lossy lossy lossless per-row lossless

**Legend:**

  • **lossless** — byte-identical round-trip within the format's expressive range.
  • **‡** — CX → XML is lossless when consumers preserve the `cx:` namespace attributes.
  • **¶** — lossless within the expressive range of that format.
  • **▣** — lossless under `--lossless`: the `$tag` structure envelope + value carriers ([`spec/conversions.md §2.2.1 / §2.3.1`](../../spec/conversions.md)) recover element documents byte-identically (strict-canonical eq). The default (idiomatic) lane keeps the ◊ losses. Program directives are outside every lane's lossless domain.
  • **◊** — CX → JSON/YAML/TOML default lane with CXDM container Items (Array, Map, Sequence-as-Item): non-string Map keys are coerced to strings on emit (lossy); the Sequence-vs-Array distinction is lost (in lossless mode both recover, via the `cx:key-type` sidecar and the `cx:seq` carrier).
  • **§** — Delimited (CSV/TSV/PSV) is well-defined but lossy: type metadata is lost on emit; comments, anchors, multi-document, mixed content, and PIs are stripped or error; element names are lost on parse (recovered via schema or auto-typing — see delim-types).
  • *** — YAML anchors and aliases are resolved (expanded) on parse; not reconstructed in CX output. Data content is preserved.
  • **\\*\\*** — TOML has no mixed content or comments, so CX output contains only what TOML can express; no loss within TOML's range.

**Practically:** you can store data as JSON for the JS ecosystem, convert to CX for selection / transformation / hashing, then convert back to JSON for delivery — no information loss, no re-formatting drift, no string-mangling in between. The CSV / TSV / Markdown projections are documented one-way exports for their respective ecosystems; round-trip back into CX recovers structure but not the lossy-on-emit metadata. The CXCol / Parquet bridges are lossless for table-shaped data and one-way for everything else.

The hash is taken over **strict canonical bytes** (see canonical-strict), not over the surface form. Two documents that round-trip identically through any projection hash to the same SHA-256 — including across surfaces. This is the foundation of CX's content-addressing story (see hash).