Skip to content
DocumentationPorted data plane
On this page

Page resources

Open Markdownllms.txtView source

Last updated

Ported data plane

One includes a bounded, ported data plane: its own recording format, a columnar chunk model, chunk store and index, query and dataframe, entity database, transforms, analytics, video, catalog, catalog wire, tile addressing, and one importer for an external recording format. These are One-owned Data atoms with their own contracts and tests. They are not a second authoring surface, and they do not turn Arrow or any external format into One semantics.

Internal record formats, not authored surfaces

One's recording formats are internal record formats. They are produced and consumed by owning Data atoms; they are not .one syntax and are not something an application authors directly.

  • ONE1 is a single-chunk frame: a magic value, a version, the entity-path length, the entity-path bytes, and an Arrow IPC stream.
  • ONER is a recording:
Plain text
ONER = RECORDING_MAGIC
     | version            (u32 little-endian)
     | chunk_count        (u32 little-endian)
     | repeated [ length   (u32 little-endian)
                | chunk frame ]
  • ONEB is a StoreBundle blob: recordings plus an optional blueprint, with a per-recording revision.

Each reader bounds its inputs and rejects bad magic, unsupported version, truncation, and trailing bytes rather than accepting a partial or over-long record. Arrow is used directly as the columnar chunk layout inside these One-owned atoms. A columnar representation provider seam is introduced only when a second real columnar realization exists.

Columnar chunks and storage

The columnar atoms are:

  • one-chunk — one entity path, timelines, and typed component columns, with latest-at and range access;
  • one-chunk-store — latest_at, range, row_time, timeline_rows, ordered_rows, atomic one-log-message ingestion, optimize, apply_lens, and a deterministic revision;
  • one-chunk-index — the index over stored chunks; and
  • one-columnar-schema — the physical column schema shared by the atoms above.

Query, dataframe, and entities

The query and entity atoms are one-query, one-dataframe (QueryPlan, Table, AggregateFn), one-analytical-query, and one-entity-db. EntityDb exposes table, latest table, query_latest, aggregates, window_aggregate, group_aggregate, analytical_query, manifest, and ingest. one-transform owns transforms, and one-entity owns EntityPath. The experience-oriented views described in Experience and presentation compose these same atoms rather than re-owning entity or timeline meaning.

Analytics, video, catalog, and tiles

  • one-analytics — bounded time-bucket aggregate with MAX_SCAN_ROWS and MAX_BUCKETS, plus descriptive statistics. These statistics are descriptive data; they are never authority or evidence.
  • one-video — EncodedVideoFrame, VideoCodec, decoded RGBA8 frames, and an injectable VideoDecoder. One owns the encoded-frame and decoder contract; no decoder runtime is owned here, and a concrete decoder remains a removable realization.
  • one-catalog — Catalog, EntryId, EntryName, Segment, and EntryKind::Dataset | Table, all bounded. one-adapter-catalog-wire maps the catalog to a wire form of a 4-byte little-endian length plus canonical JSON, with an in-band fault. The wire adapter owns no HTTP, TLS, or authorization.
  • one-map-tiles — an Experience atom with TileCoord, MAX_ZOOM = 24, tiles_for_rect, longitude and latitude conversions bounded to ±85.05112878 and ±180, MAX_TILES = 4096, and a bounded TileFetcher. The atom owns no network, cache, or authority; a network tile provider is a removable realization.

Generated and imported material

standard_telemetry_module_typescript is deterministic generated TypeScript source for the standard telemetry model. It is a projection, not a second model and not a second semantic source.

one-adapter-rrd is a real importer for one external recording format. It parses RRF2 framing plus log_msg.proto messages over LZ4 and Arrow IPC and maps them to one-chunk. It is not the One format and not an embedded external runtime. ImportedRecording::unsupported_components reports every component the mapping could not represent, so loss is explicit rather than silent.

The pure telemetry atoms build for wasm32-unknown-unknown, which activates the browser execution profile's data layer. A browser renderer or product remains inactive.

Relationship to One Database

This ported data plane is not One Database. The native transactional, temporal, and reactive database described in One Database is a separate, still-deferred realization. Do not present a native SQL database as available here, and do not present these bounded atoms as a general durable transaction engine. The broader physical, storage, and movement model remains owned by Data and state.

Illustrative syntax is not a data format

An authored state declaration describes meaning; it does not describe ONE1, ONER, columns, or Arrow layout. The .one syntax below is illustrative end-product syntax and is not exercised by the ported data atoms:

One
one 1

semantic acme.telemetry
    domain one.data@1

    state Telemetry@1
        stores EntitySamples
        retention 30d

The Rust API references on this page (crate and type names such as one-chunk, EntityDb, and TileCoord) are demonstrated by their owning crates and tests.

Next: read Data and state or One Database.

Normative owner & evidence

The normative owner is systems/data/AGENTS.md. Tile addressing is additionally owned by systems/experience/AGENTS.md, under the repository composition law in AGENTS.md.

Demonstrated today: the ONE1/ONER/ONEB record formats and their rejection bounds; the columnar chunk, store, index, schema, query, dataframe, entity, and transform atoms; one-analytics; the one-video contract; one-catalog and its wire adapter; one-map-tiles; the generated telemetry TypeScript module; and the one-adapter-rrd importer with explicit unsupported components. Each is exercised by its owning crate.

Illustrative or deferred: the one.data@1 authoring snippet above, a native SQL database, a concrete network tile or video provider, a browser renderer or product, and any second columnar representation provider.