Sync Protocol v1

This page is the specification of sync protocol v1: the protocol atlas-sync speaks to every client of an app backend. Atlas desktop, the TypeScript SDK's sync client and any client you write speak the same protocol. Where it says a side must do something, a conforming implementation does it, and the conformance suite checks it.

Every frame, op and declaration has a JSON Schema generated from the service's own types, in the @remotehost/sync-protocol package (schema/client/*.json, schema/server/*.json, schema/op/*.json, schema/declaration.json), with TypeScript types generated from them.

The examples on this page show each frame's shape. Long values (ids, nonces, proofs) are shortened with ….

1. Overview

One protocol, in two parts:

• Core: what any client implements, and what this page specifies: devices, workspaces and groups, catch-up, live changes, writes, refusals, and the record ops of declared backends.
• Atlas extensions: frames and fields only Atlas's own clients use, listed in section 13. Other clients must not send them.

Versioning. A hello names the protocol major (protocol: 1). Within a major, frames, fields, events, codes and reasons are only ever added. A client must ignore members and events it does not know; a service ignores members it does not know. A new major is a new protocol; a copy that does not speak the client's major answers reconnect or closes.

Features. welcome.features names the optional parts a copy serves: backends (declared backends and record ops), blobs and adapters. A client must not use a part its copy does not name.

2. Transport and framing

• URL: wss://sync.remotehost.ai/v1/connect (staging: wss://sync-staging.remotehost.ai/v1/connect).
• Frames are JSON text messages. Binary messages close the socket (1003).
  • A client sends {"id": <integer>, "method": "<method>", "params": {…}}.
  • The service sends events, {"event": "<event>", …}, and answers to a request, {"id": <the request's id>, "result": …} or {"id": …, "error": …}.
• Order. The service handles a socket's frames in the order they arrive, and answers in that order.
• Size. welcome.max_frame_bytes caps a client frame; a larger one closes the socket (1009). A client must split pushes to fit.
• Compression. permessage-deflate, when the client offers it.
• Liveness. A client sends ping ({"method": "ping", "params": {}}) when it has sent nothing for a while; the service answers {"event": "pong"}. A socket silent past the service's limit is closed (1001). reconnect {after_ms, reason} asks a client to connect again after after_ms.

3. Authentication

A sync token is a short-lived (5 minute) at+jwt for the sync audience, minted for a person's own session by POST /v1/auth/sync-token on the API. API keys, agent tokens and service accounts cannot mint one (403 person_session_required).

Header path. A native client sends Authorization: Bearer <token> on the upgrade. A token the service does not accept is refused 401 before the upgrade. The first frame is the challenge, naming the token's subject:

{"event": "challenge", "nonce": "q3Zf…", "subject": "0190f6a2-…"}

Frame path. A browser cannot set headers on a WebSocket upgrade, so an upgrade without Authorization is accepted unauthenticated for 10 seconds. Its challenge has no subject. The client must send auth next, before hello:

{"id": 1, "method": "auth", "params": {"token": "eyJ…", "dpop": "eyJ…"}}
{"id": 1, "result": {"subject": "0190f6a2-…"}}

Anything else first, a token the service does not accept, or nothing within 10 seconds closes the socket 4401.

DPoP. A token bound to a key (cnf.jkt) is accepted on the frame path only, with a DPoP proof (RFC 9449: htm: GET, htu the connect URL, ath, and nonce the challenge's nonce). On the header such a token is refused 401 with {"code": "use_auth_frame"}.

Refresh. Before a token expires, a client sends auth.refresh {token, dpop?} on the open socket; it is answered as auth is. A token for another subject closes the socket 4401.

4. Devices

A device is the client's identity in a workspace, minted by the service.

• Mint. A client with no device makes a P-256 key and sends, in its hello's workspace entry, device: null, device_key (the public key as a JWK) and proof: a compact ES256 JWS, header {"alg": "ES256", "typ": "device-registration+jwt"}, claims {"kind": "registration", "jkt": …, "nonce": …}, where jkt is the key's RFC 7638 thumbprint and nonce the challenge nonce. The signature must be low-S.
• Return. A client holding a device sends device: {id, secret, generation, proof}, the proof typed device-hello+jwt over {"kind": "hello", "device_id", "generation", "nonce"}.
• Rotation. Every welcome issues the device's next secret (welcome.device.{id, secret, generation}). The client must store it durably, then confirm it as its first frame: ack {workspace, acked_upto, generation}, answered ack_ok {workspace, generation}.
• Fork and rollback. A secret older than the one confirmed means two clients hold one device (a copied store): the service answers forbidden {reason: "stale_secret"} or device_conflict {reason}, and the client must register a new device. device_conflict {reason: "rollback"} means the client's store went back in time.
• Reconcile. After a conflict or a restore, device.results {workspace, device_id, from_seq, upto} returns what the old device applied and refused, by device seq with each op's hash (device_results), so the client resends only what never landed.
• Outbox epochs. A client whose store was restored or reset opens a new outbox epoch with outbox_epoch_register {workspace, device_id, prev_epoch, reason, client_nonce, sig} (sig typed atlas-outbox-epoch+jwt), answered outbox_epoch_registered.
• Device kind. device_kind: "replica" (the default) keeps a store; "ephemeral" is a memory store's device, purged once idle.

5. Workspaces, backends, scopes and groups

After the challenge (and auth, on the frame path) a client sends hello:

{"id": 2, "method": "hello", "params": {
  "protocol": 1, "app_version": "my-app/1.0", "op_versions": {"min": 1, "max": 4},
  "schema_version": 1, "profile": "full",
  "workspaces": [{"workspace": "0190f6a1-…", "device": null,
    "device_key": {"kty": "EC", "crv": "P-256", "x": "…", "y": "…"}, "proof": "eyJ…",
    "next_device_seq": 1, "acked_upto": 0, "cursors": null,
    "backends": ["issues", "checklist"]}]}}

schema_version is the client's store schema; core clients send 1, and the service ignores it in protocol 1. Each workspace is answered welcome, forbidden {workspace, reason}, or hello_refused (below):

{"event": "welcome", "protocol": 1, "workspace": "0190f6a1-…", "subject": "0190f6a2-…",
 "device": {"id": "0190f6a3-…", "secret": "…", "generation": 1},
 "device_epoch": 1, "outbox_epoch": 1, "last_device_seq": 0,
 "op_versions": {"min": 1, "max": 4}, "mode": "catch_up", "cut": {"0190f6a4-…/0": 12},
 "groups": [{"id": "0190f6a4-…", "kind": "workspace", "scope": {"kind": "workspace", "id": "0190f6a1-…"},
   "shards": 1, "layout_version": 1}],
 "features": ["backends"], "max_frame_bytes": 1048576}

Groups. A workspace's rows are held in groups, each with a kind: workspace, team, user, or backend (one per enabled declared backend). A group is split into shards. groups {workspace, add, update, remove} tells a client a group appeared, changed layout or is no longer readable; a client drops a removed group's rows.

Partial sync. backends names the backends a client holds (absent or []: ["issues"]). scopes: [{"kind": "team", "id"}] narrows Issues to those teams' groups plus the workspace group (absent: every group the caller reads). A group not asked for, not enabled or not readable is never welcomed. A client whose backends or scopes change sends a new hello for that workspace.

Refused hellos. A workspace whose entry asks what the copy does not serve is answered hello_refused, with code invalid and field the member it names:

{"event": "hello_refused", "workspace": "0190f6a1-…", "code": "invalid",
 "field": "backends", "message": "this copy does not serve backend \"checklist\""}

• field: "backends": a backend the copy does not serve (one it does not know, or one not served there, such as a test backend in production; a copy without the backends feature serves issues alone).
• field: "scopes": a scope of another kind than team, or whose id is not a team's id.

Only that workspace is refused. The socket stays open, the hello's other workspaces are answered as ever, and nothing changes for the refused one: no device is minted, held or ended, and a stream an earlier hello started goes on. The refusal is final for that hello: a client must not send it again unchanged, and should report it (the SDK's status refused). A client that does not know the event ignores it, as it ignores every event it does not know.

6. Catch-up

From welcome.cut, the service sends each group's rows at or below the cut:

• rows {workspace, group, shard, layout_version, rows, tombstones}: pages of row images ({entity, id, rev, row}) and tombstones ({entity, id, rev, state}). A client applies each by rev: a row replaces what it holds only if its rev is higher.
• A row's values. A declared record's row is {id, <fields…>, created_at, updated_at}, each field its JSON value as its type says: a bool as true or false, a json field as its object, array or scalar, a blob as its descriptor object, the rest as numbers or text. A native entity's row holds its columns as its store keeps them: numbers, text and null, with flags as 0 or 1. change rows (section 7) are the same.
• rows_end {workspace, group, shard, layout_version, upto_seq, coverage} ends a shard. coverage.complete says whether every row was sent, or only those coverage.filter chose (the instant set); fetch {what: "page"} loads the rest on demand.
• reset {workspace, group?, reason} tells a client to drop a group's rows and cursors: its catch-up starts again from zero.

Change numbers and cursors. A workspace's stream is the union of its groups' shard streams. Each (group, shard) numbers its commits with seq: per shard, gap-free and in commit order. A client keeps cursors {layout: {<group>: <layout_version>}, seqs: {"<group>/<shard>": <seq>}} and sends it back in hello; the service resumes exactly after it. Taken whole, the cursors are the workspace's change number: an opaque value that only moves forward (the SDK exposes it as one base64url string).

Splits. A shard that grows past its bound is split. The service announces it with groups {update: [{id, shards, layout_version, …}]} before any frame of the new layout; a client that sees a layout it does not know sends layout_req {workspace, group}, answered with the same update.

7. Live changes

After catch-up a client is live. Each commit reaches it as

{"event": "change", "workspace": "0190f6a1-…", "group": "0190f6a4-…", "shard": 0,
 "layout_version": 1, "from_seq": 12, "to_seq": 13,
 "rows": [{"entity": "checklist.list", "id": "0190f6b0-…", "rev": 13,
   "row": {"title": "Groceries", "archived": false}}]}

covering (from_seq, to_seq] exactly. entity is a native entity's name (issue, label, …) or "<backend>.<type>"; a row with row: null and a state is a tombstone. caught_up {workspace, group, shard, seq} says a shard is current. A client that missed a range (a gap in from_seq) sends resume {workspace, group, shard, from_seq}; the service pages (from_seq, head] as change frames.

8. Writes

A client writes by pushing ops from its outbox:

{"id": 3, "method": "push", "params": {"workspace": "0190f6a1-…", "epoch": 1, "outbox_epoch": 1,
 "acked_upto": 0, "ops": [{"device_seq": 1, "entry_nonce": "kq7…", "op_version": 4,
   "at_client": "2026-11-01T18:00:00Z", "origin": "app",
   "op": {"op": "put_record", "backend": "checklist", "type": "list", "id": "0190f6b0-…",
     "fields": {"title": "Groceries"}}}]}}

• Envelope. device_seq numbers ops per device from 1, with no gaps; entry_nonce is 16 random bytes (base64url), made once when the op enters the outbox; origin must be app for a core client (a person acting through an app). An op without origin is refused invalid {reason: "origin_required"}. batch (an id) makes ops all-or-none.
• Idempotence. An op is applied exactly once by (device, device_epoch, device_seq) and the op's bytes. A client must resend an op's bytes exactly as first encoded; a resent op is answered as it first was.
• Results. push_result {workspace, results} answers each op by device_seq: {"status": "applied", "writes": [{group, shard, seq}], "created"?}, {"status": "refused", "code", "reason"?, "field"?, "message", "current"?} or {"status": "rejected", "code", …}. A push may be partly applied.
• Rejections. rejected {workspace, code, from_device_seq, expected?, retry_after_ms?, reason?} stops the push at that op: nothing from it on was processed. A client resends from from_device_seq (or expected, for gap) after retry_after_ms.
• Window. At most 256 ops per push and a window of unacknowledged ops; past it the push is rejected window_full. ack {workspace, acked_upto} frees the window.

9. Refusals and rejections

A refusal is final: the op will never apply, and the client must drop it from its outbox and roll back its optimistic change.

Reasons refine a code. The list only grows; a client must read a reason it does not know as other: origin_required, origin_not_allowed, backend_not_enabled, unknown_backend, unknown_type, unknown_field, cascade_too_large, blob_missing, blob_too_large, blob_mismatch, adapter_refused, restricted, use_auth_frame, too_large, seq_replayed, nonce_replayed, epoch_retired, outbox_epoch_required, entry_nonce_required, and Atlas's own (agent_name_*, approval_*, agent_owner_op, mixed_actions, text_*).

A rejection is transient: retry later, from the op it names.

10. The conflict rule

The service orders every write. A write applies to the current state field by field, and the last writer in the service's order wins. expect makes a field compare-and-swap (precondition_failed with current when it does not hold). Sets, such as an issue's labels, merge per element. Issues' ops keep their own rules (a claim, a state machine). Clients never resolve conflicts: they apply the rows the service sends, by rev, over their optimistic state.

11. Record ops and declarations

A declared backend's records are written with three ops, validated against its declaration:

• put_record {backend, type, id, fields} makes record id (a UUIDv7 the client makes) from fields;
• update_record {backend, type, id, set, expect?} sets fields, expect checking each named field first;
• delete_record {backend, type, id} soft-deletes it, applying each reference's on_delete (cascade, set_null, restrict; a cascade over 1,000 rows is refused cascade_too_large).

A field the type lacks is refused invalid {reason: "unknown_field", field}; a type the backend lacks, unknown_type; a value that breaks the field's rules, invalid {field}. Their rows arrive as <backend>.<type> entities.

12. Issues ops

Issues is a native backend: its ops are atlas-core's, and its declaration (apps/atlas/backends/issues/backend.json) lists each with its JSON Schema. This table is generated from it.

13. Atlas extensions

Reserved for Atlas's own clients. Other clients must not send them, and the conformance suite does not test them:

• push op members agent_name and approval; origin values window, cli_owner and local_agent;
• events org_settings, pause and pause_lifted;
• text.ops_since, text.ops, text.state and the edit_text op;
• subscribe, entered and left (query subscriptions, not built);
• fetch kinds other than page.

14. Conformance

@remotehost/sync-conformance holds the protocol's scenarios: scripts of client frames and the server frames expected, with placeholders for what the service chooses. It is dependency-free JavaScript on Node 22.

Against a server:

node packages/sync-conformance/bin/run-server.mjs \
  --url wss://sync-staging.remotehost.ai/v1/connect \
  --workspace <workspace id> --member <subject> --token-file <file holding a sync token>

It exits 1 on any failed scenario, naming the step, what was expected and what arrived. --capture <file> keeps every frame each scenario sent and received (tokens and device secrets replaced). Against a client, fakeServer(scenario) plays the service's side of a scenario on 127.0.0.1 for the client under test.

The scenarios cover authentication on both paths, origin, device mint, rotation and fork, writes applied and refused, resend idempotence and gaps, catch-up from zero and from cursors, layout updates, partial sync by backend and scope and its refused hellos, and every record op's validation on the test backend checklist.