# Architecture How the pieces fit, why each boundary is where it is, and what happens on the wire when somebody types a character. This document is for people changing argala, evaluating it, or writing a second implementation of the protocol. If you only want to _use_ it, start with [GUIDE.md](https://docs.thinkingmachinelabs.dev/argala/guide.md). If you want the exact wire format, read [PROTOCOL.md](https://docs.thinkingmachinelabs.dev/argala/internals/protocol.md); this file explains the shape, that file is normative. **Contents** 1. [The one-sentence version](#the-one-sentence-version) 2. [The three packages](#the-three-packages) 3. [Identities, graphs and storage keys](#identities-graphs-and-storage-keys) 4. [The crypto boundary](#the-crypto-boundary) 5. [Anatomy of a record](#anatomy-of-a-record) 6. [The sync loop](#the-sync-loop) 7. [Key ceremonies](#key-ceremonies) 8. [Epochs and rotation](#epochs-and-rotation) 9. [Storage layout](#storage-layout) 10. [The relay's request pipeline](#the-relays-request-pipeline) 11. [The append-only log](#the-append-only-log) 12. [Blobs and retention](#blobs-and-retention) 13. [Failure and recovery](#failure-and-recovery) 14. [Trust boundaries](#trust-boundaries) 15. [Extension seams](#extension-seams) 16. [Deployment shapes](#deployment-shapes) 17. [How it is tested](#how-it-is-tested) --- ## The one-sentence version Every device holds the real database in plaintext. The server holds an ordered log of opaque `(nonce, ciphertext)` pairs and wrapped keys it cannot open. The interesting engineering lies in getting the right key onto the right device without ever handing that key to the server. ```mermaid graph LR subgraph Device A A1[app code] A2[ArgalaEngine] A3[(local store
plaintext)] A1 --- A2 --- A3 end subgraph Relay R1[Fastify plugin] R2[(SQLite
ciphertext only)] R1 --- R2 end subgraph Device B B1[app code] B2[ArgalaEngine] B3[(local store
plaintext)] B1 --- B2 --- B3 end A2 -- sealed ops --> R1 R1 -- sealed ops --> B2 B2 -- sealed ops --> R1 R1 -- sealed ops --> A2 style R2 fill:#2b2b2b,color:#eee style A3 fill:#1f4d2b,color:#eee style B3 fill:#1f4d2b,color:#eee ``` Green is plaintext. Grey is ciphertext. There is no third colour: the relay never sees a key, a plaintext record, or an author's name. --- ## The three packages ```mermaid graph TD core["@argala/core
crypto boundary, wire protocol, key schemas
no I/O, isomorphic"] client["@argala/client
device engine + local stores
browsers, native shells, Node"] relay["@argala/relay
the blind relay
a Fastify plugin you mount"] client --> core relay --> core style core fill:#26343f,color:#eee style client fill:#1f4d2b,color:#eee style relay fill:#2b2b2b,color:#eee ``` The dependency arrow only ever points at `core`, and that is enforced: the test suite scans the relay's own source for any import that could decrypt, and fails the build if one appears. `@argala/relay` does not depend on `@argala/client` and never will — the two ends share a protocol, not code paths. | package | holds | never holds | | ---------------- | ----------------------------------------------------------------------------------------------- | --------------------------------------------------- | | `@argala/core` | AES/ECDH/ECDSA/HKDF wrappers, canonical JSON, zod schemas, `LIMITS`, AAD and storage-key naming | any network or disk call | | `@argala/client` | keyrings, the sync loop, LWW, local stores, blobs, invites, recovery | anything the app should decide (schemas, sessions) | | `@argala/relay` | ordering, quotas, rate limits, the device roster, wrapped-key custody | a decrypt routine, a graph key, a record's contents | ### Inside `@argala/client` ```mermaid graph TD E[ArgalaEngine] T[Transport] GS[GraphStore] KS[KeyStore] K[KindRegistry] E --> T E --> GS E --> KS E --> K T --> HT[HttpTransport] GS --> M[MemoryGraphStore] GS --> I[IdbGraphStore] GS --> O[OpfsGraphStore
SQLite-WASM in a worker] GS --> S[SqlGraphStore] S --> N[NodeSqliteGraphStore] KS --> MK[MemoryKeyStore] KS --> IK[IdbKeyStore] KS --> FK[FileKeyStore
server-side agents] ``` Four seams, four interfaces. The `ArgalaEngine` implementation is the only place that knows how they fit together, so mistakes in that file are expensive. --- ## Identities, graphs and storage keys argala does not own your users. It asks the host app two questions (`Directory`) and takes one answer per request (`authenticate`). ```mermaid classDiagram class ArgalaIdentity { +memberId: string +deviceId: string } class Directory { +membersOfGroup(groupId) +isGroupMember(groupId, memberId) +isActiveMember(memberId) } class RosterDevice { +deviceId: string +memberId: string +pubJwk: string +sigJwk: string +revokedAt: number|null } ArgalaIdentity --> RosterDevice : names one ``` A **member** is a person or account, as defined by your system. A **device** is one installation of an app. Keys belong to devices, never to members — that is what makes revocation meaningful. ### Three graph shapes ```mermaid graph TD subgraph "u — personal" U["engine.openGraph('u')
storage key: u:<memberId>"] end subgraph "g:<id> — shared" G["engine.openGraph('g:acme')
storage key: g:acme
membership from Directory"] end subgraph "d:<id> — pairwise" D["sam addresses d:alex
alex addresses d:sam
storage key: d:alex:sam (sorted)"] end ``` The pairwise case deserves careful attention. Each side addresses the graph by the _other_ party's ID, but both must resolve to one row in the relay and one AAD value in the cryptographic layer, so the storage key sorts the pair. Get this wrong and the two sides seal data into slots the other cannot read, with no error anywhere. ```mermaid flowchart LR A["storageKeyFor('d:alex', 'sam')"] --> S1["sort(['sam','alex'])"] B["storageKeyFor('d:sam', 'alex')"] --> S2["sort(['alex','sam'])"] S1 --> K["d:alex:sam"] S2 --> K ``` The same function computes it on both ends — it lives in `@argala/core` precisely so it cannot drift. --- ## The crypto boundary Everything cryptographic lives in `argala-core/src/crypto.ts`, and every primitive is WebCrypto. Nothing here is hand-rolled; the layout around it is. ```mermaid graph TD subgraph "per record / per blob" AES["AES-256-GCM
fresh 96-bit nonce
AAD = storageKey/recordId"] end subgraph "per device" ECDH["ECDH P-256 + HKDF-SHA256
wrap a graph key to a device pubkey"] ECDSA["ECDSA P-256
sign the record, inside the ciphertext"] end subgraph "per passphrase" PBKDF["PBKDF2-SHA256, 310k iterations
invites and recovery bundles"] end subgraph "per personal id" TAG["HKDF then HMAC-SHA256
truncated to 132 bits"] end ``` ### Why the signature is inside the ciphertext If the signature rode outside, the relay would learn which device wrote each record — the social graph, in the clear, forever. So a record is wrapped in an envelope, the envelope is what gets sealed, and the relay sees one opaque blob. ```mermaid graph LR R["record
{id, kind, data, at, by}"] --> C["canonicalJson"] C --> SB["skm-record-v1 · storageKey · canonical JSON
newline-separated, as bytes"] SB --> SIG["ECDSA sign"] R --> ENV SIG --> ENV["envelope {v, r, d, s}"] ENV --> SEAL["AES-256-GCM
AAD = storageKey/recordId"] SEAL --> OP["op {recordId, rev, e, n, c}"] style OP fill:#2b2b2b,color:#eee ``` `canonicalJson` sorts keys recursively and drops `undefined` exactly as `JSON.stringify` does, so two devices that built the same object in a different order produce the same bytes to sign. Without it, a signature would verify or not depending on property insertion order — a bug that appears months later, on one platform. ### Why AAD, and what it buys Every seal is bound to `/` (or `/blob/`). A relay that shuffles rows, or replays one record's ciphertext into another record's slot, produces a decryption **failure**, never a wrong answer. ```mermaid sequenceDiagram participant M as malicious relay participant D as device M->>D: ciphertext of r1, served in the slot of r2 D->>D: open(key, ct, aad="g:acme/r2") D--xD: GCM tag mismatch D->>D: emit record.rejected {reason: shape}, skip, keep syncing ``` Note the last line: a bad record is skipped, not fatal. One corrupt row must never wedge a sync. ### Personal ids are derived, not named A record in your personal graph that refers to a shared record must never be named after it. `pin_` sitting beside `g:acme/` lets the relay join the two and read off what every member tracks, without decrypting anything. ```mermaid graph LR PK["personal graph key"] --> HK["HKDF info='skm-pid-v1'"] HK --> MAC["HMAC-SHA256"] KIND["'pin'"] --> MAC TGT["'doc_manifesto'"] --> MAC MAC --> ID["p_ + 132 bits
deterministic, unlinkable"] ``` Deterministic, so upsert-by-id still merges across your own devices. Derived from epoch 1 of the personal graph, so it survives rotation, a newly linked device, and a recovery import. Two members pinning the same record produce unrelated ids. `engine.personalId(kind, target)` is the only correct way to name such a record. --- ## Anatomy of a record ```mermaid classDiagram class GraphRecord { +id: string +kind: string +data: object +at: number "revision AND timestamp" +by: string «member» +deleted: true "tombstone, optional" } class RecordEnvelope { +v: 1 +r: GraphRecord +d: deviceId "optional" +s: signature "optional" } class GraphOpIn { +recordId: string +rev: number +e: epoch "optional" +n: nonce +c: ciphertext +k: kind "optional, LEAKS" +sh: shard "optional, LEAKS" } class GraphOpOut { +seq: number } RecordEnvelope --> GraphRecord : wraps GraphOpIn --> RecordEnvelope : seals GraphOpOut --|> GraphOpIn ``` `at` is both the record's timestamp and its **revision**. That dual role is deliberate — one number, monotonic per device, is all last-write-wins needs — and it carries two rules that are easy to get wrong: 1. **Monotonic per device.** Two writes inside one millisecond must not stamp the same number, or the relay drops the second and the device keeps content nobody else has. 2. **Above what it replaces.** Editing a record another device wrote from a clock a few milliseconds ahead must stamp above _that_ revision, not just above this device's own high-water mark. Otherwise the relay drops the edit as a last-write-wins loser, the local store keeps it, and the pull cursor has already passed the relay's copy — the edit lives on one device forever, with no error anywhere. `npm run argala:soak` exposed the second rule; code inspection did not. **Tombstones are records.** The relay never deletes a row; a deletion is a record with `deleted: true` and an empty `data`. Anything else would make "deleted on device A, edited on device B" unresolvable. --- ## The sync loop One graph at a time, coalesced so eight concurrent callers produce one run. ```mermaid flowchart TD START([syncGraph]) --> EPOCH{epoch stale
or no key?} EPOCH -- yes --> RK[refreshKeys] EPOCH -- no --> PUSH RK --> HASKEY{key now?} HASKEY -- no --> DONE HASKEY -- yes --> PUSH PUSH[pushPending] --> BEHIND{behind an
epoch?} BEHIND -- yes --> HOLD[hold writes
emit keys.behind] --> PULL BEHIND -- no --> BATCH[seal a batch
bounded by count AND bytes] --> SEND[POST push] SEND --> MORE{queue
empty?} MORE -- no --> BATCH MORE -- yes --> PULL PULL[pullAll] --> SUBS[for each subscription] SUBS --> GET[GET pull?after=cursor] GET --> NOTE[note the relay's epoch] NOTE --> OPS[for each op] OPS --> V{open + verify} V -- ok --> LWW{newer than local?} V -- reject --> SKIPPED[emit record.rejected] V -- defer --> STOP[stop, keep cursor] LWW -- yes --> STAGE[stage for upsertMany] LWW -- no --> NEXT[next op] STAGE --> NEXT SKIPPED --> NEXT NEXT --> CURSOR[advance cursor] CURSOR --> AGAIN{more?} AGAIN -- yes --> GET AGAIN -- no --> CATCHUP{epoch ahead
of keyring?} CATCHUP -- yes --> RK2[refreshKeys] --> GRANT CATCHUP -- no --> GRANT[grantPending] GRANT --> DONE([sync.done]) STOP --> DONE ``` Four details in that diagram are essential: **A push is bounded by bytes as well as by op count.** Two hundred records at the maximum ciphertext size total roughly 140 MB, and the resulting 413 is not retryable — a queue that stops draining and stays stopped. **A device that is behind an epoch holds its writes.** Sealing under the previous key would be readable by whoever the rotation excluded. Unsent is recoverable; sent under the wrong key is not. **Meeting an op sealed under a missing epoch stops the pull _before_ that op, without advancing the cursor**, fetches keys, and tries once more in the same sync. Skipping it would lose the record forever. **"Cannot verify" is not "does not verify."** If the roster cannot be fetched — offline, rate-limited — the batch is deferred, not dropped. Dropping would silently lose data that is sitting right there. ### Last-write-wins, and the tie ```mermaid sequenceDiagram participant A as device A participant R as relay participant B as device B Note over A,B: both edit r1 in the same millisecond, rev = 500 A->>R: push r1 rev 500 (A's content) R->>R: no existing row → store B->>R: push r1 rev 500 (B's content) R->>R: existing rev 500 >= 500 → drop silently R-->>B: accepted 0 B->>R: pull R-->>B: op r1 rev 500 (A's content) B->>B: local.at == record.at AND content differs
→ take the relay's copy Note over A,B: converged on A's content ``` The relay keeps the **first** of two equal revisions. The device's comparison is therefore `local.at <= record.at` on a difference, not `<`: a strict `<` would leave B holding its own version forever, with no error and no way back. ### The echo optimisation, and its one rule A device that just pushed a record will receive it again on the next pull. Decrypting and verifying its own writes is unnecessary, so the engine remembers `(graphId, recordId) → rev` and skips an op that matches exactly. The rule: **mark echoes only when EVERY op in the batch landed.** `rejected` lists refusals, not the ops that quietly lost a last-write-wins comparison — and a record that lost is one where the relay's copy is somebody else's. Marking that as our own echo would skip the one record we most need to read. --- ## Key ceremonies ### Creating a graph key ```mermaid sequenceDiagram participant D as device participant R as relay D->>R: GET /graphs/g:acme/devices R-->>D: roster (pubJwk per active device) D->>D: generate a 256-bit graph key D->>D: wrap it to every active device's pubJwk D->>R: POST /graphs/g:acme/keys {epoch: 1, wraps} alt somebody else got there first R-->>D: 200 — it sees "epoch 1" and treats it as a grant else fresh R-->>D: 200 — epoch 1 created end D->>R: GET /graphs/g:acme/keys R-->>D: whatever actually landed D->>D: adopt what the relay holds, never what we sent ``` That read-back is not merely defensive programming; it fixes a real race. A concurrent create is answered **200, not 409** — the relay sees a known epoch and treats the request as a grant, ignoring the duplicate wrap. A device that trusted that 200 would keep a key nobody else has: every record it writes unreadable, every record it reads undecipherable, and no error anywhere. ### Linking a second device No private key ever moves between machines. ```mermaid sequenceDiagram participant N as new device participant R as relay participant H as a device holding the keys N->>N: generate ECDH + ECDSA keypairs N->>N: selfSig = sign(fingerprint) under its own signing key N->>R: PUT /devices/ {pubJwk, sigJwk, selfSig} Note over R: set-once. Rebinding an id is refused (409). N->>R: GET /graphs/g:acme/keys R-->>N: latestEpoch 3, wraps: [] — nothing for me yet N->>N: status = waiting, showing its pairing code H->>R: GET /graphs/g:acme/keygaps R-->>H: [{deviceId: new, pubJwk, sigJwk, selfSig, epochs: [1,2,3]}] H->>H: fingerprint the triple → is this device approved here? Note over H: possession-pinned, human-approved, or blocked H->>H: wrap each epoch to the PUBLISHED pubkey H->>R: POST /graphs/g:acme/keys {epoch, wraps, excluding} Note over H,R: pubkeys come from the roster, never from
the asking device — a forged request cannot redirect a key N->>R: GET /graphs/g:acme/keys R-->>N: three wraps N->>N: unwrap with its own private key → status ok ``` `grantPending` runs automatically during every sync performed by a device that holds the keys (`autoGrant`, on by default). This makes linking feel automatic instead of like a ceremony someone must remember to perform. ### The approval gate A relay cannot mint a graph key, but it can add a row to the roster: a device ID it invented, published with a wrapping key whose private half it holds. Every honest device then sees a gap and helpfully fills it. Nothing appears broken, but the relay gains access to the graph key. Gating `grantPending` alone would not help. A planted device does not need to be granted — it can wait for `rotate()`, which is what you run when a laptop is stolen. So the gate sits on the one function every wrap goes through: ```mermaid flowchart TD subgraph one["wrapsFor() — the only place a key is wrapped to another device"] IN["createGraphKey · grantPending · rotate"] --> FP["fingerprint the key material
we were HANDED, not a cached roster"] FP --> PIN{"pin for this device?"} PIN -- "approved, fingerprint matches" --> W["wrap"] PIN -- "approved, fingerprint changed" --> CONF["pin: conflicted
device.conflict
blocked"] PIN -- "rejected / conflicted" --> BLOCK["blocked"] PIN -- none --> POP{"proof of possession?"} POP -- yes --> AUTO["pin: approved via 'possession'"] --> W POP -- no --> PEND["device.pending
blocked"] end BLOCK --> MODE{"linking mode"} PEND --> MODE CONF --> MODE MODE -- "'approve'" --> EXC["named in excluding
on the key put"] MODE -- "'observe' (default)" --> ANYWAY["wrapped anyway,
linking.unenforced once per boot"] ``` **It fingerprints what it is handed.** Splitting "the bytes a human approved" from "the bytes we encrypt to" is exactly how a relay would serve an honest key to the approval screen and a substituted one to the wrap. **It never throws.** A device nobody approved must not be able to stop a rotation, or the gate becomes a way to disable the only remedy argala has for a compromised device. Blocked devices are named in `excluding`, and the relay accepts the rotation as complete. #### Why proof of possession makes the gate usable ```mermaid graph LR R1["a record device X wrote"] --> O["we opened it
under the graph key"] O --> V["its signature verifies under
the sigJwk the roster publishes for X"] V --> C1["X held the graph key.
The relay cannot fake this — the
signature is inside ciphertext
it cannot produce."] SS["X's selfSig over its own fingerprint"] --> V2["verifies under the same sigJwk"] V2 --> C2["the published WRAPPING key belongs
to the same holder as the signing key"] C1 --> PIN["pin X, via 'possession', no prompt"] C2 --> PIN ``` Without this, every already-linked device, every invited member and every recovered laptop would raise a prompt on upgrade day — and a product that asks a hundred questions on Monday gets its questions dismissed for the rest of its life. The pin is written where the evidence appears, inside record verification, and not where a key gap appears: an already-linked fleet has no gaps, so nothing would ever trigger it. #### What a pin can be ```mermaid stateDiagram-v2 [*] --> unknown unknown --> approved: possession, or a human said yes unknown --> rejected: a human said no approved --> conflicted: published keys changed rejected --> unknown: forgetDevice() conflicted --> unknown: forgetDevice() approved --> unknown: forgetDevice() ``` `approved` is the only state in which a key is wrapped, and it is not terminal: the fingerprint is re-checked on every wrap, so an approved device whose published keys change lands in `conflicted` on the next sync rather than keeping the grant it already had. Pins live in the local `KeyStore` under `pin:` and are **never synced**. A pin is the one record in this system that must not be something the relay — or another member — can write; sync it and the question becomes "who approved the approval", which is the same question one level up. `forgetDevice()` is the anti-kill-switch. A conflicted pin blocks a device forever, so a relay that rewrote one roster row could otherwise brick an honest laptop with no way back. There has to be a path from any state to "ask me again". The human-facing half is a **pairing code**: twelve Crockford base32 characters over a PBKDF2 stretch of the fingerprint. It is one-sided — computed from the joining device's own keys on one screen, and from what the relay published on the other — because a symmetric safety-number construction would let a relay controlling both screens run a birthday attack across two candidate sets at once. ### Invites: a keyring that rides beside the door code ```mermaid graph LR I["invite string"] --> SPLIT{"split on '.'"} SPLIT --> DC["doorCode
goes to your server"] SPLIT --> KS["keySecret
never leaves the two humans"] KS --> PB["PBKDF2 310k"] PB --> OPEN["open the sealed keyring"] DC --> SRV["your invite table:
who is this for?"] ``` The server learns the door code and can answer "is this invite valid" — it never sees the half that opens the keyring. Losing the whole string to a network attacker is a real compromise; losing your server's database is not. ### Recovery bundles ```mermaid graph TD KR["every graph keyring on this device"] --> P["payload {keyrings}"] P --> S["AES-GCM under PBKDF2(passphrase, salt)"] S --> B["base64 bundle"] B -.->|import on a NEW device| N["new device keypair
+ known graph keys"] style N fill:#1f4d2b,color:#eee ``` The bundle carries **graph keys only** — never a device keypair. A restored machine is a _new_ device that happens to already know the graph keys. This keeps two machines from signing as one and preserves meaningful revocation. Importing first wipes the local record store: a store partially populated by a previous identity is worse than an empty one. --- ## Epochs and rotation An epoch is a generation of a graph's key. Rotation mints the next one and wraps it to everybody who remains. ```mermaid stateDiagram-v2 [*] --> NoKey: graph exists, nobody has made a key NoKey --> Epoch1: createGraphKey Epoch1 --> Epoch2: rotate (a device was revoked) Epoch2 --> Epoch3: rotate Epoch1 --> Epoch1: grant — a new device is wrapped in Epoch2 --> Epoch2: grant note right of NoKey relay reports epoch 0 end note note right of Epoch2 old epochs are KEPT. Records sealed under epoch 1 must still open. end note ``` Old epochs are never thrown away. A record written last year is still sealed under the key from last year, so a keyring is a map of epoch → key, and a device that lacks an old epoch appears in `keygaps` until somebody fills it. ### A rotation that would strand somebody is refused ```mermaid sequenceDiagram participant D as device participant R as relay D->>R: POST keys {epoch: 4, wraps for 3 of the 4 devices} R->>R: eligible = active devices of current members R->>R: missing = eligible - covered - excluding R-->>D: 400 incomplete-rotation {missing: ['sam-phone']} Note over R: refused, with the IDs named —
not accepted with a warning ``` The alternative is a graph silently split in half, which is the failure mode nobody notices until a week of messages is unreadable on one laptop. `excluding` is the one way out, and it is a decision rather than a gap: the caller names the devices it is stranding on purpose — the ones the approval gate blocked. The relay refuses an exclusion naming a device that is not eligible, or that the same request also wraps to (`400 invalid-exclusion`), and reports the exclusions on the `graph.rotate` event so a host app can alarm on a rotation that left somebody behind. Without this field, the completeness rule and the approval gate deadlock each other and rotation stops being possible at all. ### A rotation reaches the other devices on their next sync ```mermaid sequenceDiagram participant A as device A participant R as relay participant B as device B A->>R: revoke sam-phone A->>R: POST keys {epoch: 2, wraps to A and B} Note over B: B has not re-opened the graph.
It only syncs. B->>R: POST push (still under epoch 1) R-->>B: {accepted, cursor, now, epoch: 2} B->>R: GET pull R-->>B: {ops, cursor, more, epoch: 2} B->>B: noteEpoch(2) — ahead of my keyring B->>R: GET keys R-->>B: the epoch-2 wrap B->>B: next write is sealed under epoch 2 ``` Before the relay reported the epoch, B learned about it only after receiving a record it could not open — or when the periodic check ran, up to a minute later — and kept sealing under exactly the key the rotation was performed to take away. A guarantee that takes effect once somebody notices is not much of one. ### The hostile-member case Anyone holding a graph's key can rotate it, and the relay **cannot** check that a wrap decrypts because the relay itself cannot decrypt. A malicious member can therefore hand another device a wrap it cannot open. argala makes that failure recoverable rather than terminal: ```mermaid flowchart LR W["wrap arrives"] --> TRY{unwrap} TRY -- ok --> ADOPT["adopt the epoch"] TRY -- fails --> DISCARD["DELETE keys/:epoch
emit keys.unopenable"] DISCARD --> GAP["reappears in keygaps"] GAP --> REGRANT["an honest holder re-grants"] REGRANT --> W ``` Keeping an unopenable wrap would be worse than discarding it: the device would sit one epoch behind forever, holding every write, with the relay insisting it had been served. Cryptographic write control — signed rotations against a policy — is the real answer and is not built. See SECURITY.md. --- ## Storage layout ### On the relay ```mermaid erDiagram graph_meta ||--o{ graph_records : "graph_key" graph_meta ||--o{ graph_keys : "graph_key" graph_meta ||--o{ graph_blobs : "graph_key" graph_meta ||--|| graph_seq : "graph_key" graph_meta ||--o{ graph_log : "graph_key" devices ||--o{ graph_keys : "device_id" graph_meta { text graph_key PK int latest_epoch int rotation_required int records "usage, maintained incrementally" int bytes int blob_bytes } graph_records { text graph_key PK text record_id PK int seq "ordering" int rev "LWW" int epoch "which key sealed it" text nonce text ciphertext text kind "NULL unless the app opted in" text shard "NULL unless the app opted in" int size int at } graph_keys { text graph_key PK int epoch PK text device_id PK text wrapped "sealed TO a device pubkey" int created_at } devices { text device_id PK text member_id text pub_jwk "ECDH" text sig_jwk "ECDSA" text label int created_at int revoked_at } graph_blobs { text graph_key PK text blob_id PK text nonce int size text policy int expires_at } graph_seq { text graph_key PK int next_seq } graph_log { text graph_key PK int seq PK text record_id "append-only; see below" int rev } ``` Nothing in that schema can hold plaintext, and a test asserts it — by column name and by content, across every table. Two design notes: **Usage is counted incrementally.** `records`, `bytes` and `blob_bytes` on `graph_meta` exist because enforcing quotas with `COUNT(*)` and `SUM(size)` on every push made a graph of 100,000 records push about five times slower than an empty one. `npm run argala:bench` fails if a scan comes back. **`graph_seq` is a counter, never `MAX(seq) + 1`.** A number reused after a delete would make a puller skip a record. **Migrations are append-only.** `MIGRATIONS` runs under `PRAGMA user_version`; a migration that has shipped is never edited. The relay refuses to run against a database written by a newer build rather than guessing. ### On the device ```mermaid graph TD subgraph "GraphStore — records, plaintext" R1["records: graphId + id → GraphRecord"] R2["queue: ids waiting to push"] R3["cursors: per SUBSCRIPTION, not per graph"] end subgraph "KeyStore — secrets" K1["device:<id> → ECDH keypair"] K2["sign:<id> → ECDSA keypair"] K3["keyring:<graphId> → {epochs, current}"] end ``` They are deliberately separate. Wiping records on a recovery import must not take the keys with it, and a native shell wants the keys in the OS keychain while records stay in a database file. Cursors are keyed by _subscription_ because a device pulling `g:acme` unfiltered and `g:acme` filtered to one kind is at two different places in the same sequence. Sharing one cursor would make each subscription skip what the other consumed. ### Choosing a store ```mermaid flowchart TD START([bestStore]) --> OPFS{OPFS + SharedArrayBuffer
+ worker available?} OPFS -- yes --> O["OpfsGraphStore
SQLite-WASM, SAH-pool VFS, in a worker"] OPFS -- no --> I["IdbGraphStore
indexed by graph and kind"] NODE([Node / Electron main]) --> N["NodeSqliteGraphStore
node:sqlite on a real file"] TEST([tests, server agents]) --> M["MemoryGraphStore"] ``` OPFS is **one tab at a time**. `bestStore` falls back to IndexedDB in a second tab, so two tabs of the same app hold two different local stores. They converge through the relay, but neither sees the other's unsynced writes. Not a confidentiality problem; very much a correctness surprise, and it is in SECURITY.md for that reason. --- ## The relay's request pipeline ```mermaid flowchart TD REQ([HTTP request]) --> RL{rate limiter
per member, per bucket} RL -- over --> R429[429] RL -- ok --> AUTH{authenticate} AUTH -- null --> R401[401] AUTH -- identity --> SAFE{isSafeId
member AND device} SAFE -- no --> R400["403 malformed-identity"] SAFE -- yes --> BIND{device registered
to THIS member?} BIND -- no --> R403a["403 device-not-registered"] BIND -- revoked --> R403b["403 device-revoked"] BIND -- yes --> RESOLVE{resolve the graph} RESOLVE -- not a member --> R403c[403] RESOLVE -- unknown member --> R404[404] RESOLVE -- ok --> SCHEMA{zod body schema} SCHEMA -- fail --> R400b[400 invalid] SCHEMA -- ok --> TX["transact: BEGIN IMMEDIATE"] TX --> LOGIC[route logic] LOGIC --> POKE["poke(graph) — carries no content"] POKE --> RESP([response]) ``` `isSafeId` is not cosmetic. A member ID containing a colon would let two different pairs collide on one `d:a:b` storage key — an isolation break, not a typo — so IDs are validated before anything builds a key from them. **Every inbound frame is schema-validated before any logic touches it.** At the relay that means the HTTP body; on the device it means the _decrypted record_, because the relay cannot validate what it cannot read and any key-holding member can craft anything. Validation lives at the last boundary that can enforce it, not the first one that is convenient. ### The route surface ```mermaid graph LR subgraph devices d1["PUT /devices/:id"] d2["GET /devices"] d3["POST /devices/:id/revoke"] end subgraph records r1["POST /graphs/:gid/push"] r2["GET /graphs/:gid/pull"] r3["GET /graphs/:gid/log"] end subgraph keys k1["GET /graphs/:gid/keys"] k2["GET /graphs/:gid/keygaps"] k3["POST /graphs/:gid/keys"] k4["DELETE /graphs/:gid/keys/:epoch"] k5["GET /graphs/:gid/devices"] end subgraph blobs b1["POST /graphs/:gid/blobs/:id"] b2["GET /graphs/:gid/blobs/:id"] b3["POST /graphs/:gid/blobs/:id/ack"] b4["DELETE /graphs/:gid/blobs/:id"] end ``` The relay exposes fifteen routes. None parses a record's contents. Four handle key material, but none can open it. The log route returns the relay's own account of what it accepted, allowing devices to hold the relay accountable. --- ## The append-only log The relay cannot read a record, alter one, or forge one. It could always do two other things: decline to hand a record over, and hand over the version that was replaced. A device asking for a graph it has never seen had no way to know what it was not given. ### Why the current-record table is insufficient ```mermaid graph TD subgraph "graph_records — a table" T1["r7 · seq 90 · rev 900"] end subgraph "what actually happened" H1["r7 · seq 12 · rev 400"] --> H2["r7 · seq 51 · rev 700"] --> H3["r7 · seq 90 · rev 900"] end H3 -.->|"the row moves;
the past stops existing"| T1 style T1 fill:#3d2b2b,color:#eee ``` `graph_records` is keyed by record ID, so an update moves the row to a new sequence number and the versions it replaced cease to exist in that table. A device that joined last week and one that joined today therefore pull **different sequences of rows** — the newcomer sees `r7` once, at revision 900, and can never reconstruct what the other one saw. No chain computed over what a device pulled could ever agree between them, and without agreement there is nothing to compare. `graph_log` does not move. One row per accepted write, holding only what the relay was already handed, forever. ```mermaid erDiagram graph_log { text graph_key PK int seq PK text record_id int rev } ``` The log adds roughly sixty bytes per write, compared with the hundreds of bytes the ciphertext already requires. Refused writes are not logged — a last-write-wins loser is not a write — so sequence numbers have gaps, which matters later. ### The two checks ```mermaid sequenceDiagram participant D as device participant R as relay participant P as another device D->>R: GET pull?after=C R-->>D: ops (the relay's story) D->>R: GET log?after=L R-->>D: entries (the relay's OWN account of what it took) loop every entry with seq ≤ C D->>D: fold into the chain D->>D: do I hold this record at this rev or newer? end Note over D: no → omitted / rolled-back.
Needs no peer and no key. D->>R: push a signed checkpoint {seq, head} P->>R: pull R-->>P: D's checkpoint, sealed and signed P->>P: my head at that boundary — the same? Note over P: no → diverged.
The relay is not serving one graph to everybody. ``` The first check catches a relay that contradicts itself. The second catches a perfectly self-consistent relay that maintains a private world for each member. Such a relay passes the first check flawlessly but fails the second because the chain head depends on the log alone and two devices reading the same range must compute the same value. ### Why the chain agrees, and why boundaries are quantised ``` h₀ = SHA-256("skm-log-v1\ngenesis\n" + storageKey) hₖ = SHA-256("skm-log-v1\n" + hₖ₋₁ + "\n" + seq + "\n" + recordId + "\n" + rev) ``` Genesis is bound to the storage key, so one graph's log cannot pass as another's. Everything else is the log and nothing but the log — not what the device stored, not what it decided, not when it joined. Devices cross the log at different moments and with different page sizes, so their heads are rarely at the same position. Checkpoints are quantised to a fixed stride so they can be compared: ```mermaid graph LR subgraph "device A folds 1..70 in one page" A64["head at 64"] end subgraph "device B folds 1..40, then 41..90" B64["head at 64"] end A64 -->|compare| EQ{"equal?"} B64 --> EQ ``` The head "at boundary 64" is defined as the chain after folding every entry with `seq ≤ 64` — which is well defined whether or not an entry sits exactly on 64. That matters because gaps are normal: a push allocates its whole seq range up front and a refused op spends one without logging. A device that only recorded a head when an entry landed exactly on a boundary would, on a busy graph, have nothing in common with anybody. ### Not accusing an honest relay A detector that fires on an honest relay will eventually be switched off, so every innocent explanation is ruled out before an alarm is raised. | looks like an omission | actually | | --------------------------------------------- | -------------------------------------------------------------------------------- | | a record missing from a filtered subscription | the device asked to see less — check reports `off` | | a record this device rejected | it was refused on purpose; refusals are remembered | | entries past the pull cursor | not served yet — a pull stopped by a missing epoch, or somebody writing mid-sync | | the log starts later than the device's cursor | an operator reclaimed disk — reports `unverified` | | the log starts later than the very beginning | a device linked after a prune — anchors, and never compares heads with anybody | | no log route at all | an older relay — reports `unverified`, not an accusation | ```mermaid stateDiagram-v2 [*] --> unverified: nothing checked yet unverified --> ok: the log agrees with what was served unverified --> off: only filtered subscriptions here ok --> unverified: relay unreachable, or pruned past us ok --> alarm: contradiction, or a peer's head differs unverified --> alarm alarm --> alarm: latched — behaving afterwards proves nothing ``` `alarm` latches deliberately. A relay caught omitting one record is not cleared by behaving for the next thousand, and a warning that clears itself is a warning nobody sees. #### The anchor, and why the first fold is the dangerous one The row about a pruned log is two rows because it is two situations, and only one of them is obvious. A device that has already folded past the pruned span notices immediately: the log now starts later than its own cursor, so there is a span it verified and can no longer re-derive. It says `unverified` and stops. A device folding for the first time has no cursor to compare against. The original guard was written as "cursor > 0 AND the log starts later than the cursor", which allowed that device through. It then folded from genesis over a history that begins in the middle, arrived at a head at every boundary that no peer could reproduce, and `compareCheckpoints` read the difference as a member asserting a history nobody else shares. The result was a latching alarm that named a colleague by device and member after an operator ran a `DELETE`. The fix is an anchor. `LogState.origin` records where the chain starts, and the guard became "the log starts more than one past what we have folded", which covers both cases with one comparison. A device with a non-zero anchor: - **still folds forward**, because everything after the anchor is the relay's own account of what it accepted and is still worth checking — one `DELETE` must not switch the detector off for every device linked afterwards, which would be a better outcome for a hostile relay than being caught; - **never compares** its head with a peer's, and corroborates nobody; - **never publishes** a checkpoint, because an older build has no idea the head is anchored elsewhere and would compare the number as though it were the ordinary one — accusing the device that published it; - **never reports `ok`**, because there is a span it cannot see. Putting the anchor on the wire would let two devices with the same anchor corroborate each other again. That is a real improvement and it is not free: it needs a field that older builds can safely ignore, which is the same design question the checkpoint's optional fields already answer. ### What still gets through A relay that shows **every** member a wholly separate world, their peers' checkpoints included, is invisible in-band — the members would have to compare a head out of band. Detection is also useless on a device that never syncs, and it is off for partial sync. So the guarantee is confidentiality and authenticity, with freshness now _evidenced_ rather than _guaranteed_; the difference is deliberate and is spelled out in SECURITY.md. One honest cost: the relay now retains the shape of a record's edit history where it used to overwrite it. It never learns more about the contents — it watched each write happen either way — but it remembers. --- ## Blobs and retention Large content does not belong in a record — the record limit is 512 KB, and LWW on a 20 MB payload is a poor fit. Blobs are sealed bytes stored beside sealed records and addressed by a random ID that a record can reference. ```mermaid stateDiagram-v2 [*] --> retain: policy retain (default) [*] --> transport: policy transport [*] --> ephemeral: policy ephemeral (ttl) retain --> deleted: explicit DELETE transport --> forgotten: every member acked ephemeral --> forgotten: ttl passed forgotten --> [*] deleted --> [*] ``` The sweeper runs inside the relay by default (`sweepIntervalMs`, 60s), because retention policy that only runs when somebody remembers to schedule it is not retention policy. A multi-process deployment wants exactly one sweeper, not one per process — set `sweepIntervalMs: 0` and call `sweepBlobs()` from a single job. **Retention is intent, not enforcement.** `transport` and `ephemeral` are executed by a relay you may not control; a relay that keeps a copy is invisible to you. This is stated plainly in SECURITY.md and should be stated plainly to your users. --- ## Failure and recovery Everything here is designed around one assumption: the network will fail, and the app should continue to work. ```mermaid flowchart TD W([app writes]) --> L["local store, synchronously"] L --> Q["queue the id"] Q --> OK([app continues]) Q -.->|debounced, background| S{sync} S -- offline --> KEEP["stays queued; status.lastError set"] S -- 5xx / 429 / network --> RETRY["jittered backoff, bounded attempts"] S -- 4xx --> REFUSE["not retryable: surface it"] S -- ok --> DRAIN["dequeue"] KEEP -.->|next sync| S RETRY -.-> S ``` | failure | what happens | | ------------------------------- | ------------------------------------------------------------------------------ | | relay unreachable | writes queue; reads are local and unaffected; `status.lastError` is set | | relay returns 429 or 5xx | retried with jittered backoff, bounded attempts | | relay returns 4xx | not retried — surfaced as `status.rejected` and a `record.rejected` event | | device clock is wrong | the relay's `now` is adopted once, and the monotonic mark is reset with it | | a record fails the app's schema | skipped with `record.rejected {reason: 'schema'}`; the rest of the batch lands | | the roster cannot be fetched | the batch is deferred, cursor kept — never dropped | | a wrap cannot be opened | discarded, reported, re-granted | | the process dies mid-flush | the queue is in the store; the next boot drains it | | two tabs, OPFS taken | the second falls back to IndexedDB — see the note above | ### The clock repair, in detail ```mermaid sequenceDiagram participant D as device (clock 3 days fast) participant R as relay D->>R: push rev = now + 3 days R-->>D: rejected {reason: 'rev-too-far-ahead'}, now = D->>D: clockSkewMs = myClock - relayNow D->>D: lastAt = min(lastAt, corrected now) Note over D: BOTH halves matter. Correcting the skew
without resetting the high-water mark leaves
every later record rejected forever. D->>R: push rev = corrected R-->>D: accepted ``` --- ## Who is allowed to do what Authorization here is deliberately narrow, and the narrowness is the honest part rather than a gap. A member who holds a graph key can read it; that is arithmetic. What they cannot do is reach another device except through the relay, so the relay is the only place a permission can be enforced rather than merely asserted. ```mermaid flowchart TD M["a member's device"] --> REQ["push · blob · rotate"] REQ --> R{{"relay"}} R -->|"groupId is null?"| ALLOW["allowed — u and d: have\nnobody to have a role against"] R -->|"no capabilitiesOf?"| ALLOW R -->|"answers null?"| ALLOW R -->|"answers a list"| CHECK{"does it include\nthe capability?"} CHECK -->|yes| ALLOW CHECK -->|no| DENY["403 not-permitted"] ALLOW --> STORE[("stored — now reachable\nby every other device")] DENY --> HELD["the client HOLDS the queue\nand keeps pulling"] style DENY fill:#3d2b2b,color:#eee style STORE fill:#1f4d2b,color:#eee ``` Three of those arrows represent defaults, and reversing any of them would be a bug. A directory with no `capabilitiesOf` grants everything, because a permission model that locks out every existing user on upgrade day is not a feature. `null` means unmanaged rather than restricted, so a host app can govern one group and ignore the rest. And personal and pairwise graphs are never asked about, because a check there could only lock somebody out of their own data. **`403` must not throw on the client.** Handling it as a transport failure abandons the sync run before the pull, so a device permitted to read and not to write would gradually stop receiving anybody else's records — a permission feature behaving like a disconnection. The queue is held, the refusal is reported as `policy.readonly`, and the pull continues. **What cannot be enforced here, and why.** Per-record ownership needs to know whose record it is; restricting by kind needs to know what kind it is. Both would mean the relay reading a record, which is the one thing this system is built not to do. Those checks can only be made on a device that holds the key, which makes them non-recognition after the fact — worth having, and a different guarantee. --- ## Trust boundaries ```mermaid graph TB subgraph "trusted: the device" APP["app code"] ENG["ArgalaEngine"] PT[("plaintext store")] end subgraph "semi-trusted: the host app's server" SESS["your sessions
authenticate()"] DIR["your users
Directory"] end subgraph "untrusted: the relay" CT[("ciphertext + wrapped keys")] end subgraph "untrusted: the network" NET["TLS is the host app's job"] end APP --> ENG --> PT ENG -->|sealed ops| NET --> CT SESS -.->|names member + device| CT DIR -.->|answers membership| CT style PT fill:#1f4d2b,color:#eee style CT fill:#2b2b2b,color:#eee style NET fill:#3d2b2b,color:#eee ``` What the relay learns anyway, stated plainly: who syncs what and when, record ids and sizes, the pairwise graph list, and — only if an app opts in — kinds and shards. What it does **not** learn: record contents, authorship (signatures live inside the ciphertext), and what you read (read positions live in the personal graph, encrypted). The honest limits — rollback and omission by a malicious relay, a hurried human at the approval gate, a hostile member's denial of service, the browser delivery caveat — are in [SECURITY.md](https://docs.thinkingmachinelabs.dev/argala/internals/security.md), which is the document to read before changing anything here. --- ## Extension seams Every one of these is an interface because the alternative was guessing what your deployment looks like. ```mermaid graph LR subgraph relay DIRE["Directory
your users"] AUTH["authenticate
your sessions"] BS["BlobStore
SQLite, filesystem, S3"] RLI["RateLimiter
memory, Redis"] POKE["onPoke
your realtime"] EV["onEvent
your metrics"] end subgraph client TR["Transport
HTTP, or your own"] GS["GraphStore
five implementations"] KS["KeyStore
keychain, file, IDB"] KR["KindRegistry
your schemas"] NOW["now()
your clock"] end ``` Two of them deserve a warning: - **`MemoryRateLimiter` is per process.** Correct for one relay, wrong for four behind a load balancer. Back `RateLimiter` with something shared before you scale out. - **`FileKeyStore` is plaintext on disk.** Right for a server-side agent whose disk is already the trust boundary, wrong for a laptop — there the OS keychain is the answer, and `KeyStore` is an interface so that it can be. --- ## Deployment shapes ### Mounted in the app you already have ```mermaid graph LR B["browser app"] -->|same origin| F["your Fastify server"] F --> ARGALA["registerRelay(app, ...)"] F --> REST["your existing routes"] ARGALA --> DB[("SQLite / your DB")] F --> WS["your websocket hub"] ARGALA -.->|onPoke| WS ``` This is the common case and the deployment argala is designed for. You keep your sessions, users, and realtime infrastructure; argala adds prefixed routes and a set of tables. ### Standalone relay ```mermaid graph LR A1["app instance"] --> LB[load balancer] A2["app instance"] --> LB LB --> R1["relay 1"] LB --> R2["relay 2"] R1 --> PG[("shared database")] R2 --> PG R1 -.-> RL[("shared rate limiter")] R2 -.-> RL SW["one sweeper job"] --> PG ``` Two things change when you scale out: the rate limiter must be shared, and exactly one process should sweep blobs (`sweepIntervalMs: 0` everywhere else). ### Server-side agent ```mermaid graph LR AG["a bot / worker"] --> ENG["ArgalaEngine
MemoryGraphStore or NodeSqliteGraphStore
FileKeyStore"] ENG --> R["relay"] ``` A worker that needs to read a graph is just another device: register it, grant it, revoke it like any other. It holds real keys, so its disk is a real trust boundary. --- ## How it is tested The bugs in a system like this are the ones where everything still appears to work, so the suite is built to be adversarial rather than exhaustive. ```mermaid graph TD U["unit — core: crypto, canonical JSON, schemas"] --> V I["integration — real relay, real crypto, injected HTTP"] --> V A["adversarial — a relay that omits, rewinds, and tells two stories"] --> V F["fuzz — hostile input at every relay route"] --> V C["convergence — seeded random scenarios, 3–5 devices"] --> V V["npm run argala:verify
typecheck, lint, format, test, build"] --> SM SM["smoke — the BUILT packages, over a real socket"] --> CI([CI]) M["npm run argala:mutate
every invariant broken on purpose"] --> CI S["npm run argala:soak
hundreds of randomized scenarios"] --> CI B["npm run argala:bench
asserts a bound, not a number"] --> CI ``` Three of those are unusual enough to explain: **`npm run argala:smoke` drives the built packages over a real socket.** Tests run against `src` through a vitest alias; the published packages resolve each other through `exports`, which point at `dist`. The smoke test is what closes that gap — if you change an `exports` map or add a file, it is the one that notices. **`npm run argala:mutate` breaks each guarded line in turn** — accept a bad signature, drop the record id from the AAD, let a rotation strand a device, resolve a revision tie the other way — and fails if no test goes red. A test that has never been red is not evidence of anything. Its first run left four mutations alive, and each one was a real gap in the suite. **`npm run argala:soak` runs hundreds of randomized convergence scenarios** on a virtual clock the seed advances, so a failing seed reproduces exactly. It found the revision-floor bug and the rotation-propagation delay described above. Failing seeds are promoted into the commit suite, so a bug found once is a bug tested forever. Every scenario runs with `linking: 'approve'` and models explicit human approval rather than switching the gate off, so each run also asserts the gate stays silent afterwards, which is the property that decides whether anybody leaves it turned on. --- ## Where to go next - [docs/GUIDE.md](https://docs.thinkingmachinelabs.dev/argala/guide.md) — how to build on it - [PROTOCOL.md](https://docs.thinkingmachinelabs.dev/argala/internals/protocol.md) — the wire, normatively - [SECURITY.md](https://docs.thinkingmachinelabs.dev/argala/internals/security.md) — invariants, threat model, honest limits - [CONTRIBUTING.md](https://github.com/rishabhshuklax/bapk/blob/main/packages/CONTRIBUTING.md) — how to work on it - [ROADMAP.md](https://docs.thinkingmachinelabs.dev/argala/project/roadmap.md) — what is missing and what it would take