Architecture

How the pieces fit, why each boundary is where it is, and what happens on the wire when somebody types a character.

This document is for people changing argala, evaluating it, or writing a second implementation of the protocol. If you only want to use it, start with GUIDE.md. If you want the exact wire format, read PROTOCOL.md; this file explains the shape, that file is normative.

Contents

The one-sentence version
The three packages
Identities, graphs and storage keys
The crypto boundary
Anatomy of a record
The sync loop
Key ceremonies
Epochs and rotation
Storage layout
The relay’s request pipeline
The append-only log
Blobs and retention
Failure and recovery
Trust boundaries
Extension seams
Deployment shapes
How it is tested

The one-sentence version

Every device holds the real database in plaintext. The server holds an ordered log of opaque (nonce, ciphertext) pairs and wrapped keys it cannot open. The interesting engineering lies in getting the right key onto the right device without ever handing that key to the server.

Green is plaintext. Grey is ciphertext. There is no third colour: the relay never sees a key, a plaintext record, or an author’s name.

The three packages

The dependency arrow only ever points at core, and that is enforced: the test suite scans the relay’s own source for any import that could decrypt, and fails the build if one appears. @argala/relay does not depend on @argala/client and never will — the two ends share a protocol, not code paths.

package	holds	never holds
`@argala/core`	AES/ECDH/ECDSA/HKDF wrappers, canonical JSON, zod schemas, `LIMITS`, AAD and storage-key naming	any network or disk call
`@argala/client`	keyrings, the sync loop, LWW, local stores, blobs, invites, recovery	anything the app should decide (schemas, sessions)
`@argala/relay`	ordering, quotas, rate limits, the device roster, wrapped-key custody	a decrypt routine, a graph key, a record’s contents

Inside `@argala/client`

Four seams, four interfaces. The ArgalaEngine implementation is the only place that knows how they fit together, so mistakes in that file are expensive.

Identities, graphs and storage keys

argala does not own your users. It asks the host app two questions (Directory) and takes one answer per request (authenticate).

A member is a person or account, as defined by your system. A device is one installation of an app. Keys belong to devices, never to members — that is what makes revocation meaningful.

Three graph shapes

The pairwise case deserves careful attention. Each side addresses the graph by the other party’s ID, but both must resolve to one row in the relay and one AAD value in the cryptographic layer, so the storage key sorts the pair. Get this wrong and the two sides seal data into slots the other cannot read, with no error anywhere.

The same function computes it on both ends — it lives in @argala/core precisely so it cannot drift.

The crypto boundary

Everything cryptographic lives in argala-core/src/crypto.ts, and every primitive is WebCrypto. Nothing here is hand-rolled; the layout around it is.

Why the signature is inside the ciphertext

If the signature rode outside, the relay would learn which device wrote each record — the social graph, in the clear, forever. So a record is wrapped in an envelope, the envelope is what gets sealed, and the relay sees one opaque blob.

canonicalJson sorts keys recursively and drops undefined exactly as JSON.stringify does, so two devices that built the same object in a different order produce the same bytes to sign. Without it, a signature would verify or not depending on property insertion order — a bug that appears months later, on one platform.

Why AAD, and what it buys

Every seal is bound to <storageKey>/<recordId> (or <storageKey>/blob/<id>). A relay that shuffles rows, or replays one record’s ciphertext into another record’s slot, produces a decryption failure, never a wrong answer.

Note the last line: a bad record is skipped, not fatal. One corrupt row must never wedge a sync.

Personal ids are derived, not named

A record in your personal graph that refers to a shared record must never be named after it. pin_<docId> sitting beside g:acme/<docId> lets the relay join the two and read off what every member tracks, without decrypting anything.

Deterministic, so upsert-by-id still merges across your own devices. Derived from epoch 1 of the personal graph, so it survives rotation, a newly linked device, and a recovery import. Two members pinning the same record produce unrelated ids. engine.personalId(kind, target) is the only correct way to name such a record.

Anatomy of a record

at is both the record’s timestamp and its revision. That dual role is deliberate — one number, monotonic per device, is all last-write-wins needs — and it carries two rules that are easy to get wrong:

Monotonic per device. Two writes inside one millisecond must not stamp the same number, or the relay drops the second and the device keeps content nobody else has.
Above what it replaces. Editing a record another device wrote from a clock a few milliseconds ahead must stamp above that revision, not just above this device’s own high-water mark. Otherwise the relay drops the edit as a last-write-wins loser, the local store keeps it, and the pull cursor has already passed the relay’s copy — the edit lives on one device forever, with no error anywhere.

npm run argala:soak exposed the second rule; code inspection did not.

Tombstones are records. The relay never deletes a row; a deletion is a record with deleted: true and an empty data. Anything else would make “deleted on device A, edited on device B” unresolvable.

The sync loop

One graph at a time, coalesced so eight concurrent callers produce one run.

Four details in that diagram are essential:

A push is bounded by bytes as well as by op count. Two hundred records at the maximum ciphertext size total roughly 140 MB, and the resulting 413 is not retryable — a queue that stops draining and stays stopped.

A device that is behind an epoch holds its writes. Sealing under the previous key would be readable by whoever the rotation excluded. Unsent is recoverable; sent under the wrong key is not.

Meeting an op sealed under a missing epoch stops the pull before that op, without advancing the cursor, fetches keys, and tries once more in the same sync. Skipping it would lose the record forever.

“Cannot verify” is not “does not verify.” If the roster cannot be fetched — offline, rate-limited — the batch is deferred, not dropped. Dropping would silently lose data that is sitting right there.

Last-write-wins, and the tie

The relay keeps the first of two equal revisions. The device’s comparison is therefore local.at <= record.at on a difference, not <: a strict < would leave B holding its own version forever, with no error and no way back.

The echo optimisation, and its one rule

A device that just pushed a record will receive it again on the next pull. Decrypting and verifying its own writes is unnecessary, so the engine remembers (graphId, recordId) → rev and skips an op that matches exactly.

The rule: mark echoes only when EVERY op in the batch landed. rejected lists refusals, not the ops that quietly lost a last-write-wins comparison — and a record that lost is one where the relay’s copy is somebody else’s. Marking that as our own echo would skip the one record we most need to read.

Key ceremonies

Creating a graph key

That read-back is not merely defensive programming; it fixes a real race. A concurrent create is answered 200, not 409 — the relay sees a known epoch and treats the request as a grant, ignoring the duplicate wrap. A device that trusted that 200 would keep a key nobody else has: every record it writes unreadable, every record it reads undecipherable, and no error anywhere.

Linking a second device

No private key ever moves between machines.

grantPending runs automatically during every sync performed by a device that holds the keys (autoGrant, on by default). This makes linking feel automatic instead of like a ceremony someone must remember to perform.

The approval gate

A relay cannot mint a graph key, but it can add a row to the roster: a device ID it invented, published with a wrapping key whose private half it holds. Every honest device then sees a gap and helpfully fills it. Nothing appears broken, but the relay gains access to the graph key.

Gating grantPending alone would not help. A planted device does not need to be granted — it can wait for rotate(), which is what you run when a laptop is stolen. So the gate sits on the one function every wrap goes through:

It fingerprints what it is handed. Splitting “the bytes a human approved” from “the bytes we encrypt to” is exactly how a relay would serve an honest key to the approval screen and a substituted one to the wrap.

It never throws. A device nobody approved must not be able to stop a rotation, or the gate becomes a way to disable the only remedy argala has for a compromised device. Blocked devices are named in excluding, and the relay accepts the rotation as complete.

Why proof of possession makes the gate usable

Without this, every already-linked device, every invited member and every recovered laptop would raise a prompt on upgrade day — and a product that asks a hundred questions on Monday gets its questions dismissed for the rest of its life. The pin is written where the evidence appears, inside record verification, and not where a key gap appears: an already-linked fleet has no gaps, so nothing would ever trigger it.

What a pin can be

approved is the only state in which a key is wrapped, and it is not terminal: the fingerprint is re-checked on every wrap, so an approved device whose published keys change lands in conflicted on the next sync rather than keeping the grant it already had.

Pins live in the local KeyStore under pin:<deviceId> and are never synced. A pin is the one record in this system that must not be something the relay — or another member — can write; sync it and the question becomes “who approved the approval”, which is the same question one level up.

forgetDevice() is the anti-kill-switch. A conflicted pin blocks a device forever, so a relay that rewrote one roster row could otherwise brick an honest laptop with no way back. There has to be a path from any state to “ask me again”.

The human-facing half is a pairing code: twelve Crockford base32 characters over a PBKDF2 stretch of the fingerprint. It is one-sided — computed from the joining device’s own keys on one screen, and from what the relay published on the other — because a symmetric safety-number construction would let a relay controlling both screens run a birthday attack across two candidate sets at once.

Invites: a keyring that rides beside the door code

The server learns the door code and can answer “is this invite valid” — it never sees the half that opens the keyring. Losing the whole string to a network attacker is a real compromise; losing your server’s database is not.

Recovery bundles

The bundle carries graph keys only — never a device keypair. A restored machine is a new device that happens to already know the graph keys. This keeps two machines from signing as one and preserves meaningful revocation. Importing first wipes the local record store: a store partially populated by a previous identity is worse than an empty one.

Epochs and rotation

An epoch is a generation of a graph’s key. Rotation mints the next one and wraps it to everybody who remains.

Old epochs are never thrown away. A record written last year is still sealed under the key from last year, so a keyring is a map of epoch → key, and a device that lacks an old epoch appears in keygaps until somebody fills it.

A rotation that would strand somebody is refused

The alternative is a graph silently split in half, which is the failure mode nobody notices until a week of messages is unreadable on one laptop.

excluding is the one way out, and it is a decision rather than a gap: the caller names the devices it is stranding on purpose — the ones the approval gate blocked. The relay refuses an exclusion naming a device that is not eligible, or that the same request also wraps to (400 invalid-exclusion), and reports the exclusions on the graph.rotate event so a host app can alarm on a rotation that left somebody behind. Without this field, the completeness rule and the approval gate deadlock each other and rotation stops being possible at all.

A rotation reaches the other devices on their next sync

Before the relay reported the epoch, B learned about it only after receiving a record it could not open — or when the periodic check ran, up to a minute later — and kept sealing under exactly the key the rotation was performed to take away. A guarantee that takes effect once somebody notices is not much of one.

The hostile-member case

Anyone holding a graph’s key can rotate it, and the relay cannot check that a wrap decrypts because the relay itself cannot decrypt. A malicious member can therefore hand another device a wrap it cannot open. argala makes that failure recoverable rather than terminal:

Keeping an unopenable wrap would be worse than discarding it: the device would sit one epoch behind forever, holding every write, with the relay insisting it had been served. Cryptographic write control — signed rotations against a policy — is the real answer and is not built. See SECURITY.md.

Storage layout

On the relay

Nothing in that schema can hold plaintext, and a test asserts it — by column name and by content, across every table.

Two design notes:

Usage is counted incrementally. records, bytes and blob_bytes on graph_meta exist because enforcing quotas with COUNT(*) and SUM(size) on every push made a graph of 100,000 records push about five times slower than an empty one. npm run argala:bench fails if a scan comes back.

graph_seq is a counter, never MAX(seq) + 1. A number reused after a delete would make a puller skip a record.

Migrations are append-only. MIGRATIONS runs under PRAGMA user_version; a migration that has shipped is never edited. The relay refuses to run against a database written by a newer build rather than guessing.

On the device

They are deliberately separate. Wiping records on a recovery import must not take the keys with it, and a native shell wants the keys in the OS keychain while records stay in a database file.

Cursors are keyed by subscription because a device pulling g:acme unfiltered and g:acme filtered to one kind is at two different places in the same sequence. Sharing one cursor would make each subscription skip what the other consumed.

Choosing a store

OPFS is one tab at a time. bestStore falls back to IndexedDB in a second tab, so two tabs of the same app hold two different local stores. They converge through the relay, but neither sees the other’s unsynced writes. Not a confidentiality problem; very much a correctness surprise, and it is in SECURITY.md for that reason.

The relay’s request pipeline

isSafeId is not cosmetic. A member ID containing a colon would let two different pairs collide on one d:a:b storage key — an isolation break, not a typo — so IDs are validated before anything builds a key from them.

Every inbound frame is schema-validated before any logic touches it. At the relay that means the HTTP body; on the device it means the decrypted record, because the relay cannot validate what it cannot read and any key-holding member can craft anything. Validation lives at the last boundary that can enforce it, not the first one that is convenient.

The route surface

The relay exposes fifteen routes. None parses a record’s contents. Four handle key material, but none can open it. The log route returns the relay’s own account of what it accepted, allowing devices to hold the relay accountable.

The append-only log

The relay cannot read a record, alter one, or forge one. It could always do two other things: decline to hand a record over, and hand over the version that was replaced. A device asking for a graph it has never seen had no way to know what it was not given.

Why the current-record table is insufficient

graph_records is keyed by record ID, so an update moves the row to a new sequence number and the versions it replaced cease to exist in that table. A device that joined last week and one that joined today therefore pull different sequences of rows — the newcomer sees r7 once, at revision 900, and can never reconstruct what the other one saw. No chain computed over what a device pulled could ever agree between them, and without agreement there is nothing to compare.

graph_log does not move. One row per accepted write, holding only what the relay was already handed, forever.

The log adds roughly sixty bytes per write, compared with the hundreds of bytes the ciphertext already requires. Refused writes are not logged — a last-write-wins loser is not a write — so sequence numbers have gaps, which matters later.

The two checks

The first check catches a relay that contradicts itself. The second catches a perfectly self-consistent relay that maintains a private world for each member. Such a relay passes the first check flawlessly but fails the second because the chain head depends on the log alone and two devices reading the same range must compute the same value.

Why the chain agrees, and why boundaries are quantised

h₀ = SHA-256("skm-log-v1\ngenesis\n" + storageKey)
hₖ = SHA-256("skm-log-v1\n" + hₖ₋₁ + "\n" + seq + "\n" + recordId + "\n" + rev)

Genesis is bound to the storage key, so one graph’s log cannot pass as another’s. Everything else is the log and nothing but the log — not what the device stored, not what it decided, not when it joined.

Devices cross the log at different moments and with different page sizes, so their heads are rarely at the same position. Checkpoints are quantised to a fixed stride so they can be compared:

The head “at boundary 64” is defined as the chain after folding every entry with seq ≤ 64 — which is well defined whether or not an entry sits exactly on 64. That matters because gaps are normal: a push allocates its whole seq range up front and a refused op spends one without logging. A device that only recorded a head when an entry landed exactly on a boundary would, on a busy graph, have nothing in common with anybody.

Not accusing an honest relay

A detector that fires on an honest relay will eventually be switched off, so every innocent explanation is ruled out before an alarm is raised.

looks like an omission	actually
a record missing from a filtered subscription	the device asked to see less — check reports `off`
a record this device rejected	it was refused on purpose; refusals are remembered
entries past the pull cursor	not served yet — a pull stopped by a missing epoch, or somebody writing mid-sync
the log starts later than the device’s cursor	an operator reclaimed disk — reports `unverified`
the log starts later than the very beginning	a device linked after a prune — anchors, and never compares heads with anybody
no log route at all	an older relay — reports `unverified`, not an accusation

alarm latches deliberately. A relay caught omitting one record is not cleared by behaving for the next thousand, and a warning that clears itself is a warning nobody sees.

The anchor, and why the first fold is the dangerous one

The row about a pruned log is two rows because it is two situations, and only one of them is obvious.

A device that has already folded past the pruned span notices immediately: the log now starts later than its own cursor, so there is a span it verified and can no longer re-derive. It says unverified and stops.

A device folding for the first time has no cursor to compare against. The original guard was written as “cursor > 0 AND the log starts later than the cursor”, which allowed that device through. It then folded from genesis over a history that begins in the middle, arrived at a head at every boundary that no peer could reproduce, and compareCheckpoints read the difference as a member asserting a history nobody else shares. The result was a latching alarm that named a colleague by device and member after an operator ran a DELETE.

The fix is an anchor. LogState.origin records where the chain starts, and the guard became “the log starts more than one past what we have folded”, which covers both cases with one comparison. A device with a non-zero anchor:

still folds forward, because everything after the anchor is the relay’s own account of what it accepted and is still worth checking — one DELETE must not switch the detector off for every device linked afterwards, which would be a better outcome for a hostile relay than being caught;
never compares its head with a peer’s, and corroborates nobody;
never publishes a checkpoint, because an older build has no idea the head is anchored elsewhere and would compare the number as though it were the ordinary one — accusing the device that published it;
never reports ok, because there is a span it cannot see.

Putting the anchor on the wire would let two devices with the same anchor corroborate each other again. That is a real improvement and it is not free: it needs a field that older builds can safely ignore, which is the same design question the checkpoint’s optional fields already answer.

What still gets through

A relay that shows every member a wholly separate world, their peers’ checkpoints included, is invisible in-band — the members would have to compare a head out of band. Detection is also useless on a device that never syncs, and it is off for partial sync. So the guarantee is confidentiality and authenticity, with freshness now evidenced rather than guaranteed; the difference is deliberate and is spelled out in SECURITY.md.

One honest cost: the relay now retains the shape of a record’s edit history where it used to overwrite it. It never learns more about the contents — it watched each write happen either way — but it remembers.

Blobs and retention

Large content does not belong in a record — the record limit is 512 KB, and LWW on a 20 MB payload is a poor fit. Blobs are sealed bytes stored beside sealed records and addressed by a random ID that a record can reference.

The sweeper runs inside the relay by default (sweepIntervalMs, 60s), because retention policy that only runs when somebody remembers to schedule it is not retention policy. A multi-process deployment wants exactly one sweeper, not one per process — set sweepIntervalMs: 0 and call sweepBlobs() from a single job.

Retention is intent, not enforcement. transport and ephemeral are executed by a relay you may not control; a relay that keeps a copy is invisible to you. This is stated plainly in SECURITY.md and should be stated plainly to your users.

Failure and recovery

Everything here is designed around one assumption: the network will fail, and the app should continue to work.

failure	what happens
relay unreachable	writes queue; reads are local and unaffected; `status.lastError` is set
relay returns 429 or 5xx	retried with jittered backoff, bounded attempts
relay returns 4xx	not retried — surfaced as `status.rejected` and a `record.rejected` event
device clock is wrong	the relay’s `now` is adopted once, and the monotonic mark is reset with it
a record fails the app’s schema	skipped with `record.rejected {reason: 'schema'}`; the rest of the batch lands
the roster cannot be fetched	the batch is deferred, cursor kept — never dropped
a wrap cannot be opened	discarded, reported, re-granted
the process dies mid-flush	the queue is in the store; the next boot drains it
two tabs, OPFS taken	the second falls back to IndexedDB — see the note above

The clock repair, in detail

Who is allowed to do what

Authorization here is deliberately narrow, and the narrowness is the honest part rather than a gap. A member who holds a graph key can read it; that is arithmetic. What they cannot do is reach another device except through the relay, so the relay is the only place a permission can be enforced rather than merely asserted.

Three of those arrows represent defaults, and reversing any of them would be a bug. A directory with no capabilitiesOf grants everything, because a permission model that locks out every existing user on upgrade day is not a feature. null means unmanaged rather than restricted, so a host app can govern one group and ignore the rest. And personal and pairwise graphs are never asked about, because a check there could only lock somebody out of their own data.

403 must not throw on the client. Handling it as a transport failure abandons the sync run before the pull, so a device permitted to read and not to write would gradually stop receiving anybody else’s records — a permission feature behaving like a disconnection. The queue is held, the refusal is reported as policy.readonly, and the pull continues.

What cannot be enforced here, and why. Per-record ownership needs to know whose record it is; restricting by kind needs to know what kind it is. Both would mean the relay reading a record, which is the one thing this system is built not to do. Those checks can only be made on a device that holds the key, which makes them non-recognition after the fact — worth having, and a different guarantee.

Trust boundaries

What the relay learns anyway, stated plainly: who syncs what and when, record ids and sizes, the pairwise graph list, and — only if an app opts in — kinds and shards. What it does not learn: record contents, authorship (signatures live inside the ciphertext), and what you read (read positions live in the personal graph, encrypted).

The honest limits — rollback and omission by a malicious relay, a hurried human at the approval gate, a hostile member’s denial of service, the browser delivery caveat — are in SECURITY.md, which is the document to read before changing anything here.

Extension seams

Every one of these is an interface because the alternative was guessing what your deployment looks like.

Two of them deserve a warning:

MemoryRateLimiter is per process. Correct for one relay, wrong for four behind a load balancer. Back RateLimiter with something shared before you scale out.
FileKeyStore is plaintext on disk. Right for a server-side agent whose disk is already the trust boundary, wrong for a laptop — there the OS keychain is the answer, and KeyStore is an interface so that it can be.

Deployment shapes

Mounted in the app you already have

This is the common case and the deployment argala is designed for. You keep your sessions, users, and realtime infrastructure; argala adds prefixed routes and a set of tables.

Standalone relay

Two things change when you scale out: the rate limiter must be shared, and exactly one process should sweep blobs (sweepIntervalMs: 0 everywhere else).

Server-side agent

A worker that needs to read a graph is just another device: register it, grant it, revoke it like any other. It holds real keys, so its disk is a real trust boundary.

How it is tested

The bugs in a system like this are the ones where everything still appears to work, so the suite is built to be adversarial rather than exhaustive.

Three of those are unusual enough to explain:

npm run argala:smoke drives the built packages over a real socket. Tests run against src through a vitest alias; the published packages resolve each other through exports, which point at dist. The smoke test is what closes that gap — if you change an exports map or add a file, it is the one that notices.

npm run argala:mutate breaks each guarded line in turn — accept a bad signature, drop the record id from the AAD, let a rotation strand a device, resolve a revision tie the other way — and fails if no test goes red. A test that has never been red is not evidence of anything. Its first run left four mutations alive, and each one was a real gap in the suite.

npm run argala:soak runs hundreds of randomized convergence scenarios on a virtual clock the seed advances, so a failing seed reproduces exactly. It found the revision-floor bug and the rotation-propagation delay described above. Failing seeds are promoted into the commit suite, so a bug found once is a bug tested forever. Every scenario runs with linking: 'approve' and models explicit human approval rather than switching the gate off, so each run also asserts the gate stays silent afterwards, which is the property that decides whether anybody leaves it turned on.

Where to go next

docs/GUIDE.md — how to build on it
PROTOCOL.md — the wire, normatively
SECURITY.md — invariants, threat model, honest limits
CONTRIBUTING.md — how to work on it
ROADMAP.md — what is missing and what it would take