Architecture

How the pieces fit, why each boundary is where it is, and what happens on the wire when somebody types a character.

This document is for people changing skm, evaluating it, or writing a second implementation of the protocol. If you only want to use it, start with GUIDE.md. If you want the wire in exact terms, read PROTOCOL.md; this file explains the shape, that file is normative.

Contents

The one-sentence version
The three packages
Identities, graphs and storage keys
The crypto boundary
Anatomy of a record
The sync loop
Key ceremonies
Epochs and rotation
Storage layout
The relay’s request pipeline
Blobs and retention
Failure and recovery
Trust boundaries
Extension seams
Deployment shapes
How it is tested

The one-sentence version

Every device holds the real database in plaintext; the server holds an ordered log of opaque (nonce, ciphertext) pairs and the wrapped keys it cannot open, and the interesting engineering is all in getting the right key onto the right device without ever handing one to the server.

Green is plaintext. Grey is ciphertext. There is no third colour: the relay never sees a key, a plaintext record, or an author’s name.

The three packages

The dependency arrow only ever points at core, and that is enforced: the test suite scans the relay’s own source for any import that could decrypt, and fails the build if one appears. @skm/relay does not depend on @skm/client and never will — the two ends share a protocol, not code paths.

package	holds	never holds
`@skm/core`	AES/ECDH/ECDSA/HKDF wrappers, canonical JSON, zod schemas, `LIMITS`, AAD and storage-key naming	any network or disk call
`@skm/client`	keyrings, the sync loop, LWW, local stores, blobs, invites, recovery	anything the app should decide (schemas, sessions)
`@skm/relay`	ordering, quotas, rate limits, the device roster, wrapped-key custody	a decrypt routine, a graph key, a record’s contents

Inside `@skm/client`

Four seams, four interfaces. SkmEngine is the only thing that knows how they fit together, and it is the only file in the repo where a mistake is expensive.

Identities, graphs and storage keys

skm does not own your users. It asks the host app two questions (Directory) and takes one answer per request (authenticate).

A member is a person or an account, as your system names them. A device is one installation. Keys belong to devices, never to members — that is what makes revocation mean something.

Three graph shapes

The pairwise case is the one worth staring at. Each side addresses the graph by the other party’s id, but both must land on one row in the relay and one AAD in the crypto — so the storage key sorts the pair. Get this wrong and the two sides seal into slots the other cannot read, with no error anywhere.

The same function computes it on both ends — it lives in @skm/core precisely so it cannot drift.

The crypto boundary

Everything cryptographic lives in packages/core/src/crypto.ts, and every primitive is WebCrypto. Nothing here is hand-rolled; the layout around it is.

Why the signature is inside the ciphertext

If the signature rode outside, the relay would learn which device wrote each record — the social graph, in the clear, forever. So a record is wrapped in an envelope, the envelope is what gets sealed, and the relay sees one opaque blob.

canonicalJson sorts keys recursively and drops undefined exactly as JSON.stringify does, so two devices that built the same object in a different order produce the same bytes to sign. Without it, a signature would verify or not depending on property insertion order — a bug that appears months later, on one platform.

Why AAD, and what it buys

Every seal is bound to <storageKey>/<recordId> (or <storageKey>/blob/<id>). A relay that shuffles rows, or replays one record’s ciphertext into another record’s slot, produces a decryption failure, never a wrong answer.

Note the last line: a bad record is skipped, not fatal. One corrupt row must never wedge a sync.

Personal ids are derived, not named

A record in your personal graph that refers to a shared record must never be named after it. pin_<docId> sitting beside g:acme/<docId> lets the relay join the two and read off what every member tracks, without decrypting anything.

Deterministic, so upsert-by-id still merges across your own devices. Derived from epoch 1 of the personal graph, so it survives rotation, a newly linked device, and a recovery import. Two members pinning the same record produce unrelated ids. engine.personalId(kind, target) is the only correct way to name such a record.

Anatomy of a record

at is both the record’s timestamp and its revision. That dual role is deliberate — one number, monotonic per device, is all last-write-wins needs — and it carries two rules that are easy to get wrong:

Monotonic per device. Two writes inside one millisecond must not stamp the same number, or the relay drops the second and the device keeps content nobody else has.
Above what it replaces. Editing a record another device wrote from a clock a few milliseconds ahead must stamp above that revision, not just above this device’s own high-water mark. Otherwise the relay drops the edit as a last-write-wins loser, the local store keeps it, and the pull cursor has already passed the relay’s copy — the edit lives on one device forever, with no error anywhere.

The second rule was found by npm run soak, not by thinking about it.

Tombstones are records. The relay never deletes a row; a deletion is a record with deleted: true and an empty data. Anything else would make “deleted on device A, edited on device B” unresolvable.

The sync loop

One graph at a time, coalesced so eight concurrent callers produce one run.

Four details in that diagram carry their weight:

A push is bounded by bytes as well as by op count. 200 records at the maximum ciphertext size is 140MB, and the 413 that follows is not retryable — a queue that stops draining and stays stopped.

A device that is behind an epoch holds its writes. Sealing under the previous key would be readable by whoever the rotation excluded. Unsent is recoverable; sent under the wrong key is not.

Meeting an op sealed under a missing epoch stops the pull before that op, without advancing the cursor, fetches keys, and tries once more in the same sync. Skipping it would lose the record forever.

“Cannot verify” is not “does not verify.” If the roster cannot be fetched — offline, rate-limited — the batch is deferred, not dropped. Dropping would silently lose data that is sitting right there.

Last-write-wins, and the tie

The relay keeps the first of two equal revisions. The device’s comparison is therefore local.at <= record.at on a difference, not <: a strict < would leave B holding its own version forever, with no error and no way back.

The echo optimisation, and its one rule

A device that just pushed a record will meet it again on the next pull. Decrypting and verifying our own writes is pure waste, so the engine remembers (graphId, recordId) → rev and skips an op that matches exactly.

The rule: mark echoes only when EVERY op in the batch landed. rejected lists refusals, not the ops that quietly lost a last-write-wins comparison — and a record that lost is one where the relay’s copy is somebody else’s. Marking that as our own echo would skip the one record we most need to read.

Key ceremonies

Creating a graph key

That read-back is not defensive programming, it is the fix for a real race. A concurrent create is answered 200, not 409 — the relay sees a known epoch and treats the request as a grant, ignoring the duplicate wrap. A device that trusted that 200 would keep a key nobody else has: every record it writes unreadable, every record it reads undecipherable, and no error anywhere.

Linking a second device

No private key ever moves between machines.

grantPending runs automatically during every sync of a device that holds the keys (autoGrant, on by default). That is what makes linking feel automatic rather than like a ceremony somebody has to remember to perform.

Invites: a keyring that rides beside the door code

The server learns the door code and can answer “is this invite valid” — it never sees the half that opens the keyring. Losing the whole string to a network attacker is a real compromise; losing your server’s database is not.

Recovery bundles

The bundle carries graph keys only — never a device keypair. A restored machine is a new device that happens to already know the graph keys. That is what keeps two machines from signing as one, and what keeps revocation meaningful. Importing wipes the local record store first: a store half-filled from a previous identity is worse than an empty one.

Epochs and rotation

An epoch is a generation of a graph’s key. Rotation mints the next one and wraps it to everybody who remains.

Old epochs are never thrown away. A record written last year is still sealed under the key from last year, so a keyring is a map of epoch → key, and a device that lacks an old epoch appears in keygaps until somebody fills it.

A rotation that would strand somebody is refused

The alternative is a graph silently split in half, which is the failure mode nobody notices until a week of messages is unreadable on one laptop.

A rotation reaches the other devices on their next sync

Before the relay reported the epoch, B found out only by meeting a record it could not open — or when the periodic check came round up to a minute later — and kept sealing under exactly the key the rotation was performed to take away. A guarantee that takes effect once somebody notices is not much of one.

The hostile-member case

Anyone holding a graph’s key can rotate it, and the relay cannot check that a wrap actually decrypts — because it cannot decrypt. A malicious member can therefore hand another device a wrap it cannot open. skm makes that recoverable rather than terminal:

Keeping an unopenable wrap would be worse than discarding it: the device would sit one epoch behind forever, holding every write, with the relay insisting it had been served. Cryptographic write control — signed rotations against a policy — is the real answer and is not built. See SECURITY.md.

Storage layout

On the relay

Nothing in that schema can hold plaintext, and a test asserts it — by column name and by content, across every table.

Two design notes:

Usage is counted incrementally. records, bytes and blob_bytes on graph_meta exist because enforcing quotas with COUNT(*) and SUM(size) on every push made a graph of 100k records push about five times slower than an empty one. npm run bench fails if a scan comes back.

graph_seq is a counter, never MAX(seq) + 1. A number reused after a delete would make a puller skip a record.

Migrations are append-only. MIGRATIONS runs under PRAGMA user_version; a migration that has shipped is never edited. The relay refuses to run against a database written by a newer build rather than guessing.

On the device

They are deliberately separate. Wiping records on a recovery import must not take the keys with it, and a native shell wants the keys in the OS keychain while records stay in a database file.

Cursors are keyed by subscription because a device pulling g:acme unfiltered and g:acme filtered to one kind is at two different places in the same sequence. Sharing one cursor would make each subscription skip what the other consumed.

Choosing a store

OPFS is one tab at a time. bestStore falls back to IndexedDB in a second tab, so two tabs of the same app hold two different local stores. They converge through the relay, but neither sees the other’s unsynced writes. Not a confidentiality problem; very much a correctness surprise, and it is in SECURITY.md for that reason.

The relay’s request pipeline

isSafeId is not cosmetic. A member id containing a colon would let two different pairs collide on one d:a:b storage key — an isolation break, not a typo — so ids are validated before anything builds a key from them.

Every inbound frame is schema-validated before any logic touches it. At the relay that means the HTTP body; on the device it means the decrypted record, because the relay cannot validate what it cannot read and any key-holding member can craft anything. Validation lives at the last boundary that can enforce it, not the first one that is convenient.

The route surface

Fourteen routes. None of them parses a record’s contents; four of them handle key material and none of them can open any of it.

Blobs and retention

Large content does not belong in a record — the record limit is 512KB and LWW on a 20MB payload is a bad idea. Blobs are sealed bytes beside sealed records, addressed by a random id a record can point at.

The sweeper runs inside the relay by default (sweepIntervalMs, 60s), because retention policy that only runs when somebody remembers to schedule it is not retention policy. A multi-process deployment wants exactly one sweeper, not one per process — set sweepIntervalMs: 0 and call sweepBlobs() from a single job.

Retention is intent, not enforcement. transport and ephemeral are executed by a relay you may not control; a relay that keeps a copy is invisible to you. This is stated plainly in SECURITY.md and should be stated plainly to your users.

Failure and recovery

Everything here is designed around one assumption: the network is not there, and the app must not care.

failure	what happens
relay unreachable	writes queue; reads are local and unaffected; `status.lastError` is set
relay returns 429 or 5xx	retried with jittered backoff, bounded attempts
relay returns 4xx	not retried — surfaced as `status.rejected` and a `record.rejected` event
device clock is wrong	the relay’s `now` is adopted once, and the monotonic mark is reset with it
a record fails the app’s schema	skipped with `record.rejected {reason: 'schema'}`; the rest of the batch lands
the roster cannot be fetched	the batch is deferred, cursor kept — never dropped
a wrap cannot be opened	discarded, reported, re-granted
the process dies mid-flush	the queue is in the store; the next boot drains it
two tabs, OPFS taken	the second falls back to IndexedDB — see the note above

The clock repair, in detail

Trust boundaries

What the relay learns anyway, stated plainly: who syncs what and when, record ids and sizes, the pairwise graph list, and — only if an app opts in — kinds and shards. What it does not learn: record contents, authorship (signatures live inside the ciphertext), and what you read (read positions live in the personal graph, encrypted).

The honest limits — rollback and omission by a malicious relay, the ghost device, a hostile member’s denial of service, the browser delivery caveat — are in SECURITY.md, which is the document to read before changing anything here.

Extension seams

Every one of these is an interface because the alternative was guessing what your deployment looks like.

Two of them deserve a warning:

MemoryRateLimiter is per process. Correct for one relay, wrong for four behind a load balancer. Back RateLimiter with something shared before you scale out.
FileKeyStore is plaintext on disk. Right for a server-side agent whose disk is already the trust boundary, wrong for a laptop — there the OS keychain is the answer, and KeyStore is an interface so that it can be.

Deployment shapes

Mounted in the app you already have

The common case, and the one skm is designed for. Your sessions, your users, your realtime; skm adds routes under a prefix and a table set.

Standalone relay

Two things change when you scale out: the rate limiter must be shared, and exactly one process should sweep blobs (sweepIntervalMs: 0 everywhere else).

Server-side agent

A worker that needs to read a graph is just another device: register it, grant it, revoke it like any other. It holds real keys, so its disk is a real trust boundary.

How it is tested

The bugs in a system like this are the ones where everything still appears to work, so the suite is built to be adversarial rather than exhaustive.

Three of those are unusual enough to explain:

npm run smoke drives the built packages over a real socket. Tests run against src through a vitest alias; the published packages resolve each other through exports, which point at dist. The smoke test is what closes that gap — if you change an exports map or add a file, it is the one that notices.

npm run mutate breaks each guarded line in turn — accept a bad signature, drop the record id from the AAD, let a rotation strand a device, resolve a revision tie the other way — and fails if no test goes red. A test that has never been red is not evidence of anything. Its first run left four mutations alive, and each one was a real gap in the suite.

npm run soak runs hundreds of randomized convergence scenarios on a virtual clock the seed advances, so a failing seed reproduces exactly. It found the revision-floor bug and the rotation-propagation delay described above. Failing seeds are promoted into the commit suite, so a bug found once is a bug tested forever.

Where to go next

docs/GUIDE.md — how to build on it
PROTOCOL.md — the wire, normatively
SECURITY.md — invariants, threat model, honest limits
CONTRIBUTING.md — how to work on it
ROADMAP.md — what is missing and what it would take