We changed our minds about realtime, and here is what it took

In July we published a piece explaining why Nex did not have collaborative editing. The argument was that concurrent editing is not a feature you add but a data model you adopt, and that adopting it reaches the document format, undo, version history, every custom block and the whole write path. That argument was correct. We built it anyway, and this is what actually happened.

What changed

Not the cost. The cost was roughly what we said it would be. What changed is that we found the problem we had been avoiding was already costing us more than the fix.

Pages in Nex belong to a workspace, so two people can hold the same one open. Our editor seeded its content once and then owned it — which is what lets it keep a selection and an undo stack while the socket pushes updates underneath. The consequence was that neither person ever saw the other’s writing arrive. They overwrote each other indefinitely, both looking at a page that appeared perfectly correct.

We fixed that first, and not with realtime: a save now carries the version its editor was seeded from, and the server refuses one built on a version somebody else has replaced. That turned silent data loss into a visible, recoverable failure. It was the right first move and it bought the time to do the rest properly.

But “one of you has to reload” is a worse answer than “you can both type”, and once the failure was visible it was obvious how often it happened.

Operational transforms, not CRDTs

The July piece assumed adopting a document format built for merging. That is what we avoided. Nex stores ProseMirror JSON, and it still does — every export, every published page, the search index and the plain-text walker read exactly the field they always read.

What sits on top is an operational transform over ProseMirror steps. A keystroke produces “insert x at position 12” rather than a new copy of the document. Steps carry the version they were based on; if somebody else got there first, the server refuses and the client rebases its step over theirs and retries. That is what lets two people type in the same paragraph without one of them losing a sentence.

The reason this is tractable and a CRDT would not have been: steps are a thing ProseMirror already produces. We are not translating our documents into somebody else’s merge structure. We are sending the editor its own vocabulary.

The part that was actually hard

Not the merging. The hard part was that we had a single moment where a document was saved, and five separate subsystems hung off it: the search text, the backlink index, pruning comments whose block had been deleted, diffing mentions to send notifications, and taking a page-history snapshot.

Realtime abolishes that moment. There is no save; there is a stream of steps and a periodic snapshot. So all five moved to the snapshot, in one function, called from one place. Keeping them together is what stops them drifting apart — the failure mode otherwise is a page that is findable but has no backlinks, or has backlinks and never notified the person it mentions.

That snapshot is debounced deliberately, and the reasoning is worth stating because it is easy to get backwards: snapshots are not what makes an edit durable — steps are. Every keystroke is on the server long before a snapshot is due. A snapshot only saves a later reader from replaying history. So the interval is a cost dial, not a data-safety one, and a slow one loses nothing.

Cursors are a separate problem

The sync layer does not do cursors, and that is the correct separation: text has to be right, whereas a caret is an approximation that is useful even when slightly wrong.

The interesting difficulty is that a remote position is stale the moment it arrives. It was measured against the sender’s document; by the time it lands you may have inserted a paragraph above it, so the offset now points somewhere else.

Rather than version-tracking every caret, we leaned on something ProseMirror already does perfectly. Decorations map through every local change — so a caret is placed once and then carried along by the same machinery that moves the text it sits in, and replaced wholesale when the next update arrives a fraction of a second later. Drift is bounded by the update interval and corrects itself.

Two bugs that only appeared with a real payload

Both are worth recording because neither was visible to a type checker, a linter or a build.

The first: our editor’s own debounced save was still running alongside the sync layer. It would have written the whole document over the top of the step stream on its own schedule — reintroducing exactly the last-write-wins clobbering that sync exists to end, racing inside a single browser tab, with the loser being the version assembled from everybody’s edits. There is now a test that checks the guard exists and that it comes before the write.

The second: cursor traffic is quadratic. Every caret update invalidates the presence query for the room, and every client in the room re-runs it. Ten people typing at four updates a second is forty writes fanning out to ten subscribers — four hundred query executions a second, for decoration. The interval now grows with the size of the room, with a ceiling so a caret never simply stops.

What a published page does

Nothing, deliberately. No sync, no cursors, no presence — showing your team’s cursors to the public would leak who is working on a document and when, and a published page already says nothing about the account behind it.

What it does now do is follow. A published page was receiving every update over its subscription and rendering none of them, because the editor seeds once. That rule protects somebody’s typing, and a reader has no typing to protect — so fix a typo while somebody is reading and they see it, without reloading.

Was the July piece wrong?

About the cost, no. It reached the write path, the derived indexes, the notification path and the published page, exactly as predicted.

About the conclusion, yes — because it weighed the cost of building against the cost of nothing, and the real comparison was against the cost of two people quietly overwriting each other. We had not measured that properly when we wrote it.

We are leaving the original up. A product blog that silently deletes the posts it has grown out of is not one you can trust the current post of either.