All writing
5 min readEngineeringSearch

Search that reads the page, not just the title

Half of search should run on your machine and half on the server. Splitting it that way is what makes both halves fast.

Press ⌘K in Nex and two different kinds of search run. Titles are matched in your browser. The words inside your documents are matched on the server. Neither half would work well doing the other’s job.

Titles: on your machine, instantly

The sidebar already subscribes to the whole page tree — it has to, to draw itself — so every page title and its ancestor path are sitting in memory already. Matching them is free and needs no round trip. Typing feels instant because nothing is being asked of anything.

This is where most navigation happens. You know the page exists and you know roughly what it is called; you are not searching, you are jumping. Making that wait on a network request in order to be architecturally uniform would be a bad trade every single time.

Bodies: on the server, because they are not here

Document bodies are deliberately not in the browser. The tree the sidebar subscribes to carries titles and icons and nothing else, because anything on those rows is re-sent to every connected client whenever any of it changes. With bodies inline, one keystroke would push a copy of every other page down the socket to redraw a list of titles.

So body search is a server query, and it runs against a proper full-text index rather than scanning. Matching in a query means loading every document in the account on every keystroke — tolerable at fifty pages, impossible at five hundred.

The index cannot read the document

Documents are stored as structured JSON, and you cannot point a text index at that. It would tokenise the node names, so every page in the account would match “paragraph” and a page about headings would be indistinguishable from a page that contains one.

Nex maintains a plain-text copy of each body, written on the one hook where a body changes. Three details in that extraction are worth knowing, because each is a page that would otherwise be unfindable:

  • Mentions and page links carry their words in an attribute rather than a text node. Skip them and a page is findable by every word in it except the names of the people and pages it points at.
  • Page properties live in a block’s attributes too, so a page whose only mention of a name is its Owner field is still found by that name.
  • An unparseable body yields empty text, never an error. Every caller runs inside a write that must not fail because one row is shaped oddly.

One walker does all three, shared with the excerpt in a notification and the word count beside a version. Three copies of “what counts as text” is three chances for a page to be findable in one place and invisible in another.

Results say why they matched

A body result shows a snippet centred on the first word of your query that appears in the page, not the page’s opening sentence. Every page opens with something; eight results all showing their first line tell you nothing about which one you meant.

Archived pages never surface. The trash is somewhere things go to be forgotten, and a search that returns binned pages makes it a filing system nobody can leave.