Hallgren
← All notesNote 002 · One minute
A question, answered

Can AI work on confidential material without it leaving the building?

Yes, if the reading is done by a self-hosted local model on hardware you control and only paraphrase travels to a frontier model for judgement. Split the work by kind, not by size, and the cost stays flat too.

Filed underself-hosting

Yes, and the design is the same whether the material is a manuscript under copyright, a client's site, or a household's own records.

Bulk reading goes to a self-hosted local model on my own hardware, a Mac Studio in this case. Chunking, scene maps, passage extraction and first-pass summaries all happen there, and the source text never leaves the machine. What travels to a frontier model, when judgement is worth paying for, is paraphrase: a scene map, a candidate learning, a question and the rows retrieved to answer it. The rule is written down and binding: learnings are paraphrase, evidence may quote a sentence or two with attribution, and unpublished manuscripts are never library sources.

The split is by kind of work rather than by size. Volume is local and cheap; judgement goes to the stronger model, or to me. That is what kept the cost flat as the libraries grew, and it is why the newest library can answer a question with citations while the Wi-Fi is off.

The one operational cost is discipline about the hardware. The daily model and the large ingestion model cannot both be resident on a 32 GB machine, so an ingestion checks every scheduled agent's calendar for a safe window, swaps the model in, proves it with a one-token probe, and always swaps back. An agent that wants to read a book must ask which library it belongs to first.

My own answers, from my own work. They change when the work does.

If this is the conversation you are having at your organisation, I would rather have it with you than write another one of these.

Start a conversation