Why we let AI read your documents but never file them

There is an obvious way to build document auto-filing, and we did not build it.

The obvious way is to hand the whole job to a model: here is a PDF, work out what it is and put it somewhere sensible. It demos beautifully. It is also the version that files a 1099 into the wrong client's 2024 folder in March, confidently, where nobody looks until someone asks why the return is short.

The split

So in LedgerOS a model answers exactly one question about an arriving document: what is this, and how sure am I?

That is the whole job. It does not choose a folder, does not decide which client the document belongs to on its own judgment, and does not get to act on a guess. The folder is then computed from your own filing rules — in code, deterministically, from that answer plus the document's origin.

Folder templates carry the rules: what auto-files into them, what the portal shows, how long things are kept. You set them once per client type. From then on, filing is an outcome rather than a task.

Why the split matters

It matters at exactly one moment: when the model is unsure.

If the model is doing the filing, low confidence still produces a filing decision — just a worse one. There is no natural place for the system to stop, because the thing making the decision is the thing that is uncertain, and its output looks the same either way.

Because placement is separate and fails closed, low confidence produces something different: the document lands in Unfiled, with a count on it, in a queue somebody looks at.

What arrives, and from where

Three paths, one pipeline:

• A desktop agent watches the folders your tax software exports into. Returns and e-file authorizations file themselves without anyone dragging a file. Identity there is the folder path — the agent walks up to the client folder and reads the structure below it against your rules.
• The client portal. Uploads land on the same path as everything else, already attached to the client who sent them.
• Direct upload, including in bulk.

Everything is hashed on the way in, so the same file captured twice does not become two documents. Every replacement is a new version with an audit trail rather than an overwrite.

The same rule, elsewhere

Once you have decided the model never gets the last word, it shows up everywhere.

On a captured signature set, a model names which form is on each page and says how confident it is. It does not place fields, read figures, or decide to send anything.

In tax research, the model drafts an answer from an allow-list of IRS and state sources — and then a second pass goes back out, re-fetches each source it cited, and checks whether that text actually supports the sentence it was attached to. Citations carry the result: verified, unverified, or unfetched.

The honest version

This design costs us something. A more aggressive system files more documents without asking, and in a demo that looks better than a queue with four things in it.

We took the trade anyway, because you are the one who signs the return. The useful question about AI in this profession is not how clever the model is. It is what happens when it is wrong, and whether you can tell.

More on how AI works here, including the things it is not allowed to do.