Every retrieval prototype works on the first day. You point it at a folder, ask a question it was always going to answer, and the response is fluent and correct. What it does not tell you is what happens six months in, when the folder holds four thousand documents of nine different kinds, half of them superseded, and nobody can say which.

That is a corpus problem, not a model problem, and swapping in a better model does not touch it. So on client projects we kept doing the same thing by hand: giving the knowledge a shape before indexing any of it. okflib is that practice turned into a library.

The Open Knowledge Format

A unit of knowledge is a markdown bundle with YAML frontmatter. Markdown because it survives everything. It diffs in git, a human can read it in a terminal, and a language model ingests it without a parsing step that can go wrong. Frontmatter because the things you need in order to use a document are not in its prose.

The frontmatter is where the useful questions get answered. Where did this come from. When was it true. Who vouches for it. What supersedes it. These are the trust signals, and they are the difference between a corpus and a folder.

Why trust signals earn their place

A retrieval system that treats every chunk as equally authoritative will confidently cite a draft from two years ago, because the draft happens to be the nearest match in vector space. Relevance is not the same as authority, and embeddings only measure the first one.

Carrying the second one explicitly gives the pipeline something to rank with, something to filter on, and something to show the reader. When a system says "this came from the signed version of the policy, dated March, superseding the one you are thinking of", the answer is no more fluent; it is simply usable.

Ingestion built for the thing that reads it

The last piece is that bundles are meant to be read by a model. okflib stores, queries and generates them, so the same format serves the whole lifecycle: what a human writes, what a pipeline indexes, and what a model gets handed at inference time. One shape, rather than a conversion at each step, and conversions are where provenance quietly goes missing.

It is deliberately small. It came out of a problem we had to solve for ourselves, and it does that one job.

okflib on GitHub