Building a Document Assistant That Is Allowed to Say "Not Found"

The worst habit of document chatbots is confidence. When the answer isn't in your files, many of them write one anyway, in the same calm tone as a real answer. For HR policies, payroll rules or contracts, a wrong answer is worse than no answer.

Knowledge Assistant is built around the opposite rule. It answers from your own documents, shows the excerpt it used, and when the documents don't cover the question it says so.

The pipeline

Upload/Import -> Parse -> Chunk -> Embed -> pgvector + full-text
              -> hybrid top-K (RRF) -> one LLM call -> answer + citations

Everything lives in one Postgres database: documents, chunks, vectors and chat history. That's one system to run, back up and secure instead of three.

Why vector search alone wasn't enough

Embeddings are good at meaning. Ask "how many days off do I get?" and they'll find the paragraph about annual leave even though none of the words match.

They're weaker at exact things: an employee ID, a contract number, a person's name. That's where plain keyword search is strong.

So retrieval runs both, pgvector similarity and Postgres full-text search, and merges the two ranked lists with Reciprocal Rank Fusion. The textbook version of RRF fits in a few lines:

def rrf(rankings, k=60):
    scores = {}
    for ranking in rankings:
        for rank, chunk_id in enumerate(ranking, start=1):
            scores[chunk_id] = scores.get(chunk_id, 0) + 1 / (k + rank)
    return sorted(scores, key=scores.get, reverse=True)

A chunk that ranks well in either list gets near the top. One that ranks well in both wins. There's no need to normalise two very different scoring systems or tune weights between them.

Tables need their column names

When you import a spreadsheet or a database table, each row becomes text. Store only the values and Alice | 4200 means nothing to a retriever. Keep the column names, name: Alice | salary: 4200, and "What is Alice's salary?" finds the right row.

Citations, and permission not to know

Every answer comes with the sources it was built from. The streaming endpoint sends the sources first, then the answer token by token over SSE, so the user can see what the model is reading while it writes.

The plain API makes the "not found" case explicit. POST /api/query returns answer, found, citations and model. When found is false, the interface says plainly that the documents don't cover it.

Follow-up questions are condensed into standalone search queries first, so "and for part-time staff?" still retrieves the right policy.

Khmer, end to end

A lot of Cambodian business documents are scanned PDFs in Khmer. The Docker image ships Tesseract with English and Khmer language packs, so scanned files are OCR'd on upload. For retrieval, bge-m3 embeddings handle Khmer well and run locally through Ollama.

Fully offline when it matters

With Ollama, no API key is needed and nothing leaves your server:

ollama pull gemma3:4b
ollama pull bge-m3
cp .env.example .env    # set LLM_PROVIDER=ollama and EMBEDDING_MODEL=bge-m3
docker compose up --build

Each model provider (Ollama, Gemini, Claude, OpenAI, or any OpenAI-compatible API) is a small adapter, so switching is one environment variable.

Measure, don't guess

The repo includes an evaluation harness. Each case is a question, the source that should be cited, and text the answer should contain:

cd backend && python -m scripts.evaluate eval/demo.json

Every case is scored on found, source cited and answer contains. The run exits non-zero if any source check fails, so it can block a deploy in CI. Build the eval file from real user questions, and every retrieval or model change gets a before-and-after number instead of a feeling.

Source: hoysengleang/local-model-free-form. The case study is here.