Situation:
Company knowledge often sits in documents nobody has time to search. I built a RAG (retrieval-augmented generation) application that answers questions from such documents, using either a local or a cloud language model. The answers had to stay grounded in the documents, and the AI provider had to be replaceable.
Task:
As architect and developer, I designed the full pipeline on .NET 10. The goal was reliable answers only from the documents, an honest "I don't know" when the documents have no answer, and freedom to switch AI providers.
Action:
- Designed two flows in C# 14: indexing (read, chunk, sanitize, embed) and asking (embed, retrieve, check, prompt, answer). Every stage is its own single-responsibility class.
- Built structure-aware chunking that joins Markdown headings with their paragraphs, plus a sanitizer that removes noise before embedding.
- Implemented retrieval with cosine similarity, a TopK limit and a MinScore threshold.
- Stored the embeddings in PostgreSQL with pgvector, behind the same search contract as the in-memory search.
- Added a guardrail: if no chunk passes the threshold, the app answers "I don't know" without calling the LLM.
- Built grounded prompts with numbered sources, so the agent (Microsoft Agent Framework) answers only from the retrieved context and can cite it.
- Programmed against Microsoft.Extensions.AI abstractions, so local Ollama (qwen3:4b, embeddinggemma) and Google Gemini (gemini-2.5-flash, gemini-embedding-001) run behind one configuration setting. API keys stay outside code and settings records.
- Calibrated the MinScore threshold separately for each embedding model, using in-scope and out-of-scope test questions.
Result:
- Delivered a working RAG application that answers from private documents and switches between local and cloud models with one setting.
- Kept data in-house in local mode: Ollama runs without any external key, so documents stay on local hardware.
- Rejected unrelated questions before any LLM call, which saves cost and response time.
- Tuned the threshold for each embedding model, so relevant questions pass and unrelated ones are rejected.
- Moved storage to PostgreSQL with pgvector by changing only the indexing and retrieval classes.
Planned:
Expose the pipeline as a REST API with FastEndpoints, so other systems can send a question and receive an answer with its sources.
Technologies Used:
.NET 10, C# 14, Microsoft Agent Framework, Microsoft.Extensions.AI, Ollama, OllamaSharp, Google Gemini (OpenAI-compatible API), PostgreSQL, pgvector, RAG, Vector Embeddings, Cosine Similarity, System.Numerics.Tensors, Microsoft.Extensions.Configuration
This project is in progress and changes regularly. Features and details may differ from this description.
Other references


