RAG Chatbot Assistant
A document-grounded chatbot API that answers questions from PDF and Word files using retrieval-augmented generation, with per-user conversation memory stored in PostgreSQL.
By Jawher Ennaceur
View on GitHub- RAG
- LLM Applications
- LangChain
- FastAPI
- Vector Databases
- NLP
The Problem
General-purpose LLMs don't know the contents of a private document set and may invent answers. I needed an assistant that answers from the supplied documents, admits when it can't, and remembers each user's earlier questions.
The Approach
An ingestion pipeline loads PDF and Word files (PyMuPDF and Unstructured loaders), splits them into 500-character chunks with 50 characters of overlap, embeds them with a multilingual sentence-transformer model, and stores them in ChromaDB. At query time, a history-aware retriever first rewrites follow-up questions into standalone ones using Gemini 2.0 Flash, then fetches the top 3 chunks. A second prompt instructs the model to answer briefly from those chunks and to say it lacks enough information when the answer isn't there. Each user's message history is persisted in PostgreSQL, and the whole flow is exposed through a FastAPI endpoint.
Results
- End-to-end RAG pipeline: ingestion, retrieval, generation and an HTTP API
- Follow-up questions are rewritten into standalone queries before retrieval
- The prompt constrains answers to the retrieved documents, with an explicit fallback instead of guessing
- Supports PDF and Word documents and multilingual embeddings
- Per-user conversation history persisted in PostgreSQL
Tech Stack
FastAPI
LangChain
ChromaDB
PostgreSQL