Frieren RAG System

Overview

A retrieval-augmented generation (RAG) system that answers questions about characters from Frieren: Beyond Journey's End, using data parsed from the series wiki. It covers the whole pipeline: loading the data, chunking it, embedding it with OpenAI or Google models, and storing the vectors in memory or in ChromaDB. The chunks it retrieves ground the answer, which a separate LLM writes. You can use it from the command line or through a streaming web API with a simple static frontend.

The pipeline

Ingest · once, at setup

Wiki text51 character files
Chunk1,000 chars, 200 overlap
EmbedOpenAI or Gemini
Vector storein-memory or ChromaDB

Answer · for every question

QuestionCLI or POST /query
Embedsame model as the chunks
Retrievetop 5 from the store
Promptcontext + question
LLMtemperature 0.1
Answerstreamed back

The highlighted steps share one vector store: ingest fills it once, and every question reads from it.

The work splits into two phases.

1. Ingest

Load: 51 plain-text files, one per character, parsed from the series wiki.
Chunk: LangChain's RecursiveCharacterTextSplitter cuts each file into 1,000-character chunks with 200 characters of overlap, so a fact that spans a boundary still appears whole in at least one chunk.
Embed: each chunk becomes a vector, using OpenAI's text-embedding-3-small or a Gemini embedding model.
Store: vectors go into either a small in-memory store or a ChromaDB collection.

2. Answer

Embed the question with the same model used for the chunks.
Retrieve the five most similar chunks. The in-memory store ranks them with a hand-written cosine similarity; ChromaDB runs the search itself.
Build the prompt: the chunks become a "Context" section, followed by the user's question.
Generate: the LLM (gpt-4o-mini by default, or a Gemini model) answers at temperature 0.1. With OpenAI models the answer streams to the client as it is written; with Gemini it is generated in full, then sent down the same stream.

Keeping answers grounded

The prompt tells the model to answer only from the retrieved context and to say "I cannot find the answer in the provided information." when the context doesn't cover the question. An optional "educated guess" mode relaxes this, but the model must then say at the end which parts were inferred. Callers can also ask for the answer in another language.

Swappable parts

Chunk size, overlap, top-K, the LLM, the embedding model and the vector store are all environment variables, so you can compare setups without changing code. The in-memory store needs no setup, which is handy for development. ChromaDB keeps the vectors between runs; docker compose up starts the API alongside a Chroma container with a persistent volume.

Interfaces

CLI for asking questions from the terminal.
HTTP API: POST /query on Express 5 streams the answer as chunked plain text.
Static frontend and API docs served from the same server.