Initial commit: RAG pipeline for semantic search over personal journal archive

Vector search with cross-encoder re-ranking, hybrid BM25+vector retrieval,
incremental index updates, and multiple LLM backends (Ollama local, OpenAI API).
This commit is contained in:
Eric 2026-02-20 06:02:28 -05:00
commit e9fc99ddc6
43 changed files with 7349 additions and 0 deletions

View file

@ -0,0 +1,54 @@
Query: --query Passages that quote Louis Menand.
Vector: 20, BM25: 20, overlap: 7, merged: 33, re-ranked to: 15
Response:
The provided excerpts contain several references to Louis Menand's ideas and quotes. Here are the specific passages:
- In file_path: ./data/2025-11-04.txt, you can find a direct quote from Louis Menand: "We created God, and then pretended that God
created us. We hypostatized our own concept and turned it into something “out there” whose commandments (which we made up) we
struggle to understand and obey. We are supplicants to our own fiction."
- In file_path: ./data/2025-02-14.txt, there is a reference to Menand's quote about the human tendency to create and interact with
abstract ideas as if they are tangible realities.
- In file_path: ./data/2022-08-14.txt, another excerpt from Menand's work is mentioned: "We created God, and then we pretended
that God created us."
- In file_path: ./data/2025-07-27.txt, the author discusses Menand's quote about the creation of fictions and the role of
organization in human culture.
- In file_path: ./data/2024-09-06.txt, there is a mention of Menand's writing style, describing him as "witty and serious."
- In file_path: ./data/2025-02-14.txt, the poem takes a wistful tone, hinting at the fragility of human attachment to concepts,
which is reminiscent of existential crises and Menand's ideas on hypostasis.
- In file_path: ./data/2025-07-27.txt, the author further elaborates on Menand's quote, emphasizing the human tendency to create
and interact with fictions.
- In file_path: ./data/2024-08-03.txt, there is a reference to Louis Menand's book "The Free World" and its exploration of art,
literature, and culture in the 20th century.
All these excerpts contribute to understanding Louis Menand's ideas on hypostasis, human creation of fictions, and the complex
relationship between stories, beliefs, and reality.
Files contributing to the answer:
- ./data/2025-11-04.txt
- ./data/2025-02-14.txt
- ./data/2022-08-14.txt
- ./data/2025-07-27.txt
- ./data/2024-09-06.txt
- ./data/2025-02-14.txt
- ./data/2025-07-27.txt
- ./data/2024-08-03.txt
Source documents:
2025-11-04.txt ./data/2025-11-04.txt 3.303
2025-02-14.txt ./data/2025-02-14.txt 3.144
2022-08-14.txt ./data/2022-08-14.txt 2.183
2021-07-04.txt ./data/2021-07-04.txt 1.595
2025-02-14.txt ./data/2025-02-14.txt 1.504
2025-07-27.txt ./data/2025-07-27.txt 0.171
2024-07-26.txt ./data/2024-07-26.txt -0.969
2024-09-06.txt ./data/2024-09-06.txt -1.661
2024-04-08.txt ./data/2024-04-08.txt -2.017
2025-02-05.txt ./data/2025-02-05.txt -2.027
2022-04-17.txt ./data/2022-04-17.txt -2.054
2023-07-01.txt ./data/2023-07-01.txt -2.284
2023-01-06.txt ./data/2023-01-06.txt -2.315
2024-09-04.txt ./data/2024-09-04.txt -2.379
2024-08-03.txt ./data/2024-08-03.txt -2.391
Query processed in 94 seconds.