Initial commit: RAG pipeline for semantic search over personal journal archive
Vector search with cross-encoder re-ranking, hybrid BM25+vector retrieval, incremental index updates, and multiple LLM backends (Ollama local, OpenAI API).
This commit is contained in:
commit
e9fc99ddc6
43 changed files with 7349 additions and 0 deletions
13
NOTES.md
Normal file
13
NOTES.md
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
|
||||
|
||||
Simple query in ChatGPT produced
|
||||
|
||||
Metric | Best For | Type | Notes
|
||||
-- | -- | -- | --
|
||||
Cosine Similarity | L2-normalized vectors | Similarity | Scale-invariant
|
||||
Dot Product | Transformer embeddings | Similarity | Fast, especially on GPUs
|
||||
Euclidean Distance | Raw vectors with meaningful norms | Distance | Sensitive to scale
|
||||
Jaccard | Sparse binary or set-based data | Similarity | Discrete features
|
||||
Soft Cosine | Sparse with semantic overlap | Similarity | Better for text-term overlap
|
||||
Learned Similarity | Fine-tuned deep models | Varies | Best accuracy, slowest retrieval
|
||||
|
||||
Loading…
Add table
Add a link
Reference in a new issue