Command-line tools for marked-up PDFs and scanned handwritten notes: pdf-scrub, pdf-notes-md, scan-notes-md (Ollama by default, Claude on request), scan-notes-index (per-page search metadata), and scan-notes-stats. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011ZSvbNaLhgFuTm7wmkgbWs
5.1 KiB
5.1 KiB
CLAUDE.md -- pdf-tools
Small command-line utilities for marked-up PDFs and scanned handwritten
notes. Python 3.11+, managed with uv. User-facing documentation is in
README.md.
Layout
src/pdf_tools/
scrub.py pdf-scrub
extract_pdf.py pdf-notes-md
extract_scan.py scan-notes-md: CLI, page rendering, Claude backend
ollama.py scan-notes-md Ollama backend: prompt, host selection
companion.py scan-notes-md companion writer, driven by CompanionSpec
scan_config.py scan-notes-md config file and defaults
scan_stats.py per-page stats log and scan-notes-stats summary
scan_index.py scan-notes-index: per-page metadata files from transcripts
common.py shared helpers
scripts/ shell wrappers (Quick Actions, batch-scan.sh, batch-index.sh)
docs/ findings, e.g. ollama-context-and-gpu.md
tests/ pytest; run with `uv run pytest`
scan-notes-md rules
- No cloud fallback.
--backend ollamamust never call Claude or any other cloud service, andollama.pymust not importanthropic. If no Ollama host is usable, exit with an error (status 3). The tests fail if Claude is called on an Ollama run; keep them that way. - Nothing personal in the code. The package is meant to be published.
Folder paths, host names, organization names, title tables, and name
lists belong in the user's config file, never in
src/or the defaults.scan-notes.example.tomluses placeholder names. - Confidential folders. Anything under a
[[confidential]]root defaults to Ollama, and Claude is refused without--allow-cloud. This check is based on location, not on--companion. Keep the backend decision in Python, not in the shell wrappers. Reject unknown config keys, so a typo cannot silently drop a protection. - No stray copies. Page images are rendered in memory. Nothing is written outside the scan's folder except to the system temp directory, and temp files are removed. Do not log transcript text or images.
- Companion edits are narrow. In an existing companion, change only
the
### Draft transcript (unverified)block and thetranscribed_by:field. Layouts come from[confidential.companion]tables; add new layout features asCompanionSpecoptions, not as named special cases. - One prompt. The Ollama instruction is
HANDWRITING_PROMPTinollama.py.build_promptappends configuredprompt_extratext (top level, then per folder) and the name list. Keep domain guidance, such as the drawing and equation rules, in config, not in the base prompt. The Claude backend keeps its ownTRANSCRIBE_PROMPTinextract_scan.py. - Record what we learn. Context size, GPU memory, speed, and model
failures go in
docs/ollama-context-and-gpu.mdwith a date, model, GPU, and Ollama version. The stats log holds numbers and page labels only, never transcript text. - Looping pages. A
RepetitionErrorpage is retried withREPEAT_RETRIESand then replaced byREPEAT_PLACEHOLDER; first attempts stay at temperature 0. - Batch runs.
scripts/batch-scan.shskips PDFs that have a.md.extract_scancheckpoints after every page to a hidden.<stem>.scan-partial.jsonnext to the scan, keyed on inputs, model, prompt, and dpi, and removes it once the output is written. - Config, not code. Hosts, models, timeouts, folders, layouts, and
per-folder names live in
~/.config/pdf-tools/scan-notes.toml; generic defaults are inscan_config.py. Keepscan-notes.example.tomlloading (a test checks it). Command-line flags override the file. - HTTP is standard library. Ollama calls go through
ollama._http_json(urllib), which the tests replace. Do not add an HTTP or Ollama SDK.
scan-notes-index rules
- Ollama only. It has no Claude backend and uses the same host
selection and no-fallback rule as
--backend ollama. - Generated text is labeled. Summaries and key points are interpretation, unlike the transcript; page files mark them machine-generated and unverified. The prompt tells the model never to add reference details that are not on the page; keep that.
- Prompt and schema.
INDEX_PROMPTandschema()inscan_index.pyare the base; names,prompt_extra, andkindscome from[index]in the config. Answers are constrained with Ollama'sformat(JSON schema) throughollama.chat_json, which retries loops, output cut off at the cap, and invalid JSON. - Resume by file. A page whose file exists is skipped unless
--force. A failed page is reported, the others are written, and the exit status is 4. The stats log gets"task": "index"records with counts only.
Conventions
- Add dependencies with
uv add(oruv add --dev); keep them few. - Status and progress go to stderr; document output goes to stdout or files.
- US English spelling. No emojis in code, output, or documentation.
- Tests must not touch the network or real confidential files: fake
_http_json, usetmp_path, and declare temporary confidential folders in a config file passed throughPDF_TOOLS_SCAN_CONFIG.