pdf-tools/CLAUDE.md
Eric Furst e464871b69 Initial public release of pdf-tools
Command-line tools for marked-up PDFs and scanned handwritten notes:
pdf-scrub, pdf-notes-md, scan-notes-md (Ollama by default, Claude on
request), scan-notes-index (per-page search metadata), and
scan-notes-stats.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011ZSvbNaLhgFuTm7wmkgbWs
2026-11-03 07:25:51 -05:00

5.1 KiB

CLAUDE.md -- pdf-tools

Small command-line utilities for marked-up PDFs and scanned handwritten notes. Python 3.11+, managed with uv. User-facing documentation is in README.md.

Layout

src/pdf_tools/
  scrub.py          pdf-scrub
  extract_pdf.py    pdf-notes-md
  extract_scan.py   scan-notes-md: CLI, page rendering, Claude backend
  ollama.py         scan-notes-md Ollama backend: prompt, host selection
  companion.py      scan-notes-md companion writer, driven by CompanionSpec
  scan_config.py    scan-notes-md config file and defaults
  scan_stats.py     per-page stats log and scan-notes-stats summary
  scan_index.py     scan-notes-index: per-page metadata files from transcripts
  common.py         shared helpers
scripts/            shell wrappers (Quick Actions, batch-scan.sh, batch-index.sh)
docs/               findings, e.g. ollama-context-and-gpu.md
tests/              pytest; run with `uv run pytest`

scan-notes-md rules

  • No cloud fallback. --backend ollama must never call Claude or any other cloud service, and ollama.py must not import anthropic. If no Ollama host is usable, exit with an error (status 3). The tests fail if Claude is called on an Ollama run; keep them that way.
  • Nothing personal in the code. The package is meant to be published. Folder paths, host names, organization names, title tables, and name lists belong in the user's config file, never in src/ or the defaults. scan-notes.example.toml uses placeholder names.
  • Confidential folders. Anything under a [[confidential]] root defaults to Ollama, and Claude is refused without --allow-cloud. This check is based on location, not on --companion. Keep the backend decision in Python, not in the shell wrappers. Reject unknown config keys, so a typo cannot silently drop a protection.
  • No stray copies. Page images are rendered in memory. Nothing is written outside the scan's folder except to the system temp directory, and temp files are removed. Do not log transcript text or images.
  • Companion edits are narrow. In an existing companion, change only the ### Draft transcript (unverified) block and the transcribed_by: field. Layouts come from [confidential.companion] tables; add new layout features as CompanionSpec options, not as named special cases.
  • One prompt. The Ollama instruction is HANDWRITING_PROMPT in ollama.py. build_prompt appends configured prompt_extra text (top level, then per folder) and the name list. Keep domain guidance, such as the drawing and equation rules, in config, not in the base prompt. The Claude backend keeps its own TRANSCRIBE_PROMPT in extract_scan.py.
  • Record what we learn. Context size, GPU memory, speed, and model failures go in docs/ollama-context-and-gpu.md with a date, model, GPU, and Ollama version. The stats log holds numbers and page labels only, never transcript text.
  • Looping pages. A RepetitionError page is retried with REPEAT_RETRIES and then replaced by REPEAT_PLACEHOLDER; first attempts stay at temperature 0.
  • Batch runs. scripts/batch-scan.sh skips PDFs that have a .md. extract_scan checkpoints after every page to a hidden .<stem>.scan-partial.json next to the scan, keyed on inputs, model, prompt, and dpi, and removes it once the output is written.
  • Config, not code. Hosts, models, timeouts, folders, layouts, and per-folder names live in ~/.config/pdf-tools/scan-notes.toml; generic defaults are in scan_config.py. Keep scan-notes.example.toml loading (a test checks it). Command-line flags override the file.
  • HTTP is standard library. Ollama calls go through ollama._http_json (urllib), which the tests replace. Do not add an HTTP or Ollama SDK.

scan-notes-index rules

  • Ollama only. It has no Claude backend and uses the same host selection and no-fallback rule as --backend ollama.
  • Generated text is labeled. Summaries and key points are interpretation, unlike the transcript; page files mark them machine-generated and unverified. The prompt tells the model never to add reference details that are not on the page; keep that.
  • Prompt and schema. INDEX_PROMPT and schema() in scan_index.py are the base; names, prompt_extra, and kinds come from [index] in the config. Answers are constrained with Ollama's format (JSON schema) through ollama.chat_json, which retries loops, output cut off at the cap, and invalid JSON.
  • Resume by file. A page whose file exists is skipped unless --force. A failed page is reported, the others are written, and the exit status is 4. The stats log gets "task": "index" records with counts only.

Conventions

  • Add dependencies with uv add (or uv add --dev); keep them few.
  • Status and progress go to stderr; document output goes to stdout or files.
  • US English spelling. No emojis in code, output, or documentation.
  • Tests must not touch the network or real confidential files: fake _http_json, use tmp_path, and declare temporary confidential folders in a config file passed through PDF_TOOLS_SCAN_CONFIG.