furst/AdminAnalytics
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Update CLAUDE.md status, structure, and run commands to reflect built project

Browse source 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X9jG4CUFcH6n4vFvgtwndY
This commit is contained in:
emfurst 2026-10-18 09:48:20 -04:00
commit 2cd7aa1c9d
1 changed files with 22 additions and 12 deletions
Download patch file Download diff file Expand all files Collapse all files

34
CLAUDE.md
View file

@ -4,14 +4,15 @@ University administrative cost benchmarking project using public data (IRS 990,
## Project status ## Project status
Currently in planning. Phase 1 (Data Acquisition) is planned but not yet built. See `phase1_plan.md` for the full implementation plan and `administrative_analytics_scope_v0.1.md` for project scope. Phase 1 (data acquisition) and the dashboard are built and deployed at adminalytics.org. The CLI ingests IPEDS, IRS 990, BLS CPI-U, EADA athletics, and a staff-directory scrape into DuckDB; the Dash dashboard has tabs for overview, compensation, staffing, current headcount, endowment, philanthropy, athletics, and about. Next planned iteration: peer/AAU institution comparisons.
## Architecture ## Architecture
- **Language:** Python - **Language:** Python
- **Database:** DuckDB for Phase 1 (single-file, zero-config). Migrate to PostgreSQL in Phase 3 when the dashboard needs concurrent access. - **Database:** DuckDB (single-file, zero-config). The dashboard opens it read-only; migrating to PostgreSQL is an option if concurrent writes become necessary.
- **Package manager:** uv - **Package manager:** uv
- **CLI framework:** typer or click (TBD) - **CLI framework:** typer
- **Dashboard:** Dash + Plotly, served by gunicorn in production
- **Testing:** pytest - **Testing:** pytest
## Data sources ## Data sources
@ -22,7 +23,7 @@ Currently in planning. Phase 1 (Data Acquisition) is planned but not yet built.
| IPEDS | CSV bulk downloads | Institution directory (HD), finance by function (F1A/F2), staffing (S/SAL), enrollment (EF) | | IPEDS | CSV bulk downloads | Institution directory (HD), finance by function (F1A/F2), staffing (S/SAL), enrollment (EF) |
| EADA (Equity in Athletics) | Excel via ope.ed.gov/athletics API | Athletics expenses by sport, coaching salaries, participants (keyed by UNITID, FY2007+) | | EADA (Equity in Athletics) | Excel via ope.ed.gov/athletics API | Athletics expenses by sport, coaching salaries, participants (keyed by UNITID, FY2007+) |
| BLS CPI-U | Flat file or API | Consumer Price Index for inflation-adjusted compensation analysis | | BLS CPI-U | Flat file or API | Consumer Price Index for inflation-adjusted compensation analysis |
| Admin office web pages (stretch) | HTML scraping | Staff directory headcounts | | Admin office web pages | HTML scraping | Staff directory headcounts |
## Key concepts ## Key concepts
@ -33,7 +34,7 @@ Currently in planning. Phase 1 (Data Acquisition) is planned but not yet built.
- IRS 990 XML schemas change across tax years. Use the Master Concordance File or IRSx library to handle XPath variations. - IRS 990 XML schemas change across tax years. Use the Master Concordance File or IRSx library to handle XPath variations.
- IPEDS variable names change across years. Always parse the accompanying data dictionary; never hardcode variable names. - IPEDS variable names change across years. Always parse the accompanying data dictionary; never hardcode variable names.
## Planned project structure ## Project structure
``` ```
src/admin_analytics/ src/admin_analytics/
@ -43,7 +44,10 @@ src/admin_analytics/
irs990/ # 990 download, XML parsing, Schedule J extraction, university filtering irs990/ # 990 download, XML parsing, Schedule J extraction, university filtering
ipeds/ # IPEDS download, dictionary parsing, finance/HR/enrollment loading ipeds/ # IPEDS download, dictionary parsing, finance/HR/enrollment loading
bls/ # CPI-U fetcher and loader bls/ # CPI-U fetcher and loader
scraper/ # Stretch: admin office headcount scraper eada/ # EADA athletics download and loader
scraper/ # Admin office staff-directory headcount scraper
dashboard/ # Dash app (app.py), queries, components, pages/, assets/, wsgi.py
validation.py # Data validation checks
data/raw/ # Downloaded files (gitignored) data/raw/ # Downloaded files (gitignored)
tests/ tests/
fixtures/ # Sample XML/CSV files for tests fixtures/ # Sample XML/CSV files for tests
@ -51,15 +55,21 @@ tests/
## Build & run ## Build & run
Not yet implemented. When built, the CLI will support:
``` ```
admin-analytics ingest ipeds --year-range 2005-2024 uv run admin-analytics ingest ipeds --year-range 2005-2024
admin-analytics ingest irs990 --year-range 2005-2024 uv run admin-analytics ingest irs990 --year-range 2019-2025
admin-analytics ingest cpi uv run admin-analytics ingest cpi
admin-analytics ingest eada --year-range 2007-2025 uv run admin-analytics ingest eada --year-range 2007-2025
admin-analytics ingest all uv run admin-analytics ingest scrape
uv run admin-analytics ingest all
uv run admin-analytics validate
uv run admin-analytics dashboard # dev server on 127.0.0.1:8050
uv run gunicorn -w 2 -b 127.0.0.1:8050 admin_analytics.dashboard.wsgi:server
uv run pytest
``` ```
Restart the dashboard after ingesting; it holds read-only DuckDB connections.
## Conventions ## Conventions
- Raw data tables are prefixed with `raw_` (e.g., `raw_institution`, `raw_990_schedule_j`) - Raw data tables are prefixed with `raw_` (e.g., `raw_institution`, `raw_990_schedule_j`)