Reproduce · inspect · adapt
Your corpus.
Your environment.
An accountable workflow.
The website makes the method visible. The Python package is where researchers and their agents run, inspect and adapt an analysis. Begin with the synthetic acceptance check, then make each research decision explicit.
01 / A check that costs nothing
Start with an inspectable result.
Use Python 3.11 or newer; the recorded full ML run uses Python 3.12 on Windows. In your fork or extracted source directory, create a virtual environment and install the package. These commands install the analysis core; add [ml] for discovery.
python -m venv .venv
# Activate .venv using your shell's activation command.
python -m pip install ".[dev]"
topicllm --profile quickcheck --output-dir runs/quickcheckThe acceptance check uses generator truth and real inference code. Expect 5,872 classified rows, five prevalence rows, 15 comparisons and all 15 rounded percentages matching the reported table. It makes no API calls and does not evaluate a classifier.
02 / Give your agent a contract
Fork first. Preserve the evidence.
Open the fork in your preferred coding agent and provide this task. The repository’s AGENTS.md, machine-readable protocol and methods define the sequence, artifacts and review boundaries.
Read AGENTS.md, REPRODUCE.md, docs/protocol.json and the methods.
Run the offline synthetic quickcheck before changing anything.
Verify protected source checksums and record the exact commit and environment.
Prepare a plan for my corpus, including exclusions, model settings and API cost.
Run only within my approved budget. Retain raw responses, usage and checkpoints.
Assemble the Stage 3 review packet and stop for actual researcher decisions.
Do not invent human labels, reviewer identities or historical model results.
Validate on an independent benchmark, report every theme's performance,
and export the complete audit record before making scientific claims.03 / Exercise the complete synthetic pipeline
Run every stage, with its provenance intact.
python -m pip install ".[ml,dev]"
topicllm --profile synthetic-e2e --votes 20 --budget-usd 5 --max-api-calls 7000 --workers 6 --output-dir runs/synthetic-liveSet OPENAI_API_KEY privately in your environment first. The full synthetic profile runs the configured 50-draw discovery search, naming, generator-based test mapping, the 250-row benchmark, the winning classifier across the corpus, and inference. Twenty naming votes are a disclosed validation budget; the historical protocol used 5,000. Generated mapping is always marked human_review_performed: false.
Before paid execution, add --plan to inspect planned calls. Resume in the same output directory only with unchanged inputs and settings. Successfully recorded calls are cached; uncertain attempts retain their budget reservation. See the validation record for measured results and limitations.
04 / Move to a real research question
Keep the scientific decisions with researchers.
Use a documented corpus with a nonempty tweet column, stable IDs and ISO dates for period comparisons. Supply your own independently annotated benchmark. Run discovery and naming, then review the sampled documents, proposed themes and ambiguous votes. Record the actual reviewers and bind approval to the final theme map.
The pipeline rejects stale review records, synthetic benchmark reuse on research data, unresolved classifications and changed checkpoint inputs. Technical gate passage is a minimum condition: researchers must judge class support, precision, sensitivity and the consequences of errors for their question.
New research requires a declared sampling frame, eligibility rules, duplicate policy, preprocessing, interpretation protocol, validation criteria and analysis plan. Read the full reproduction guide before scaling.
What this release establishes
The dissertation and archival source papers remain unchanged. The guided website reconstruction matches the reported rounded prevalence by construction. New synthetic discovery and live predictions are measured and reported separately. Original private records, original ballots and the historical fitted model are unavailable. LIWC sentiment analysis and Study 2 structural equation modeling are outside this five-stage package.
Explore the visual demonstration · Inspect source discrepancies · Website data handling and limits