Skip to content

doe: a build log#

doe is the study and exam platform I run on my homelab. It started on 12 July 2026 as "FALKEN", a small Flask app for sitting a weekly exam in a browser (ff18acc). Eleven weeks and 243 commits later it holds lessons, a versioned question bank, six question types, AI and deterministic grading, spaced repetition and a daily Python drill, and it serves two learners: me, working towards an Open University computing degree, and a friend who is a nursing student (f22161e, [2026-08-16-doe-project-history.md]). A lot of it was written by Claude, Anthropic's model, working under my direction: 110 of the 243 commits carry a Co-Authored-By line naming a Claude model, the first of them f93c7e5 on 7 August. This log describes the records as they stood on 27 September 2026; nobody checked the running system while writing it.
First commit ff18acc, 2026-07-12, "initial webapp scaffold — gunicorn, schema, GET /exam/"
Latest commit 92f9a0d, 2026-09-27, "drill 38: ten rung-4 labs, sitting 176 on doe_prod"
Commits 243 (23 in July, 112 in August, 108 in September)
Earlier names FALKEN; the repo falken-exam-webapp, merged with a second repo exam-webapp on 3 August (e971a4f); rebranded "WOPR : Department of Education" (5d56fcf)
Stack Python, Flask blueprints under gunicorn (f8f73a0); PostgreSQL [2026-08-16-doe-project-history.md]; Anthropic API for short-answer grading (6624db9) and paper generation (34bff3a, hardened in bd58fc6); Chart.js 4.4.3 served locally (bdf7748); Pyodide 0.26.4 for Python in the browser (f30001a); pypdf for PDF upload (20b5325)
Where it runs Production on CT120 (doe_prod, doe.service); development on CT136 (doe_dev, doe-dev.service) [REBUILD-doe.md]; published behind a login through a Pangolin/newt tunnel [2026-08-16-doe-project-history.md]
Who wrote it Me, directing. Claude is co-author on 110 commits: Opus 5 on 97, Opus 5 "1M context" on 2, Fable 5.1 on 5, Opus 4.8 on 3, Opus 5.5 on 3. 80 commits carry a Claude Code session link, from 8e47afe on 8 September

What I set out to do#

Before there was any code, there was a regime. On 27 June I sat a twenty-question "Week 1 Diagnostic" written as one markdown file and graded in a single pass. By 4 July the second week landed within four points of the first, so the measurement was worth keeping. On 10 July I wrote the process down before the software. The problems the manual regime had shown were concrete. The marking scheme must never reach the examinee before grading. Papers pile up faster than they can be found. Results only help if they compare across weeks [2026-08-16-doe-project-history.md].

So the first goal was narrow: sit the week's exam in a browser, from a database of concepts and sealed answers, without dropping to a terminal. The first four commits did that (ff18acc, bb9e312, 4d5cf79).

The goal then changed several times, and the changes are the spine of this log.

  • From exam tool to study platform, late July. Progress tracking, AI grading and a login arrived together (6624db9, 34bff3a). Within a day the app had a second user and every query was filtered by owner (fcd0530, 0b0d371); "tenancy is live and the nursing student has an account" (5d82c13).
  • Retired, then not, 2 and 3 August. On 2 August the project was retired because it had failed its own test, content to tests to grades, and its concepts had been harvested from chat logs rather than from any syllabus. The next day that was amended rather than withdrawn: what had failed was which topics got examined, not the idea [2026-08-03-authority-seed-phoenix-exam-app-course-correction.md].
  • From testing to teaching, 6 and 7 August. Two failures on one day showed that doe could not tell "never taught" from "forgotten" [2026-08-06-note-doe-honest-scope.md]. The answer was a rule in code: the generator refuses material that has not been taught (e2728f7), and a small learn mode records when material was presented (20061ba).
  • The sourcing rule, August. The nursing lesson contract was amended so that the course fixes scope and a cited public authority may supply the grounding (19b6f57, c6bb386). Later documents cite this as the standing rule: syllabus sets scope, trusted authority supplies grounding, the student is never the source of supply [2026-09-05-proposal-external-syllabus-as-exam-source.md].
  • Built for the second learner, mid-August. She needed to turn her own lecture material into lessons and a paper without help. That became one upload button (af46778), the first purpose on the platform to close [2026-08-19-progress-report-doe-and-completion.md].
  • From papers to cards, late August. A quick test of cards with hints and explanations (f0e5e4c), then a daily drill with spaced repetition (dc1e675).
  • "doe has no user", 5 September. A finding that day said both of us had been choosing to sit questions inline in chat rather than in doe, and a proposal built on it opens "doe failed for want of scope" [2026-09-05-proposal-external-syllabus-as-exam-source.md]. The same weekend a sixth question type, lab, ran my Python drills in the browser, and the drill venue moved to production on 6 September [DECISIONS-doe.md, D12]. The finding stands as filed, though the commits after it show the heaviest use the platform has had.

Timeline#

Twelve stages, cut where the log itself changes direction. Commit counts are inclusive. Failures are told in full once, under What went wrong; the timeline points to them.

Stage 1: First light (12 to 13 July; ff18acc to 4d5cf79, 4 commits)#

The scaffold was gunicorn, a schema and GET /exam/<week> (ff18acc). The same evening added a questions table, a concept_questions junction and a certification column, taking the corpus to 66 concepts across Fundamentals, CCNA, CISSP, Kali and a CompTIA warmup (bb9e312), plus a migrations/ directory (5ea552a). Next morning the exam route read from the database, ordered "deficit-first", and a week-4 bank of 19 questions went in (4d5cf79).

Then the log goes quiet: no commits between 13 and 27 July. The hand-written history says the week-4 exam was sat on the platform on 18 July and that on 24 July the scope widened to a dashboard, progress tracking and flashcards [2026-08-16-doe-project-history.md]. The next commit shows why git was silent: the running container had moved ahead of the repository.

Stage 2: Catching the repo up, and making it safe for two (27 July to 2 August; 6624db9 to 4332cb1, 22 commits)#

6624db9 "brings the repo up to the state running on CT120, which was ahead of the working copy": AI short-answer grading, a progress page, and templates never tracked before. It also took a hardcoded database password out of the code (see What went wrong). The next day 34bff3a did the same for session auth, an admin console and paper generation, noting that production "was the only complete copy of the app" and checking all ten files by md5 first.

Tenancy came in two steps on 28 July. fcd0530 added owner_id to five tables, saying outright that queries were not yet filtered. Fifteen minutes later 0b0d371 filtered every non-admin query, made owner_id a required argument in the generator so that forgetting it raises, and listed its own gaps, among them a concept primary key that could not hold two owners' copies of one concept. Scripts moved onto a surrogate key, concept_pk, on 31 July (10ed54d); the SQL that created it "was run from /tmp and was not kept", so a full schema dump became the new baseline (3101fc8).

Small fixes followed from use (a44d5a3, 5d82c13, 236c9c1). Four routes that look backwards (confirm, results-detail, grading-bundle, result) resolved to the next paper due rather than the one sat; the commit called its fix "scaffolding, not architecture" (0fe69b3). A disposable week-90 sitting became the harness for anything touching a live paper (44ac0a5), and building it exposed the grader injection defect (0d0eda2, dd8f7c0; see What went wrong).

Generation grew up too. can_generate separated generating from admin rights (f0e8613). An allocation gate stopped a paper asking for more questions than its bank supports, after a run the structural check failed "after both models had been paid" (bd58fc6). The guess flag became its own column instead of a [GUESS] prefix on the stored answer (17155df), and the multiple-choice key was chosen in Python rather than by the model (23647ea). 4332cb1 stopped a single verifier flag from blocking a whole paper, because "this is a two-person CCNA and homelab study aid, not a certifying instrument".

Stage 3: Retirement, refactor and one repo (2 to 3 August; 4b26b96 to 44b6978, 6 commits)#

On 2 August I retired the project. The retirement document is not among these sources, but the seed that amended it quotes its reasons: FALKEN had failed its criterion, content to tests to grades, and its concepts came from chat logs, one of them "rhetorical shorthand harvested as an examinable topic" [2026-08-03-authority-seed-phoenix-exam-app-course-correction.md].

The code kept moving. A second repository, exam-webapp, was baselined from the container (4b26b96), the 989-line app.py was split into eight modules with the route map verified identical (f8f73a0), a twelve-check smoke test was added (106064c), and corpus upload shipped with an end-to-end proof as user 2 (6bf7ac3).

On 3 August the seed "PHOENIX" separated two things the retirement had run together. Concept provenance, which topics get examined, had failed. Content provenance, the text questions are generated from, had never been attempted for networking and security, because those fetchers did not exist. "Po named this. The model had asserted the opposite and was wrong." The seed set the rules: concepts come only from a published syllabus, material from primary sources per domain, and "no new webapp features until a paper reaches a human" [2026-08-03-authority-seed-phoenix-exam-app-course-correction.md].

The same afternoon e971a4f merged the two repositories. exam-webapp had the newer code but no migrations; falken-exam-webapp had all nineteen SQL files but the old monolith. The merge chose the older repo as truth because "application code is recoverable; migrations are not". Questions started carrying a source_url from the corpus loader, with the migration written after the column had been applied by hand on CT120 that morning (44b6978).

The seed names the other learner's units as her course's unit codes as first recorded [2026-08-03-authority-seed-phoenix-exam-app-course-correction.md]; later commits use her course's unit codes as later commits give them (f22161e, e77928d), and the commits are the better record.

Stage 4: Teach, then test (6 to 10 August; dffdda1 to bd5d3cf, 30 commits)#

By 6 August the app is doe. The week counter left sitting labels, because "the counter encoded one paper, one sitter, one week; all three assumptions have broken", and a void status arrived (dffdda1), which turned out the next day to be written and never read (105262e). Preparing the nursing student's first paper exposed a learner hardcoded as a computing student and, on a fresh corpus, concept selection in alphabetical order (f22161e).

On 6 August a scoping note had named the gap under several failures: doe tests, and never issues, so it could not tell forgotten from never taught [2026-08-06-note-doe-honest-scope.md]. Its smallest answer landed on 7 August as require_taught in the generator (e2728f7) and a /learn page that presents a lesson and records the exposure in the same act (20061ba). The first agent pipeline arrived the same day: a Sonnet lesson-writer and a read-only Opus lesson-QA as Claude Code project agents, with tools listed explicitly "because omitting the field grants all tools, not none" (f952bc1). Its first lesson went through two rejections before approval (f93c7e5).

On 8 August every route was re-keyed on sitting_id (bab084a). A nursing lesson pipeline arrived with a stricter gate, "a clinically true fact she was never taught is a BLOCKER" (19b6f57, c6bb386), and networking lessons moved to primary standards cited by section (1543b43).

9 August was a volume day. The Ethical Hacker scope entered from Cisco NetAcad's published Scope and Sequence as 38 topic concepts, with the course supplying scope and open authorities supplying claims (23e8132); a minute later a correction recorded that the PDF was stale, not sparse, and that topic 6.10 does exist (95a266d). Then came 25 Ethical Hacking lessons (afa5313), 36 Linux, Python and git lessons (b03f36f), and 35 across Kubernetes, Docker and three clouds, after triage rejected 38 of 73 source documents (072a6fa). The same day a review agent caught an answer-key ordering defect (1cd2dea) and a provenance column split evidence from assertion (9498b18).

Stage 5: A surface the second learner could use (12 to 17 August; 05b5db4 to 4e5538a, 27 commits)#

The nursing student could generate papers but had no visible way to supply material, "no error, no 403 and no log line" (4ecd6e4). /corpus and /generate became one page, /material, in three numbered steps and plain words, "because the person using this page is sitting nursing exams, not operating a corpus" (af89d3c). PDF upload followed, reporting pages with no text as probable diagrams (20b5325).

The shared wopr.css 2.1.0 and self-hosted fonts arrived, with a lint measuring 40 errors and no template changed (40a22be, 7db5652). On 14 August a day of image and PDF figure work was found patched straight into the container (66f4c5a; see What went wrong). On 15 August submitted papers became readable at /papers, a read-only blueprint with the answer key absent by construction (9b743ec), which shipped with no CSS until a missing CSP nonce was found (7ee4866).

The night of 15 to 16 August replaced the material page with one button. /upload takes a PDF or pasted text, infers the subject, writes concepts, lessons and checks, and makes a paper. It added an authored provenance class, "a real weakening, taken knowingly" of the teach-then-test gate (af46778). She ran it on her own study material that night, and display defects she found by looking were fixed within the hour (bf3896b, 518420d). Light mode (2756822) and the rebrand from FALKEN (5d56fcf) followed.

On 16 August the agent pipeline was generalised by subject, two truncated uploads were split and re-ingested with a byte-for-byte proof that the parts rejoin, and a question-writer and question-QA pair was added in which the writer may not choose an answer letter at all (976ce9c). Its first run banked fifteen approved items (5684d7b). 17 August was reconciliation (a127265, ce246c7, ff0d73e, 4e5538a). In among it, the separate "confirm" step after grading was deleted because recording moved into submit (1091c83), which I had called redundant ceremony the day before [2026-08-16-delegable-doe-ui-polish.md].

Stage 6: Question types, the quick test and a daily drill (19 to 26 August; 7515bee to 4a05686, 26 commits)#

7515bee landed five question types (multiple choice, short answer, cloze, match, ordering), deterministic partial credit for the three objective types, and an attested provenance so a person holding a document can state it was taught. The five types then reached no sitter until I asked (89d6a47).

On 21 August the quick test arrived: a card sitting drawn from the bank so it opens instantly, marked one card at a time, with a hint and an explanation, and the key released only after the answer is written (f0e5e4c). 5a40fea supplied hints and explanations and created the first development environment, after a day in which "a migration, a schema change and a model-driven backfill all ran against the live database with the nursing student's material in it, because there was no alternative". It used synthetic data, because a dev database seeded from a prod dump "would put the nursing student's PII in a second place while claiming to be safe" (5a40fea, 5fbef4f). The bank's options field held two shapes, so the quick test returned zero questions for the nursing student; it was migrated and constrained, and code became an attachment any question can carry (7af2ec9, 66c94f9).

On 22 August the R1 remix, "sit a paper again", shuffled multiple-choice options and refused the shuffle when a rationale contained a bare letter A to D. It also fixed the register's defect A3: quick_new had been dropping concept links, so every quick sitting was invisible to per-concept performance (898372f, ac8490b). The same night brought a schema that builds doe from empty, a demo seed and a requirements.txt captured from production's venv (9036427, 194075e, 10a01b0); the four-environment runbook's stated reason for a demo tier was a sellable artefact [2026-08-22-runbook-doe-four-environments.md].

The daily drill landed on 22 August: topic chips, size presets, one daily per owner, and write-back to each concept's next_due, status and streak (dc1e675, c8b19ad). It shipped with a dead "Everything" button (45349c4) and with timeouts counting as wrong answers (ac8a80c), both under What went wrong.

On 23 August one shared API helper and resume-on-reload arrived (7e5f09a). My drill 16 re-sit became the first drill loaded into the bank (2707e1c), and 4a05686 is the first commit to name the database doe_prod.

The rename from exam_wopr happened on 24 August, and a defect register opened the same day because eight known defects "lived in chat, which is not a store". Four were found by sitting drill 17, among them B1, a link that opened a quick sitting in the paper renderer, and B3, a code block that rendered as a fifth option; the housekeeping rename turned up six more [2026-08-24-register-doe-open-defects.md]. B1 was fixed on 30 August (bc3a41a).

Stage 7: Two axes, every type on a card, and the first fan-out (27 to 30 August; 5324632 to ab4b3d6, 17 commits)#

The maintenance window on 27 August gave the repo an AGENTS.md. Until then every agent working in ~/repos/doe had loaded no rules at all (5324632, [AGENTS.md]). On 28 August the guess checkbox became a Confident / Guess pair shown after an answer is picked, because in sitting 130 flagged answers ran 2 of 4 and unflagged 6 of 11, "no separation at all" (cd118ed). Chart.js was vendored with its hash checked (bdf7748).

On 29 August I ruled that sittings.intent was carrying selection, grading timing and, by implication, layout. Two additive columns, layout (card or paper) and grading (live or deferred), were added with defaults that reproduce existing behaviour (5d45f96), made reachable from both creation paths (af20a1e) and exposed as controls that say which types each combination can draw (504cd5a). Mode chips collapsed into three format presets, and /quick got its first ten markup assertions (45ee2f0). The nav became Learn, New Test, Revise, Upload (44e7fed). The same evening found doe_dev unable to read its own data (e23f432; see What went wrong).

On 30 August the card client learned to render and mark cloze, match and ordering through a registry in static/qtypes.js. Partial credit now reaches the score while the spaced-repetition schedule still needs full correctness: "a score should reward what was known, a schedule should not" (a0138f1). The deferred-grading card was cut, with InstaGrade and "AInalysis After" as the two named modes (bc3a41a).

The same day ran the first competitive fan-out: two models write explanations independently, each critiques the other, and only agreement lands (1363f04). Twenty-seven items landed in doe_dev and six were held, "held is a finished state, not a spillover" (6dedaf5, 6c2d85c, 8e09f8d). The landing commit also corrected two documents that placed the development database on CT120: "Po's instruction named 136 and was correct against two documents that said otherwise" (8e09f8d). No commit records the move itself; on 24 August both databases were on CT120 [2026-08-24-register-doe-open-defects.md], and by 29 August doe_dev was on CT136 (e23f432).

Stage 8: Question generation and a new question model (31 August to 1 September; 611587d to 9dc4099, 16 commits)#

413 of my 512 concepts had never had a question attached, and my complaint was "I have memorised the crop" (611587d). A fan-out on a new branch, qgen-run-1, generated candidates for 146 untouched concepts. Two writer models (Sonnet and Fable) worked blind to each other, an adversarial reviewer judged every item, and duplicate thresholds were measured from 47,895 existing pairs. Triage moved from a local model to code because a 4k context would silently truncate a batch and return "a confident pass" (611587d). Run 1 finished early on 1 September: 438 items, 252 landed, 146 held, 40 flagged, 8 duplicated within batches (3b59a2f). Its findings file matters more than its questions (580e578, 092bed0; see What went wrong).

The same night the question model changed. 968a3e2 reversed "the 2026-07-12 ruling that questions are sitting-tied": 1,271 question rows held only 558 distinct items, and deleting a sitting deleted its questions. Now questions holds identity, question_versions holds content and is append-only, question_sources allows many citations, and a sitting points at the version it actually showed. The migration was verified inside its own committing transaction, run on development first (968a3e2); the rebuild notes set it out as eight named assertions [REBUILD-doe.md]. On production 61 void sittings could finally be purged and run 1's 252 items loaded, taking the selectable bank from 443 to 695 (1306e91). The 173 items loaded before the answer-leak check existed were screened afterwards and re-derived with the record gap named; the earlier backlog file was marked superseded, not deleted (e1614c2, c5217ad, [DECISIONS-doe.md, D8, D9]).

REBUILD-doe.md, with a "Known broken" section, and the append-only DECISIONS-doe.md closed the stage (3d2340f, 9dc4099). The log then goes quiet from 1 September 11:00 until 5 September 22:08.

Stage 9: A source registry, lab questions and the repo that did not match (5 to 9 September; a4fbd60 to 5ea7639, 21 commits)#

In the silent days a sixth question type, lab, was added: the sitter writes Python that runs in a browser worker on Pyodide, with a starter and a difficulty rung from 1 to 5. On 6 September the drill venue moved to production [DECISIONS-doe.md, D12]. D13, the same day, split which types a card selects from which it accepts: a lab is answerable but unmarked, and writes its verdict and score as NULL, "not false and zero", so an unmarked lab and a zero-scored lab differ in the record [DECISIONS-doe.md, D13].

The first commit after the gap filed a source registry to qgen-run-1 rather than main, because main did not match production (a4fbd60). Overnight into 6 September a fan-out for the nursing student's week 6 ran through Sonnet workers, a no-model machine gate, an Opus adversary and a Fable judge; the gate first rejected 28 of 28 items on its own defects (e3d3e31). Two sets landed 56 items with 3 held, into doe_dev only (e77928d, 666340e). These are the first commits with a Fable 5.1 trailer (6d6f94a onward).

On 6 September I measured the drift properly: no branch held what production ran (3ad2b4d, [DECISIONS-doe.md, D15]). 3cc2b24 adopted the deployed files as source, including the whole Pyodide wrapper behind the lab type; Pyodide itself, 14 MB of binaries, was pinned and documented rather than committed (f30001a).

daily-smoke.py finally refused to run outside doe_dev (8e47afe). A law-and-ethics lesson pipeline for her course was set up with jurisdiction as a blocker, since its failure mode is "a correct statement from the wrong jurisdiction" (7cdf799). A ten-hour run produced nine lessons and 36 checks, leaving 52 of 61 concepts without a lesson (7937243, 262ccbf), and a throughput review found roughly seventeen minutes of frontier inference per multiple-choice question (97a4cfd). On 9 September the quick test was found dead since 5 September (7f6ecb8) and the law-ethics checks unreachable (5ea7639). A stem gate, "a question has to ask something", replaced a verb check that would have refused 41 sound questions of 42 flagged, and was wired into every path that authors a new stem, leaving the option-permutation copy path deliberately ungated (d35a064, cfb59c2).

Stage 10: One grading architecture, and drilling on production (10 to 17 September; a14a55a to 6cd642f, 34 commits)#

Drills were now built as ops scripts and loaded straight into production (eae781d, a14a55a). The eleven agent definitions were committed (0a5c74e).

05e27b1 put every question type behind one grading registry: grading became "a property of when the verdict arrives", with a grading_state per response, a pending poll with "no daemon, no queue, no websocket", a lab grader, and one writer of results. dbeb4fd moved the paper path onto it. results.grading_source dropped "deferred" for words that separate a sitting with nothing markable from one that scored zero (fcbeeed).

On 10 and 11 September came duplicate concepts merged (e565755), an "Enhanced" remix and a concept_performance view (1ce9cb3), one enforced shape for multiple-choice options (dcd82de), a dev-to-prod sitting promotion path (5e052a2), and a sitting ceiling of 50 (82c1bea). On 12 September a byte-accurate refresh of doe_dev from production would have deleted 46 dev-only questions, 36 of them the nursing student's, so they were promoted first by natural key, "never by identity" (28fe589). The refresh still dropped the source registry (2fc01db), and a tier marker now stops a refreshed dev passing for production (5680dc2).

13 September fixed "7 of 0" on the finish card (8e7e71b), put "Sit again" on it (601b2de), and retired three "intersection" domains so that a slash in a domain now means parent and child (c1cc48c, [2026-09-13-proposal-doe-taxonomy-one-axis.md]). Drills 22 to 28 followed, with five to ten labs each (6eaf34a, f35679c); drill 24 scored seven of seven cold, and rung 4 began with drill 25 (a117f04). An elapsed timer (97b1058) and a fuller finish summary (6cd642f) arrived on 17 September; those two and drill 28 (a8e0070) are the three commits with an Opus 4.8 trailer.

Stage 11: The explanation backfill and a cleaner bank (18 to 20 September; 4b51dd7 to c36d10d, 29 commits)#

Drills 29 and 30 (drill 30 had fifteen labs) and a 35-question weekly test were built on 18 September; the weekly was cut from 50 because "50 is possibly a bit long for the weekly. i'll save that for cram sessions" (4b51dd7, d8d2fdf, 9c4e5ab).

That night began a backfill of explanations and hints for 777 items. A writer and two blind adversarial reviewers were calibrated over four prompt revisions (21a4ff2, 8225c72). I then ruled that all 777 should run without spending the calibration sample, and the commit records the cost: the disagreement threshold is never measured (47b7ad6). Batch by batch on both databases, it reached 583 of 775 by 03:25 on 19 September (db234cd through 47a0158).

On 19 September stem repairs landed as version 2 rows, because 15 of the 24 held items had already been sat (2d34c8d); dev was refreshed from prod, this time created explicitly as UTF8 (bec16d4), the shuffle guard was widened from letters to ordinals (b7f59db), questions gained a retired_at so an item can leave circulation without being deleted (3d2089a), and rationales that name a letter or position are now refused when written (1076967). On 20 September the same backfill ran for the nursing student's items under the stricter sourcing rule: 153 of 159 written, six held because no registered source supports them, with citations refused unless the quote is verbatim in an offered passage (c36d10d).

Stage 12: Drilling on it (20 to 27 September; 6a0e912 to 92f9a0d, 11 commits)#

The last week is mostly drills 31 to 38 and tests, each generated by a script that runs every starter and model answer before writing SQL (6a0e912, aa8c054, fd8b7b8, 0e1b15d, 10b6cb0, 92f9a0d). The mid-week test of 23 September and the weekly of 26 September were both 20 questions, built through the same API a person would use (0e1b15d, 10b6cb0). Drills 35 and 36 have no commit of their own; they are mentioned only in later messages (10b6cb0, df09864).

The fixes are the ones sitting forced: the card clock now counts from first open, not from when the sitting was built (fd8b7b8); the lab grader is sent each item's stored checks (4c095fc); hint records were corrected (65af827, aaa1714, b8585b1). The last functional change warns once when a lab is submitted with the starter unchanged (df09864), the first commit with an Opus 5.5 trailer. Drill 37's message holds the rung at 4 "until the lab grader is fixed" (10b6cb0). The latest commit loads drill 38 as sitting 176 (92f9a0d).

Features#

What doe does now, per the records. Each row gives the commits that introduced the feature, then the main rework.

Content: concepts, sources and the question bank#

Feature Commits
Concepts per owner, filed by domain with tags; a slash in a domain means parent and child bb9e312, fcd0530; surrogate key 10ed54d; c1cc48c
Versioned questions: questions, question_versions, question_sources, sitting_questions; repairs land as new versions; retired_at takes an item out of circulation 968a3e2, 1306e91; 2d34c8d, 3d2089a
Six question types: mc, sa, cloze, match, ordering, lab 7515bee; lab [DECISIONS-doe.md, D12], committed in 3cc2b24
Code and figures as attachments on any question 7af2ec9, 66f4c5a; 4bedbcb
Hints and explanations; staged hints for labs f0e5e4c, 5a40fea; backfills 8e09f8d, db234cd to 47a0158, c36d10d
Source registry with immutable stored text a4fbd60; migration on both tiers 2fc01db
Authoring gates: a stem must ask something; a rationale may not name a letter or position; mc options have one positional shape d35a064, cfb59c2, 1076967, b7f59db, dcd82de

Teaching#

Feature Commits
Teach-then-test gate: the generator refuses untaught material; evidence kept apart from assertion e2728f7; 9498b18; authored af46778, attested 7515bee
Lessons with practice checks at /learn 20061ba, ad110d3, e565755
Agent pipelines: lesson and question writer/QA pairs, nursing and law-ethics variants f952bc1, 19b6f57, 976ce9c, 7cdf799; definitions tracked 0a5c74e
Upload: a PDF or pasted text becomes concepts, lessons and a paper in one action af46778, folding in af89d3c and 20b5325

Sitting and grading#

Feature Commits
Long-form paper keyed on sitting id, with a leave-page warning ff18acc, bab084a, 236c9c1
Card test ("quick"): one question at a time, key released after answering; format presets f0e5e4c; all objective types a0138f1; 45ee2f0; size slider to 50 82c1bea
Layout and grading as separate properties of a sitting 5d45f96, af20a1e, 504cd5a; deferred card cut bc3a41a
Daily drill with spaced-repetition write-back; confidence stated after the answer dc1e675, ac8a80c, cd118ed
Lab cards: Python in the browser on Pyodide, Run and Submit, unchanged-starter warning; an unmarked lab records NULL, not zero [DECISIONS-doe.md, D12, D13]; f30001a, f463689, df09864
Remix: sit again (shuffle) and Enhanced (new questions on the same concepts); "Sit again" on the finish card 898372f, 1ce9cb3, 601b2de
AI short-answer grading with blank-scores-zero and a fixed denominator 6624db9, 0d0eda2, dd8f7c0
Deterministic partial credit for cloze, match and ordering 7515bee, a0138f1
One grading registry with pending, graded, failed and unmarked states 05e27b1, dbeb4fd, fcbeeed; lab grader against stored checks 4c095fc, dev only
Finish panel and elapsed clock 8e7e71b, 6cd642f, 97b1058; 185429e, fd8b7b8

Records and platform#

Feature Commits
Progress ledger and chart, four views; unscored rows marked "Completed", not a number 6624db9, a44d5a3, bdf7748; [DECISIONS-doe.md, D14]
Paper review at /papers, answer key never sent 9b743ec, 7ee4866
One results row per sitting, one writer 969d56f, 05e27b1
concept_performance view 1ce9cb3
Session auth, owner tenancy, generation permission apart from admin 34bff3a, 0b0d371, f0e8613, 4ecd6e4
Blueprints; shared estate design f8f73a0; e600fe7, 7db5652, 2756822
Dev and prod tiers, tier marker, dev refresh, row and sitting promotion, guarded smoke test 5a40fea, 5fbef4f, 5680dc2, bec16d4, 28fe589, 5e052a2, 8e47afe
Schema that builds from empty, demo seed, dependency list, rebuild and decisions documents 9036427, 194075e, 10a01b0, 3d2340f, 9dc4099

What went wrong#

The commit messages on this project are unusually candid. What follows is the set that shaped how the work is done.

The repository kept falling behind the running system#

What happened. Again and again the only complete copy of doe was the container (6624db9, 34bff3a): a learn-page rewrite never committed (ad110d3); eight migrations left in scratch (61b96f1); a day of image work patched into the container (66f4c5a); a module that existed only there (ce246c7); nine hours of card work on dev, so that "a promotion from the repo would have shipped yesterday's code to prod" (bc3a41a). By 6 September no branch held what production ran, and the closest branch was the question-generation working branch, 31 diff lines short where main was 177 [DECISIONS-doe.md, D15], [2026-09-06-note-doe-repo-deployment-drift.md]. Of 63 deployed files, 10 differed and 4 were absent from the repo (3cc2b24).

How it was found. Not by git status. On 14 August it showed one modified file where md5 showed ten (66f4c5a); on 17 August "git status reported clean; only an md5 comparison against the running file caught it" (a127265). A drift checker reported zero drift on a file changed three times in one night, because it checked a hand-written file list (bf3896b).

What fixed it. Each time, container to repo, with md5 at every hop; on 6 September the deployed files were adopted wholesale (3cc2b24). The drift checker now enumerates from git ls-files (bf3896b).

What changed. Byte-identical comparison against the running system became the only accepted parity check; 39 commits mention md5. The prior question, how code reaches the containers at all, is recorded as open (3cc2b24, [DECISIONS-doe.md, D15]). It cost schema on 12 September too: the source registry existed only on dev, as a note had warned [2026-09-06-note-doe-prod-missing-source-id.md]. A byte-accurate refresh dropped it, and the commit's own account is that it found a column null on every row and "called it an orphan without checking the feature it belonged to" (2fc01db).

A database password in history#

The first commit carried the database password as a literal. It was removed from the code on 27 July, with the message stating that the literal "remains in history" and that the purge was "deferred, not forgotten"; some other copies were affected too (6624db9). Stale backups in the application directory also carry the vulnerable grader, where a later grep would find them [hygiene-backlog.md]. The sources do not confirm this was resolved.

The grader could be told what mark to give#

What happened. The grading prompt put the stem and the answer in one block with the rules. A dummy question reading "Leave this one blank" scored full marks for a blank answer (0d0eda2), found while building a test harness (44ac0a5).

What fixed it. First, prompt hardening, verified with one direct call and declared fixed (0d0eda2). Through the real submit flow the same code awarded 3.0 for a blank answer: "That verification was insufficient... a prompt-level guard is probabilistic and cannot be relied on as a control." Two deterministic guards followed: blanks score zero before any API call, and the denominator is a server constant (dd8f7c0).

What changed. Verification through the real path, not a unit call. The message names the remaining gap: a non-blank answer can still be over-marked if the model obeys an instructing stem, now bounded to two marks (dd8f7c0).

Answer keys the model chose badly, again and again#

  • A paper came out keyed ABCD ABCD ABCD: the model had been asked to "vary" the letter, and a perfect cycle satisfies that. The key is now drawn in Python (23647ea).
  • On 9 August, 67 of 71 new lessons had their checks keyed A, B, C, D in order. "The cause was my own instruction": the brief said "letters A1 B1 C1 D1", and a distribution audit reported a perfect spread, which is exactly what a fixed sequence produces. A review agent caught it (1cd2dea).
  • On 16 August all 522 lesson checks ran A 34.9% down to D 18.6%, z = 5.2, so the question-writer was barred from emitting a letter (976ce9c). Its seeded permutation was wrong twice before it was right, each fix "narrower than the defect". The same tool rebuilt each record from a field whitelist and silently dropped concept_ids, so every question would have landed linked to no concept; all twelve tests passed, "because every one of them checks that bad input is refused and none checks that good input survives intact" (5684d7b).
  • Rationales that name an option break when options are shuffled. The shuffle refused bare letters from 22 August (898372f); on 19 September three items were found shuffling while their rationale said "the second option" or "the last option", and one, qv 639, had named the correct answer among "the two middle options" it dismissed even before any shuffle (b7f59db). 152 of 1,236 rationales were rewritten by hand, which is why letters and positions are now refused at write time (1076967).

What changed: no model picks a key on this platform any more, and the checks moved from distribution to sequence and then to write time.

The retirement that was wrong about why#

The retirement on 2 August (Stage 3) diagnosed a real failure and drew the wrong conclusion, that the idea was dead. The amendment kept the record and fixed the premise. The history written on 16 August calls it "the most instructive episode in the project's history" [2026-08-16-doe-project-history.md].

Never taught looked the same as forgotten#

Sitting 19 asked about AWS and Azure, never taught, and doe read that as decay [2026-08-06-note-doe-honest-scope.md]. The fix (e2728f7, 20061ba) then misfired twice. On the old corpus it left one domain examinable, so a paper drew 25 of 25 questions from networking and a backfill granted "taught" to concepts examined in real papers (afc5fac). An earlier backfill had marked all 143 nursing concepts as taught, which hid the three nursing lessons written next; the app reported "all lessons learned". A separate provenance column now keeps demonstrated, examined and asserted apart (9498b18), and each later class (authored, attested) was added with its weakening stated (af46778, 7515bee).

Content that was true but not sourced, or sourced but copied#

  • The nursing pipeline's first batch approved nothing: 24 blockers, every one a true statement with no source the learner had been given, because the harvester read only HTML and fetched US references (19b6f57, c6bb386).
  • Two AWS lessons reproduced documentation verbatim while their notes certified they did not; a shingle-overlap test found it. The other seven carry the same self-certified note and are untested (bfa1e19).
  • Two uploads were cut off at 60,000 characters, so two lessons had been written from a table of contents; two of four concept descriptions examined closely were wrong, and 89 remain unexamined. Provenance blockers tracked the ratio of requested length to available source: 3.1 times gave five, 0.5 times gave none (976ce9c).
  • In the generation fan-out the recurring blocker was a correct key with a false explanation, and WebFetch invented an RFC quotation (580e578).
  • Two approved law-ethics files "shipped internal addresses in copy she reads"; it became a build gate (7937243).
  • The sharpest self-criticism is in the throughput review: "Nobody has sat these questions. Nothing in the run measured whether any of them improves recall. The work optimised for defensibility and not at all for effect" (97a4cfd).

What changed: cited authority per claim, verbatim-quote checks at write time for the nursing material (c36d10d), and "a fetch summary is not evidence" (580e578).

Features that worked and could not be reached#

  • The nursing student could generate papers but could not see the page to supply material, with no error anywhere (4ecd6e4).
  • 118 lesson checks rendered as nothing, because the upload flow stored options as an object and the template expected an array (bf3896b). On 11 September "[object Object]" showed on 36 question versions across nine lessons, for the same class of reason (dcd82de).
  • Five question types passed thirty tests and reached zero sitters. "Found by Po asking whether the new formats had been wired, not by the suite" (89d6a47).
  • A correct matching answer showed 0% (ad22e8c); the paper page had never been sent code listings (4bedbcb); the learn page promised four practice questions and served one (e565755).
  • The "Everything" chip did nothing on first press; "py_compile, node --check, the Jinja parse, forty server-side assertions and two smoke suites all passed on the broken code, on dev AND on prod" (45349c4).
  • 36 law-ethics checks were written and unreachable, because a CTE could not see rows inserted by a sibling CTE, while the verification printed a clean report (5ea7639).

What changed: a CSP assertion on every page, markup assertions, client-side logic tests lifted from the shipped template (45349c4, 0b6eb15), a check that fetches the rendered page with a session (bc3a41a), and the rule that "a capability is not shipped until a caller in the product asks for it" [2026-08-19-progress-report-doe-and-completion.md].

A test tool that wrote to production#

daily-smoke.py writes sittings, ledger rows and schedule state. On 23 August it consumed the real daily drill on both databases; sitting 80 was repaired and its schedule write-back left as it was, by decision, and the commit said "the suite needs a guard before it runs anywhere again" (7e5f09a). The leftover partial result then made that paper show 40% instead of 80%, because two results rows existed and every reader took whichever came first (969d56f). The same commit records that a dev sitting was voided as a "duplicate daily" without reading its owner; it belonged to the nursing student (7e5f09a). config.py defaulted the database name to production, so the script "does not need to name prod to hit prod" [2026-08-24-register-doe-open-defects.md]. The guard came on 8 September, sixteen days after the incident: env file and live connection must agree, and the database must be doe_dev, or it exits with no writes (8e47afe).

Timeouts that rewrote the schedule#

Once the daily drill wrote back to schedules, a card that timed out was recorded as wrong, so a phone left face-down would mark a concept weak. Fixed the same evening (ac8a80c). The runbook's correction states the general point: "adding a consumer changes the meaning of every producer that already existed, and the blast radius of stage F was never enumerated" [2026-08-22-runbook-doe-quick-drill.md].

Late and quiet failures in the plumbing#

  • doe_dev was created SQL_ASCII because the cluster's template was, so any query returning a non-ASCII byte threw. The health endpoint touched no text and stayed green, "which is why session 002 closed on an isolation proof that never asked for a non-ASCII character". The root cause was left open (e23f432); the 19 September refresh named UTF8 explicitly (bec16d4).
  • The question-entity migration kept an old function name as an alias, so five smoke and bench scripts imported cleanly and failed only when run: "the alias makes the failure late and quiet rather than immediate and loud" [DECISIONS-doe.md, D4]. They were repaired on 9 and 10 September (2e6dea7, e75d9fd).
  • psycopg2 treats a query as a format string. A percent sign inside a SQL comment broke /finish for every sitting on 30 August (bc3a41a), and another made /api/quick/new fail on every call from 5 to 9 September. The second fix removed the conditions: one static query, named parameters, no prose inside SQL (7f6ecb8).
  • On a model reply wrapped in a code fence, the paper path's short-answer half vanished and the ledger took the objective half as the whole score: "Not a zero - a plausible number with half the paper deleted, and nothing said so." (dbeb4fd)
  • The lab grader was never sent each item's stored checks, so four drill sittings carry marks that contradict them (4c095fc): "a lab score from this platform is not yet a trustworthy number" (aa8c054).
  • progress.scored existed, but both write paths passed a literal True, so an unmarked sitting showed as "null/100" [DECISIONS-doe.md, D14]. D14 says it first showed on drill 24, but D14 was committed on 6 September (3ad2b4d) and the drill 24 commit is dated 13 September (f35679c), so the record does not settle which sitting it was.

Records that could not say what had been checked#

The fan-out state files record which check a reviewer objected on, so "a PASS row and a never-screened row are byte-identical" [DECISIONS-doe.md, D5]. A screening script read only one of three stored option shapes and "silently dropped 826 of 1,114 answers", shortlisting 4 pairs where the corrected one found 29 (e1614c2). In the 777-item backfill, reviewers were never asked whether a stem was answerable, so nine of eleven defective stems were misdiagnosed as hint failures (8f9e59f, 4d93c6b). A hint taken before paging away was not recorded, so "a stored zero there is not evidence a card was sat cold" (185429e), and the lab editor lost two drill-30 cards when the sitter paged away (f463689, 6a0e912).

Mistakes in the record about the record#

Several commits exist only to correct an earlier claim. 6f6c3e5 corrects 1363f04, which called a rule new when it was mine from July: "Absence needs a denominator, and two is not one." 092bed0 corrects a per-writer figure published from estimate. 6b6b937 corrects drill 23's header, which claimed one variable had changed when two had. Drill 25 was contaminated because its constructs were in the message carrying the link (a117f04), and drill 26's missing answer took three repair scripts, two kept "because they name the faults" (af5da7e). 7e81c86 moves a misfiled document, quoting my objection that "spraying random detritus that literally ignores every convention including naming and domain is deeply unhelpful to the point of harmful". The documents also contradict the commits: the quick-drill runbook says style-lint.py "does not exist" and "Nothing has been linting anything" [2026-08-22-runbook-doe-quick-drill.md], while commits on 13 and 23 August report style-lint runs (7db5652, 7e5f09a); the record does not settle where the lint ran.

Dead ends and reversals#

The week counter as route key (bab084a). /corpus and /generate, then /material, then /upload within three days (af89d3c, af46778). A CI-CD domain name applied on dev and backed out (c1cc48c). Questions owned by one sitting, a July ruling reversed by measurement in September (968a3e2).

Where it stands#

This is what the latest commits and documents say as of 27 September 2026, not a reading of the running system.

Running. Production is doe_prod on CT120 and development is doe_dev on CT136 [REBUILD-doe.md], last refreshed from production on 19 September (bec16d4). On 19 September production held 1,252 live questions and 2 retired for me, and 389 for the nursing student (3d2089a). The backfill left one of my items outstanding (5cdc563) and six of hers held for want of a registered source (c36d10d).

In use. I sit a rung-4 lab drill most days and a 20-question weekly or mid-week test (0e1b15d, 10b6cb0); the latest is drill 38, sitting 176 (92f9a0d). The rung-4 exit test is a sitting made entirely of coupled reversals, cold, no hints, 8 of 10 or better (4b51dd7); a proposed two-at-80% promotion rule is recorded as proposed and not implemented (aa8c054).

Open problems the sources name. Each is named open in its source, and no later commit in this log records a fix.

  • The lab grader fix is on dev and not yet deployed; one case still needs "a deterministic check runner rather than prompt work" (4c095fc). Until then the rung is held (10b6cb0).
  • How code reaches the containers without passing through a commit (3cc2b24, [DECISIONS-doe.md, D15]). The database password in git history (6624db9).
  • The fan-out state files cannot record which checks were applied [DECISIONS-doe.md, D5]; 28 items from the check-15 screening are listed for repair [DECISIONS-doe.md, D8]; 563 of 1,114 of my question versions had no concept link at that screening (e1614c2).
  • functional-smoke.py and remix-smoke.py query tables the version migration removed, and views_exam.py maps only four option positions, so a five-option question is always marked wrong (dcd82de).
  • Posting finish to an already graded sitting leaves a blank page (185429e). A full reload still loses a lab draft (f463689).
  • The card picker can draw 352 of my questions against 552 linked to concepts, "a gap of 200 [that] is real and is an open thread" (4f8fc0d).
  • Domains need a display name stored apart from the filing path, and the tag vocabulary has four words doing one job [2026-09-13-proposal-doe-taxonomy-one-axis.md].
  • Seven AWS lessons are untested for copying (bfa1e19).
  • From the 24 August register: trailing commas that kill papers with a JSONDecodeError (A1); verifier verdicts that are produced and lost (A2); CT120's Postgres listening on all interfaces (C1); and a config default that names a database rather than failing (C3) [2026-08-24-register-doe-open-defects.md].

Planned next. Deploying the lab grader fix (4c095fc). The throughput proposal's order: a structured authority store, a mechanical pre-QA gate, generated commissions, then measurement against results, with the warning that "if throughput does not improve several-fold, the architecture is wrong" (97a4cfd). Resolving source ids at promotion to production, which "is not implemented" [2026-09-06-note-doe-prod-missing-source-id.md]. The case-study question type, 34 of 50 marks in one of the nursing student's exams, was scoped as a design problem on 19 August, and the same report says "nothing about TM111 is modelled" [2026-08-19-progress-report-doe-and-completion.md]; neither appears in a later commit. Of the four-environment plan, the empty-schema build, the demo seed and a separate dev container exist (9036427, 194075e, [REBUILD-doe.md]); the log records no demo-dev container.

What I'd tell someone building one#

  1. Check parity against what is running, byte for byte. Where drift was measured, git status missed it and md5 against the container found it (66f4c5a, a127265, bc3a41a, 3cc2b24).
  2. A guard that depends on the model's cooperation is not a guard. Prompt hardening passed one test and failed the real submit path; arithmetic that does not ask the model held (dd8f7c0).
  3. Never let the model choose the answer key. Choose in code, and check order as well as distribution (23647ea, 1cd2dea, 976ce9c, 5684d7b).
  4. A capability is not shipped until something in the product calls it. Five question types passed thirty tests and reached nobody (89d6a47).
  5. Look at the screen. The defects the suites missed were found by a person (4ecd6e4, 7ee4866, 45349c4).
  6. Test that good input survives, not only that bad input is refused. Twelve refusal tests passed on a tool that dropped every concept link (5684d7b).
  7. Keep evidence apart from assertion, and null apart from zero. One field meaning "answered correctly" and "declared known" hid three lessons (9498b18); an unmarked lab must not look like a failed one [DECISIONS-doe.md, D13].
  8. When you add a consumer of old data, re-read what produces it. Schedule write-back turned a harmless timeout into a wrong answer (ac8a80c, [2026-08-22-runbook-doe-quick-drill.md]).
  9. Tools that write need a hard guard, not a comment. The daily smoke test consumed real drills and left a duplicate score behind (7e5f09a, 969d56f, 8e47afe).
  10. A detector that finds almost nothing is probably broken. Measure its coverage before you report the absence (e1614c2, [DECISIONS-doe.md, D6]).

Sources#

All under sources/doe/.

  • gitlog.txt: all 243 commits, ff18acc (2026-07-12) to 92f9a0d (2026-09-27), read in full.
  • 2026-08-16-doe-project-history.md: used for the period before the first commit and for the gap between 13 and 27 July, checked against the log where they overlap.
  • AGENTS.md, DECISIONS-doe.md, REBUILD-doe.md, hygiene-backlog.md.
  • prj/: 2026-08-03-authority-seed-phoenix-exam-app-course-correction.md, 2026-08-06-note-doe-honest-scope.md, 2026-08-16-delegable-doe-ui-polish.md, 2026-08-19-progress-report-doe-and-completion.md, 2026-08-22-runbook-doe-four-environments.md, 2026-08-22-runbook-doe-quick-drill.md, 2026-08-24-register-doe-open-defects.md, 2026-09-05-proposal-external-syllabus-as-exam-source.md, 2026-09-06-note-doe-prod-missing-source-id.md, 2026-09-06-note-doe-repo-deployment-drift.md, 2026-09-13-proposal-doe-taxonomy-one-axis.md.
  • prj/2026-09-14-defect-falken-pg-dump-retention-evicts-real-dumps.md: read, and not used. It says itself that it is a treasury defect; the job only ran on doe's container, so it belongs in the treasury log. The register's A4 is likewise a treasury item.

Gaps in the record: no commits between 13 and 27 July, or between 1 September 11:00 and 5 September 22:08. The lab type and Pyodide runtime were deployed on 5 September and first committed on 6 September (3cc2b24). The move of the development database from CT120 to CT136 is in no commit. The retirement document of 2 August, the 5 September "no user" finding and DRILL-LOG.md are referenced by these sources but are not among them.