Skip to content

Evaluations

Status: Current — convention set 2026-08-27. Owner: delan

Outside assessments of the project — usually an LLM asked to review the architecture, the story, the card design, or the Kickstarter readiness. This is where they land. Before this directory existed they landed at the repository root, untracked and outside version control.

The convention

Filename: YYYY-MM-DD-<topic>.md — dated, kebab-case topic. 2026-08-27-class-faction-cards.md, not EVALUATION-cards-FINAL-v2.md.

Status block, directly under the H1, stating four things:

> **Status:** Point-in-time assessment, YYYY-MM-DD. Not canon and not a plan.
> **Provenance:** which model, and what prompt or scope it was given.
> **Verification:** which claims were spot-checked against the repo, and what held.
> **Known errors:** anything later found wrong — appended, never silently fixed.

The rules

  1. An evaluation is never edited to change its findings. It is a record of what someone thought on a date. If it turns out to be wrong, append a Known Errors note to the status block. Do not quietly correct the body — that destroys the record of how the project was seen at the time.
  2. Superseded, not updated. A newer assessment gets a new dated file. The old one stays.
  3. Not canon, and not a backlog. An evaluation cannot decide anything. If you accept a recommendation, it goes into docs/official/todo-inventory.md or story/bible/ as a real decision, and that is what future work follows.
  4. Spot-check before acting. Every evaluation here has been checked at least shallowly against the repo, and the status block says how far that went. The August 2026 card evaluation read the wrong file and drew a materially wrong conclusion from it — see its Known Errors note. Outside reviewers are misled by the same documentation drift that misleads you.
  5. Say which model wrote it. Not to discount it — to spot patterns when three evaluations agree because they share a failure mode rather than because they are right.

How to get a good one

The reviewer's output is only as good as what it can see. Before commissioning one:

  • Point it at START-HERE.md first, especially the Traps section.
  • Say explicitly that _archive/ is not canon and docs/proposals/ is not built.
  • Say that data/content/cards.json is the card list. This is the single mistake most likely to be repeated.

Current evaluations

Date Topic Model Notes
2026-08-28 Overall concept Grok 4.6 Reassessment. Concept B+, shipped structure C. Launch odds if sequenced vs if launched now.
2026-08-28 Game story Grok 4.6 Bible as concept A-; shippable story D+. Live opening is the pre-rebuild truth dump.
2026-08-28 Game architecture Grok 4.6 Core A-. WPF Quick Play is a real loop (todo screen-absence lines are stale). Two run loops; three card truths.
2026-08-28 Game assets Grok 4.6 584/476 is real. Identity mostly in text; 108 empty payloads are the identity cards. Harness cannot balance that.
2026-08-28 Suggestions Grok 4.6 Sequencing only. Not a backlog until copied into todo-inventory.
2026-08-27 Architecture, story, Kickstarter Grok Sequencing argument is sound. Two known errors noted in its status block. WPF playability understated relative to 2026-08-28.
2026-08-27 Class, faction, cards Grok Strong design critique. One material inventory error — read the bootstrap seed, not the live card pool. Identity advice still usable; inventory superseded by 2026-08-28 assets eval.