Correctness first.
Efficiency second.

What works today

One full challenge and three quick exercises. Each checks a JSON artifact you paste or upload, with deterministic, visible checks that run in your browser. Use any AI assistant you like outside VeebCoder; the site has no model UI and makes no model calls. Every artifact must be valid JSON with no repeated key inside one object; a repeated key is rejected rather than silently resolved.

School timetable (full challenge)

Your timetable must place all 102 sessions of the one disclosed instance, each exactly once, and every published rule must hold at the same time: exact format and known IDs, sessions inside the day, qualified teachers, teacher and room availability, room capacity and equipment, and no cohort, teacher or room double-booking. The result is all-or-nothing. “Valid timetable” means zero errors. The X / 102 count is progress, not a score: it counts distinct sessions placed with well-formed fields and known IDs, even if they still break a rule. Unplaced sessions are grouped and listed in full; up to 100 other errors are listed with a JSON path.

Quick exercises (four checks each)

Triage, release planning and budget routing each run four named checks on one small fixture, with a short reason for each failed check. A result of 4/4 means the artifact passed this fixture only. Some checks combine requirements, so a partial score is a hint, not a grade.

Neither result proves a general algorithm, unseen test performance, or your model’s capability. These are public practice fixtures, not hidden or secure evaluations.

What we measure

1. Correctness: the number of acceptance checks passed. Read each check’s scope; some checks combine requirements.
2. Tokens: optional, self-reported total across all attempts, including input and output. Missing is unknown, never zero.
3. Cost and time: optional, self-reported USD and elapsed seconds. No inferred pricing, made-up measurements, or live rankings.

What is not enabled

No model calls, account system, public submissions, code execution, hidden tests, verified telemetry, or leaderboard. Prompts and artifacts remain in browser memory until you export them. Avoid sensitive data. Hosting providers may retain ordinary request logs.

The path to trusted evaluation

Versioned challenge suites and isolated, short-lived runners with network disabled, strict CPU/memory/time limits, read-only fixtures, and signed results. Never execute submitted code in the web server. Route authorized model calls through a provider gateway that records provider-reported usage, model version, cached/reasoning tokens, retries, price snapshot, and elapsed time.

Future comparisons will first group by verified correctness on the same suite version, then compare tokens, cost, and time separately. A composite score must not hide failed requirements. Self-reported runs will remain separate from verified runs.