Drop max_tokens cap on codegen; qwen3 reasoning truncation left content empty

qwen38-iq3s reasons extensively before emitting the skill body. A max_tokens
cap truncated the hidden reasoning (finish_reason: length) leaving content
empty, so the body write failed with 'skill body is empty'. Omit max_tokens
so the model runs to completion (~7 min); reasoning is filtered automatically
since only content is read. Client timeout default raised to 1200s.
This commit is contained in:
Denton Social
2026-09-24 01:20:18 -05:00
parent 440e49e76e
commit 9e365446a3
4 changed files with 64 additions and 20 deletions
+10 -3
View File
@@ -185,9 +185,16 @@ unit tests (24) + box integration tests (2).
### codegen (skill bodies, box)
- Skill **bodies** are written by a separate OpenAI-compatible model, configured
under `codegen` in config.json (default model `qwen38-iq3s`, the 12G 27B
IQ3_S GGUF — huge/slow; a 3-bit 27B write can take 30-120s). Title +
description for new skills still come from the **small** decision model
(`engine.generate`); only the runnable code body uses codegen.
IQ3_S GGUF — huge/slow). Title + description for new skills still come from
the **small** decision model (`engine.generate`); only the runnable code body
uses codegen.
- **Do NOT cap `max_tokens`** on the codegen call. qwen38-iq3s reasons first
and a cap truncates the hidden reasoning, leaving `content` empty
(`finish_reason: length`) and the body write fails with "skill body is
empty". Unbounded, it runs to completion in ~7 min (~40k chars of reasoning
then the code); the client reads only `content`, so reasoning is filtered
automatically. The client default timeout is 1200s — raise `codegen.timeout`
in config if a harder prompt needs more.
- Bodies are persisted to `data/skills/<category>/<name>.py` (gitignored) and
loaded back at startup via `importlib`, so skills stay runnable across
restarts. `SKILL.md` at the repo root is the contract the codegen model is