Author runnable skill bodies via OpenAI-compatible codegen model
create_skill now writes a real predict/act body: the small decision model still authors title + description (engine.generate), then a larger OpenAI-compatible model (default qwen38-iq3s) writes the runnable code against the SKILL.md contract. Bodies persist to data/skills/<cat>/<name>.py, are hot-loaded via importlib, merged into the running tree, and the request re-dispatches to the new leaf. The dashboard decision-flow view shows the title/description with a writing badge while the body is being written. Codegen failure degrades to a navigable stub.
This commit is contained in:
@@ -4,6 +4,7 @@ __pycache__/
|
|||||||
data/decisions.jsonl
|
data/decisions.jsonl
|
||||||
data/runs.jsonl
|
data/runs.jsonl
|
||||||
data/categories.json
|
data/categories.json
|
||||||
|
data/skills/
|
||||||
data/drafts/
|
data/drafts/
|
||||||
.pytest_cache/
|
.pytest_cache/
|
||||||
config.json
|
config.json
|
||||||
|
|||||||
@@ -24,9 +24,12 @@ queue.py urgency max-heap (desc weight, FIFO seq), age pulls toward 1.0
|
|||||||
skills.py tree + registry (email.compose, response.reject, tracking.check),
|
skills.py tree + registry (email.compose, response.reject, tracking.check),
|
||||||
navigation = SemIf choices per level (logged), create_category
|
navigation = SemIf choices per level (logged), create_category
|
||||||
and create_skill author + register stubs via the decision model
|
and create_skill author + register stubs via the decision model
|
||||||
in generation mode
|
in generation mode; SkillBodyStore + materialize_skill persist
|
||||||
|
and hot-load runnable skill bodies from data/skills/
|
||||||
skill.py loop: observe -> predict -> act -> observe -> assess (LLM)
|
skill.py loop: observe -> predict -> act -> observe -> assess (LLM)
|
||||||
engine.py SemIfEngine -> semif_phase1.llamacpp_backend (lazy import)
|
engine.py SemIfEngine -> semif_phase1.llamacpp_backend (lazy import)
|
||||||
|
codegen.py CodegenClient (OpenAI-compatible) writes runnable skill bodies
|
||||||
|
against SKILL.md; parse/validate (compile + predict/act)
|
||||||
llm.py OpenAI-compatible client for self-assessment (stdlib urllib)
|
llm.py OpenAI-compatible client for self-assessment (stdlib urllib)
|
||||||
log.py decisions.jsonl rows {state, question, options, predicted_probs,
|
log.py decisions.jsonl rows {state, question, options, predicted_probs,
|
||||||
selected, observed_outcome, label_source}
|
selected, observed_outcome, label_source}
|
||||||
@@ -116,12 +119,16 @@ unit tests (24) + box integration tests (2).
|
|||||||
validate (accuracy/ECE on a held-out slice, prompt-hash regression), swap the
|
validate (accuracy/ECE on a held-out slice, prompt-hash regression), swap the
|
||||||
pinned model revision. GPU offload: train on a beefier GPU; the running agent
|
pinned model revision. GPU offload: train on a beefier GPU; the running agent
|
||||||
keeps a frozen inference revision until a swap validates.
|
keeps a frozen inference revision until a swap validates.
|
||||||
- `create_skill` branch: now live, mirroring `create_category`. The decision
|
- `create_skill` branch: live, mirroring `create_category`. The decision
|
||||||
model, driven in normal generation mode via `SemIfEngine.generate`, proposes a
|
model, driven in normal generation mode via `SemIfEngine.generate`, proposes a
|
||||||
specific skill title + description for the chosen category; the stub is
|
specific skill title + description for the chosen category; the stub is
|
||||||
persisted to `data/categories.json` (under that category's `skills` list) and
|
persisted to `data/categories.json` (under that category's `skills` list) and
|
||||||
merged into the running tree as a leaf. Still deferred: a real skill body —
|
merged into the running tree as a leaf. Since Sep 2026 the leaf also gets a
|
||||||
opencode authoring at a tree leaf remains future work.
|
real runnable body: a larger OpenAI-compatible model (`codegen`, default
|
||||||
|
`qwen38-iq3s`) writes `predict`/`act` code against `SKILL.md`, persisted to
|
||||||
|
`data/skills/` and hot-loaded, then the request re-dispatches to the new
|
||||||
|
skill. Authoring is still a single pass — validating/reusing written bodies
|
||||||
|
across runs is future work.
|
||||||
- Queue persistence (durable across restarts).
|
- Queue persistence (durable across restarts).
|
||||||
- Event/timer intake sources beyond typed input.
|
- Event/timer intake sources beyond typed input.
|
||||||
- Concurrency: SemIf shared-state mode (`score_shared` / `SerialPrefixScorer`)
|
- Concurrency: SemIf shared-state mode (`score_shared` / `SerialPrefixScorer`)
|
||||||
@@ -175,6 +182,25 @@ unit tests (24) + box integration tests (2).
|
|||||||
- If generation hangs with no log output, restart the service
|
- If generation hangs with no log output, restart the service
|
||||||
(`sudo systemctl restart ollama`) — the ROCm runner can wedge.
|
(`sudo systemctl restart ollama`) — the ROCm runner can wedge.
|
||||||
|
|
||||||
|
### codegen (skill bodies, box)
|
||||||
|
- Skill **bodies** are written by a separate OpenAI-compatible model, configured
|
||||||
|
under `codegen` in config.json (default model `qwen38-iq3s`, the 12G 27B
|
||||||
|
IQ3_S GGUF — huge/slow; a 3-bit 27B write can take 30-120s). Title +
|
||||||
|
description for new skills still come from the **small** decision model
|
||||||
|
(`engine.generate`); only the runnable code body uses codegen.
|
||||||
|
- Bodies are persisted to `data/skills/<category>/<name>.py` (gitignored) and
|
||||||
|
loaded back at startup via `importlib`, so skills stay runnable across
|
||||||
|
restarts. `SKILL.md` at the repo root is the contract the codegen model is
|
||||||
|
prompted with — change it only with intent, it shapes every generated body.
|
||||||
|
- **Trust boundary**: generated skill code is executed locally (it is imported
|
||||||
|
as a module and its `predict`/`act` run in-process). The box is the intended
|
||||||
|
target; treat the endpoint as trusted.
|
||||||
|
- Flow in `scheduler._create_skill`: small model authors title+description →
|
||||||
|
trace `skill_writing` (dashboard shows title/description + a "writing skill
|
||||||
|
body…" badge) → sync codegen write → `materialize_skill` → hot-merge into the
|
||||||
|
tree → bounded re-dispatch so the request runs the new skill. Codegen failure
|
||||||
|
leaves a navigable stub and returns a graceful `create_skill` result.
|
||||||
|
|
||||||
### Code principles
|
### Code principles
|
||||||
- **No mocking.** The decision engine is always real SemIf; the LLM is always a
|
- **No mocking.** The decision engine is always real SemIf; the LLM is always a
|
||||||
real endpoint. Pure unit tests touch data-structure math only (queue ordering,
|
real endpoint. Pure unit tests touch data-structure math only (queue ordering,
|
||||||
@@ -199,7 +225,8 @@ unit tests (24) + box integration tests (2).
|
|||||||
|
|
||||||
- `python3 -m pytest tests/ -q --ignore=tests/integration` — anywhere, fast.
|
- `python3 -m pytest tests/ -q --ignore=tests/integration` — anywhere, fast.
|
||||||
Includes the dashboard API tests (`tests/test_dashboard_api.py`), which spin
|
Includes the dashboard API tests (`tests/test_dashboard_api.py`), which spin
|
||||||
up the stdlib HTTP server on an ephemeral port with the engine never loaded.
|
up the stdlib HTTP server on an ephemeral port with the engine never loaded,
|
||||||
|
and `tests/test_codegen.py` for prompt/parse/validate + body store round-trips.
|
||||||
- `tests/integration/` — box only; requires real SemIf + real ollama.
|
- `tests/integration/` — box only; requires real SemIf + real ollama.
|
||||||
- After touching scheduler/skills/engine, re-run both; the integration tests are
|
- After touching scheduler/skills/codegen/engine, re-run both; the integration
|
||||||
the only end-to-end verification.
|
tests are the only end-to-end verification.
|
||||||
@@ -41,7 +41,7 @@ All decisions are SemIf calls: `{state, question, options[]}`. State is the curr
|
|||||||
|
|
||||||
- Structure: categories → skills → actions. Top level listed at each level.
|
- Structure: categories → skills → actions. Top level listed at each level.
|
||||||
- Navigation is a chain of SemIf choices, one per level, descending until a leaf skill matches.
|
- Navigation is a chain of SemIf choices, one per level, descending until a leaf skill matches.
|
||||||
- Navigation offers a `create_category` suggestion at the category level and a `create_skill` suggestion at the leaf level. Both are live: the decision model, driven in normal generation mode, authors a title + description (broad bucket for a category, single specific action for a skill), and the stub is persisted to the category registry and merged into the running tree. The new skill becomes a leaf immediately; a real skill body (opencode authoring a skill manifest) remains deferred to v2.
|
- Navigation offers a `create_category` suggestion at the category level and a `create_skill` suggestion at the leaf level. Both are live: the decision model, driven in normal generation mode, authors a title + description (broad bucket for a category, single specific action for a skill), and the stub is persisted to the category registry and merged into the running tree. For a new skill the stub is then promoted to a runnable body: a separate, larger OpenAI-compatible model writes the `predict`/`act` code against the `SKILL.md` contract, persisted under `data/skills/` and hot-loaded, and the request re-dispatches to the new leaf.
|
||||||
|
|
||||||
### Skill manifest
|
### Skill manifest
|
||||||
- name, category, description, allowed inputs, action list, cost budget, decision log reference.
|
- name, category, description, allowed inputs, action list, cost budget, decision log reference.
|
||||||
@@ -97,7 +97,7 @@ Every skill run follows the same loop:
|
|||||||
- contains_request: yes
|
- contains_request: yes
|
||||||
- interrupt_current: yes
|
- interrupt_current: yes
|
||||||
- skill_selection: tracking => (no leaf) => create_skill
|
- skill_selection: tracking => (no leaf) => create_skill
|
||||||
- opencode authors `tracking.check_delivery` from the input + registry conventions → manifest registered → requeue → skill_selection: tracking => check_delivery(input)
|
- the codegen model authors `tracking.check_delivery` (title+description from the decision model; `predict`/`act` body from a larger OpenAI-compatible model per `SKILL.md`) → body persisted + registered → requeue → skill_selection: tracking => check_delivery(input)
|
||||||
|
|
||||||
### Example 5 (self-assessment + dream)
|
### Example 5 (self-assessment + dream)
|
||||||
- current_process: email compose
|
- current_process: email compose
|
||||||
|
|||||||
@@ -0,0 +1,132 @@
|
|||||||
|
# SKILL.md — the contract for new skills
|
||||||
|
|
||||||
|
This file is the authoritative spec for what a new skill is and how its code
|
||||||
|
body must be written. It is fed verbatim to the code-generation model so every
|
||||||
|
generated skill is consistent, and it is read by humans who want to know what a
|
||||||
|
good skill looks like.
|
||||||
|
|
||||||
|
## What a skill is
|
||||||
|
|
||||||
|
A skill is a **leaf** in the agent's skill tree, reached by a chain of SemIf
|
||||||
|
decisions (category -> skill). It is one specific, single-purpose action the
|
||||||
|
agent can take — never a broad bucket (that is a category's job). It runs the
|
||||||
|
standard skill loop: observe -> predict -> act -> observe -> assess.
|
||||||
|
|
||||||
|
A skill is two things:
|
||||||
|
|
||||||
|
1. **A manifest** — the registry entry that makes it navigable and describes
|
||||||
|
what it does.
|
||||||
|
2. **A code body** — a runnable Python module implementing the `predict` and
|
||||||
|
`act` phases.
|
||||||
|
|
||||||
|
## Manifest schema
|
||||||
|
|
||||||
|
Registered in `data/categories.json` (and mirrored in the running tree). Fields:
|
||||||
|
|
||||||
|
| Field | Meaning |
|
||||||
|
| -------------------- | -------------------------------------------------------------------- |
|
||||||
|
| `name` | `category.skill` dotted snake_case id. Lowercase letters, digits, `_` and `.` only (`[a-z0-9_]+(?:\.[a-z0-9_]+)*`). Single purpose. |
|
||||||
|
| `category` | The top-level category bucket the skill lives under. |
|
||||||
|
| `description` | One to two sentences: what the skill does, for navigation. |
|
||||||
|
| `allowed_inputs` | What the skill accepts / needs as input context. |
|
||||||
|
| `actions` | The concrete actions the skill takes. |
|
||||||
|
| `cost_budget` | Relative budget for one run; a run that exceeds it fails fast. |
|
||||||
|
| `decision_log_ref` | Reference to the decision rows this skill logged during a run. |
|
||||||
|
|
||||||
|
Only `name` and `description` are required for a stub; the rest fill in as the
|
||||||
|
skill is exercised.
|
||||||
|
|
||||||
|
## Code body contract
|
||||||
|
|
||||||
|
The generated module is persisted to `data/skills/<category>/<name>.py` and
|
||||||
|
imported at runtime. It must satisfy **all** of the following:
|
||||||
|
|
||||||
|
### Required functions
|
||||||
|
|
||||||
|
```python
|
||||||
|
def predict(ctx, request) -> Prediction:
|
||||||
|
"""Forecast + make any SemIf sub-decisions. Return the prediction."""
|
||||||
|
|
||||||
|
def act(ctx, request, prediction) -> ActionResult:
|
||||||
|
"""Execute the action. Return the result + new state."""
|
||||||
|
```
|
||||||
|
|
||||||
|
- `ctx` is an `ActionContext` with `ctx.engine` (the real SemIf engine) and
|
||||||
|
`ctx.config` (the agent config dict).
|
||||||
|
- `request` is the `Request` being handled.
|
||||||
|
- `Prediction(text: str, decisions: list)` and
|
||||||
|
`ActionResult(action_log: str, new_state: str)` are imported from
|
||||||
|
`semif_agent.skills`; return those exact types. `decisions` carries any
|
||||||
|
`(DecisionRequest, DecisionResult)` pairs made during predict so they are
|
||||||
|
logged as training rows.
|
||||||
|
|
||||||
|
### Rules (hard requirements)
|
||||||
|
|
||||||
|
- **Stdlib only.** No third-party imports, no files outside the project. The
|
||||||
|
core agent is pure-stdlib and runs on the thin dev box.
|
||||||
|
- **No mocking.** Sub-decisions use the real engine: build a
|
||||||
|
`DecisionRequest(state, question, options=[Option(id, description), ...])`
|
||||||
|
and call `ctx.engine.call(decision)`; return it inside `Prediction.decisions`.
|
||||||
|
- **Never swallow the request.** If the skill cannot act, return an
|
||||||
|
`ActionResult` with a short `action_log` explaining why and set `new_state`
|
||||||
|
back to `request.text`.
|
||||||
|
- **Write files under configured data dirs only** (e.g. `ctx.config["drafts"]`),
|
||||||
|
never anywhere else on disk.
|
||||||
|
- **Fail fast on budget.** Keep the work small; do not loop or retry in code.
|
||||||
|
- **Names match the manifest.** The module is imported as its manifest name;
|
||||||
|
the functions are `predict` and `act` exactly.
|
||||||
|
|
||||||
|
## Conventions
|
||||||
|
|
||||||
|
- Single purpose, single file, single module.
|
||||||
|
- Avoid duplicating an existing skill in the same category.
|
||||||
|
- `predict` resolves ambiguity (arguments, recipients, targets) with SemIf
|
||||||
|
sub-decisions, mirroring how `email.compose` resolves its recipient.
|
||||||
|
- `act` performs the concrete action and writes a human-readable `action_log`
|
||||||
|
that the self-assessment LLM can judge.
|
||||||
|
|
||||||
|
## Acceptance criteria
|
||||||
|
|
||||||
|
A generated skill is accepted only if:
|
||||||
|
|
||||||
|
1. It compiles (`compile(..., "exec")` succeeds) and defines both `predict`
|
||||||
|
and `act`.
|
||||||
|
2. Its `name` matches the manifest regex and its `category` is given.
|
||||||
|
3. Its body imports nothing outside the stdlib and the agent package.
|
||||||
|
4. It uses `ctx.engine` (never mocks) and returns proper `Prediction` /
|
||||||
|
`ActionResult` types.
|
||||||
|
5. It is single-purpose and does not duplicate an existing category leaf.
|
||||||
|
|
||||||
|
## Worked example
|
||||||
|
|
||||||
|
`email.compose` resolves its recipient with a SemIf sub-decision, then writes a
|
||||||
|
draft file:
|
||||||
|
|
||||||
|
```python
|
||||||
|
def predict(ctx, request):
|
||||||
|
contacts = read_contacts(ctx.config) # config-driven, local data
|
||||||
|
decision = DecisionRequest(
|
||||||
|
state=f"{request.text} [current process: none]",
|
||||||
|
question="Which contact is the intended recipient?",
|
||||||
|
options=[Option(c["name"], c.get("description", "")) for c in contacts]
|
||||||
|
+ [Option("none", "None of the listed contacts.")],
|
||||||
|
)
|
||||||
|
result = ctx.engine.call(decision)
|
||||||
|
return Prediction(text=f"recipient is {result.selected}",
|
||||||
|
decisions=[(decision, result)])
|
||||||
|
|
||||||
|
def act(ctx, request, prediction):
|
||||||
|
recipient = prediction.text.removeprefix("recipient is ")
|
||||||
|
drafts = Path(ctx.config.get("drafts", "data/drafts"))
|
||||||
|
drafts.mkdir(parents=True, exist_ok=True)
|
||||||
|
target = drafts / f"{request.id}.txt"
|
||||||
|
target.write_text(f"To: {recipient}\nBody: {request.text}\n")
|
||||||
|
return ActionResult(
|
||||||
|
action_log=f"email.compose: wrote draft {target} for {recipient!r}.",
|
||||||
|
new_state=f"Draft written to {target.name} for {recipient}.",
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
Write skill bodies in this shape: resolve ambiguity in `predict` via
|
||||||
|
`ctx.engine`, do the work in `act`, keep both stdlib-only, and return the
|
||||||
|
proper types.
|
||||||
@@ -12,6 +12,13 @@
|
|||||||
"threads": 8
|
"threads": 8
|
||||||
},
|
},
|
||||||
"llm": {"base_url": "http://localhost:11434/v1", "model": "qwen3.5:4b"},
|
"llm": {"base_url": "http://localhost:11434/v1", "model": "qwen3.5:4b"},
|
||||||
|
"codegen": {
|
||||||
|
"_comment": "OpenAI-compatible model that writes runnable skill bodies. Larger/slower than the decision or self-assessment model.",
|
||||||
|
"base_url": "http://localhost:11434/v1",
|
||||||
|
"model": "qwen38-iq3s",
|
||||||
|
"timeout": 600
|
||||||
|
},
|
||||||
|
"skill_bodies": "data/skills",
|
||||||
"skills": {
|
"skills": {
|
||||||
"email": {"cost_budget": 1.0},
|
"email": {"cost_budget": 1.0},
|
||||||
"contacts": "data/contacts.json",
|
"contacts": "data/contacts.json",
|
||||||
|
|||||||
@@ -18,6 +18,7 @@ import sys
|
|||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
from .dream import dream as run_dream
|
from .dream import dream as run_dream
|
||||||
|
from .codegen import CodegenClient
|
||||||
from .engine import EngineConfig, EngineUnavailable, SemIfEngine
|
from .engine import EngineConfig, EngineUnavailable, SemIfEngine
|
||||||
from .llm import LLMClient
|
from .llm import LLMClient
|
||||||
from .log import DecisionLog
|
from .log import DecisionLog
|
||||||
@@ -45,6 +46,14 @@ def build_scheduler(config: dict) -> tuple[Scheduler, dict]:
|
|||||||
base_url=config.get("llm", {}).get("base_url", "http://localhost:11434/v1"),
|
base_url=config.get("llm", {}).get("base_url", "http://localhost:11434/v1"),
|
||||||
model=config.get("llm", {}).get("model", "qwen2.5:3b"),
|
model=config.get("llm", {}).get("model", "qwen2.5:3b"),
|
||||||
)
|
)
|
||||||
|
codegen_cfg = config.get("codegen", {})
|
||||||
|
codegen = CodegenClient(
|
||||||
|
base_url=codegen_cfg.get(
|
||||||
|
"base_url", config.get("llm", {}).get("base_url", "http://localhost:11434/v1")
|
||||||
|
),
|
||||||
|
model=codegen_cfg.get("model", "qwen38-iq3s"),
|
||||||
|
timeout=float(codegen_cfg.get("timeout", 600.0)),
|
||||||
|
)
|
||||||
log = DecisionLog(config.get("log", "data/decisions.jsonl"))
|
log = DecisionLog(config.get("log", "data/decisions.jsonl"))
|
||||||
trace = TraceLog(config.get("trace", "data/runs.jsonl"))
|
trace = TraceLog(config.get("trace", "data/runs.jsonl"))
|
||||||
scheduler = Scheduler(
|
scheduler = Scheduler(
|
||||||
@@ -55,6 +64,7 @@ def build_scheduler(config: dict) -> tuple[Scheduler, dict]:
|
|||||||
tau=float(config.get("tau", 0.6)),
|
tau=float(config.get("tau", 0.6)),
|
||||||
max_reentries=int(config.get("max_reentries", 3)),
|
max_reentries=int(config.get("max_reentries", 3)),
|
||||||
trace=trace,
|
trace=trace,
|
||||||
|
codegen=codegen,
|
||||||
)
|
)
|
||||||
return scheduler, config
|
return scheduler, config
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,169 @@
|
|||||||
|
"""Skill code-body generation through an OpenAI-compatible API.
|
||||||
|
|
||||||
|
The decision engine authors a skill's title + description with the small model
|
||||||
|
(engine.generate). Writing the runnable body is a separate step: a larger
|
||||||
|
OpenAI-compatible model (e.g. qwen38-iq3s on ollama) is prompted with the
|
||||||
|
SKILL.md contract plus the request and the existing tree, and must reply with
|
||||||
|
valid Python implementing `predict` / `act`. Stdlib-only HTTP, mirroring
|
||||||
|
`llm.py`.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import ast
|
||||||
|
import json
|
||||||
|
import urllib.error
|
||||||
|
import urllib.request
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from .decisions import Request
|
||||||
|
from .skills import SkillDraft, tree_summary
|
||||||
|
|
||||||
|
DEFAULT_CONTRACT = Path(__file__).resolve().parent.parent / "SKILL.md"
|
||||||
|
|
||||||
|
|
||||||
|
class CodegenError(RuntimeError):
|
||||||
|
"""The codegen endpoint could not be reached."""
|
||||||
|
|
||||||
|
|
||||||
|
def read_skill_contract(path: str | None = None) -> str:
|
||||||
|
contract = Path(path) if path else DEFAULT_CONTRACT
|
||||||
|
if not contract.is_file():
|
||||||
|
raise CodegenError(f"skill contract not found: {contract}")
|
||||||
|
return contract.read_text()
|
||||||
|
|
||||||
|
|
||||||
|
class CodegenClient:
|
||||||
|
"""Minimal OpenAI-compatible chat client for writing skill bodies."""
|
||||||
|
|
||||||
|
def __init__(self, base_url: str, model: str, timeout: float = 600.0):
|
||||||
|
self.base_url = base_url.rstrip("/")
|
||||||
|
self.model = model
|
||||||
|
self.timeout = timeout
|
||||||
|
|
||||||
|
def chat(self, messages: list[dict], max_tokens: int = 2048, temperature: float = 0.0) -> str:
|
||||||
|
url = f"{self.base_url}/chat/completions"
|
||||||
|
body = json.dumps(
|
||||||
|
{
|
||||||
|
"model": self.model,
|
||||||
|
"messages": messages,
|
||||||
|
"temperature": temperature,
|
||||||
|
"max_tokens": max_tokens,
|
||||||
|
}
|
||||||
|
).encode("utf-8")
|
||||||
|
request = urllib.request.Request(
|
||||||
|
url, data=body, headers={"Content-Type": "application/json"}
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
with urllib.request.urlopen(request, timeout=self.timeout) as response:
|
||||||
|
payload = json.loads(response.read().decode("utf-8"))
|
||||||
|
except urllib.error.URLError as exc:
|
||||||
|
raise CodegenError(
|
||||||
|
f"codegen endpoint unreachable at {url}: {exc}. Is your local server running?"
|
||||||
|
) from exc
|
||||||
|
return payload["choices"][0]["message"]["content"]
|
||||||
|
|
||||||
|
|
||||||
|
def build_skill_body_prompt(
|
||||||
|
request: Request,
|
||||||
|
category: str,
|
||||||
|
draft: SkillDraft,
|
||||||
|
tree: dict,
|
||||||
|
contract: str,
|
||||||
|
) -> list[dict]:
|
||||||
|
"""Messages for the code-generation model.
|
||||||
|
|
||||||
|
The small model already chose the title + description; the big model only
|
||||||
|
writes the runnable body against the SKILL.md contract, informed by the
|
||||||
|
request, the category, and the existing skills so it avoids duplication.
|
||||||
|
"""
|
||||||
|
existing = ", ".join(s.name for s in tree.get(category, [])) or "(none)"
|
||||||
|
system = (
|
||||||
|
"You write runnable skill bodies for a local agent. The contract below "
|
||||||
|
"is authoritative: follow it exactly.\n\n"
|
||||||
|
f"{contract}"
|
||||||
|
)
|
||||||
|
user = (
|
||||||
|
f"Request: {request.text}\n"
|
||||||
|
f"Category: {category}\n"
|
||||||
|
f"Skill name: {draft.name}\n"
|
||||||
|
f"Skill description: {draft.description}\n"
|
||||||
|
f"Existing skills in this category: {existing}\n"
|
||||||
|
f"Existing categories:\n{tree_summary(tree)}\n"
|
||||||
|
"Write the Python module body now. Reply with ONLY valid Python code "
|
||||||
|
"defining `predict` and `act`. No prose, no markdown fences, no JSON."
|
||||||
|
)
|
||||||
|
return [
|
||||||
|
{"role": "system", "content": system},
|
||||||
|
{"role": "user", "content": user},
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def parse_skill_body(raw: str) -> str:
|
||||||
|
"""Extract and validate a Python skill body from the model's reply.
|
||||||
|
|
||||||
|
Accepts bare code, ```fenced``` code, or JSON {"code": "..."}. The body
|
||||||
|
must parse and must define module-level `predict` and `act` functions.
|
||||||
|
Returns the cleaned source. Raises ValueError otherwise.
|
||||||
|
"""
|
||||||
|
text = raw.strip()
|
||||||
|
if "code" in text[:400] and "{" in text and "}" in text:
|
||||||
|
start, end = text.find("{"), text.rfind("}")
|
||||||
|
try:
|
||||||
|
parsed = json.loads(text[start : end + 1])
|
||||||
|
candidate = parsed.get("code")
|
||||||
|
if isinstance(candidate, str):
|
||||||
|
text = candidate.strip()
|
||||||
|
except (ValueError, AttributeError):
|
||||||
|
pass
|
||||||
|
for fence in ("```python", "```py", "```"):
|
||||||
|
start = text.find(fence)
|
||||||
|
if start != -1:
|
||||||
|
text = text[start + len(fence):]
|
||||||
|
close = text.rfind("```")
|
||||||
|
if close != -1:
|
||||||
|
text = text[:close]
|
||||||
|
break
|
||||||
|
text = text.strip()
|
||||||
|
if not text:
|
||||||
|
raise ValueError("skill body is empty")
|
||||||
|
try:
|
||||||
|
module = ast.parse(text, filename="<generated>")
|
||||||
|
except SyntaxError as exc:
|
||||||
|
raise ValueError(f"skill body is not valid Python: {exc}") from exc
|
||||||
|
names = {node.name for node in module.body if isinstance(node, ast.FunctionDef)}
|
||||||
|
for required in ("predict", "act"):
|
||||||
|
if required not in names:
|
||||||
|
raise ValueError(f"skill body must define a module-level `{required}` function")
|
||||||
|
return text
|
||||||
|
|
||||||
|
|
||||||
|
def generate_skill_body(
|
||||||
|
client: CodegenClient,
|
||||||
|
request: Request,
|
||||||
|
category: str,
|
||||||
|
draft: SkillDraft,
|
||||||
|
tree: dict,
|
||||||
|
contract: str | None = None,
|
||||||
|
max_tokens: int = 2048,
|
||||||
|
) -> str:
|
||||||
|
"""Author a skill body with the big model; retries once on invalid output."""
|
||||||
|
contract_text = contract if contract is not None else read_skill_contract()
|
||||||
|
messages = build_skill_body_prompt(request, category, draft, tree, contract_text)
|
||||||
|
last_error: Exception | None = None
|
||||||
|
for attempt in range(2):
|
||||||
|
try:
|
||||||
|
raw = client.chat(messages, max_tokens=max_tokens)
|
||||||
|
return parse_skill_body(raw)
|
||||||
|
except ValueError as exc:
|
||||||
|
last_error = exc
|
||||||
|
messages = messages + [
|
||||||
|
{
|
||||||
|
"role": "user",
|
||||||
|
"content": (
|
||||||
|
f"That was rejected: {exc}. Reply with ONLY the Python code "
|
||||||
|
"now — no prose, no fences, no JSON."
|
||||||
|
),
|
||||||
|
}
|
||||||
|
]
|
||||||
|
raise ValueError(f"skill body rejected twice: {last_error}")
|
||||||
@@ -9,6 +9,7 @@ from __future__ import annotations
|
|||||||
|
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
|
|
||||||
|
from .codegen import CodegenClient, CodegenError, generate_skill_body
|
||||||
from .decisions import DecisionRequest, Option, Request
|
from .decisions import DecisionRequest, Option, Request
|
||||||
from .engine import SemIfEngine
|
from .engine import SemIfEngine
|
||||||
from .llm import LLMClient
|
from .llm import LLMClient
|
||||||
@@ -21,12 +22,15 @@ from .skills import (
|
|||||||
CreateCategory,
|
CreateCategory,
|
||||||
CreateSkill,
|
CreateSkill,
|
||||||
Skill,
|
Skill,
|
||||||
|
SkillBodyStore,
|
||||||
build_skills,
|
build_skills,
|
||||||
build_tree,
|
build_tree,
|
||||||
compose_state,
|
compose_state,
|
||||||
generate_category,
|
generate_category,
|
||||||
generate_skill,
|
generate_skill,
|
||||||
|
materialize_skill,
|
||||||
merge_registry,
|
merge_registry,
|
||||||
|
merge_skill_bodies,
|
||||||
navigate,
|
navigate,
|
||||||
)
|
)
|
||||||
from .trace import TraceLog
|
from .trace import TraceLog
|
||||||
@@ -55,6 +59,7 @@ class DispatchResult:
|
|||||||
summary: str
|
summary: str
|
||||||
skill: str | None = None
|
skill: str | None = None
|
||||||
decisions_logged: int = 0
|
decisions_logged: int = 0
|
||||||
|
body_written: bool = False
|
||||||
|
|
||||||
|
|
||||||
class Scheduler:
|
class Scheduler:
|
||||||
@@ -67,6 +72,7 @@ class Scheduler:
|
|||||||
tau: float = 0.6,
|
tau: float = 0.6,
|
||||||
max_reentries: int = 3,
|
max_reentries: int = 3,
|
||||||
trace: TraceLog | None = None,
|
trace: TraceLog | None = None,
|
||||||
|
codegen: CodegenClient | None = None,
|
||||||
):
|
):
|
||||||
self.engine = engine
|
self.engine = engine
|
||||||
self.llm = llm
|
self.llm = llm
|
||||||
@@ -75,6 +81,7 @@ class Scheduler:
|
|||||||
self.config = config
|
self.config = config
|
||||||
self.tau = tau
|
self.tau = tau
|
||||||
self.max_reentries = max_reentries
|
self.max_reentries = max_reentries
|
||||||
|
self.codegen = codegen
|
||||||
self.queue = UrgencyQueue(
|
self.queue = UrgencyQueue(
|
||||||
max_size=int(config.get("queue", {}).get("max_size", 100)),
|
max_size=int(config.get("queue", {}).get("max_size", 100)),
|
||||||
age_rate=float(config.get("queue", {}).get("age_rate", 0.0)),
|
age_rate=float(config.get("queue", {}).get("age_rate", 0.0)),
|
||||||
@@ -82,7 +89,9 @@ class Scheduler:
|
|||||||
self.skills = build_skills(config)
|
self.skills = build_skills(config)
|
||||||
self.tree = build_tree(self.skills)
|
self.tree = build_tree(self.skills)
|
||||||
self.registry = CategoryRegistry(config.get("category_registry", "data/categories.json"))
|
self.registry = CategoryRegistry(config.get("category_registry", "data/categories.json"))
|
||||||
|
self.body_store = SkillBodyStore(config.get("skill_bodies", "data/skills"))
|
||||||
merge_registry(self.tree, self.registry.read())
|
merge_registry(self.tree, self.registry.read())
|
||||||
|
merge_skill_bodies(self.tree, self.body_store, self.registry.read())
|
||||||
self.ctx = ActionContext(engine=self.engine, config=config)
|
self.ctx = ActionContext(engine=self.engine, config=config)
|
||||||
self.runner = SkillRunner(self.ctx, self.llm, self.log)
|
self.runner = SkillRunner(self.ctx, self.llm, self.log)
|
||||||
self.current: Process | None = None
|
self.current: Process | None = None
|
||||||
@@ -210,22 +219,32 @@ class Scheduler:
|
|||||||
|
|
||||||
# ---- dispatch ----
|
# ---- dispatch ----
|
||||||
|
|
||||||
def _dispatch(self, request: Request) -> DispatchResult:
|
def _dispatch(self, request: Request, _depth: int = 0) -> DispatchResult:
|
||||||
navigation = navigate(self.engine, self.log, self.trace, request, self.tree)
|
navigation = navigate(self.engine, self.log, self.trace, request, self.tree)
|
||||||
if isinstance(navigation, CreateCategory):
|
if isinstance(navigation, CreateCategory):
|
||||||
return self._create_category(request)
|
return self._create_category(request)
|
||||||
if isinstance(navigation, CreateSkill):
|
if isinstance(navigation, CreateSkill):
|
||||||
return self._create_skill(request, navigation.category)
|
created = self._create_skill(request, navigation.category)
|
||||||
outcome = self.runner.run(navigation, request)
|
if created.kind == "create_skill" and created.body_written and _depth < self.max_reentries:
|
||||||
|
requeued = _requeue(request, request.text)
|
||||||
|
self.trace.append(
|
||||||
|
"requeued", request.id, text=request.text, reason="skill created"
|
||||||
|
)
|
||||||
|
return self._dispatch(requeued, _depth=_depth + 1)
|
||||||
|
return created
|
||||||
|
return self._run_skill(navigation, request)
|
||||||
|
|
||||||
|
def _run_skill(self, skill: Skill, request: Request) -> DispatchResult:
|
||||||
|
outcome = self.runner.run(skill, request)
|
||||||
if outcome.error:
|
if outcome.error:
|
||||||
self.trace.append(
|
self.trace.append(
|
||||||
"error", request.id, skill=navigation.name, message=outcome.error
|
"error", request.id, skill=skill.name, message=outcome.error
|
||||||
)
|
)
|
||||||
return DispatchResult(kind="error", summary=f"skill error: {outcome.error}")
|
return DispatchResult(kind="error", summary=f"skill error: {outcome.error}")
|
||||||
self.trace.append(
|
self.trace.append(
|
||||||
"assessed",
|
"assessed",
|
||||||
request.id,
|
request.id,
|
||||||
skill=navigation.name,
|
skill=skill.name,
|
||||||
success=outcome.success,
|
success=outcome.success,
|
||||||
summary=outcome.summary,
|
summary=outcome.summary,
|
||||||
updated_request=outcome.updated_request,
|
updated_request=outcome.updated_request,
|
||||||
@@ -235,8 +254,8 @@ class Scheduler:
|
|||||||
self.trace.append("requeued", request.id, text=outcome.updated_request)
|
self.trace.append("requeued", request.id, text=outcome.updated_request)
|
||||||
return DispatchResult(
|
return DispatchResult(
|
||||||
kind="ran",
|
kind="ran",
|
||||||
summary=f"{navigation.name}: {'ok' if outcome.success else 'failed'} — {outcome.summary}",
|
summary=f"{skill.name}: {'ok' if outcome.success else 'failed'} — {outcome.summary}",
|
||||||
skill=navigation.name,
|
skill=skill.name,
|
||||||
decisions_logged=outcome.decisions_logged,
|
decisions_logged=outcome.decisions_logged,
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -275,7 +294,14 @@ class Scheduler:
|
|||||||
)
|
)
|
||||||
|
|
||||||
def _create_skill(self, request: Request, category: str) -> DispatchResult:
|
def _create_skill(self, request: Request, category: str) -> DispatchResult:
|
||||||
"""Author a new skill leaf stub with the decision model in generation mode."""
|
"""Author a new skill leaf with the decision model in generation mode.
|
||||||
|
|
||||||
|
The small model writes the title + description; a larger OpenAI-
|
||||||
|
compatible model then writes the runnable body against SKILL.md. The
|
||||||
|
stub is registered first so the leaf is navigable even if the body
|
||||||
|
write fails; a successful write is merged into the tree as a runnable
|
||||||
|
skill and the request re-dispatches to it.
|
||||||
|
"""
|
||||||
from .engine import EngineUnavailable
|
from .engine import EngineUnavailable
|
||||||
|
|
||||||
try:
|
try:
|
||||||
@@ -296,21 +322,78 @@ class Scheduler:
|
|||||||
kind="error",
|
kind="error",
|
||||||
summary=f"create_skill failed: {draft.name} already exists",
|
summary=f"create_skill failed: {draft.name} already exists",
|
||||||
)
|
)
|
||||||
|
|
||||||
self.registry.register_skill(category, draft.name, draft.description)
|
self.registry.register_skill(category, draft.name, draft.description)
|
||||||
self.tree.setdefault(category, []).append(
|
self.tree.setdefault(category, []).append(
|
||||||
Skill(name=draft.name, category=category, description=draft.description)
|
Skill(name=draft.name, category=category, description=draft.description)
|
||||||
)
|
)
|
||||||
|
self.trace.append(
|
||||||
|
"skill_writing",
|
||||||
|
request.id,
|
||||||
|
category=category,
|
||||||
|
skill=draft.name,
|
||||||
|
description=draft.description,
|
||||||
|
model=self.codegen.model if self.codegen else None,
|
||||||
|
)
|
||||||
|
|
||||||
|
if self.codegen is None:
|
||||||
|
self.trace.append(
|
||||||
|
"skill_created",
|
||||||
|
request.id,
|
||||||
|
category=category,
|
||||||
|
skill=draft.name,
|
||||||
|
description=draft.description,
|
||||||
|
body=None,
|
||||||
|
written=False,
|
||||||
|
)
|
||||||
|
return DispatchResult(
|
||||||
|
kind="create_skill",
|
||||||
|
summary=f"created stub {category}.{draft.name}: {draft.description} (no codegen configured)",
|
||||||
|
skill=draft.name,
|
||||||
|
)
|
||||||
|
|
||||||
|
try:
|
||||||
|
draft.code = generate_skill_body(
|
||||||
|
self.codegen, request, category, draft, self.tree
|
||||||
|
)
|
||||||
|
skill = materialize_skill(draft, category, self.body_store)
|
||||||
|
except (CodegenError, ValueError) as exc:
|
||||||
|
self.trace.append(
|
||||||
|
"error",
|
||||||
|
request.id,
|
||||||
|
phase="create_skill",
|
||||||
|
category=category,
|
||||||
|
message=f"skill body write failed: {exc}",
|
||||||
|
)
|
||||||
|
return DispatchResult(
|
||||||
|
kind="create_skill",
|
||||||
|
summary=f"created stub {category}.{draft.name}: {draft.description} (body write failed: {exc})",
|
||||||
|
skill=draft.name,
|
||||||
|
)
|
||||||
|
|
||||||
|
skills = self.tree.setdefault(category, [])
|
||||||
|
for index, existing in enumerate(skills):
|
||||||
|
if existing.name == draft.name:
|
||||||
|
skills[index] = skill
|
||||||
|
break
|
||||||
|
else:
|
||||||
|
skills.append(skill)
|
||||||
|
skills.sort(key=lambda s: s.name)
|
||||||
|
body_path = self.body_store.body_path(category, draft.name).as_posix()
|
||||||
self.trace.append(
|
self.trace.append(
|
||||||
"skill_created",
|
"skill_created",
|
||||||
request.id,
|
request.id,
|
||||||
category=category,
|
category=category,
|
||||||
skill=draft.name,
|
skill=draft.name,
|
||||||
description=draft.description,
|
description=draft.description,
|
||||||
|
body=body_path,
|
||||||
|
written=True,
|
||||||
)
|
)
|
||||||
return DispatchResult(
|
return DispatchResult(
|
||||||
kind="create_skill",
|
kind="create_skill",
|
||||||
summary=f"created skill {category}.{draft.name}: {draft.description}",
|
summary=f"created skill {category}.{draft.name}: {draft.description}",
|
||||||
skill=draft.name,
|
skill=draft.name,
|
||||||
|
body_written=True,
|
||||||
)
|
)
|
||||||
|
|
||||||
def status(self) -> str:
|
def status(self) -> str:
|
||||||
|
|||||||
@@ -12,6 +12,7 @@ Only the real skills live here; navigation uses the real decision engine.
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import importlib.util
|
||||||
import json
|
import json
|
||||||
import os
|
import os
|
||||||
import re
|
import re
|
||||||
@@ -95,6 +96,7 @@ class SkillDraft:
|
|||||||
|
|
||||||
name: str
|
name: str
|
||||||
description: str
|
description: str
|
||||||
|
code: str = ""
|
||||||
|
|
||||||
|
|
||||||
class CategoryRegistry:
|
class CategoryRegistry:
|
||||||
@@ -130,6 +132,113 @@ class CategoryRegistry:
|
|||||||
self.path.write_text(json.dumps(categories, indent=2) + "\n")
|
self.path.write_text(json.dumps(categories, indent=2) + "\n")
|
||||||
|
|
||||||
|
|
||||||
|
class SkillBodyStore:
|
||||||
|
"""Persists runnable skill bodies as one Python file per skill.
|
||||||
|
|
||||||
|
Layout: <base>/<category>/<name>.py. Bodies are written by the codegen step
|
||||||
|
and loaded back at startup so skills stay runnable across restarts.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, path: str = "data/skills"):
|
||||||
|
self.path = Path(path)
|
||||||
|
|
||||||
|
def write(self, category: str, name: str, code: str) -> Path:
|
||||||
|
directory = self.path / category
|
||||||
|
directory.mkdir(parents=True, exist_ok=True)
|
||||||
|
target = directory / f"{name}.py"
|
||||||
|
target.write_text(code.rstrip() + "\n")
|
||||||
|
return target
|
||||||
|
|
||||||
|
def body_path(self, category: str, name: str) -> Path:
|
||||||
|
return self.path / category / f"{name}.py"
|
||||||
|
|
||||||
|
def list_bodies(self) -> list[tuple[str, str]]:
|
||||||
|
if not self.path.is_dir():
|
||||||
|
return []
|
||||||
|
bodies = []
|
||||||
|
for directory in sorted(self.path.iterdir()):
|
||||||
|
if not directory.is_dir():
|
||||||
|
continue
|
||||||
|
for module in sorted(directory.glob("*.py")):
|
||||||
|
bodies.append((directory.name, module.stem))
|
||||||
|
return bodies
|
||||||
|
|
||||||
|
|
||||||
|
def load_skill_module(category: str, name: str, base: str = "data/skills"):
|
||||||
|
"""Import a persisted skill body and return its module."""
|
||||||
|
path = Path(base) / category / f"{name}.py"
|
||||||
|
module_name = f"_skill_{category}_{name}".replace("-", "_")
|
||||||
|
spec = importlib.util.spec_from_file_location(module_name, path)
|
||||||
|
if spec is None or spec.loader is None:
|
||||||
|
raise ValueError(f"cannot load skill module: {path}")
|
||||||
|
module = importlib.util.module_from_spec(spec)
|
||||||
|
spec.loader.exec_module(module)
|
||||||
|
return module
|
||||||
|
|
||||||
|
|
||||||
|
def materialize_skill(
|
||||||
|
draft: SkillDraft, category: str, store: SkillBodyStore
|
||||||
|
) -> Skill:
|
||||||
|
"""Persist the draft's code body and build a runnable Skill from it."""
|
||||||
|
if not draft.code:
|
||||||
|
raise ValueError(f"skill {draft.name} has no code body to materialize")
|
||||||
|
store.write(category, draft.name, draft.code)
|
||||||
|
try:
|
||||||
|
module = load_skill_module(category, draft.name, store.path)
|
||||||
|
except Exception as exc:
|
||||||
|
raise ValueError(f"skill {category}.{draft.name} body failed to import: {exc}") from exc
|
||||||
|
if not callable(getattr(module, "predict", None)) or not callable(
|
||||||
|
getattr(module, "act", None)
|
||||||
|
):
|
||||||
|
raise ValueError(f"skill {category}.{draft.name} body must define predict and act")
|
||||||
|
return Skill(
|
||||||
|
name=draft.name,
|
||||||
|
category=category,
|
||||||
|
description=draft.description,
|
||||||
|
predict=module.predict,
|
||||||
|
act=module.act,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def merge_skill_bodies(
|
||||||
|
tree: dict[str, list[Skill]], store: SkillBodyStore, registry: dict[str, dict]
|
||||||
|
) -> int:
|
||||||
|
"""Upgrade persisted skill bodies in the tree to runnable skills.
|
||||||
|
|
||||||
|
A body file makes a stub leaf executable; where the registry entry was lost
|
||||||
|
(or never written), the category is created and the description falls back
|
||||||
|
to the skill name. Returns the number of skills made runnable.
|
||||||
|
"""
|
||||||
|
upgraded = 0
|
||||||
|
for category, name in store.list_bodies():
|
||||||
|
description = ""
|
||||||
|
entry = registry.get(category, {})
|
||||||
|
for skill in entry.get("skills", []):
|
||||||
|
if skill.get("name") == name:
|
||||||
|
description = skill.get("description", "")
|
||||||
|
try:
|
||||||
|
module = load_skill_module(category, name, store.path)
|
||||||
|
except Exception:
|
||||||
|
continue
|
||||||
|
skill = Skill(
|
||||||
|
name=name,
|
||||||
|
category=category,
|
||||||
|
description=description or name,
|
||||||
|
predict=module.predict,
|
||||||
|
act=module.act,
|
||||||
|
)
|
||||||
|
skills = tree.setdefault(category, [])
|
||||||
|
for index, existing in enumerate(skills):
|
||||||
|
if existing.name == name:
|
||||||
|
skills[index] = skill
|
||||||
|
break
|
||||||
|
else:
|
||||||
|
skills.append(skill)
|
||||||
|
skills.sort(key=lambda s: s.name)
|
||||||
|
upgraded += 1
|
||||||
|
return upgraded
|
||||||
|
|
||||||
|
|
||||||
def compose_state(request: Request, current: str | None = None) -> str:
|
def compose_state(request: Request, current: str | None = None) -> str:
|
||||||
parts = [request.text]
|
parts = [request.text]
|
||||||
if current:
|
if current:
|
||||||
|
|||||||
@@ -121,6 +121,11 @@ function eventRow(evt) {
|
|||||||
div.textContent = `queued (${evt.label})`;
|
div.textContent = `queued (${evt.label})`;
|
||||||
} else if (evt.kind === "preempted") {
|
} else if (evt.kind === "preempted") {
|
||||||
div.textContent = `preempted ${evt.preempted}`;
|
div.textContent = `preempted ${evt.preempted}`;
|
||||||
|
} else if (evt.kind === "skill_writing") {
|
||||||
|
div.textContent = `writing skill ${evt.skill} — ${evt.description} (${evt.model || "codegen"})`;
|
||||||
|
} else if (evt.kind === "skill_created") {
|
||||||
|
div.textContent = `created ${evt.skill}${evt.written ? " (body written)" : " (stub)"}`;
|
||||||
|
div.title = evt.body || evt.description || "";
|
||||||
}
|
}
|
||||||
return div;
|
return div;
|
||||||
}
|
}
|
||||||
@@ -327,6 +332,36 @@ function eventNode(evt) {
|
|||||||
body.textContent = `interrupted ${evt.preempted}, requeued with state`;
|
body.textContent = `interrupted ${evt.preempted}, requeued with state`;
|
||||||
} else if (evt.kind === "dropped") {
|
} else if (evt.kind === "dropped") {
|
||||||
body.textContent = evt.reason || "";
|
body.textContent = evt.reason || "";
|
||||||
|
} else if (evt.kind === "skill_writing") {
|
||||||
|
node.classList.add("writing");
|
||||||
|
const title = document.createElement("div");
|
||||||
|
title.className = "skill-title";
|
||||||
|
title.textContent = evt.skill;
|
||||||
|
body.appendChild(title);
|
||||||
|
const desc = document.createElement("div");
|
||||||
|
desc.className = "muted";
|
||||||
|
desc.textContent = evt.description || "";
|
||||||
|
body.appendChild(desc);
|
||||||
|
const badge = document.createElement("span");
|
||||||
|
badge.className = "writing-badge";
|
||||||
|
badge.textContent = "writing skill body…";
|
||||||
|
node.appendChild(badge);
|
||||||
|
} else if (evt.kind === "skill_created") {
|
||||||
|
const title = document.createElement("div");
|
||||||
|
title.className = "skill-title";
|
||||||
|
title.textContent = evt.skill;
|
||||||
|
body.appendChild(title);
|
||||||
|
const desc = document.createElement("div");
|
||||||
|
desc.className = "muted";
|
||||||
|
desc.textContent = evt.description || "";
|
||||||
|
body.appendChild(desc);
|
||||||
|
if (evt.body) {
|
||||||
|
const p = document.createElement("div");
|
||||||
|
p.className = "muted";
|
||||||
|
p.textContent = `body: ${evt.body}`;
|
||||||
|
node.appendChild(p);
|
||||||
|
}
|
||||||
|
node.classList.add(evt.written ? "ok" : "stub");
|
||||||
} else {
|
} else {
|
||||||
body.textContent = evt.summary || evt.text || "";
|
body.textContent = evt.summary || evt.text || "";
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -283,6 +283,26 @@ select {
|
|||||||
|
|
||||||
.node.event-node .evt-kind { text-transform: uppercase; color: var(--muted); font-size: 10px; }
|
.node.event-node .evt-kind { text-transform: uppercase; color: var(--muted); font-size: 10px; }
|
||||||
|
|
||||||
|
.node.event-node .skill-title { font-weight: 700; }
|
||||||
|
.node.event-node.writing { border-left: 3px solid var(--warn); }
|
||||||
|
.node.event-node.stub { border-left: 3px solid var(--muted); }
|
||||||
|
|
||||||
|
.writing-badge {
|
||||||
|
display: inline-block;
|
||||||
|
margin-top: 8px;
|
||||||
|
padding: 2px 8px;
|
||||||
|
border: 1px solid var(--warn);
|
||||||
|
border-radius: 10px;
|
||||||
|
color: var(--warn);
|
||||||
|
font-size: 10px;
|
||||||
|
animation: writing-pulse 1.2s ease-in-out infinite;
|
||||||
|
}
|
||||||
|
|
||||||
|
@keyframes writing-pulse {
|
||||||
|
0%, 100% { opacity: 1; }
|
||||||
|
50% { opacity: 0.35; }
|
||||||
|
}
|
||||||
|
|
||||||
/* ---------- inspector / tree / status ---------- */
|
/* ---------- inspector / tree / status ---------- */
|
||||||
|
|
||||||
.right { border-right: none; }
|
.right { border-right: none; }
|
||||||
|
|||||||
@@ -12,10 +12,20 @@ from pathlib import Path
|
|||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
from semif_agent.cli import build_scheduler, load_config
|
from semif_agent.cli import build_scheduler, load_config
|
||||||
|
from semif_agent.codegen import CodegenClient, generate_skill_body
|
||||||
from semif_agent.decisions import Request
|
from semif_agent.decisions import Request
|
||||||
from semif_agent.dream import dream
|
from semif_agent.dream import dream
|
||||||
from semif_agent.engine import EngineUnavailable
|
from semif_agent.engine import EngineUnavailable
|
||||||
from semif_agent.skills import CategoryDraft, SkillDraft, generate_category, generate_skill
|
from semif_agent.skills import (
|
||||||
|
CategoryDraft,
|
||||||
|
SkillBodyStore,
|
||||||
|
SkillDraft,
|
||||||
|
build_skills,
|
||||||
|
build_tree,
|
||||||
|
generate_category,
|
||||||
|
generate_skill,
|
||||||
|
materialize_skill,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def require_real(config: dict):
|
def require_real(config: dict):
|
||||||
@@ -140,6 +150,38 @@ def test_generate_skill(tmp_path):
|
|||||||
assert draft.name and draft.description
|
assert draft.name and draft.description
|
||||||
|
|
||||||
|
|
||||||
|
def test_generate_skill_body_codegen(tmp_path):
|
||||||
|
"""A real OpenAI-compatible model writes a runnable skill body.
|
||||||
|
|
||||||
|
Slow: uses the big codegen model (qwen38-iq3s by default). Run this one in
|
||||||
|
the background and poll — long-lived ssh sessions get SIGHUP'd.
|
||||||
|
"""
|
||||||
|
config = load_config()
|
||||||
|
require_real(config)
|
||||||
|
codegen_cfg = config.get("codegen", {})
|
||||||
|
client = CodegenClient(
|
||||||
|
base_url=codegen_cfg.get("base_url", "http://localhost:11434/v1"),
|
||||||
|
model=codegen_cfg.get("model", "qwen38-iq3s"),
|
||||||
|
timeout=float(codegen_cfg.get("timeout", 600.0)),
|
||||||
|
)
|
||||||
|
tree = build_tree(build_skills({"skills": {}}))
|
||||||
|
draft = SkillDraft(
|
||||||
|
name="check_service",
|
||||||
|
description="Check whether a service is reachable.",
|
||||||
|
)
|
||||||
|
code = generate_skill_body(
|
||||||
|
client,
|
||||||
|
Request("is my home server reachable right now?"),
|
||||||
|
"tracking",
|
||||||
|
draft,
|
||||||
|
tree,
|
||||||
|
)
|
||||||
|
print(f"generated {len(code)} bytes of skill body")
|
||||||
|
store = SkillBodyStore(str(tmp_path / "skills"))
|
||||||
|
skill = materialize_skill(draft, "tracking", store)
|
||||||
|
assert callable(skill.predict) and callable(skill.act)
|
||||||
|
|
||||||
|
|
||||||
def test_create_skill_empty_category_does_not_wedge(tmp_path):
|
def test_create_skill_empty_category_does_not_wedge(tmp_path):
|
||||||
"""A dispatch that lands on an empty category must not leave the scheduler wedged.
|
"""A dispatch that lands on an empty category must not leave the scheduler wedged.
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,219 @@
|
|||||||
|
"""Pure-stdlib tests for skill code-body generation.
|
||||||
|
|
||||||
|
Prompt building, draft parsing/validation, body persistence + import, and
|
||||||
|
tree hot-merge all run without SemIf or a real LLM. The only network usage is a
|
||||||
|
throwaway stdlib HTTP server that stands in for an OpenAI-compatible endpoint —
|
||||||
|
the CodegenClient itself is real, not mocked.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import json
|
||||||
|
import threading
|
||||||
|
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from semif_agent.codegen import (
|
||||||
|
CodegenClient,
|
||||||
|
build_skill_body_prompt,
|
||||||
|
generate_skill_body,
|
||||||
|
parse_skill_body,
|
||||||
|
read_skill_contract,
|
||||||
|
)
|
||||||
|
from semif_agent.decisions import Request
|
||||||
|
from semif_agent.skills import (
|
||||||
|
SkillBodyStore,
|
||||||
|
SkillDraft,
|
||||||
|
build_skills,
|
||||||
|
build_tree,
|
||||||
|
load_skill_module,
|
||||||
|
materialize_skill,
|
||||||
|
merge_skill_bodies,
|
||||||
|
merge_registry,
|
||||||
|
)
|
||||||
|
|
||||||
|
GOOD_BODY = """\
|
||||||
|
from semif_agent.decisions import DecisionRequest, Option
|
||||||
|
from semif_agent.skills import ActionResult, Prediction
|
||||||
|
|
||||||
|
def predict(ctx, request):
|
||||||
|
return Prediction(text="ok", decisions=[])
|
||||||
|
|
||||||
|
def act(ctx, request, prediction):
|
||||||
|
return ActionResult(action_log="probe ran", new_state=request.text)
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
def test_read_skill_contract_loads_contract():
|
||||||
|
text = read_skill_contract()
|
||||||
|
assert "predict" in text and "act" in text
|
||||||
|
assert "data/skills" in text
|
||||||
|
|
||||||
|
|
||||||
|
def test_build_skill_body_prompt_includes_contract_request_and_draft():
|
||||||
|
tree = build_tree(build_skills({"skills": {}}))
|
||||||
|
draft = SkillDraft(name="probe", description="Probe the service.")
|
||||||
|
messages = build_skill_body_prompt(
|
||||||
|
Request("check if the service is up"), "tracking", draft, tree, "THE CONTRACT"
|
||||||
|
)
|
||||||
|
assert messages[0]["role"] == "system"
|
||||||
|
assert "THE CONTRACT" in messages[0]["content"]
|
||||||
|
joined = messages[1]["content"]
|
||||||
|
assert "check if the service is up" in joined
|
||||||
|
assert "probe" in joined
|
||||||
|
assert "tracking.check" in joined
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"raw",
|
||||||
|
[
|
||||||
|
GOOD_BODY,
|
||||||
|
"```python\n" + GOOD_BODY + "\n```",
|
||||||
|
json.dumps({"code": GOOD_BODY}),
|
||||||
|
"Here you go:\n```python\n" + GOOD_BODY + "\n```\nHope that helps.",
|
||||||
|
'Sure: ' + json.dumps({"code": GOOD_BODY}) + ' (that was it)',
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_parse_skill_body_accepts_forms(raw):
|
||||||
|
code = parse_skill_body(raw)
|
||||||
|
assert "def predict" in code and "def act" in code
|
||||||
|
|
||||||
|
|
||||||
|
def test_parse_skill_body_rejects_empty():
|
||||||
|
with pytest.raises(ValueError):
|
||||||
|
parse_skill_body("")
|
||||||
|
|
||||||
|
|
||||||
|
def test_parse_skill_body_rejects_invalid_python():
|
||||||
|
with pytest.raises(ValueError):
|
||||||
|
parse_skill_body("def predict(:\n pass")
|
||||||
|
|
||||||
|
|
||||||
|
def test_parse_skill_body_rejects_missing_functions():
|
||||||
|
with pytest.raises(ValueError):
|
||||||
|
parse_skill_body("def predict(ctx, request):\n return None")
|
||||||
|
|
||||||
|
|
||||||
|
def test_parse_skill_body_rejects_missing_act():
|
||||||
|
with pytest.raises(ValueError):
|
||||||
|
parse_skill_body("def predict(ctx, request):\n return None\nx = 1")
|
||||||
|
|
||||||
|
|
||||||
|
def test_body_store_roundtrip(tmp_path):
|
||||||
|
store = SkillBodyStore(str(tmp_path / "skills"))
|
||||||
|
assert store.list_bodies() == []
|
||||||
|
store.write("tracking", "probe", GOOD_BODY)
|
||||||
|
assert store.list_bodies() == [("tracking", "probe")]
|
||||||
|
target = store.body_path("tracking", "probe")
|
||||||
|
assert target.is_file()
|
||||||
|
assert "def predict" in target.read_text()
|
||||||
|
|
||||||
|
|
||||||
|
def test_load_skill_module_exposes_predict_act(tmp_path):
|
||||||
|
store = SkillBodyStore(str(tmp_path / "skills"))
|
||||||
|
store.write("tracking", "probe", GOOD_BODY)
|
||||||
|
module = load_skill_module("tracking", "probe", store.path)
|
||||||
|
assert callable(module.predict) and callable(module.act)
|
||||||
|
|
||||||
|
|
||||||
|
def test_materialize_skill_builds_runnable_skill(tmp_path):
|
||||||
|
store = SkillBodyStore(str(tmp_path / "skills"))
|
||||||
|
draft = SkillDraft(name="probe", description="Probe the service.", code=GOOD_BODY)
|
||||||
|
skill = materialize_skill(draft, "tracking", store)
|
||||||
|
assert skill.name == "probe"
|
||||||
|
assert skill.category == "tracking"
|
||||||
|
assert callable(skill.predict) and callable(skill.act)
|
||||||
|
|
||||||
|
|
||||||
|
def test_materialize_skill_requires_code(tmp_path):
|
||||||
|
store = SkillBodyStore(str(tmp_path / "skills"))
|
||||||
|
draft = SkillDraft(name="probe", description="Probe the service.")
|
||||||
|
with pytest.raises(ValueError):
|
||||||
|
materialize_skill(draft, "tracking", store)
|
||||||
|
|
||||||
|
|
||||||
|
def test_materialize_skill_rejects_import_failure(tmp_path):
|
||||||
|
store = SkillBodyStore(str(tmp_path / "skills"))
|
||||||
|
bad = "def predict(ctx, request):\n return None\n"
|
||||||
|
draft = SkillDraft(name="probe", description="Probe.", code=bad)
|
||||||
|
with pytest.raises(ValueError):
|
||||||
|
materialize_skill(draft, "tracking", store)
|
||||||
|
|
||||||
|
|
||||||
|
def test_merge_skill_bodies_upgrades_stub(tmp_path):
|
||||||
|
store = SkillBodyStore(str(tmp_path / "skills"))
|
||||||
|
store.write("tracking", "probe", GOOD_BODY)
|
||||||
|
tree = build_tree(build_skills({"skills": {}}))
|
||||||
|
registry = {"tracking": {"description": "", "skills": [{"name": "probe", "description": "Probe."}]}}
|
||||||
|
merge_registry(tree, registry)
|
||||||
|
upgraded = merge_skill_bodies(tree, store, registry)
|
||||||
|
assert upgraded == 1
|
||||||
|
skill = next(s for s in tree["tracking"] if s.name == "probe")
|
||||||
|
assert callable(skill.predict) and callable(skill.act)
|
||||||
|
assert skill.description == "Probe."
|
||||||
|
|
||||||
|
|
||||||
|
def test_merge_skill_bodies_creates_missing_category(tmp_path):
|
||||||
|
store = SkillBodyStore(str(tmp_path / "skills"))
|
||||||
|
store.write("brand_new", "ping", GOOD_BODY)
|
||||||
|
tree = build_tree(build_skills({"skills": {}}))
|
||||||
|
upgraded = merge_skill_bodies(tree, store, {})
|
||||||
|
assert upgraded == 1
|
||||||
|
assert tree["brand_new"][0].name == "ping"
|
||||||
|
|
||||||
|
|
||||||
|
class _FakeOpenAI(BaseHTTPRequestHandler):
|
||||||
|
reply: str = GOOD_BODY
|
||||||
|
|
||||||
|
def do_POST(self):
|
||||||
|
length = int(self.headers.get("Content-Length") or 0)
|
||||||
|
self.rfile.read(length)
|
||||||
|
body = json.dumps({"choices": [{"message": {"content": self.reply}}]}).encode("utf-8")
|
||||||
|
self.send_response(200)
|
||||||
|
self.send_header("Content-Type", "application/json")
|
||||||
|
self.send_header("Content-Length", str(len(body)))
|
||||||
|
self.end_headers()
|
||||||
|
self.wfile.write(body)
|
||||||
|
|
||||||
|
def log_message(self, format, *args):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def _fake_server(reply: str) -> tuple[ThreadingHTTPServer, str]:
|
||||||
|
handler = type("Handler", (_FakeOpenAI,), {"reply": reply})
|
||||||
|
httpd = ThreadingHTTPServer(("127.0.0.1", 0), handler)
|
||||||
|
threading.Thread(target=httpd.serve_forever, daemon=True).start()
|
||||||
|
return httpd, f"http://127.0.0.1:{httpd.server_address[1]}/v1"
|
||||||
|
|
||||||
|
|
||||||
|
def test_generate_skill_body_end_to_end(tmp_path):
|
||||||
|
httpd, base = _fake_server(GOOD_BODY)
|
||||||
|
try:
|
||||||
|
client = CodegenClient(base_url=base, model="test", timeout=10)
|
||||||
|
tree = build_tree(build_skills({"skills": {}}))
|
||||||
|
draft = SkillDraft(name="probe", description="Probe the service.")
|
||||||
|
code = generate_skill_body(client, Request("is the service up?"), "tracking", draft, tree)
|
||||||
|
assert "def predict" in code and "def act" in code
|
||||||
|
finally:
|
||||||
|
httpd.shutdown()
|
||||||
|
httpd.server_close()
|
||||||
|
|
||||||
|
|
||||||
|
def test_generate_skill_body_retries_then_fails(tmp_path):
|
||||||
|
httpd, base = _fake_server("this is not python at all")
|
||||||
|
try:
|
||||||
|
client = CodegenClient(base_url=base, model="test", timeout=10)
|
||||||
|
tree = build_tree(build_skills({"skills": {}}))
|
||||||
|
draft = SkillDraft(name="probe", description="Probe the service.")
|
||||||
|
with pytest.raises(ValueError):
|
||||||
|
generate_skill_body(client, Request("is the service up?"), "tracking", draft, tree)
|
||||||
|
finally:
|
||||||
|
httpd.shutdown()
|
||||||
|
httpd.server_close()
|
||||||
|
|
||||||
|
|
||||||
|
def test_codegen_client_unreachable_raises(tmp_path):
|
||||||
|
client = CodegenClient(base_url="http://127.0.0.1:1/v1", model="test", timeout=2)
|
||||||
|
tree = build_tree(build_skills({"skills": {}}))
|
||||||
|
draft = SkillDraft(name="probe", description="Probe the service.")
|
||||||
|
with pytest.raises(Exception):
|
||||||
|
generate_skill_body(client, Request("is the service up?"), "tracking", draft, tree)
|
||||||
@@ -146,3 +146,41 @@ def test_submit_trace_event_recorded_even_when_engine_missing(tmp_path):
|
|||||||
assert "submit" in kinds
|
assert "submit" in kinds
|
||||||
finally:
|
finally:
|
||||||
server.close()
|
server.close()
|
||||||
|
|
||||||
|
|
||||||
|
def test_skill_writing_and_created_events_in_payload(tmp_path):
|
||||||
|
scheduler = build_scheduler(tmp_path)
|
||||||
|
scheduler.trace.append("submit", "run-9", text="track my package")
|
||||||
|
scheduler.trace.append(
|
||||||
|
"skill_writing",
|
||||||
|
"run-9",
|
||||||
|
category="tracking",
|
||||||
|
skill="track_live",
|
||||||
|
description="Follow a package in real time.",
|
||||||
|
model="qwen38-iq3s",
|
||||||
|
)
|
||||||
|
scheduler.trace.append(
|
||||||
|
"skill_created",
|
||||||
|
"run-9",
|
||||||
|
category="tracking",
|
||||||
|
skill="track_live",
|
||||||
|
description="Follow a package in real time.",
|
||||||
|
body="data/skills/tracking/track_live.py",
|
||||||
|
written=True,
|
||||||
|
)
|
||||||
|
server = Server(scheduler)
|
||||||
|
try:
|
||||||
|
status, payload = server.get("/api/trace")
|
||||||
|
assert status == 200
|
||||||
|
run = next(r for r in payload["runs"] if r["run_id"] == "run-9")
|
||||||
|
kinds = [e["kind"] for e in run["events"]]
|
||||||
|
assert "skill_writing" in kinds and "skill_created" in kinds
|
||||||
|
writing = next(e for e in run["events"] if e["kind"] == "skill_writing")
|
||||||
|
assert writing["skill"] == "track_live"
|
||||||
|
assert "real time" in writing["description"]
|
||||||
|
assert writing["model"] == "qwen38-iq3s"
|
||||||
|
created = next(e for e in run["events"] if e["kind"] == "skill_created")
|
||||||
|
assert created["written"] is True
|
||||||
|
assert created["body"] == "data/skills/tracking/track_live.py"
|
||||||
|
finally:
|
||||||
|
server.close()
|
||||||
Reference in New Issue
Block a user