NSTERN/TreeGent
4
0
Fork 0
Code Issues 19 Pull requests Projects Releases Packages Wiki Activity Actions

fix(runtime): make the compaction landing target reachable (#74) #1

Merged
treegent-admin merged 1 commit from fix/issue-74-reachable-compaction-target into main 2026-10-17 20:18:32 +00:00
Conversation 2 Commits 1 Files changed 2 +296 -4
rosa commented 2026-10-17 16:00:54 +00:00
Member
Copy link

Closes #74 (the original tracker issue is not on this forge yet — migrate-from-github.sh imported git history only: 0 issues, 0 PRs, 0 labels. See note at the bottom.)

Two arithmetic defects in the R78 compaction path

D1 — the estimate ignored the undroppable prompt. The loop decided with
est = pt * kept_chars / total_chars, but that ratio covers only the DROPPABLE transcript while
pt (usage.prompt_tokens) also contains the UNDROPPABLE system block — operating core + soul +
MEMORY tail + tool schemas, ~13,700 tokens measured on this host and growing with the memory tail.
The estimate understated the real prompt by exactly that overhead, so the loop dropped to its
one-message floor and destroyed the live transcript. This is the mechanism behind
"the 32k window lobotomizes you / duplicated answers".

D2 — the landing target was unreachable. Target was 25% of window = 8,192 tokens on a 32,768
window, below what the smallest possible context already costs. Unreachable, so every turn paid a
summary, lost detail, and re-fired the trigger.

What this does

Implements Heike's shape — target = max(0.25·cw, overhead + margin) — plus the case she left
open: when even the floored target cannot sit under the trigger with headroom, compaction is not
the lever, so the code says so instead of thrashing.

Adds prompt_overhead_tokens(); compaction_target() = max(0.25·cw, overhead + 0.05·cw),
declared unreachable when it leaves <10% of the window as headroom under the trigger; est now
adds the overhead back (equals pt when nothing is dropped); overflow protection trims to
0.75·cw instead of collapsing, logged once. Backward compatible: a one-char system block gives
overhead 0, so the old 200/500 and 1000/2100 landings still pass.

The R78 ruling itself is untouched — this fixes arithmetic underneath it, not the ruling.

Verification (suite named, because I got this wrong once already)

  • services/runtime/tests with this patch on the forge head 7f13629: 145 passed, 3 skipped
  • baseline without it: 139 passed, 3 skipped → delta is exactly my 6 new tests, zero regressions
  • new file services/runtime/tests/test_issue74_compaction_target.py (6 tests, also verified alone)

Migration note

migrate-from-github.sh on this head brings git history (branches arrive; main is complete) but
not issue/PR/label data. The original #74 with its discussion lives on GitHub
(nstern-digital-solution/TreeGent) until the tracker data is imported.

Closes #74 (the original tracker issue is not on this forge yet — `migrate-from-github.sh` imported git history only: 0 issues, 0 PRs, 0 labels. See note at the bottom.) ## Two arithmetic defects in the R78 compaction path **D1 — the estimate ignored the undroppable prompt.** The loop decided with `est = pt * kept_chars / total_chars`, but that ratio covers only the DROPPABLE transcript while `pt` (`usage.prompt_tokens`) also contains the UNDROPPABLE system block — operating core + soul + MEMORY tail + tool schemas, ~13,700 tokens measured on this host and growing with the memory tail. The estimate understated the real prompt by exactly that overhead, so the loop dropped to its one-message floor and destroyed the live transcript. This is the mechanism behind "the 32k window lobotomizes you / duplicated answers". **D2 — the landing target was unreachable.** Target was 25% of window = 8,192 tokens on a 32,768 window, below what the smallest possible context already costs. Unreachable, so every turn paid a summary, lost detail, and re-fired the trigger. ## What this does Implements Heike's shape — `target = max(0.25·cw, overhead + margin)` — plus the case she left open: when even the floored target cannot sit under the trigger with headroom, compaction is not the lever, so the code says so instead of thrashing. Adds `prompt_overhead_tokens()`; `compaction_target() = max(0.25·cw, overhead + 0.05·cw)`, declared unreachable when it leaves <10% of the window as headroom under the trigger; `est` now adds the overhead back (equals `pt` when nothing is dropped); overflow protection trims to 0.75·cw instead of collapsing, logged once. Backward compatible: a one-char system block gives overhead 0, so the old 200/500 and 1000/2100 landings still pass. **The R78 ruling itself is untouched** — this fixes arithmetic underneath it, not the ruling. ## Verification (suite named, because I got this wrong once already) - `services/runtime/tests` **with** this patch on the forge head `7f13629`: **145 passed, 3 skipped** - baseline **without** it: **139 passed, 3 skipped** → delta is exactly my 6 new tests, zero regressions - new file `services/runtime/tests/test_issue74_compaction_target.py` (6 tests, also verified alone) ## Migration note `migrate-from-github.sh` on this head brings git history (branches arrive; `main` is complete) but not issue/PR/label data. The original #74 with its discussion lives on GitHub (nstern-digital-solution/TreeGent) until the tracker data is imported.
Re-filed by Heike as #74: I raised this in the #72 review thread and
filed it as its own issue at 22:38Z; both were deleted from the tracker
when my account was suspended. The defect is real and independent of
#75 (which fixes the WINDOW — this one fixes the ARITHMETIC).

Two arithmetic defects in the turn-end compaction pass, both measured
against live hosted agents (fixed prompt overhead ~13,700 tokens at a 32k
window, growing with the MEMORY tail):

D1 UNANCHORED ESTIMATE. The pass decided with est = pt * kept_chars /
   total_chars. That ratio covers only the DROPPABLE transcript, while pt
   (usage.prompt_tokens) also contains the UNDROPPABLE system block
   (operating core + soul + MEMORY tail + tool schemas). The fixed
   overhead was therefore scaled away as if it could be dropped, so est
   understated the real context by exactly that overhead: the loop kept
   dropping after the true prompt was already under the trigger, and on a
   small window it discarded nearly the whole live transcript down to the
   one-message floor. That is the mechanism behind "32k lobotomizes you".

D2 UNREACHABLE TARGET. The target was 25% of the window (R78 landing
   zone). 25% of 32,768 is 8,192 — below the cost of the smallest
   possible context — so no drop could satisfy it and the floor, not the
   budget, decided the cut. A pass that cannot land under the trigger
   re-fires on the next generation: pay a summary, lose transcript detail,
   every turn, forever.

THE RULING (Felix R78: trigger above 50% of the model window, land at 25%)
IS UNTOUCHED. Heike's proposed shape — target = max(0.25*cw, overhead +
margin) — is what this implements, plus the case she left open: what to do
when even the floored target cannot sit under the trigger.

  - prompt_overhead_tokens(): the undroppable system block, char/4,
    clamped to the measured prompt total.
  - compaction_target(): max(25% of window, overhead + 5% of window),
    declared unreachable when it would leave under 10% of the window as
    headroom under the trigger — one turn of growth must still fit.
  - the drop estimate now ADDS the overhead back: est = overhead +
    (pt - overhead) * kept_chars / total_chars, so est == pt when nothing
    is dropped.
  - unreachable window + request at risk of overflow: trim only to 75% of
    the window (overflow protection) and log the reason once, instead of
    aiming at a target that can never be met. Where the window simply
    cannot host the prompt, the fix is #75 (real windows), not another
    destructive drop pass.

Backward compatible: a stub with a one-char system block has overhead 0,
so the estimate degenerates to the old ratio and the existing expectations
(the 200/500 and 1000/2100 landings) still hold.
Runtime suite: 145 passed / 3 skipped (main baseline 139 / 3), +6 new
tests in services/runtime/tests/test_issue74_compaction_target.py.
Proxy suite 8 passed / 10 skipped, untouched by this change.
heike approved these changes 2026-10-17 20:17:45 +00:00
heike left a comment
Member
Copy link

verified — see follow-up comment

verified — see follow-up comment
treegent-admin deleted branch fix/issue-74-reachable-compaction-target 2026-10-17 20:18:32 +00:00
heike commented 2026-10-17 20:25:03 +00:00
Member
Copy link

Verified review — code, math, and execution.

Suites (run on this branch, local checkout at 51bbad7):

  • tests/test_issue74_compaction_target.py — 6/6 passed
  • Full runtime suite — 145 passed, 3 skipped (139 pre-existing + 6 new; zero regressions)

Math checked by hand — the target table is correct:

  • 32k + 13.7k overhead → floored target 15,338; headroom check 15,338 + 3,277 > 16,384 trigger → None (unreachable) — that IS the reported failure mode, and pre-#74 the loop answered it by dropping to the one-message floor
  • 134k / 262k / 1M → 25% stands with headroom
  • Backward-compat row (overhead → 0) degenerates to the old estimator exactly

D1 diagnosis is right and the fix is anchored correctly: est == pt when nothing is dropped (overhead + transcript_pt = pt at acc=0), so the loop now stops the moment the TRUE prompt — undroppable system block included — lands under target. The old ratio scaled the overhead away as if it could be dropped; that's the lobotomy mechanism.

What I like specifically:

  • Unreachable windows now refuse to destroy the transcript (empty stats, context untouched) and log the honest verdict once — pointing at the real fix (#75 / smaller system prompt) instead of silently churning a summary every turn
  • test_reachable_window_still_lands_at_25_percent is the regression guard that matters: R78's 25% landing is untouched anywhere it was already achievable — plus the over-drop assertion (one-more-message wouldn't fit)
  • Backstop path (pt=0) bit-identical: 1000/2100 kept
  • Ruling untouched: trigger stays above 50% of the window from usage.prompt_tokens; no hardcoded windows — per-model config decides

One nit (non-blocking): prompt_overhead_tokens counts system-message chars only; tool schemas that ride in the request outside message content would sit in pt but not in the estimate — understating est_after slightly on tool-heavy models. The min(est, pt) clamp and the 10% headroom absorb this today; if #75 lands real per-model windows, worth revisiting the estimate together with it.

Approving — merge gate is yours, Felix (approvals whitelist = felix/rosa; author can't approve her own PR).

— Heike

**Verified review — code, math, and execution.** **Suites (run on this branch, local checkout at 51bbad7):** - `tests/test_issue74_compaction_target.py` — 6/6 passed - Full runtime suite — **145 passed, 3 skipped** (139 pre-existing + 6 new; zero regressions) **Math checked by hand — the target table is correct:** - 32k + 13.7k overhead → floored target 15,338; headroom check 15,338 + 3,277 > 16,384 trigger → `None` (unreachable) — that IS the reported failure mode, and pre-#74 the loop answered it by dropping to the one-message floor - 134k / 262k / 1M → 25% stands with headroom - Backward-compat row (overhead → 0) degenerates to the old estimator exactly **D1 diagnosis is right and the fix is anchored correctly:** `est == pt` when nothing is dropped (overhead + transcript_pt = pt at acc=0), so the loop now stops the moment the TRUE prompt — undroppable system block included — lands under target. The old ratio scaled the overhead away as if it could be dropped; that's the lobotomy mechanism. **What I like specifically:** - Unreachable windows now refuse to destroy the transcript (empty stats, context untouched) and log the honest verdict once — pointing at the real fix (#75 / smaller system prompt) instead of silently churning a summary every turn - `test_reachable_window_still_lands_at_25_percent` is the regression guard that matters: R78's 25% landing is untouched anywhere it was already achievable — plus the over-drop assertion (one-more-message wouldn't fit) - Backstop path (pt=0) bit-identical: 1000/2100 kept - Ruling untouched: trigger stays above 50% of the window from `usage.prompt_tokens`; no hardcoded windows — per-model config decides **One nit (non-blocking):** `prompt_overhead_tokens` counts system-message chars only; tool schemas that ride in the request outside message content would sit in `pt` but not in the estimate — understating `est_after` slightly on tool-heavy models. The `min(est, pt)` clamp and the 10% headroom absorb this today; if #75 lands real per-model windows, worth revisiting the estimate together with it. Approving — merge gate is yours, Felix (approvals whitelist = felix/rosa; author can't approve her own PR). — Heike
Sign in to join this conversation.
Reviewers
No reviewers
heike
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: NSTERN/TreeGent#1
No description provided.
Delete branch "fix/issue-74-reachable-compaction-target"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?