Memory & Data Audit — 2026-04-27

Status: snapshot of the VPS at the time of audit · mindx.pythai.net / 168.231.126.58 · disk 96 G used, 700 MB free (100 %).

Scope: every subdirectory under /home/mindx/mindX/data/ — what writes it, what reads it, what prunes it (if anything), and where it sits in the cognitive-ascent chain.

Why: the disk filled twice in 24 hours despite freeing 5 GB. The naive read is "delete more." The honest read is the memory pipeline mindX has is partly built and partly imagined, and the gap between the two is what's filling the disk.

Where the bytes are

/home/mindx/mindX/data
├── memory/stm/                                   30 G    ← bulk
│   └── bdi_agent_mastermind_strategy_mastermind_prime/   28 G    ← ONE AGENT
├── memory/agent_workspaces/                      21 G    ← ORPHAN tier (read by nobody)
│   └── mastermind_prime/                         20 G    ← SAME AGENT
├── memory/ltm/                                   35 M    ← consolidated knowledge
├── memory/archive/                              103 M    ← already-pruned-but-never-deleted
├── memory/dreams/                               1.3 M    ← dream reports
├── logs/                                         87 M    ← rotatable
├── ceo_backup/                                   54 M    ← versioned
├── system_state/                                 56 M    ← snapshots
├── improvement_backlog.json                      16 M    ← audit trail
└── (all others)                                  <5 M
                                                  ────
                                                  ~52 G  in two paths owned by ONE agent (mastermind)

184 815 JSON files are ≥ 14 days old. Only 7 are ≥ 30 days old. The 30-day prune threshold currently in memory_agent.prune_stm() is the wrong cliff: by the time STM hits 30 d it's already off the dream cycle's analysis window. The IPFS-offload eligibility (14 d) is the right cliff; offload just isn't running because no Lighthouse / nft.storage keys are vaulted.

Per-agent concentration

The disk problem is not 30 GB spread across 19 agents — it's two paths owned by one agent:

Whatever fix lands needs to bind the mastermind tightly. The other 18 agents would not have triggered this audit on their own.

The cognitive-ascent map

The three gaps

Gap 1 — The orphan tier (21 GB)

agent_workspaces/{agent}/process_traces/process_trace.jsonl is written on every call to memory_agent.log_process() and read by nothing in the codebase. The grep is empty: no consumer, no pruner, no rotation. It just grows. The mastermind workspace alone holds 20 GB of these.

Either the consumer was planned but never written, or it was a debug instrumentation that became permanent. Either way it's the silent disk killer.

Gap 2 — IPFS offload dormant (would drain 30 GB)

agents/storage/offload_projector.py is fully wired. Phase 8 of the dream cycle calls it. The eligibility predicate (agents/storage/eligibility.py) correctly identifies any STM dir ≥ 14 d old and ≥ 5 GB cumulative. It would gzip-pack each (agent, date), push to IPFS via Lighthouse + nft.storage, verify sha256, mark the pgvector row with content_cid + offload_tx_hash, and (with dry_run=False) delete the local copy.

It doesn't run because data/config/vault_bankon/ doesn't have lighthouse_api_key or nftstorage_api_key. Phase 8 logs cold-tier: no IPFS keys configured — skipping every dream cycle.

This is the single highest-leverage fix for the disk problem: vault one Lighthouse free-tier key (https://lighthouse.storage; free up to 5 GB; ~5 minutes to obtain) and the dream cycle starts draining ≥ 14 d STM automatically.

Gap 3 — Wisdom tier write-only (the loop is open)

The training-data export shipped earlier today produces _training.jsonl files alongside the dream insights — finetuning-ready chat-completion rows. Each dream cycle writes ~91 such rows. They're correctly persisted at the right path with the right shape.

Nothing reads them. No fine-tune pipeline, no perception loop ingestion, no THOT mint of verified rows, no embed-into-pgvector for retrieval. The substrate is correct; the consumer is missing.

The cognitive ascent stops at file write. The system dreams but doesn't teach itself from dreams.

Retention recommendations (per tier)

Recommended actions, in order of leverage

1. Vault one Lighthouse free-tier API key. Phase 8 of dream cycle starts running. ~30 GB drains over a few cycles. 5-minute operator action; no code change.
2. Tier-aware prune_stm — change the 30 d single-threshold to per-tier (Phase 3 of the implementation plan). 14 d → archive instead of 30 d gives Phase 8 something to push.
3. Orphan pruner — rotate process_trace.jsonl files at 100 MB; archive at 30 d; delete at 90 d. Frees the 21 GB without losing recent traces.
4. /insight/memory/audit endpoint — replaces ad-hoc SSH du commands with a live computed view. Operator-facing.
5. Wisdom-tier consumer (agents/cognition/wisdom_loader.py) — read _training.jsonl, embed, store with cognition_tier='wisdom', surface to BDI perceive() so plans get a "RELEVANT WISDOM" preamble. Closes the cognitive-ascent loop.

What this audit does not change

Audit conducted 2026-04-27. Updates to this file should be appended (not in-place edits) so the historical view is preserved.

ssh root@168.231.126.58 \
  'rm -f /home/mindx/mindX/data/memory/archive/bdi_agent_mastermind_strategy_mastermind_prime/workspace_traces/process_trace.1.20260427.jsonl.gz \
        /home/mindx/mindX/data/memory/agent_workspaces/bdi_agent_mastermind_strategy_mastermind_prime/process_trace.1.jsonl'

disk:           80 % (was 100 %) · 21 GB free
freed:          ~19.5 GB (the mastermind orphan)
mastermind STM: 26.8 GB (unchanged; <14 d age)
archive:        1.3 GB (175k files from Stage 1)
mindX:          active throughout

Appendix added 2026-04-27 ~21:00 UTC. The audit document remains the canonical record of the gap analysis; the appendix names what was acted on.*