PG_MEMPROJ: The Agent's Working Memory

We gave agents perfect documents. We gave them a mirror. Now we need to give them a working memory that fits.

The Trilogy So Far

Two extensions down, one critical gap remaining.

PG_FACETS solved retrieval. Hybrid BM25 plus embedding search, Roaring Bitmap multi-dimensional filtering, all colocated inside PostgreSQL — giving agents the right documents in milliseconds without a separate search infrastructure. The question "which documents?" is answered.

PG_DGRAPH solved structure and self-knowledge. Directed graphs of knowledge nodes, competency coverage scores, gap detection against domain ontologies — giving agents a traversable model of what they know, what they can do, and where their blind spots are. The question "am I equipped to reason about what I just found?" is answered.

Both extensions exist in production (internally this spring 2026). And yet agents built on top of them still routinely fail at the last step before the model ever sees a single token. Not because they retrieved the wrong documents. Not because they lack a self-model. But because the bridge between having the right knowledge and feeding the right knowledge to the model, at the right cost, in the right order does not exist.

That bridge is pg_memproj.

The Token Budget Problem

A Bounded Resource in an Unbounded Problem Space

Large language models are fundamentally token-bounded. Every turn has a cost. Every word injected into the context window is a deliberate choice — or should be. In practice, most agent architectures treat context assembly as an afterthought: retrieve documents, dump them into the prompt, hope the model figures out what matters.

This works in demos. It fails in production.

At scale, a compliance agent handling a GDPR audit might have access to hundreds of relevant facets, a knowledge graph with thousands of edges, and a session history spanning dozens of turns. The total available context is measured in megabytes. The model's window is measured in kilobytes. Something has to decide what gets in — and that decision, made naively, introduces noise, inflates cost, and confuses the model with competing signals.

The problem compounds in multi-step workflows. An agent executing a twelve-step compliance checklist doesn't need the full ontology at step nine. It needs the current step, its prerequisites, the constraints that apply to it, and the two or three facts most relevant to the query at hand. Everything else is noise at a cost.

Two Bad Choices — and One Structural Answer

Without a projection layer, an agent building its own context faces two equally bad options.

The first option is to inject everything available. Full document text, raw graph dumps, complete session history. Token cost explodes. The model receives noise alongside signal. Reasoning quality degrades not from lack of information but from excess of it. This is the context pollution problem.

The second option is to recompute context from canonical sources on every turn — re-running retrieval, re-traversing the graph, re-assembling the relevant facts. Latency climbs. Redundant computation accumulates. And the output is brittle: if source data changes mid-session, the agent may receive inconsistent context across consecutive turns.

The structural answer is a projection layer that sits between canonical sources and consumption: materialize a read-optimized, query-shaped view from authoritative data, maintain it incrementally, and serve it at O(1) cost with provenance attached.

That is exactly what pg_memproj is.

What pg_memproj Is — And What It Isn't

This distinction matters enough to state plainly before going further.

pg_memproj is not a fourth database. It is not a separate product. It does not own data. It runs inside the same PostgreSQL instance as pg_facets and pg_dgraph, sharing the same transaction log, the same operational surface, the same ACID guarantees.

pg_memproj is a projection layer. Its job is to take canonical facts from pg_facets and canonical relations from pg_dgraph, transform them into a format optimized for LLM consumption, and serve that format efficiently. When sources change, projections are invalidated. The inverse is never true — pg_memproj has no write path back to canonical stores.

The role separation is enforced by four normative rules that govern the entire memory stack:

• R1: pg_dgraph is authoritative for global relations between entities. pg_memproj never writes graph edges.
• R2: pg_memproj may mirror graph relations as DSL lines for context packs, but provenance — a source_ref pointing to the canonical UUID — is mandatory on every line.
• R3: pg_memproj must never become a second writer for the same logical edge. One source of truth per relation, always.
• R4: When both graph context and memproj propositions appear in the same prompt, the architecture enforces clear layering. Two competing sources of truth in one context window is a failure mode, not a feature.

These rules are the difference between a well-designed memory architecture and a system that eventually contradicts itself.

The Complete Memory Stack

The three extensions occupy mutually exclusive roles, together forming a complete epistemic stack:

The design principle is explicit: do not duplicate. Each layer does one job. pg_facets supplies truth in document form. pg_dgraph supplies truth in relational form. pg_memproj supplies neither — it supplies delivery, shaped for the model's consumption constraints.

The Proposition DSL — Technical Foundation

Why a DSL at All?

The central technical claim of pg_memproj is that agents reason better on a stable, line-oriented, typed surface than on raw JSON blobs or graph dumps.

This is worth examining from first principles. A model receiving a raw JSON object of 4KB does not fail to parse it — it parses it fine. The issue is different. The JSON carries structural noise: brackets, quotation marks, nested keys, field names that repeat across every object.

None of that is reasoning content. It consumes tokens. It provides no ranking signal. There is no way to say "this fact matters more than that fact for this query" inside a raw JSON dump — the model has to infer it or ignore it.

A graph dump is worse for context injection. It is complete and traversable — exactly the properties that make it valuable for pg_dgraph — but those properties become liabilities when you need to inject twenty relevant propositions into a 2,000-token budget. A graph dump does not rank. It does not summarize. It is a data structure, not a communication medium.

The Proposition DSL solves this with a line-oriented format where each line is a self-contained, typed, rankable unit:

textFACT: agent:42 has active_sessions=3
GOAL: project:compliance-audit reach status=COMPLETE by 2026-04-01
STEP: validate all GDPR article-49 clauses before submitting
CONSTRAINT: token_budget_remaining < 2000 → switch to compact mode

Each line carries four things: a type (FACT, GOAL, STEP, CONSTRAINT), a content statement in plain language, a source_ref pointing to the canonical UUID in pg_facets or pg_dgraph, and a ranking weight computed at projection time. The line is the unit of selection — you can filter by type, sort by weight, and inject exactly the top N most relevant propositions for a given query, with surgical precision.

Projectors — The Write Side

Projectors are the writers of the pg_memproj system. They are event-driven processes that run after canonical writes and transform source data into DSL lines stored in memory_projections.

Three projectors cover the primary surfaces:

OntologyProjector fires after a successful Persister.Apply on the facet store. It reads ontology artifacts and registry entries — which live as canonical facets — and emits FACT: lines representing their key propositions. An 800-node GDPR ontology becomes a set of ranked FACT: lines. An agent querying for "GDPR data transfer obligations" gets back the 15 most relevant ones, not all 800.

PlanProjector fires after a plan task update. It reads the current state of a multi-step workflow and emits GOAL: and STEP: lines with their completion status. The agent tracking a 12-step compliance checklist at step 9 receives the current step, its dependencies, and any CONSTRAINT: lines that apply — not the entire plan.

SnapshotProjector fires on SessionStart and Heartbeat events. It captures the agent's current operational state and emits it as a structured projection, ready for the monitoring pattern described below.

The write path is strictly unidirectional:

textpg_facets ──┐
├── Projectors ──→ memory_projections ──→ Consumers
pg_dgraph ──┘

No consumer — not the agent, not the CLI, not the prompt assembler — writes back through this chain. Canonical sources are never modified by downstream consumers.

memproj_pack_context — The Read Side

The primary consumption API is a single SQL function:

sqlSELECT pack_text
FROM memproj_pack_context(
user_id := 'agent:42',
query := 'GDPR data transfer obligations',
limit := 20
);

This returns a pre-ranked, pre-formatted block of DSL text — typically 80 to 200 bytes — ready to inject directly into a context window. No reconstruction. No parsing. No intermediate representation. The ranking considers semantic relevance to the query, recency, type weights, and the agent's current user_id scope (workspace, project, or session).

The result is not a document. It is a curated reasoning surface.

Three Actors, Three Jobs

The Reasoning Agent

The reasoning agent's job is: "Give me ontology and plan state in a small, ranked bundle for this intent."

Without pg_memproj, this requires the agent to call pg_facets for documents, traverse pg_dgraph for structural context, assemble the results, deduplicate, rank, and format — on every turn. With pg_memproj, it calls PackContext once and receives a pre-assembled bundle already shaped for the current query.

For structured traversal, the ontology_navigate tool can attach DSL propositions directly to graph traversal results — the agent gets both the relation structure (from pg_dgraph) and the pre-projected propositions (from pg_memproj) in a single coordinated response. Graph as structure, memproj as retrieval-shaped memory.

The Supervisor — Cockpit Without Log Streaming

The supervisor's job is: "Give me a real-time snapshot of system health without reading raw event logs."

The dashboard pattern integrates all three extensions into a monitoring architecture with negligible token cost.

Step 1 — Metrics as facets. The orchestrator's existing hook points

(SessionStart, Pre/PostToolUse, Heartbeat) update key-value facets in pg_facets at O(1) cost:

textagent:42:active_sessions → 3
agent:42:avg_latency_ms → 124
project:compliance:state → RUNNING
global:token_budget → 8423

Step 2 — Graph as discovery layer. Rather than hard-coding every facet key, pg_dgraph stores agent → has_facet → <facet_key> edges. The agent asks the graph "give me all facets attached to me" and receives a compact list. No knowledge of facet naming conventions required.

Step 3 — Memproj materializes the summary.

sqlSELECT json_data
FROM memoproj_agent_status
WHERE agent_id = 'agent:42';
-- → { "active": true, "latency_ms": 124, "tokens_left": 8423, "state": "IDLE" }

Approximately 80 bytes. No log streaming. No event replay. No aggregation pipeline. A single read returns a pre-computed JSON blob that the agent parses locally and acts on — throttle if latency exceeds 500ms, pause if project state is ERROR, switch to compact mode if token budget drops below threshold.

The MFO layer adds a higher-level health flag — project:compliance:health → GREEN / YELLOW / RED — updated when the orchestrator detects anomalies like consecutive failed tool calls. This flag appears in the same projection, alongside per-agent metrics, in one read.

The Human Operator

The operator's job is: "Debug what the pack would inject for user X and query Q."

MindCLI exposes three read-only commands:

• memproj inspect — shows exactly which propositions would be injected for a given user_id and query, with their weights and source references
• memproj project — lists all active projections scoped to a workspace, project, or session
• memproj stats — reports bitmap cardinality and projection volumes

These commands answer the question every AI system architect eventually asks: "Why did the agent say that?" Not by reading logs and reconstructing context after the fact, but by deterministically reproducing the exact ranked bundle the agent received. The reasoning becomes auditable before it happens, not only after it goes wrong.

Challenging the Premise

"This is just a materialized view. PostgreSQL already has those."

PostgreSQL materialized views are batch-refreshed, monolithic, and carry no per-row provenance. When a single source fact changes, the entire view must be refreshed — you cannot invalidate a specific set of DSL lines because you cannot trace which lines derived from which source row. pg_memproj attaches source_ref — a canonical UUID — to every DSL line individually. When a facet is updated, only the projections derived from that specific UUID are invalidated. The rest remain valid and cached. This is incremental invalidation with provenance, not a database view.

"The model handles JSON fine. Why a proprietary DSL?"

It does handle JSON. The issue is not parsing — it is selection, ranking, and token efficiency. A 4KB JSON blob injected into a 2,000-token budget consumes roughly 60% of it and arrives as an unranked flat structure. The model receives no signal about which fields matter for the current query. The Proposition DSL is designed as a pre-ranked selection medium: each line is an independent unit with a weight, a type, and a provenance. PackContext selects the top N by relevance before injection. The model receives exactly what it needs — not everything that exists.

"PG_DGRAPH already has structural context. Why mirror it into memproj?"

Rule R3 exists precisely to prevent the naive answer to this question. pg_memproj does not duplicate graph edges. It mirrors them in read-only DSL form for LLM consumption, with mandatory provenance. The distinction: pg_dgraph is authoritative for the relation A → B. It can be traversed, updated, and queried. pg_memproj can express that relation as FACT: A requires B for a prompt pack — but it does not own that relation. If pg_dgraph updates or deletes the edge, the corresponding DSL line is invalidated. One source of truth. Two consumption surfaces.

"A smart agent should be able to derive all this context itself from canonical sources."

That is exactly the anti-pattern pg_memproj exists to eliminate. Deriving context from canonical sources on every turn means re-running retrieval, re-traversing the graph, re-ranking results — hundreds of milliseconds of latency and thousands of tokens of overhead, turn after turn. The intelligence of an agent should be applied to reasoning about context, not reconstructing it. Pre-computed, incrementally maintained, O(1)-readable projections move that cost out of the reasoning loop entirely.

Use Cases

Ontology Injection Without the Tax

A compliance agent must reason about a GDPR ontology covering 800 concepts — articles, obligations, enforcement bodies, cross-reference dependencies. Without memproj, every relevant turn requires a partial graph traversal and document retrieval pass costing hundreds of milliseconds and several thousand tokens. With the OntologyProjector running continuously, the agent calls PackContext("GDPR data transfer obligations", limit=15) and receives 15 ranked FACT: lines in under 5 milliseconds. The reasoning window is clean. The cost is negligible.

Multi-Step Plan Tracking at Scale

A regulatory audit workflow spans twelve sequential steps with dependencies, constraints, and conditional branches. The PlanProjector maintains a live projection of the current plan state. At step 9, the agent calls PackContext and receives STEP: validate article-49 clauses, the two CONSTRAINT: lines that apply to this step, and the GOAL: for the overall project — not the entire twelve-step plan. Completed steps sink to the bottom of the ranking. Blocking constraints surface at the top. The agent's context reflects exactly where it is in the workflow.

Agent Self-Regulation in Real Time

An agent executing a token-heavy analysis task monitors its own operational state before each expensive action. The SnapshotProjector maintains memoproj_agent_status continuously. The agent reads its snapshot — 80 bytes — before calling a tool: if latency_ms > 500, it throttles parallel tool calls. If token_budget_remaining < 2000, it switches to compact mode. If project:state = ERROR, it pauses and escalates with a structured reason. These decisions require no log analysis, no event replay, no supervisor intervention. The agent governs itself from a pre-materialized readout.

Multi-Agent Competency-Aware Routing with Context Handoff

In a multi-agent system, when pg_dgraph routes a task to the agent with the highest coverage score for the required knowledge path, pg_memproj handles the context handoff.

The receiving agent does not need to rebuild context from scratch — the projections relevant to the task are already materialized and scoped to the task's project_id. The handoff is a PackContext call, not a full state transfer.

Auditable Reasoning for Regulated Environments

A financial services firm deploys an agent to assist with loan underwriting. A decision is challenged by a regulator.

The compliance team runs memproj inspect agent:underwriting "debt-to-income threshold applicability" and retrieves the exact ranked propositions the agent was injected with at decision time — their weights, their types, their source UUIDs pointing back to the canonical regulation facets and graph edges that generated them.

The reasoning chain is traceable from model output back to source data without reconstructing logs. This is not post-hoc debugging — it is architectural auditability.

The Complete Stack

Every component now has a defined role. No overlaps, no gaps.

textIncoming Agent Task
│
▼
[PG_DGRAPH]
"Am I equipped? What are my knowledge gaps?"
"Which entities and relations are relevant?"
"What is the shortest path to the required competency?"
│
├── Gap node detected → escalate with structured reason
│
▼ (equipped, or partial coverage confirmed)
[PG_FACETS]
"Which canonical documents match this task?"
"Hybrid BM25 + embedding retrieval, facet-filtered"
│
▼
[PG_MEMPROJ — Projectors]
Canonical sources → Proposition DSL
FACT: / GOAL: / STEP: / CONSTRAINT:
Ranked by relevance, provenanced to source UUID
Incrementally maintained, not recomputed
│
▼
memproj_pack_context(user_id, query, limit)
→ 80–200 bytes, pre-ranked, injection-ready
│
▼
LLM Context Window
Compact, ranked, traceable
│
▼
Response / Action / Structured Escalation

The progression is deliberate: find → structure → compress → reason.

PG_FACETS answers what exists. PG_DGRAPH answers how it relates and whether the agent is equipped. PG_MEMPROJ answers what the model actually needs to know right now, at what cost, with what proof of origin.

The Trilogy Is Complete

Three extensions. One PostgreSQL instance. No external systems.

The analogy to human cognition is not decorative. Working memory in cognitive science is the system that holds and manipulates a small amount of information currently relevant to the task at hand — drawing from long-term memory, structured by schema, bounded by capacity.

The failure mode when working memory is bypassed or overloaded is not ignorance. It is noise-induced error: the right knowledge is available, but the system cannot surface the relevant subset when it is needed.

This is the exact failure mode pg_memproj is built to prevent.

An agent equipped with all three extensions does not search in noise. It does not drown in its own context. It knows what it knows (pg_facets), understands the structure of that knowledge and its own limits (pg_dgraph), and receives exactly what it needs to reason — compact, ranked, and traceable back to source (pg_memproj).

The architecture does not make agents infallible. It makes their failures explicit, bounded, and recoverable. It makes their reasoning auditable. It makes their costs predictable. And it does all of this inside a single PostgreSQL instance that your infrastructure already runs.

PG_MEMPROJ is in active development as part of the MindFlight Open Source stack, alongside PG_FACETS and PG_DGRAPH. If you are building multi-agent systems that need to operate reliably in production — especially in regulated, high-stakes, or token-constrained environments — reach out for early access.