Per-tenant knowledge

In plain language

A client's documents belong to the client. They are stamped with the client's identity at intake, retrievable only by that client's Solutions, shareable only by an explicit recorded grant, and erasable on offboarding with proof of what was removed. Messy material — PDFs, decks, spreadsheets, transcripts — enters through a governed pipeline that extracts, normalises, deduplicates and stages it for operator approval before it becomes retrievable, with spend ceilings so ingestion cannot run away.

The mechanism

The knowledge invariants (ADR-0010, per-tenant knowledge design).

INV-KNOW-01 — Owned. Every knowledge artifact carries owningTenantId, stamped at intake, immutable; chunks and vectors inherit it at write time.
INV-KNOW-02 — Isolated. A run serving tenant T retrieves only from sources T owns or has been granted. Both retrieval entry points take a mandatory serving-tenant argument; the vector filter is owningTenantId ∈ (servingTenant ∪ granted) composed with the source filter. Absent or unknown tenant → refuse, never default.
INV-KNOW-03 — Private by default; sharing always explicit. A KnowledgeGrant is a record (platformSourceId, capabilityId), admin-mutated and ledgered. Client-owned sources are ungrantable — no code path exists.
INV-KNOW-04 — Tenant-relative binding. A knowledge component binds to well-known source names resolved per tenant, so one authored Capability serves many clients against each client's own corpus.
INV-KNOW-05 — Erasable. Offboarding provably removes all of a tenant's knowledge.
INV-KNOW-06 — Audited. Every retrieval's provenance row carries the serving tenant, the owning tenant(s) and the grants exercised.
INV-KNOW-07 — Extraction is delegated and attributed.
INV-KNOW-08 — Sources are permanent identities.

Fail-closed on both paths. The buffered lane computes the allowed source set from pure authority (computeAllowedSourceIds) and refuses outside it; an empty allowed set is an honest empty answer, never a leak. The streamed lane resolves the same way. Leak rigs run in CI as executable invariants: tenant A's corpus plus tenant B's run yields zero cross-retrieval, with and without grants; revocation takes effect on the next invoke; hostile grants and foreign binds are refused.

The Normalizer (ADR-0029). Governed ingestion is knowledge-substrate machinery, not a construction layer. Input arrives by presigned S3 PUT to tenant-prefixed, KMS-encrypted keys (never bytes through our API); an extraction router sends text and markdown through passthrough, PDFs and images to Amazon Textract (page and bounding-box anchors), DOCX and CSV to pure-JS extractors; the normalised ExtractedText artifact is persisted beside the chunks so retrieval can cite pages; content-hash idempotence makes a duplicate upload a no-op and per-document supersedure means a refresh never leaves competing copies. Orchestration is a Step Functions job with a dead-letter queue and per-step idempotency; terminal status is derived from the vector write. Extracted formats become retrievable only after operator approval of the normalised preview.

Spend ceilings. A per-document Textract ceiling of $1.00 and a per-tenant daily ingestion ceiling of $10.00 are enforced as the last gate before a job starts; the refusal names the ceiling and the remedy (retry after 00:00 UTC, split the corpus, or an operator-approved exception). Textract pages and embedding tokens are metered into ingestion cost records. Large PDFs (50 MB) are streamed, not buffered.

Offboarding erases in a fail-closed order. The offboard-client operation (dry-run by default, refuse-before-write) erases fifteen categories in an order that severs access first — grants, then entitlements, then vectors, chunk objects, source blobs, the knowledge records leaf to root, Desk deliverables / turns / sessions / configs, workflow run snapshots, memberships (users disabled) — so an interruption mid-run always leaves the corpus non-retrievable. What is deliberately retained is stated in the runbook and covered by the client contract: the append-only trust ledger, cost records, immutable published registries, disabled user accounts and the retired Solution record. "If a client requires ledger content itself to be expunged, that is a legal/contract decision, not an operator action."

Conversational memory rides the same substrate (ADR-0030). A Desk's durable engagement memory is documents in a per-Desk memory source owned by the client's tenant — inheriting staged review, the document browser, retrieval with provenance, census and erasure with zero new machinery. A working window and a versioned session digest handle the live session; fresher Knowledge outranks Memory, with the conflict disclosed.

Design notes

Why the platform tenant is an ordinary tenant (ADR-0010, ADR-0036). A default pool is where isolation quietly fails. Making the platform's own corpus a first-class owner whose use is a recorded grant means the first grant exercised in production was the platform's, under the same rule as everyone else's.
Why no client corpus before erasure existed. Ingesting confidential material is responsible only when there is a verified path to delete it; the erasure tool was built and rehearsed in production before the first client corpus.
Why staged review (ADR-0029). Extraction is lossy and messy; nothing becomes retrievable — or durable memory — without a governed approval of what was actually extracted.

Sources

matterandgas-com/docs/architecture/per-tenant-knowledge-design-v1.md — INV-KNOW-01…08.
matterandgas-com/docs/adr/0010, 0029, 0030.
matterandgas-com/amplify/functions/knowledge-ingest/dispatch.ts — the ceilings.
matterandgas-com/docs/runbooks/offboard-a-client.md, knowledge-ingestion-recovery.md.
matterandgas-com/tests/unit/ — the leak rigs.