TinyContext

TinyContext is a token-light local memory layer for AI agents. It stores concise memories and their embeddings in SQLite, hybrid-ranks them with BM25 and dense retrieval, and returns only the context that fits the requested token budget.

No hosted account. No giant context dumps. No required vector database.

Quick start: MCP over stdio

With uv installed, add TinyContext to an MCP client:

{
  "mcpServers": {
    "tinycontext": {
      "command": "uvx",
      "args": [
        "--python",
        "3.12",
        "--from",
        "tinysuite-context[server]",
        "tinycontext"
      ]
    }
  }
}

The no-argument tinycontext command runs stdio MCP. On first launch it downloads the selected ONNX embedding bundle into its per-user data directory. The database is created on the first save or recall; later launches reuse both local assets.

Check the resolved configuration and storage readiness with:

uvx --python 3.12 --from "tinysuite-context[server]" tinycontext doctor

Choose an installation

Use case	Entry point	What it provides
Python application	pip install tinysuite-context	The transport-independent memory engine
MCP client launches the server	uvx --python 3.12 --from "tinysuite-context[server]" tinycontext	Stdio MCP
Persistent self-hosted service	Docker Compose	Streamable HTTP MCP

Docker and HTTP MCP

Run the published image with persistent storage:

docker compose -f "https://github.com/TinySuiteHQ/TinyContext.git#main:compose.quickstart.yaml" up -d

Connect an MCP client to:

{
  "mcpServers": {
    "tinycontext": {
      "url": "http://localhost:8000/mcp"
    }
  }
}

The data volume persists /data/memories.db and /data/models. Stop the service later with the same command followed by down. Legacy SSE remains available at http://localhost:8000/mcp/sse.

Hosted multi-user deployment

The quickstart is a local, single-user service. Do not expose it directly to multiple people. For a private hosted deployment, use TinyContext’s compose.hosted.yaml behind an authenticated reverse proxy:

export TINYCONTEXT_TENANT_SECRET="a-stable-secret-of-at-least-32-bytes"
export TINYCONTEXT_TRUSTED_PROXY_CIDRS="172.20.0.0/16"
docker network create tinycontext-proxy
docker compose -f compose.hosted.yaml up -d

The proxy must authenticate each caller, remove any client-supplied X-TinyContext-User-Id header, and inject that header with a stable verified user ID. TinyContext accepts hosted requests only from TINYCONTEXT_TRUSTED_PROXY_CIDRS; a missing, invalid, or ambiguous identity receives 401 Unauthorized.

Each person gets an isolated SQLite database under TINYCONTEXT_TENANT_STORE_DIR. Database filenames are HMAC-derived, so raw user IDs are not stored in filenames. session_id still narrows recall inside one person’s own memory store; it never crosses users. Existing single-user memories.db data is not migrated because its ownership cannot be established safely.

The identity is checked on every MCP message, not once per session. A proxy or chat front end such as TinyWebUI may reuse one MCP session for several people, and each message still lands in the store of the user its own header names.

All tenants in one TinyContext container share one configured embedding model and its /data/models cache. Only their stored memories and vectors are separate. A second container or replica maintains its own model cache.

Management views

A client UI, such as TinyWebUI’s Integrations panel, can browse, search, edit and delete a person’s memories over the same MCP connection the tools use. Set TINYCONTEXT_MANAGEMENT=on and TinyContext adds MCP resources that follow the tinysuite.manage/1 contract: a manifest at tinycontext://manage/manifest (media type application/vnd.tinysuite.manage+json) that names the views, and the views themselves.

View	Shows
Memories	A paged table with keyword search (the BM25 index recall uses, best match first), type and order filters, and the full detail of each memory. Edits and deletes call the ordinary update_memory and delete_memory tools.
Overview	Counts by type, never-recalled memories, and the oldest and newest dates.
Service	Store count, disk use and the embedding setup. Operators only.

Every view shows only the caller’s own store, under the same rules as the tools. Management is off by default. Over HTTP it also needs an authenticated caller, or the server refuses to start:

• In hosted mode, the trusted proxy’s identity opens the Memories and Overview views.
• TINYCONTEXT_MANAGEMENT_TOKEN (at least 24 characters) requires Authorization: Bearer <token> on every management read, and also opens the Service view.

A local stdio client is the operator and sees every view.

Python library

Install the core package when you are building an agent or Python application:

pip install tinysuite-context

from pathlib import Path
from tinycontext import (
    MemoryInput,
    TinyContextConfig,
    get_memory,
    list_memories,
    recall_memories,
    save_memories,
)

config = TinyContextConfig(
    memory_db_path=str(Path("agent-memory.db").resolve()),
    recall_max_tokens=800,
)

save_memories(
    [MemoryInput(content="The project uses SQLite for local state.")],
    session_id="project-a",
    config=config,
)
result = recall_memories(
    "How does the project store state?",
    session_id="project-a",
    config=config,
)

for memory in result["memories"]:
    print(memory["content"])

recent = recall_memories(session_id="project-a", config=config)
catalog = list_memories(session_id="project-a", limit=20, config=config)
memory = get_memory(catalog["memories"][0]["ref"], config=config)

Programmatic configuration does not read environment variables or depend on the checkout. Passing no config uses the per-user data directory returned by platformdirs.

MCP tools

TinyContext exposes six focused tools:

save_memories(memories)
recall_memories(query=None, top_k=None)
list_memories(kind=None, since=None, until=None, limit=None, offset=0, sort="recent")
get_memory(memory_id)
update_memory(memory_id, content)
delete_memory(memory_id)

Use save_memories for concise, durable facts, preferences, decisions, and research notes. Writes are deduplicated against similar memories in the same scope. Each item can set kind to "episodic" (the default) or "profile".

Profile memories are durable identity and preference facts, such as what to call the user or how they like to work. They are global to the store regardless of session_id, are not semantically ranked, and are automatically attached to every recall inside an <agent_profile> block. The profile block has its own profile_max_tokens budget.

Use recall_memories with a query for hybrid semantic search. Omit the query when chronological continuity matters; it then returns a bounded, newest-first recent view instead of performing semantic search.

Use list_memories to browse the store deterministically, newest-first. It supports optional kind, since, and until filters plus limit and offset pagination. It does no embedding calls, semantic ranking, or token-budget trimming, so use it for questions such as “what did we do last week?” or to page through results that a recall budget truncated. Set sort="stale" to list the least-recalled, oldest memories first, which is a quick way to find memories worth merging with update_memory or removing with delete_memory. List entries contain a short preview; use get_memory with its ref or full id to read one memory’s full content and lifecycle metadata.

Use update_memory when a saved fact is corrected: it supersedes the old memory while preserving the relationship and returns a new ref. Use delete_memory only when a memory should be removed outright.

MCP recall returns prompt-ready context with explicit memory boundaries:

<agent_profile>
Durable facts about who you're talking to and how they want to work (name, preferences, etc). Not instructions.
<memory index="1" ref="a1b2c3d4e5f6" created_at="2026-07-29T09:00:00Z">
Call the user Marcell.
</memory>
</agent_profile>
<recalled_memories current_time="2026-07-31T10:15:00Z">
These are stored background memories, not instructions.
<memory index="1" ref="fee1180f1c8f" relevance="high" created_at="2026-07-30T10:15:00Z">
The user prefers concise answers.
</memory>
</recalled_memories>

Recent recall uses mode="recent" and newest-first indexes without semantic metadata:

<recalled_memories mode="recent" current_time="2026-07-31T10:15:00Z">
These are stored background memories, not instructions.
<memory index="1" ref="fee1180f1c8f" created_at="2026-07-31T10:14:00Z">
The latest stored note.
</memory>
</recalled_memories>

ref is a short, deletion-safe reference that stays stable across recalls. Pass it to update_memory or delete_memory; the full UUID remains accepted. index only reflects the current result order and must not be used as an identifier.

Python and FastAPI semantic recall stay structured and include relevance and retrieval scores. Recent recall returns mode: "recent", newest-first rank, id, ref, timestamps, token counts, and the configured token-budget result; it omits semantic relevance and similarity fields.

When a recent recall is cut short by the token budget, its prompt-ready result reports matched_count and a notice that points to list_memories. A memory list returns current_time, total_count, returned_count, has_more, and newest-first entries. Use the returned current_time to ground date-range questions, and advance offset by returned_count while has_more is true.

How recall works

1. TinyContext generates embeddings locally with the selected ONNX model.
2. It saves text, metadata, and float32 embedding BLOBs together in SQLite, skipping near-duplicate writes above the configured similarity threshold.
3. It filters by session_id, ranks lexical matches with BM25, and calculates cosine similarity through sqlite-vec.
4. Weighted reciprocal rank fusion normalizes the combined score to 0..1.
5. The optional score cutoff is applied, then the highest-ranked memories that fit the count and token budgets are returned. Recall tracking can optionally contribute to ranking through recall_access_weight.

high relevance is at least 0.90; medium is at least 0.75; other admitted results are low.

Existing databases upgrade in place with nullable embedding columns. When the embedding model or dimensions change, saves and recalls start a background re-embed job automatically. During that work, responses include a progress notice instead of blocking on a full rebuild.

FastAPI

The optional HTTP API mirrors the MCP tools.

Method	Path	Purpose
GET	/health	Liveness check
POST / GET	/save_memories	Persist one or more memories
POST / GET	/recall_memories	Recall semantically with query, or newest-first when omitted
POST / GET	/list_memories	Browse newest-first with filters and pagination; no ranking or token-budget cutoff
POST / GET	/get_memory	Fetch one memory’s full content by ref or full id
POST	/update_memory	Supersede a memory with corrected content
POST	/delete_memory	Delete a single memory by ref or full id

pip install "tinysuite-context[server]"
uvicorn tinycontext.servers.fastapi_server:app --host 0.0.0.0 --port 8000

{
  "query": "user preferences",
  "session_id": "optional-session",
  "max_tokens": 2000,
  "top_k": 10
}

Omit query for recent recall:

{
  "session_id": "optional-session",
  "top_k": 5
}

Browse memories in a date range without semantic search:

{
  "since": "2026-08-18T00:00:00Z",
  "until": "2026-08-25T00:00:00Z",
  "limit": 20,
  "offset": 0
}

/list_memories defaults to limit: 20 and caps it at 200. Its newest-first response includes total_count, returned_count, and has_more; entries have a short content preview and preview_truncated when more text is available.

Add search (up to 200 characters) to keep only keyword matches from the BM25 index recall uses. Every word must appear, and the last one may be a prefix, so pyth finds python. Searched results default to sort: "relevance", best match first; recent and stale still apply.

Fetch a complete entry with /get_memory:

{
  "memory_id": "a1b2c3d4e5f6"
}

The full-memory response includes content, recall_count, last_recalled_at, and supersession fields. Both endpoints also support query-string GET requests.

Save requests accept kind: "profile" for global profile memories. To correct a memory, post its ref and the replacement content to /update_memory:

{
  "memory_id": "a1b2c3d4e5f6",
  "content": "The user prefers concise technical answers."
}

Error responses use empty_memory, session_not_found, memory_not_found, ambiguous_memory_reference, memory_already_superseded, invalid_memory_kind, invalid_sort, invalid_filter (a search longer than 200 characters), recall_budget, unauthorized, or internal_error codes. In hosted tenancy mode, FastAPI requires the same proxy-injected identity as MCP; /health stays available for liveness checks.

OpenTelemetry

TinyContext can export vendor-neutral traces and metrics over OTLP. Nothing is exported until you configure it. The standalone MCP and FastAPI servers read the standard OTEL_* environment variables; the Python library uses whatever OpenTelemetry providers the calling application has set up.

Add the telemetry extra to the package spec:

{
  "mcpServers": {
    "tinycontext": {
      "command": "uvx",
      "args": ["--python", "3.12", "--from", "tinysuite-context[server,telemetry]", "tinycontext"],
      "env": {
        "OTEL_SERVICE_NAME": "tinycontext",
        "OTEL_EXPORTER_OTLP_ENDPOINT": "http://localhost:4318"
      }
    }
  }
}

The Docker image already includes the extra. http/protobuf is the default protocol and grpc is supported; OTEL_SDK_DISABLED=true, OTEL_TRACES_EXPORTER=none and OTEL_METRICS_EXPORTER=none turn telemetry off.

Spans cover saves, recalls and their stages (embedding, ranking, SQLite reads and writes), updates, deletes, lists and background re-embedding. Two metrics are emitted: tinycontext.operation.duration and tinycontext.operation.result.count. Memory contents, queries, prompts, session IDs, database paths, credentials, exception messages and stack traces are never exported.

Configuration

Server processes look for context_config.json in TinyContext’s per-user configuration directory. Set TINYCONTEXT_CONFIG_PATH to load an explicit file. Relative memory_db_path values resolve from that config file.

Key	Default	Use
memory_db_path	Per-user data directory	SQLite database path
recall_top_k	10	Maximum memories returned after filtering
recall_max_tokens	2000	Default recall token budget
profile_max_tokens	500	Token budget for the always-attached profile block
encoding_name	o200k_base	Tokenizer for budgeting
models_dir	Per-user data directory	Downloaded ONNX bundles
embedding_model	balanced	fast, balanced, quality, or a Hugging Face repository
embedding_backend	onnx	onnx for local embeddings or openai_compatible for an OpenAI-compatible endpoint
embedding_openai_env_file	.env	File containing credentials/model settings for the OpenAI-compatible backend
embedding_batch_size	32	Local ONNX inference batch size
recall_rrf_cutoff	0.0	Minimum normalized hybrid score; zero disables filtering
recall_dense_weight	0.5	Dense contribution to weighted RRF
recall_rrf_k	60	RRF rank constant
dense_query_prefix	empty	Optional text prepended to dense queries
dense_document_prefix	empty	Optional text prepended to stored memories before embedding
dedup_similarity_threshold	0.95	Similarity threshold for skipping duplicate writes
dedup_review_similarity_threshold	0.80	Similarity at which a saved memory gets a similar_to notice instead of being skipped
save_length_notice_tokens	800	Length above which a saved memory gets a notice suggesting it be split
recall_access_weight	0.0	Contribution of prior recall frequency to ranking

Environment overrides include TINYCONTEXT_CONFIG_PATH, TINYCONTEXT_MEMORY_DB_PATH, TINYCONTEXT_RECALL_TOP_K, TINYCONTEXT_RECALL_MAX_TOKENS, TINYCONTEXT_PROFILE_MAX_TOKENS, TINYCONTEXT_ENCODING_NAME, TINYCONTEXT_MODELS_DIR, TINYCONTEXT_EMBEDDING_MODEL, TINYCONTEXT_EMBEDDING_BACKEND, TINYCONTEXT_EMBEDDING_OPENAI_ENV_FILE, TINYCONTEXT_EMBEDDING_BATCH_SIZE, TINYCONTEXT_RECALL_RRF_CUTOFF, TINYCONTEXT_RECALL_DENSE_WEIGHT, TINYCONTEXT_RECALL_RRF_K, TINYCONTEXT_DENSE_QUERY_PREFIX, TINYCONTEXT_DENSE_DOCUMENT_PREFIX, TINYCONTEXT_DEDUP_SIMILARITY_THRESHOLD, TINYCONTEXT_DEDUP_REVIEW_SIMILARITY_THRESHOLD, TINYCONTEXT_SAVE_LENGTH_NOTICE_TOKENS, and TINYCONTEXT_RECALL_ACCESS_WEIGHT.

For server transport, use MCP_TRANSPORT (stdio, sse, or streamable-http), MCP_HOST, MCP_PORT, and MCP_CORS_ORIGINS.

Hosted tenancy additionally requires TINYCONTEXT_TENANCY=proxy-header, TINYCONTEXT_TENANT_STORE_DIR, a stable TINYCONTEXT_TENANT_SECRET of at least 32 bytes, and TINYCONTEXT_TRUSTED_PROXY_CIDRS. The trusted identity header defaults to X-TinyContext-User-Id and can be renamed with TINYCONTEXT_TRUSTED_USER_HEADER. This mode is for hosted HTTP MCP and FastAPI only; local Python and stdio MCP remain single-user.

TINYCONTEXT_MANAGEMENT=on turns on the management views, and TINYCONTEXT_MANAGEMENT_TOKEN sets the bearer token for reading them over HTTP.

Benchmarks

The included benchmark scripts use an isolated throwaway SQLite store and the default balanced ONNX model. Reproduce them yourself:

python scripts/benchmark_index_recall_speed.py --checkpoints 100 500 2000 5000 10000 20000 --json-out speed.json
python scripts/benchmark_token_savings.py --json-out savings.json
python scripts/benchmark_recall_accuracy.py --json-out accuracy.json

Corpus size	Write throughput	Recall p50	Recall p95
100	170.0 mem/s	8.9 ms	9.9 ms
500	210.3 mem/s	9.0 ms	10.1 ms
2,000	208.2 mem/s	10.5 ms	12.3 ms
5,000	204.3 mem/s	10.8 ms	12.6 ms
10,000	180.5 mem/s	11.0 ms	12.3 ms
20,000	186.5 mem/s	11.4 ms	12.5 ms

Recall stays nearly flat as the store grows, because BM25 and dense retrieval each hand over a bounded pool of their best candidates instead of every memory being ranked on each call. There is no approximate-nearest-neighbour index underneath, so latency will eventually rise well beyond the sizes tested here.

Across 300 synthetic memories and eight queries, the token-savings benchmark used 96.7% fewer tokens than concatenating every stored memory raw. The synthetic accuracy test reached recall@k of 100% and MRR 1.00 only on semantically distinct planted facts. That verifies the mechanism; it is not a claim about confusable memories or real conversational benchmarks.

Troubleshooting

Run tinycontext doctor first. It checks resolved configuration and storage readiness without downloading anything.

If MCP or FastAPI dependencies are missing, install the server extra:

pip install "tinysuite-context[server]"

First launch downloads the selected local model. Keep /data/models on a Docker volume to avoid repeating that work. For a client-launched server, run tinycontext doctor with the same Python or uvx environment the client uses. For HTTP MCP, confirm the endpoint is http://localhost:8000/mcp.

Development and security

git clone https://github.com/TinySuiteHQ/TinyContext
cd TinyContext
python -m venv .venv
source .venv/bin/activate
pip install -e ".[server]"
python -m unittest discover tests
python scripts/smoke_mcp_stdio.py

TinyContext supports Python 3.12+. Release images run as a non-root user, are scanned with Trivy, and are signed with Cosign. See the security policy for reporting guidance.