Back to seenX
In development
Reference

The stack & architecture

SQLite stores the memory, vectors find meaning, and MCP connects assistants to the same engine.

Production target · implementation varies by layer

One engine, two interfaces

The native command bar and an authorized assistant retrieve the same records through one local engine. The Vercel website is a static explanation and sample preview; it is not the personal-memory service.

Chosen responsibilities and current implementation status
LayerTechnologyRole and status
WebsiteNext.js + React + TypeScriptWorking static preview on Vercel; separate from personal files.
Prototype enginePythonOCR, CLIP, hybrid rank fusion, and extraction caching exist; durable memory and service APIs are next.
Persistent memorySQLite + FTS5Planned catalog, exact text index, notes, source versions, permissions, and typed relationships.
Semantic retrievalLocal image/text vectorsExisting CLIP baseline; note embeddings and sqlite-vec evaluation are planned. Unrelated embedding spaces stay separate.
Native interfaceSwiftUI + AppKit + ScreenCaptureKit + VisionPlanned capture, command bar, source permissions, OCR boxes, and annotation UI.
Production coreRustDependency-free scaffold today; migrate measured behavior gradually.
Assistant accessMCPRead-only V1 plan using the shared engine; no native bridge runs on this website.

Why SQLite and vectors work together

SQLite is the durable catalog for image revisions, selected text, notes, region coordinates, sources, typed relationships, permissions, and jobs. FTS5 finds exact words. Local vector retrieval finds visually or semantically related evidence. A candidate extension such as sqlite-vec can keep vector search embedded; it is not a selected or installed production dependency today.

A lightweight ontology

Explicit types and links explain what each piece of context belongs to: an annotation refers to a region; that region belongs to an image revision; the image belongs to a collection or source. These links can live in SQLite without a separate graph database. Inference must remain a reviewable suggestion rather than a silent fact.

Native capture and annotation

SwiftUI/AppKit is the intended command-bar and capture-HUD interface. ScreenCaptureKit and Vision are capture/OCR candidates within supported macOS versions. Preserve image orientation and OCR coordinates. Browser URLs and originating-app details require a supported integration and permissions.

Python first, gradual Rust production

Python already supplies CLIP vectors, literal OCR/PDF search, rank fusion, and an extraction cache. It does not yet implement the persistent memory catalog or API. Stabilize real annotation and retrieval contracts in Python, connect native/MCP clients, then port measured behavior to Rust. The final desktop target does not require users to install Python.

Agent interface and evaluation harness

The read-only MCP adapter returns matching evidence and requested images/crops. The external client runs the agent; seenX supplies memory. A test harness evaluates whether a request finds the right image, region, quote, and note, including no-match and access-denied cases. A separate autonomous agent runtime is not needed for that.

Portable library

The V1 durability plan includes complete-library or context-only archives, stable identities and annotation coordinates, optional passphrase protection, and verified restore on another Mac. Credentials and machine-specific grants stay out of archives. Rebuild incompatible derived indexes instead of losing notes. Transfer and restore details →

Current availability

The website supports pretend screenshots, authored text boxes, spatial notes in browser storage, and JSON export. Rust is a scaffold and the native app is documentation only. Durable native storage, real text-box OCR, FTS5, semantic note embeddings, the local service, and MCP remain planned work.

Follow the ordered milestones →