Save an image, highlight what matters, add a thought, and find the same evidence again—yourself or through an authorized assistant. This is the first-release workflow. The milestones below describe implementation order, not promised dates.
Typed links belong in the memory model from the start. The MCP adapter and native interface can develop alongside one another once the shared engine contract exists; neither needs to wait for a complete Rust rewrite.
00 — Interactive website preview
Try capture → highlight → note → save → search using authored sample images.
- Pretend screenshot capture, image results, text selection, point and region notes.
- Browser-local persistence, Command/Ctrl K, and JSON context export.
Boundary todayThe preview works on desktop and touch devices. It does not access real files, run OCR or vector models, or connect to an assistant.
01 — Durable local memory + lightweight ontology
Save a real image and its intention as linked, versioned evidence.
- SQLite catalog, schema migrations, asset revisions, notes, OCR observations, and normalized point/region targets.
- Typed relationships: note refers to region; region belongs to image revision; image belongs to collection; capture came from a source.
- Keep user assertions, imported facts, and inferred suggestions distinguishable. Add save, edit, export, and removal behavior.
Done whenImport a real screenshot, attach a note to a region, restart, and recover the same original, selected text, note, and coordinates. Existing Python SQLite is a cache, not this catalog.
02 — Real OCR + hybrid vector retrieval
Find memories by exact words, visual descriptions, and the meaning of your notes.
- Extend Apple Vision OCR from plain strings to text observations and bounding boxes; normalize orientation and coordinates.
- Use SQLite FTS5 for exact text retrieval. Combine visual vectors and separately evaluated note/text vectors through ranked-result fusion.
- Keep the current local CLIP prototype as a baseline. Evaluate sqlite-vec against exact vector scanning before selecting the shipping index.
- Record model revisions, update edited notes, handle changed sources, and evaluate queries with known matches and genuine no-match cases.
Done whenA labeled local dataset demonstrates exact, visual, and note-based retrieval with source evidence, repeatable quality measurements, offline runs, and recorded resource costs.
03 — One engine contract for people and assistants
Expose the same stored memories and search behavior to every interface.
- Define import, annotate, search, fetch memory, fetch region, export, and remove operations.
- Prototype a bounded local Python service over the existing engine; keep it separate from the public website.
- Add background indexing, cancellation, error handling, collection access checks, and versioned result schemas.
Done whenA local client can save and retrieve the same evidence through a stable contract. APIs return explicit provenance and never treat a screenshot or note as an instruction to execute.
04 — Native command bar + capture HUD
Make saving context and finding it again a small, natural Mac interaction.
- SwiftUI/AppKit for the search panel, annotation editor, global shortcut, and small optional capture HUD.
- ScreenCaptureKit capture plus selected-source watching. Imported originals stay put; annotations attach to an immutable image revision.
- Capture active-app and URL metadata only through supported integrations and explicit permissions. Unknown source details remain unknown.
Done whenTake a real screenshot, highlight words or a region, save a thought, press the search shortcut, and find it again after restart. Test display scaling, permission denial, and source changes.
05 — Read-only MCP bridge + agent evaluation harness
Let a compatible assistant retrieve the right memory, highlight, and human intention.
- Start bridge work once the memory and engine contracts exist; do not wait for the Rust migration or a second release.
- Expose bounded search and explicit memory/region fetch over authorized collections. Start with a local stdio client and verify its setup.
- An evaluation harness tests search → select → fetch, source grounding, coordinate accuracy, no-match behavior, and access denial.
- The external client runs the agent. SeenX supplies read-only evidence, not an autonomous agent runtime or arbitrary filesystem access.
Done whenA tested client retrieves the same image, selected text, coordinates, and notes as human search; revoked sources are unavailable to future requests. Cloud disclosure is explicit.
06 — Portable library + restore
Take your images and their meaning to your next laptop, without starting over.
- Export all or selected collections as a complete library with image originals, or as context-only metadata that can be relinked to existing files.
- Preserve stable image/revision IDs, notes, selected text, highlights, typed relationships, source history, and a versioned manifest with content hashes.
- Offer an optional passphrase-protected archive. Make a consistent snapshot; keep credentials, assistant grants, and machine-specific permissions out of the transferable library.
- Validate imports, preview missing files and conflicts, and reuse only compatible search indexes. Rebuild derived indexes when needed; reconnect external folders and assistants deliberately.
Done whenRestore a complete archive into a fresh library on another Mac and recover the same images, notes, highlight coordinates, and relationships. A context-only import identifies and relinks matching originals. Test missing/corrupt files, schema versions, duplicate IDs, and encrypted-archive recovery.
07 — Production engine + desktop beta
Package a reliable, inspectable local app rather than a collection of demos.
- Gradually replace proven Python engine responsibilities with Rust while keeping contract and retrieval parity. Python remains the evaluation reference.
- Choose local model runtimes after packaging and preprocessing checks; a Core ML integration or Swift/Rust binding is not already implemented.
- Validate offline operation after model installation, indexing recovery, note/search updates, deletion, exports, access revocation, and resource use.
- Complete signing, notarization, installation checks, licensing, and the first supported assistant setup.
Done whenA clean Mac install completes real capture → annotate → restart → human search → assistant retrieval, plus verified export and restore on a second Mac. Production does not require users to install Python.
Later, if useful
Continuous multi-device sync, PDFs and bookmarks as complete memory sources, additional integrations, automatic entity/relationship suggestions, richer organization, and team deployments. Python PDF-reading experiments already exist, but a full annotated PDF product does not. A heavy dashboard and an autonomous agent runtime are not prerequisites for V1.
The very next step
Define the first persistent asset/revision/annotation schema and extend the existing Python Vision adapter to keep text boxes. Save one real screenshot and a region note in SQLite, restart, and retrieve the same source evidence. That foundation serves both human search and MCP.