System · architecture
Three lanes: index time, storage, query time.
skygrep index or
skygrep watch. Storage is the only shared state between the two
lanes. Query time runs per call to skygrep search.
0.6.0 batches chunks across file boundaries before embedding them. Full,
incremental, explicit-file, and watch indexing share the same bounded batch
contract (SKYGREP_INDEX_BATCH_SIZE=64, clamped to
1..512) while preserving deterministic file/chunk order. Watch
mode also refreshes file embeddings and removes stale chunks when a file
becomes empty.
System · recall
0.5.13 adds bounded candidate recall before final ranking.
The intent router ranks retrieval lanes, but it is no longer the only
component that decides which files can be seen. For semantic, lexical,
and mixed queries, skygrep now builds a cheap candidate file pool from
independent signals before the expensive cascade makes the final choice:
explicit include scope, indexed path tokens, indexed symbols, SQLite
chunk text, and a bounded rg -il -F pass.
Candidate recall is additive. It can vote likely files into the pool,
but semantic scoring, reranking, deduplication, and diversification still
decide ordering. For --content and agent calls, the selected
file can also carry a small same-file support pack: related constants,
symbol definitions, or assertion anchors that make the returned context
sufficient without widening to a raw tree dump.
Agent confidence is calibrated from the evidence actually available.
confidence_basis records whether the summary used both lexical
and retrieval evidence or retrieval-only signals; non-lexical queries never
receive lexical coverage credit. For synthesized document answers, a final
authority policy keeps current living references separate from unnamed
historical snapshots.
In 0.7.0, direct and daemon agent-context requests call the same
run_agent_context_search implementation. Strict requests add an
independent corpus-wide semantic pass and a source/index timestamp check;
inconclusive evidence is returned explicitly rather than being upgraded from
first-pass confidence. The daemon also validates that the requested project,
database, and lexical scope stay inside the server's configured boundary.
Equivalent macOS /var and /private/var paths are
coalesced before source priors and final ranking.
System · storage
Two SQLite tables joined 1-to-1.
storage.init_db. The
chunks table holds source location and verbatim text;
vectors holds the embedding as a binary float32 buffer.
The two tables are kept in lockstep at write time
(store_chunks_batch inserts both rows in the same
transaction) and joined on id at read time. There is no
declared foreign key constraint; the join is enforced by application
code.
System · ranking
Cosine, lexical adjustment, dedup, diversify.
