`reach query`¶

Synthesize, convert, and inspect labeled evaluation query datasets across JSON, JSONL, and CSV formats.

An evaluation dataset consists of realistic user prompts paired with ground-truth target skill expectations and difficulty metrics.

Synopsis¶

# Synthesize queries for all skills (defaults to saving in .reach/queries.json)
reach query

# Synthesize directly in JSONL or CSV format
reach query -o queries.jsonl
reach query -o queries.csv

# Synthesize for a specific skill corpus or target skill
reach query ./my-skills --count 3 --adversarial

# Synthesize queries with interactive browser review before saving
reach query ./my-skills --review

# Convert an existing query dataset between formats offline (0 LLM tokens)
reach query .reach/queries.json -o queries.csv
reach query queries.csv -o .reach/queries.json
reach query queries.json -o queries.jsonl

# Inspect query difficulty, leak detection, and citations
reach query --queries .reach/queries.json --leaks --citations

Options¶

Synthesis & Targets¶

Option	Type	Default	Description
`[TARGET]`	Path	Auto-discovered	Skill directory to draft queries for, or existing query file (`.json`, `.jsonl`, `.csv`) to convert.
`--skills`, `-s`	Path	Auto-discovered	Path to skills directory or catalog.
`--skill`	String	All	Specific skill name(s) to draft queries for (repeatable).
`--count`	Integer	`3`	Number of queries to draft per target skill.
`--generator-model`	String	`gemini-3.8-flash`	Model used to draft synthetic queries.
`--generator-agent`	Choice	Auto-detected	Agent driver used to draft synthetic queries (`antigravity-cli`, `antigravity-sdk`, `claude-code`, `goose`, `keyword`, `pi`).
`--generator-arm`	Choice	`None`	Prompt synthesis arm (`content` or `framing`).
`--adversarial`	Flag	`false`	Synthesize near-miss negative queries sharing target vocabulary.
`--adversarial-count`	Integer	`1`	Number of adversarial negative queries per target.
`--top-rivals`	Integer	`3`	Maximum number of rival skills to include in prompt context.
`--draft-concurrency`	Integer	`1`	Parallel concurrency for query generation workers.
`--review`	Flag	`false`	Launch interactive browser review for drafted queries before saving.

Output & Formats¶

Option	Type	Default	Description
`--out`, `-o`	Path	`.reach/queries.json`	Where to write the query set (format auto-inferred from `.json`, `.jsonl`, or `.csv`).
`--format`, `-f`	Choice	`json`	Explicit output format: `json`, `jsonl`, or `csv`. When `--out` is omitted with `csv` or `jsonl`, output is printed to stdout.
`--force`	Flag	`false`	Overwrite destination query set file if it already exists.
`--sync`	Flag	`false`	Backfill missing skills and refresh updated skills in an existing destination query set.
`--dry-run`	Flag	`false`	Preview prompts and token budget without making model calls.
`--config`	Path	`reach.toml`	Explicit path to a configuration file.
`--global`	Flag	`false`	Include user-level global skill directories (`~`) during discovery.
`--quiet`, `-q`	Flag	`false`	Suppress non-essential progress output.

Inspection & View¶

Option	Type	Default	Description
`--queries`	Path	-	Query set path to inspect or convert.
`--leaks`	Flag	`false`	Add a column flagging potential target skill name leakage in query text.
`--citations`	Flag	`false`	Add a column quoting each query's grounding passage (requires `.reach/queries-citations.json`).
`--agent`	Choice	Auto-discovered	Agent runtime used for skill corpus discovery.

Agent Registry Options¶

Option	Type	Default	Description
`--project`, `-p`	String	None	Google Cloud project ID hosting the Agent Registry.
`--location`	String	`"global"`	Agent Registry location endpoint (`global`, `us`, `eu`).
`--publisher`	String	None	Filter registry skills by publisher identifier (e.g. `cloud.google.com`).
`--registry`	Flag	`false`	Target Google Cloud Agent Registry instead of local workspace.
`--fresh`	Flag	`false`	Bypass cached metadata and re-fetch latest skill definitions.
`--no-cache`	Flag	`false`	Run without reading or writing local disk cache.

Field Mapping (Custom CSV / JSONL)¶

Option	Type	Default	Description
`--text-column`	String	-	Column holding query text.
`--id-column`	String	-	Column holding query ID (numbered automatically when absent).
`--kind-column`	String	-	Column holding query kind.
`--expected-skill-column`	String	-	Column holding expected target skill name.
`--acceptable-skills-column`	String	-	Column holding neutral router or helper skills.
`--notes-column`	String	-	Column holding per-query notes.
`--separator`	String	`,`	Delimiter separating multiple skills in a cell.
`--id-prefix`	String	-	Prefix prepended to query IDs to prevent collisions.
`--catalog`	String	`all`	Target catalog ID to associate with imported queries.
`--notes`	String	`""`	Provenance notes or reviewer comments describing this query set.

Subcommands¶

`reach query draft`¶

Draft synthetic evaluation queries targeting resident skills without converting existing datasets.

reach query draft [TARGET] [OPTIONS]

# Draft queries targeting skills in a local directory
reach query draft ./my-skills

# Sync missing skills and refresh updated skills in an existing query set
reach query draft ./my-skills --sync

# Draft queries with interactive browser review before saving
reach query draft ./my-skills --review

Option	Type	Default	Description
`--count`	Integer	`3`	Target evaluation queries generated per skill.
`--sync`	Flag	`false`	Backfill missing skills and refresh updated skills in an existing query set.
`--review`	Flag	`false`	Open interactive browser review session before saving.
`--generator-agent`	Choice	Auto-detected	Generation agent runtime (`antigravity-cli`, `antigravity-sdk`, `claude-code`, `goose`, `keyword`, `pi`).
`--generator-model`	String	`gemini-3.8-flash`	LLM model identifier for query drafting.
`--generator-arm`	Choice	`None`	Prompt synthesis arm (`content` or `framing`).
`--top-rivals`	Integer	`3`	Maximum number of rival skills to include in prompt context.
`--draft-concurrency`	Integer	`1`	Parallel concurrency for query generation workers.
`--out`, `-o`	Path	-	Output path for generated queries JSON (defaults to `.reach/queries.json`).

`reach query view`¶

Render an evaluation query set as a terminal table with difficulty ranks, leakage flags, and citations.

reach query view <QUERIES> [OPTIONS]

Format Conversion & Exchange¶

Reach supports offline format conversion directly through reach query without calling an LLM:

# Export JSON to CSV for spreadsheet review in Google Sheets or Excel
reach query .reach/queries.json -o queries.csv

# Print CSV rows directly to stdout
reach query .reach/queries.json -f csv

# Export JSON to JSONL for streaming evaluation pipelines
reach query .reach/queries.json -o queries.jsonl

# Import edited CSV back into canonical Reach JSON
reach query queries.csv -o .reach/queries.json --catalog all

# Map external CSV datasets with non-standard column headers
reach query external.csv -o .reach/queries.json --text-column prompt --expected-skill-column tool

reach eval also natively accepts .yaml/.yml, .jsonl, and .csv files via --queries, so conversion is purely optional:

reach eval --queries queries.yaml
reach eval --queries queries.jsonl
reach eval --queries queries.csv

Query Schema (`Query`)¶

Each entry in a .reach/queries.json (or .yaml / .jsonl / .csv dataset) conforms to the Query model. When hand-authoring JSON or YAML files, top-level catalog_id defaults to "all" and query_id (or legacy "id", which is normalized to "query_id" on load) is required; when importing CSV or JSONL files via reach query, missing IDs are numbered automatically as q-1, q-2, ...:

Field	Type	Default	Description
`query_id`	`str`	—	Unique non-empty identifier for the evaluation query (accepts legacy `"id"` on load; numbered automatically as `q-1`, `q-2`, ... during CSV/JSONL import).
`text`	`str`	—	Realistic user prompt text presented to the agent runtime.
`kind`	`QueryKind \\| None`	`None`	Structural category (`implicit`, `contextual`, `neighbor_negative`, or `out_of_scope`).
`expected_skill`	`str \\| None`	`None`	Ground-truth target skill expected to be invoked, or `None` for out-of-scope queries.
`acceptable_skills`	`tuple[str, ...]`	`()`	Optional neutral helper or router skills (e.g. `finding-google-skills`) that consume turns at runtime but are neither rewarded as TP nor penalized as FP.
`notes`	`str`	`""`	Author notes, rationale, or difficulty context.

Provenance & Per-Skill Sync Digests (`QuerySetProvenance`)¶

When saved as JSON (.reach/queries.json), the top-level QuerySet wrapper includes a provenance block (QuerySetProvenance) that tracks how queries were synthesized and records per-skill content digests:

{
  "catalog_id": "all",
  "queries": [...],
  "provenance": {
    "origin": "generated",
    "generator_model": "gemini-3.8-flash",
    "queries_per_target": 3,
    "bodies_digest": "8f21b39e4c01",
    "skill_digests": {
      "cloud-run-basics": "2a2f37e47243",
      "gcloud": "7d17b03e9bff"
    }
  }
}

skill_digests (dict[str, str]): Maps each drafted skill name to a 12-character SHA-256 digest of its SKILL.md markdown body excluding YAML frontmatter. Optimizing a skill's frontmatter description (reach optimize) does not mark its benchmark queries stale, whereas modifying a skill's instructions or examples triggers a warning in reach query and selectively refreshes that skill when running reach query draft --sync or reach sweep.
bodies_digest (str): Aggregate 12-character SHA-256 digest across all (skill_name, skill_body_digest) pairs at generation time.

reach query¶