Extractor options — operator runbook

Plan 87 WI-5. The Secruna extractor is the LLM that classifies AI systems during the discovery + HITL pipeline. The extractor is pluggable per tenant; this doc lists the supported providers, their tradeoffs, and the quick-setup procedures.

Provider matrix

Provider	Where it lives	Latency (typical)	Cost / 1k input tokens	Cost / 1k output tokens	Structured output reliability	Data residency	Setup steps
anthropic	Anthropic-hosted cloud (US / EU)	800 ms	€0.003	€0.015	high	Anthropic's region (configurable in Console)	Set ANTHROPIC_API_KEY in cp-api env. Default model claude-sonnet-4.
azure_openai	Customer's Azure subscription	1200 ms	€0.0025	€0.010	high	Configurable per AOAI region	Provision AOAI resource + deployment; set AZURE_OPENAI_* env vars.
ollama	Local network OR cloudflared tunnel	2500 ms (varies)	€0.0	€0.0	medium (7B drifts from JSON)	On-prem / customer-managed	brew install ollama; ollama pull qwen2.5:14b-instruct; set OLLAMA_*.
byo	Customer-managed HTTPS endpoint	1500 ms (varies)	depends	depends	depends on customer model	Customer's network	POST /tenant/settings/extractor with provider=byo + endpoint + auth.
mock	In-process (never leaves cp-api)	<1 ms	€0.0	€0.0	deterministic	n/a	Set EXTRACTOR_PROVIDER=mock. Never use in production.

Quick-setup: Ollama via Cloudflare tunnel

The cheapest path to a real (non-mock) extractor for a Secruna demo or a low-volume tenant. Five minutes end-to-end.

# 1. Install Ollama + cloudflared
brew install cloudflared ollama

# 2. Pull a JSON-capable model. qwen2.5:14b-instruct is the recommended
#    minimum — smaller models drift from JSON schema enough that the
#    extractor service rejects ~5% of responses.
ollama pull qwen2.5:14b-instruct

# 3. Run Ollama locally
ollama serve &

# 4. Expose it via cloudflared. The CLI prints a URL of the form
#    https://random-words.trycloudflare.com — copy it; we'll paste it
#    into the Secruna UI next.
cloudflared tunnel --url http://localhost:11434

Then in the Secruna dashboard:

1. Open /settings/tenant (Tenant settings).
2. Scroll to Extractor (advanced).
3. Pick Self-hosted (Ollama).
4. Paste the https://random-words.trycloudflare.com URL into Base URL.
5. Set Model to qwen2.5:14b-instruct.
6. Click Test connection. A green checkmark means the tunnel is reachable and the model answered the probe.
7. Click Save.

Platform-admin alternative

A platform admin can drive the same flow without per-tenant input:

1. Set OLLAMA_TUNNEL_URL=https://random-words.trycloudflare.com on the cp-api Container App.
2. Open /admin/extractors.
3. Find the tenant row, click Swap preset, pick Use Ollama tunnel, click Apply preset.

The preset stamps {provider: "ollama", endpoint: "<tunnel>", model: "qwen2.5:14b-instruct"} onto the tenant's settings.extractor JSONB and writes a tenant.extractor.swapped audit row.

Failover chain best practices

The extractor system supports a fallback chain (Plan 87 WI-3). Common useful patterns:

• [ollama_tunnel, azure_openai] — primary is the cheap self-hosted model; when the laptop sleeps (tunnel drops) the system falls back to Azure OpenAI without user intervention. Recommended default for demo / pilot tenants where the founder runs Ollama on their laptop.

• [azure_openai_eu, azure_openai_us] — primary is the EU-resident AOAI deployment; the US-resident deployment kicks in when the EU region hits a quota wall. Recommended for tenants with strict EU data-residency requirements but high enough volume to warrant a burst-overflow region.

• [anthropic, mock] — anthropic as the primary, mock as the fallback. Useful for staging / preview tenants: when the Anthropic key runs out of credit, the mock provider keeps the discovery pipeline green so dashboards stay populated.

Chain length is capped at four entries total (primary + 3 fallbacks); see MAX_FAILOVER_CHAIN_LENGTH in rekognise.agent.llm.factory for the rationale. The cap keeps tail latency bounded — a 4-deep chain where every entry hits a 60s BYO timeout still completes within the discovery- worker's per-extraction budget.

Cost-conscious patterns

• Use Mock for CI tests. Set EXTRACTOR_PROVIDER=mock in any integration test fixture that exercises the extractor surface so AOAI / Anthropic are never billed during CI runs. See tests/integration/cp/admin/test_tenant_flags.py line ~57 for the pattern.

• Cache extractor responses. EXTRACTOR_CACHE_ENABLED=true (the default) wraps the resolved LLMClient in a CachedExtractor keyed on the request hash. Identical extraction requests from a re-discovery run cost €0. Per-tenant isolation is on by default; cross-tenant cache hits are an intentional cost-saving in the eval / golden-set path.

• Pick the smallest competent model. The structured-output reliability column above is calibrated against the Secruna golden set — qwen2.5:14b is the floor for self-hosted; smaller models cost more in retries than they save in tokens.

• Watch the panel. The /admin/extractors table shows per-tenant last_success_at / last_failure_at. A tenant stuck on amber (no success in the last hour but no recent failures either) is usually idle, not broken — check the tenant's discovery cadence before panicking.

Troubleshooting via the panel

The platform-admin /admin/extractors panel renders four status dots per tenant:

• Green — last_success_at <= 5 min ago. The tenant is happily extracting; nothing to do.
• Amber — last_success_at <= 1 h ago. The tenant has succeeded recently but is not currently active. Usually idle, not broken.
• Red — last_failure_at is newer than last_success_at. The last call to the extractor failed. Check the audit log for the corresponding extractor.call.failed event — the context will carry the provider's error message.
• Gray — no extractor.call.* events on file. Either the tenant has never run discovery, or audit-event emission for the extractor path hasn't been wired yet (deferred in Plan 87 WI-5; see the WI-5 spec for the follow-up work-item).

Common failure modes

Symptom	Likely cause	Fix
Test-connection returns 401 / 403	API key invalid / not rotated	Reset the per-tenant secret in Key Vault under extractor-tenant-<tenant-id>-<provider>.
Test-connection returns 5xx timeout	Self-hosted endpoint unreachable	Re-run cloudflared tunnel ...; the URL changes per run unless you use a named tunnel.
BYO endpoint returns 200 but ok=false	Model doesn't follow the BYO contract (raw text instead of JSON)	Wrap the model in a thin server that conforms to rekognise.agent.llm.byo_client.ByoExtractionRequest.
Repeated extractor.call.failed events	Hitting AOAI quota	Either set up a failover chain, or upgrade the AOAI deployment SKU.

Endpoints reference

Verb	Path	Role	Purpose
GET	/admin/extractors/status	platform admin	Global default + per-tenant overrides + provider catalog.
POST	/admin/extractors/test-preset	platform admin	Fire a probe at an ephemeral LLMClient built from request params.
POST	/admin/tenants/{tenant_id}/extractor/swap-to-preset	platform admin	Apply one of azure_openai_default / ollama_tunnel / anthropic_default / mock. Audits tenant.extractor.swapped.
GET	/tenant/settings	any	Includes the per-tenant extractor block (no API key).
PATCH	/tenant/settings/extractor	org admin	Mutate the per-tenant extractor block.
POST	/tenant/settings/extractor/test-connection	org admin	Pre-save validation against a proposed config.

Audit events

Action	Tenant scope	Emitter
tenant.extractor.updated	per-tenant	PATCH /tenant/settings/extractor
tenant.extractor.test_connection_run	per-tenant	POST /tenant/settings/extractor/test-connection
tenant.extractor.swapped	per-tenant	POST /admin/tenants/{id}/extractor/swap-to-preset
extractor.call.succeeded	per-tenant	deferred — orchestrator / HITL emitter not yet wired (Plan 87 WI-5 follow-up).
extractor.call.failed	per-tenant	deferred — orchestrator / HITL emitter not yet wired (Plan 87 WI-5 follow-up).