Przejdź do treści

Extractor options — operator runbook

Plan 87 WI-5. The Secruna extractor is the LLM that classifies AI systems during the discovery + HITL pipeline. The extractor is pluggable per tenant; this doc lists the supported providers, their tradeoffs, and the quick-setup procedures.

Provider matrix

Provider Where it lives Latency (typical) Cost / 1k input tokens Cost / 1k output tokens Structured output reliability Data residency Setup steps
anthropic Anthropic-hosted cloud (US / EU) 800 ms €0.003 €0.015 high Anthropic's region (configurable in Console) Set ANTHROPIC_API_KEY in cp-api env. Default model claude-sonnet-4.
azure_openai Customer's Azure subscription 1200 ms €0.0025 €0.010 high Configurable per AOAI region Provision AOAI resource + deployment; set AZURE_OPENAI_* env vars.
ollama Local network OR cloudflared tunnel 2500 ms (varies) €0.0 €0.0 medium (7B drifts from JSON) On-prem / customer-managed brew install ollama; ollama pull qwen2.5:14b-instruct; set OLLAMA_*.
byo Customer-managed HTTPS endpoint 1500 ms (varies) depends depends depends on customer model Customer's network POST /tenant/settings/extractor with provider=byo + endpoint + auth.
mock In-process (never leaves cp-api) <1 ms €0.0 €0.0 deterministic n/a Set EXTRACTOR_PROVIDER=mock. Never use in production.

Quick-setup: Ollama via Cloudflare tunnel

The cheapest path to a real (non-mock) extractor for a Secruna demo or a low-volume tenant. Five minutes end-to-end.

# 1. Install Ollama + cloudflared
brew install cloudflared ollama

# 2. Pull a JSON-capable model. qwen2.5:14b-instruct is the recommended
#    minimum — smaller models drift from JSON schema enough that the
#    extractor service rejects ~5% of responses.
ollama pull qwen2.5:14b-instruct

# 3. Run Ollama locally
ollama serve &

# 4. Expose it via cloudflared. The CLI prints a URL of the form
#    https://random-words.trycloudflare.com — copy it; we'll paste it
#    into the Secruna UI next.
cloudflared tunnel --url http://localhost:11434

Then in the Secruna dashboard:

  1. Open /settings/tenant (Tenant settings).
  2. Scroll to Extractor (advanced).
  3. Pick Self-hosted (Ollama).
  4. Paste the https://random-words.trycloudflare.com URL into Base URL.
  5. Set Model to qwen2.5:14b-instruct.
  6. Click Test connection. A green checkmark means the tunnel is reachable and the model answered the probe.
  7. Click Save.

Platform-admin alternative

A platform admin can drive the same flow without per-tenant input:

  1. Set OLLAMA_TUNNEL_URL=https://random-words.trycloudflare.com on the cp-api Container App.
  2. Open /admin/extractors.
  3. Find the tenant row, click Swap preset, pick Use Ollama tunnel, click Apply preset.

The preset stamps {provider: "ollama", endpoint: "<tunnel>", model: "qwen2.5:14b-instruct"} onto the tenant's settings.extractor JSONB and writes a tenant.extractor.swapped audit row.

Failover chain best practices

The extractor system supports a fallback chain (Plan 87 WI-3). Common useful patterns:

  • [ollama_tunnel, azure_openai] — primary is the cheap self-hosted model; when the laptop sleeps (tunnel drops) the system falls back to Azure OpenAI without user intervention. Recommended default for demo / pilot tenants where the founder runs Ollama on their laptop.

  • [azure_openai_eu, azure_openai_us] — primary is the EU-resident AOAI deployment; the US-resident deployment kicks in when the EU region hits a quota wall. Recommended for tenants with strict EU data-residency requirements but high enough volume to warrant a burst-overflow region.

  • [anthropic, mock] — anthropic as the primary, mock as the fallback. Useful for staging / preview tenants: when the Anthropic key runs out of credit, the mock provider keeps the discovery pipeline green so dashboards stay populated.

Chain length is capped at four entries total (primary + 3 fallbacks); see MAX_FAILOVER_CHAIN_LENGTH in rekognise.agent.llm.factory for the rationale. The cap keeps tail latency bounded — a 4-deep chain where every entry hits a 60s BYO timeout still completes within the discovery- worker's per-extraction budget.

Cost-conscious patterns

  • Use Mock for CI tests. Set EXTRACTOR_PROVIDER=mock in any integration test fixture that exercises the extractor surface so AOAI / Anthropic are never billed during CI runs. See tests/integration/cp/admin/test_tenant_flags.py line ~57 for the pattern.

  • Cache extractor responses. EXTRACTOR_CACHE_ENABLED=true (the default) wraps the resolved LLMClient in a CachedExtractor keyed on the request hash. Identical extraction requests from a re-discovery run cost €0. Per-tenant isolation is on by default; cross-tenant cache hits are an intentional cost-saving in the eval / golden-set path.

  • Pick the smallest competent model. The structured-output reliability column above is calibrated against the Secruna golden set — qwen2.5:14b is the floor for self-hosted; smaller models cost more in retries than they save in tokens.

  • Watch the panel. The /admin/extractors table shows per-tenant last_success_at / last_failure_at. A tenant stuck on amber (no success in the last hour but no recent failures either) is usually idle, not broken — check the tenant's discovery cadence before panicking.

Troubleshooting via the panel

The platform-admin /admin/extractors panel renders four status dots per tenant:

  • Green — last_success_at <= 5 min ago. The tenant is happily extracting; nothing to do.
  • Amber — last_success_at <= 1 h ago. The tenant has succeeded recently but is not currently active. Usually idle, not broken.
  • Red — last_failure_at is newer than last_success_at. The last call to the extractor failed. Check the audit log for the corresponding extractor.call.failed event — the context will carry the provider's error message.
  • Gray — no extractor.call.* events on file. Either the tenant has never run discovery, or audit-event emission for the extractor path hasn't been wired yet (deferred in Plan 87 WI-5; see the WI-5 spec for the follow-up work-item).

Common failure modes

Symptom Likely cause Fix
Test-connection returns 401 / 403 API key invalid / not rotated Reset the per-tenant secret in Key Vault under extractor-tenant-<tenant-id>-<provider>.
Test-connection returns 5xx timeout Self-hosted endpoint unreachable Re-run cloudflared tunnel ...; the URL changes per run unless you use a named tunnel.
BYO endpoint returns 200 but ok=false Model doesn't follow the BYO contract (raw text instead of JSON) Wrap the model in a thin server that conforms to rekognise.agent.llm.byo_client.ByoExtractionRequest.
Repeated extractor.call.failed events Hitting AOAI quota Either set up a failover chain, or upgrade the AOAI deployment SKU.

Endpoints reference

Verb Path Role Purpose
GET /admin/extractors/status platform admin Global default + per-tenant overrides + provider catalog.
POST /admin/extractors/test-preset platform admin Fire a probe at an ephemeral LLMClient built from request params.
POST /admin/tenants/{tenant_id}/extractor/swap-to-preset platform admin Apply one of azure_openai_default / ollama_tunnel / anthropic_default / mock. Audits tenant.extractor.swapped.
GET /tenant/settings any Includes the per-tenant extractor block (no API key).
PATCH /tenant/settings/extractor org admin Mutate the per-tenant extractor block.
POST /tenant/settings/extractor/test-connection org admin Pre-save validation against a proposed config.

Audit events

Action Tenant scope Emitter
tenant.extractor.updated per-tenant PATCH /tenant/settings/extractor
tenant.extractor.test_connection_run per-tenant POST /tenant/settings/extractor/test-connection
tenant.extractor.swapped per-tenant POST /admin/tenants/{id}/extractor/swap-to-preset
extractor.call.succeeded per-tenant deferred — orchestrator / HITL emitter not yet wired (Plan 87 WI-5 follow-up).
extractor.call.failed per-tenant deferred — orchestrator / HITL emitter not yet wired (Plan 87 WI-5 follow-up).