Eorbitt — home
Book a technical call

Capability

Applied AI

You want an assistant that answers from your own documentation. You also need to know what it will do when the answer is not in there, and to be able to prove to somebody that it is right often enough to deploy.

At a glance

Phases
4
First phase
1 to 2 weeks
Deliverables
8
Stack layers
4
Book a technical call

What we deliver

The work itself

  • Assemble an evaluation set of real questions with agreed correct answers, before any model is chosen
  • Build retrieval over your approved content, with chunking tuned against that set
  • Report accuracy, refusal rate and citation coverage as numbers, and re-report them on every change
  • Constrain the assistant to cite a source, or to say it does not know and hand to a person
  • Log every exchange with its retrieved context, so a wrong answer can be traced to a cause
  • Put a human in the loop wherever an answer commits the business to something
  • Keep the content pipeline in your hands, so correcting a document corrects the assistant
  • Re-run the evaluation set on model upgrades, so a provider change cannot silently degrade quality

How we work

What the engagement looks like

Phases, durations and what each one hands over are listed below. The stack is published in full — nothing about how this is built is a surprise once work starts.

Stack

What we build it with

  • Runtime
    TypeScript · Python · Claude API · Streaming responses over SSE
  • Data
    PostgreSQL with pgvector · Hybrid retrieval (BM25 and dense) · Document ingestion pipeline · Conversation store
  • Platform
    Cloudflare Workers · Azure Container Apps · Microsoft Graph for document sources
  • Operations
    Evaluation harness in CI · Prompt and retrieval versioning · Token and cost budgets per conversation · OpenTelemetry traces

Engagement shape

Phases and timeline

  • Evaluation set
    1 to 2 weeks
    Between 80 and 200 real questions with agreed answers, and a baseline score for doing nothing
  • Retrieval build
    2 to 4 weeks
    Ingestion pipeline, retrieval tuned against the evaluation set, and the measured score
  • Assistant and guardrails
    2 to 4 weeks
    Assistant with citations, refusal behaviour, escalation rules and conversation logging
  • Supervised pilot
    3 to 6 weeks
    Live with every answer reviewed, and a decision to widen or stop based on the review

What we need from you

  • The documentation you are willing to stand behind as correct
  • A subject-matter expert to settle disputed answers in the evaluation set
  • A decision on what the assistant is never allowed to answer
  • Someone to review pilot conversations daily while the pilot runs

After go-live

Who holds it at 3am

  • The evaluation set runs on every prompt, retrieval or model change, and the score is reported
  • Escalated conversations are reviewed weekly, and the gaps they expose become content fixes
  • Cost per conversation is tracked against a budget agreed before launch
  • If accuracy falls below the agreed floor, the assistant reverts to handing over to a person