We built an on-premises AI security assistant. Analysts ask plain-English questions about terabytes of log data and get answers in seconds, and no data leaves the customer’s environment.
What goes wrong with most AI chatbots
- Data leaves the building, sent to outside model APIs.
- It doesn’t scale: you cannot load terabytes into a prompt.
- Agent chains are slow: every extra step adds latency and another chance to fail.
- Models hallucinate, and in security a confident wrong answer is worse than none.
The model’s job is translation
Don’t bring the data to the model. Let the model write one precise question for the system that already holds the data.
Our assistant turns a question into a single, precise query and sends it to a purpose-built search engine. The search engine does what it is good at; the model does what it is good at.
- A local model runs on the customer’s own GPU. No external API calls.
- Deterministic validation checks every generated query against the structure of the data before it runs, which makes hallucinated fields and indexes structurally unlikely.
- A rules-based fallback answers when the model’s output isn’t usable.
Simple won
We built five versions and measured each against hundreds of hard questions. The simplest architecture was the fastest, cheapest and most accurate. Every “smart” layer we added made results worse.
Real sophistication is doing one hard thing well, and keeping the data where it belongs.