Writing
Notes on building agentic systems that actually reach production, and on the parts that are harder than the demos suggest.
A confidence score that routes documents well can still be uncorrelated with correctness. Measured on receipts: accuracy at 1.0 was lower than at 0.0.
We added BM25 to a dense retriever across ten probes. It moved two of them, and seven were already saturated. What that means before you build it.
Stock Presidio measured on real court judgments and synthetic PII: entity recall, the partial-redaction trap, and what the defaults do not catch.
Scanned PDFs passed every document-level quality check we threw at them while whole pages extracted as one character. Where the loss actually hides.
Raising the Presidio score threshold reads like a precision knob. Measured across a full sweep it is a cliff, and names fall off it in one step.
A benchmark that cannot fail is measuring whether you wrote a hard question. The armed-probe rule, and the counts that show whether yours is armed.
Changing the splitter recovered 19 of 25 answers. Adding overlap on top recovered 6 more, and none of them for the reason overlap is sold on.
Token accounting for a local RAG pipeline: what k costs, where prompt tokens go, what the template tax is, and the formula to price it on any provider.
A method for evaluating RAG that survives a real corpus: armed probes, controls first, and retrieval failures counted apart from generation failures.
Saturation, partial credit, abstention counted as a catch, and detectability without power. Four ways a RAG metric reports a result it did not earn.
We built the cheap guardrail, checking a cited chunk holds the value the answer asserts, then measured it against the answer failures of four configs.
How to tell which business processes suit an AI agent, which ones do not, and what an agent should never be allowed to do on its own.
The difference between agentic AI and AI agents, why it matters when you are buying, and the questions that reveal which one you are being sold.
A plain-English explanation of MCP servers: what they do, the three things they expose, how they connect to AI apps, and when you actually need one.
If something here matches a problem you are looking at, describe it and you will get a straight answer on whether agents are the right tool.