The Agentic AI Stack: Evaluating and Operating in Production (Part 3)

Part 1 looked at how Claude Code itself is built. Part 2 zoomed out to how an orchestrator like LangGraph coordinates Claude Code, ML models, and other LLMs toward an objective no single agent can meet. Part 3 covers the last mile: proving the system works, getting it into production, and keeping it trustworthy once it is live.

Two examples run through this piece because they sit at opposite ends of the same stack. The Unified Fraud Detection Platform (UFDP) is real-time, high-volume, and statistical. ITSM incident resolution is conversational and grounded in years of past tickets.

Start with the right model, not the biggest one

Not every step deserves a frontier model, so treat model choice as a rubric and start at the cheapest rung that can do the job. Deterministic steps and statistical scoring belong to code and classic ML. Simple, high-volume work such as classification or extraction can go to a small language model, run locally through a platform like Ollama or taken from a small hosted tier, or to a dedicated classifier such as Jev, which returns structured answers at a fraction of an LLM’s cost. Moderate tasks where you want control over hosting, data, and cost suit open-weight models such as Kimi K3, Qwen3, or DeepSeek V4. The frontier model is the last rung, kept for multi-step, complex logic, and even there thinking mode is a dial. Turn it on where later steps depend on getting this one right, then step down to non-thinking or a lighter model, because reasoning tokens burn cost fast. With model selection this programmable, the trade-off is simple: quality of response against cost.

Two things sit alongside the rungs rather than on them. RAG grounds answers in confidential or policy content, such as HR policy questions or ITSM incident history, so knowledge stays in retrieval and out of the model weights. We have further evolved from classical RAG to Agentic RAG to the Graph RAG, attaching a small picture that illustrates the idea and scenarios.

A framework like LlamaIndex chunks and embeds that content so the model retrieves it at query time, and GraphRAG adds explicit relationships on top of similarity search when an answer depends on how pieces of information connect. Fine-tuning shapes language, not knowledge. It matters less than it used to, as BloombergGPT showed: built for finance, it was later outperformed on many financial benchmarks by general frontier models. But it still earns its place where the language itself is specialized: legal drafting, clinical documentation, financial regulatory narratives, and brand-voice support replies in ITSM. In UFDP, detection itself stays with the ML model, while fine-tuning can shape how flagged cases are written up.

Validate at two levels

In an agentic solution there are usually two levels of validation. The first is the model level, the testing of the model itself, whether that is a GPT or Claude model, or one you have fine-tuned or picked from the open-weight and small-model options. With a frontier model used as is, that validation belongs to the provider and is not yours to repeat. The second is the agentic system level, the application built on top of the model, and this is where your team’s effort goes.

At the system level, two checks carry most of the weight. The first is trajectory: did the agent take the intended steps, call the right tools the right number of times, and fail over cleanly when something broke. The second is output correctness. You have no independent way to know whether the first agent’s answer is right, so a reflection agent acts as judge, checking it against a gold dataset of known-correct examples. A baseline of sanity checks splits along the same two levels. Helpfulness, harmlessness, honesty, and no hallucination describe the model’s behavior. Security, latency, and compliance belong to the harness and the application around it.

Both levels then feed ordinary software engineering practice: SIT, UAT, and security testing in the CI/CD pipeline. Agents can take part in that pipeline too. A coding agent such as Claude Code, for example, can be invoked from the CI/CD YAML script to run test scenarios before the next stage proceeds.

Ship it, orchestrate it, watch it

Deployment starts with containerizing the agent like any other service, but production hosting is a wider question than a container. The major cloud and data platforms now offer managed homes for agents: Mosaic AI on Databricks, which deploys agents through Model Serving next to the governed data they use; Amazon Bedrock on AWS, with AgentCore for hosting agents and their tools; and Vertex AI Agent Engine on GCP. Each takes on scaling, so more instances spin up as traffic grows without your team running that infrastructure.

Orchestration here means orchestrating the harness, the set of components that lets an agent do its job: LLMs and ML models, retrieval over your own data, and tools and APIs reached through a standard connector. A framework wires these together and manages state and control flow. LangGraph, Crew.ai, and the Claude Agent SDK are examples, MCP is a common connector layer, and LlamaIndex a common choice for the retrieval piece. The managed platforms are the engineered, cloud-native home for that harness. n8n is the business-facing counterpart above it, calling the hosted agents as endpoints and letting non-technical teams wire them, along with other modules and scripts, into flows they can run themselves.

Once live, observability keeps the system accountable. LangSmith is the familiar starting point, with Langfuse and Arize Phoenix covering similar ground: tracing agent behavior, catching drift, and attaching evaluation scores to live traffic.

Closing the stack

Part 1 built the agent. Part 2 orchestrated it toward a shared objective. Part 3 makes it fit to run: the right model for each step, two levels of validation, a path through CI/CD, and observability that catches problems before a customer does. That is the full distance from a single coding agent to a system a business can depend on.