agents
Posts tagged agents.
AI Agents Why Pass Rate Lies: Revision Rate, Trajectories, and Coverage
Pass rate flatters bad agents. Gate deploys on revision rate, trajectory scores, eval coverage, and cost per successful task—not a single green percentage.
AI Agents Why Agents Loop on Failed Tools: No-Progress Detection Beats Longer Prompts
Agents loop on failed tools because the harness never detects no-progress. Fingerprint calls, honor retryable:false, cap turns, and terminate with a reason code.
AI Agents Agentic Systems: An Operating Manual for Multi-Agent Work That Ships
An agentic system is not a chat window with tools. It is evaluators, sandboxes, state machines, memory contracts, and kill switches — built so the work survives contact with real data.
AI Agents Why Agent Demos Die in Production: Control Gaps, Not Model IQ
Demo success proves a happy path under staged tools. Production fails when the control loop — schemas, auth, evaluators, and kill switches — was never part of the demo.
AI Agents Build the Evaluator Before the Agent
If judgement and work share a context, you are grading your own homework. Build the evaluator first — criteria, evidence, ceilings — then let the agent earn autonomy.
AI Agents The Evaluator Is the Product
Agent accuracy did not come from a better prompt or a bigger model. It came from separating the thing that does the work from the thing that judges it.
AI Agents Single Agent First: Split Only When Trust, Audience, or Timing Conflicts
Start with one agent and many tools. Split only when trust, audience, or timing conflict—and prove that split with pass rate, cost, and escalate rate.
AI Agents State Machines for Agent Loops: Determinism Where It Matters
Agent loops need freedom inside a cage. Explicit states, legal transitions, revision ceilings, and escalate paths are how non-deterministic models ship deterministic ops.
AI Agents MCP vs Native Function Calling: Portability Tax vs Shortest Loop
Native function calling wins for one app's short tool loop; MCP earns the tax when tools must be shared and governed across hosts—not a LangChain swap.
AI Agents LangGraph vs CrewAI vs a Custom Loop: Choose Control, Not Fashion
Pick LangGraph, CrewAI, or a custom loop by how much control you need—then compare them on the same golden set and cost band, not on Hacker News fashion.
AI Agents Agent Memory Patterns: What to Persist, What to Forget
Agent memory is a policy, not a bigger context window. Persist identifiers and approved facts; forget scratch and failed conclusions; promote to long-term only on purpose.
AI Agents Golden Sets from Production Failures: Turn Bad Runs into Regression Fuel
Turn a bad production agent run into a regression test by harvesting the trace into a stubbed fixture with a clear pass criterion.
AI Agents Prompt Injection for Tool Agents: Stop Text from Becoming Actions
Defend production agents that read email, tickets, or pages by fencing untrusted text as data and blocking injection paths into tool calls.
AI Agents Cost Controls for Agent Fleets: Budgets, Caps, and Kill Switches
Token spend is a product feature. Per-run budgets, model tiers by state, revision caps, and kill switches keep agent fleets from becoming surprise invoices.
AI Agents Idempotent Agent Tool Writes: Retries Without Double Emails or Double Charges
Make agent tool writes safe on timeout by minting runtime idempotency keys once, then sharing them across model, harness, and HTTP retries.
AI Agents The Fractional AI CTO Model: When You Need Architecture, Not Another Chatbot
A fractional AI CTO is architecture and operating cadence for teams building agentic IP — not a chatbot retainer. When to hire one, what they own, and how it differs from a pilot.
AI Agents Scoping an Agentic Pilot That Proves Value in Five Days
A good agentic pilot is one sentence, real data, an evaluator, and a cage — finished in five days for $1,500. Here is how to scope it so you learn something true.
AI Agents Pin the Model, Gate the Upgrade: Catch Agent Drift Before Customers Do
Yes—pin production agents to explicit model IDs. Floating aliases change behavior with no deploy. Upgrade only through a golden-set gate you actually run.
AI Agents Observability for Agents: Traces, Scores, and the Dashboard Ops Actually Reads
Provider dashboards will not catch silent wrongness. Agent observability means traces with states and tool calls, evaluator scores online, and a weekly dashboard humans actually use.
AI Agents Tool Schemas Agents Follow: Descriptions, Enums, and Killing the Omnibus Tool
Agents invent arguments when schemas are vague. Write JSON Schema like agent UX—enums, required fields, property descriptions—and kill the do_anything tool.
AI Agents When Not to Build an Agent (And What to Build Instead)
Agents are for uncertain paths with clear criteria. If the path is known, build automation. If criteria are mush, fix the process. Here is how to choose without theater.
AI Agents Durable Agent Runtimes: Survive Restarts Without Calling It "Memory"
Durability is resume-correct state across crashes and human waits—not memory. Choose LangGraph, Durable Objects, or Temporal by the failure you must survive.
AI Agents Pre-Execution Policy Gates: The Kill Switch That Lives Outside the Prompt
Your agent’s kill switch is a pre-execution policy gate outside the prompt: allow, deny, or pending-approval before side effects — fail closed on outages.
AI Agents LLM-as-Judge Reliability: Calibrate the Scorer Before You Trust the Score
An LLM judge is a noisy instrument, not ground truth. Calibrate against human labels, kill position and verbosity bias, re-check after model or rubric changes.
AI Agents Agent Loop vs LLM-in-Workflow: Pick the Shape That Matches the Uncertainty
Most jobs need a workflow with one LLM step, not an agent loop. Use a three-tier test — certainty, branching, blast radius — then prove hybrid is enough.