Getting Started with AI Agents

An AI agent is a system where an LLM autonomously decides which actions to take to accomplish a goal. Unlike a single prompt → response call, an agent runs a loop: it reasons about the task, picks a tool, observes the result, and decides what to do next.

The minimum ingredients are:

• A model — usually a chat LLM that supports tool/function calling.
• Tools — functions the model can invoke (search, code exec, APIs).
• A control loop — orchestration logic that runs until the goal is met.
• Memory — short-term (conversation) and long-term (vector store).

Tool Calling / Function Calling

The model returns a structured JSON payload describing which function to invoke with what arguments. Your runtime executes it and feeds the result back.

RAG (Retrieval-Augmented Generation)

Inject relevant documents from a vector store into the prompt so the model can answer with grounded, up-to-date information. Indexed under Memory & Knowledge.

MCP (Model Context Protocol)

An open protocol from Anthropic for connecting LLMs to external tools and data sources in a standardized way. Becoming the USB-C of agent tooling.

Guardrails

Runtime checks that validate inputs, outputs, and tool calls — to prevent prompt injection, PII leaks, and policy violations. See Security & Guardrails.

Evals

Automated test suites for non-deterministic LLM behavior. Treat them like CI for your agent. See Observability & Testing.

Sandbox / Runtime

An isolated environment where the agent can safely execute generated code or browse the web. See Runtimes & Compliance.

There is no single best framework — the right pick depends on what you're building and what your team already knows. Rough heuristics:

1. Pick a model with reliable tool calling: GPT-4o, Claude 3.5 Sonnet, or Gemini 1.5 Pro. Start with a hosted API — don't self-host yet.
2. Pick a framework. Beginners: start with LangGraph or CrewAI.
3. Define 2–3 tools — e.g. web search, a calculator, and one domain-specific API. Resist the urge to add 20 tools on day one.
4. Add observability from day one. Wire up Langfuse or LangSmith before you write your first eval — you'll thank yourself.
5. Ship a thin slice behind a feature flag. Iterate on real user traces, not imagined ones.

• Always sandbox code execution. Never run model-generated code on a host with credentials.
• Always validate tool inputs. The model will eventually hallucinate a destructive call.
• Always rate-limit and budget-cap. A runaway loop can burn $1,000 in API calls in an hour.
• Always log to a trace store with PII redaction, not raw stdout.
• Always have a human-in-the-loop fallback for irreversible actions (payments, deletes, emails).