Agent component with tools

Each tool connected to the Agent component is presented to the model with a name, a description and its input schema. The model reads the question and those descriptions and emits a tool call, Langflow runs it and feeds the result back, and this loops until the model gives a final answer. So the biggest lever is the tool descriptions: one clear purpose per tool, when to use it, and what inputs it expects. I would also tighten the system instructions, for example 'use the calculator for any arithmetic', remove tools the flow does not need, and then check traces in Langfuse or LangSmith to see where it picks the wrong tool and adjust wording based on real failures rather than guesses.

The engineer's question enters through Chat Input with a session id. Message history loads earlier turns for that session so follow-ups keep context. The Agent component receives the question, system instructions (read-only, cite sources, never guess numbers) and its connected tools: Calculator, URL fetch for public status pages, API Request in tool mode limited to GET on allowlisted hosts, and a runbook retrieval flow attached as a tool. The agent calls tools, checks whether the results answer the question, and repeats up to an iteration cap. Chat Output returns the answer with each figure and its source. Langfuse or LangSmith tracing records every model and tool call. After passing Playground tests, bots call the same flow through POST /api/v1/run/<flow-id>, with tweaks per environment.

I use layers, because a prompt alone is not a guarantee. The system instructions forbid guessing and require every figure to carry its source. All arithmetic goes through the Calculator so derived numbers are also tool outputs. The answer format pairs each figure with the tool that produced it, which makes gaps visible to the reader. I add a post-check that compares numbers in the final answer against numbers in the traced tool outputs and flags or blocks mismatches. The test set includes questions where tools return nothing, and the expected answer is a plain statement that the data is not available. Finally, I sample traces regularly to catch anything the checks miss.

Instructions shape model behaviour but do not guarantee it. Injected text from a fetched page, a confusing request or a model change can all lead to an unexpected call. Real enforcement sits outside the model. API Request is configured for GET only and restricted to allowlisted hosts. The credentials it uses belong to a read-only service account scoped to metrics, so a write would fail even if attempted. Network egress rules can block other destinations. Most importantly, no write-capable tool is connected at all: if no tool can write, the agent cannot write. Traces then let us audit that every call was a read.

I start from real on-call questions and group them. Single-tool questions, multi-tool questions like the error budget plus runbook example, follow-ups such as 'and for payments?', ambiguous questions, and cases where data is missing. I also add adversarial cases: requests to restart a service, and fetched pages with injected instructions. For each case I write down the expected tools and the expected properties of the answer: figures match the tool output, sources are cited, and writes are refused. I run the cases in the Playground and inspect every tool call, not just the final text. The set is rerun whenever the model, prompt or tool descriptions change. Later I automate it through the run API.

I treat fetched content as untrusted data, never as instructions. URL fetch is limited to allowlisted public status pages, which narrows exposure. The system instructions tell the agent that tool output is information to report, not commands to follow. The strongest control is structural: there are no write tools and the credentials are read-only, so a successful injection can at worst distort the text of an answer. It cannot change production. Source citations let the engineer see where a claim came from. I keep injected-page cases in the test set. Because distorted answers are still possible, I also review traces for odd tool sequences that follow a fetch.

Separation gives me a unit I can test and change on its own. The runbook flow has its own retrieval steps, vector store and prompt, and I can check its quality in the Playground with retrieval-specific questions without the agent's behaviour getting in the way. The agent only sees one tool with a clear description, which keeps its tool list small and its choices clearer. The same flow can be reused by other agents, called directly over the API, or exposed to MCP clients when Langflow acts as an MCP server. The cost is an extra hop and another place to monitor, so I make sure tracing covers both flows and the sub-flow returns compact output with sources.

First I keep every tool read-only. API Request only does GET calls to an allowlist of internal hosts, with a service credential that has read scope, so even a confused or manipulated agent cannot change anything. Second, I treat fetched pages and API responses as untrusted text, because they can contain instructions that try to redirect the agent; the system prompt says tool output is data, not instructions, and the host allowlist limits exposure. Third, I cap iterations and response size to avoid runaway loops and cost. Fourth, I use per-user session ids so memory does not leak between people. Finally, tracing records every call, so we can audit what the agent looked at and why it answered the way it did.

Without a cap, a failing tool or a confused model can loop, burning tokens and adding latency while the engineer waits during an incident. I set the cap from Playground traces: enough for the longest legitimate multi-tool question in the test set, plus some margin. When the cap is hit, the agent should not fill gaps with guesses. It returns what it did find, with sources, states clearly what it could not determine, and suggests a clarification or a place to check manually. I also count cap hits as a metric. A rising rate usually points to a broken tool, a vague tool description or a prompt regression.

I find the trace in Langfuse or LangSmith by session id and time, then walk the steps in order. Did the agent pick the metrics tool at all? Did it pass the right service name and time window? Did the metrics API return correct data, which I can check against the source system? Did the Calculator get the right operands? Did the model copy the result accurately into the answer? That sorts the failure into one of a few causes: wrong tool choice, wrong parameters, bad upstream data, wrong math inputs, or transcription. Each has a different fix: tool descriptions, prompt, data owner, or output checks. I then add the case to the test set.

Language models can make arithmetic mistakes, especially with percentages, multi-step calculations and many decimal places. Error budget math is exactly that kind of work. The Calculator is deterministic. Its inputs and output appear in the trace, so anyone can audit how a number was derived. It also satisfies the rule that every figure must come from a tool, because derived numbers become tool outputs too. The cost is an extra tool call and some latency, which is acceptable for numbers people act on during incidents. I also have the agent show the formula it used, so the engineer can sanity-check the result quickly.

First, the API Request tool needs a sensible timeout so a hung call does not stall the whole answer. When the tool returns an error, the agent must not estimate or reuse an old number. It can retry once, which is safe because the call is a GET. It can try another source only if that source genuinely answers the same question. Otherwise it tells the engineer that the metrics source failed, includes the error, and suggests where to check directly. Any partial answer must be clearly labelled as partial. The trace records the failure, and alerts on tool error rates let us spot a broken dependency before engineers do.

I would derive the numbers rather than guess them. First, estimate the request volume: engineers times questions per shift, plus expected bot calls. From Playground traces I measure the average and p95 number of model calls per question, the tokens per call, and the tool calls per question. Cost per request is roughly model calls times tokens times the provider's current pricing. Latency is roughly the sum of sequential model and tool calls. I check rate limits on the model provider and on the metrics API, since tool calls multiply load on internal systems. Then I confirm the estimates with a load test that scripts calls against the run endpoint in staging.

Every caller authenticates with a Langflow API key, one key per bot so access can be revoked and traced to a caller. The Langflow instance should not be publicly reachable. It sits behind network controls or a gateway with TLS and rate limits. Edit rights to flows are restricted separately from run rights. Tweaks deserve special care: a caller could override the API base URL to point the tool somewhere else, or swap the model. So either only trusted callers get keys, or a gateway validates which tweaks are allowed. Caller identity should appear in traces so every answer can be attributed.

I write one when I want the rules enforced in code rather than in configuration. A custom component is a Python class with defined inputs and outputs. It can hardcode the host and allow only GET. It can validate parameters, for example checking the service name against a known list and keeping time windows bounded. It can trim responses to the fields the agent needs and normalise errors into clear messages. It can redact sensitive values before they reach the model or traces. It can also be switched to tool mode with a precise description. The trade-off is code that must be reviewed, tested and maintained, but the tool becomes narrower and safer.

The agent should refuse clearly and briefly. It explains that it is read-only, then helps in the ways it can. It shows the relevant runbook steps for a restart, with the source, and points to the proper human or tooling process for making the change. Because no write-capable tool is connected and the credentials are read-only, an actual restart is impossible even if the model misbehaved. I keep direct and indirect versions of this request in the test set, including ones hidden inside fetched content. I check in the Playground and in traces that no unexpected tool call was attempted.

I start with the traces to see where time actually goes. Common wins are clearer tool descriptions so the agent stops making wasted calls, and trimming message history so prompts stay small. Setting tool timeouts prevents slow dependencies from dominating. Short-lived caching of runbook retrieval or status page results helps, as long as each cached value carries its fetch time and the answer shows it. A faster model may work if it passes the same test set. Streaming the output improves perceived speed in chat. What I never do is skip a tool call and let the model estimate a number, because that trades correctness for speed.

They should never be hardcoded in the flow, because flow definitions get exported, shared and committed to version control. I keep them in Langflow's stored variables or in environment variables injected at deploy time, with a separate read-only service account per environment. They must never appear in prompts or tool descriptions, where the model could repeat them in an answer. I rotate them on a schedule. I also check that traces sent to Langfuse or LangSmith do not capture authorization headers, and redact them if they do. That way a leaked trace or flow export does not expose access to internal systems.

Once exposed, any authorised MCP client, such as an IDE assistant or another agent, can call the flow. That widens the audience and the range of inputs. The read-only guarantee still holds because it is enforced in the tools and credentials, not in the caller. The new risk is downstream: another agent might act on our output, for example a status reading, and take a write action we never approved. So the tool description must state clearly that it returns read-only information with sources. Access needs authentication. Usage should be monitored per client. I would also expect callers to treat our answer as advisory, not as an instruction to act.

These are separate point-in-time reads, not a transaction, so the goal is honesty about timing rather than perfect consistency. Every metric call for one answer uses the same explicit time window, such as month to date. The answer states an 'as of' time for each source. If the status page says all systems are healthy while the metrics show elevated errors, the agent reports both with their sources instead of quietly picking one. Any caching must keep the fetch timestamp so values of different ages are never presented as simultaneous. For incident decisions, the engineer then knows exactly how fresh each piece of information is.

Message history supplies the previous turn, so the agent can infer the intended question: the same error budget question, for the payments service. It should then call the tools again with the new service and never reuse checkout's figures. Two things can go wrong. History may be missing or truncated, for example if the session id changed. Or 'payments' may map to several services. In the second case the agent should ask which one is meant rather than pick one. I keep explicit follow-up cases in the test set and check in traces that the tool parameters changed correctly.

Only what follow-ups need: recent turns for that session. I trim or summarise older turns to control token cost and stay within context limits. Operations conversations can contain internal metrics and incident details, so history needs a retention policy and deletion after a defined period. Access should be limited to the people and services that need it. Session ids should be unique, hard to guess and tied to one user or one bot task, so history never leaks between engineers. Bots that ask one-off questions through the API should use a fresh session id per task, so earlier tasks do not influence the answer.

Each call must be isolated, which means unique session ids per conversation or task so contexts never mix. Capacity is the next concern. Langflow workers, the model provider's rate limits and the metrics API's limits all bound throughput. Tool calls fan out load onto internal systems, so concurrent bots can overload them. Bots should set timeouts and retry with backoff. Retries are safe here because every tool call is a read. To scale Langflow horizontally, message history and state should live in a shared database rather than on one instance. I would also apply per-key rate limits so one misbehaving bot cannot starve everyone else.

Each answer gets a record with: session id and caller identity, flow version, model, the user question, and an ordered list of tool calls. Each tool call holds the tool name, inputs, output, status, latency and timestamp. The record also holds the final answer, a mapping from each figure in the answer to the tool call that produced it, the iteration count, whether the cap was hit, and any errors. Langfuse or LangSmith traces capture most of this already. The figure-to-source mapping makes the no-invention rule checkable, and stored records can be turned into regression test cases.

A model change can alter tool choice, how arguments are formatted, and how strictly instructions are followed, so I treat it as a release, not a setting change. I pin specific model versions. I pass the candidate model as a tweak in staging and run the full test set. I then compare traces with the current model: tool sequences, iteration counts, figure accuracy and refusals of write requests. Only after it passes do I change production. Because tweaks apply per request, I also make sure callers cannot switch production to an untested model on their own.

I export the flow definition to version control and review the diffs like code, including prompts and tool descriptions. The runbook retrieval flow is a dependency, so it is versioned and promoted together with the agent flow. Separate Langflow instances or projects host each environment. Values that differ between environments, such as API base URLs and models, come from variables or tweaks rather than from edits to the flow. Flow ids can differ between instances, so callers read them from configuration. After importing, I run the test set before switching traffic. Rollback means re-importing the previous version.

First I look at the trace to see the exact query the agent sent. A poorly phrased query is a common cause, and the fix there is in the tool description or the prompt. Then I test the runbook flow on its own in the Playground with that query. Typical retrieval problems are poor chunking, missing metadata such as the service name, or weak embeddings for operations jargon. Fixes include metadata filters by service, chunking by runbook section, and returning section titles or links for citations. A relevance threshold lets the flow return 'no match' instead of weak results, so the agent can say the runbook has no guidance.

Traces capture inputs and outputs of every model and tool call, so they can contain internal metrics, hostnames, tokens or personal data from incident chats. I redact authorization headers and secrets before they are recorded. Where practical, I trim tool responses to the needed fields, which a custom component can do. Access to the tracing project is restricted, with a defined retention period. Langfuse can be self-hosted, so if policy forbids sending operational data to a third party, I would run it inside our own environment. Data-handling approval should come before tracing is turned on in production.

When the ambiguity is something tools cannot resolve: which service, which environment, or which time window. Calling tools with guessed parameters produces confident, well-cited answers to the wrong question, which is worse than asking. I write this into the instructions and cover it in the test set. The balance matters, because too many clarifications frustrate engineers during incidents. If the team has an agreed default, such as month to date for error budgets, the agent can use it. It must then state that assumption in the answer so the engineer can correct it.

With four tools, one agent is simpler. It is easier to trace, faster, and tool choice is usually reliable. As tools grow, choices degrade because descriptions start to overlap. Then I would group tools by domain into sub-flows connected as tools, the same pattern already used for the runbook retrieval. Each sub-flow can be tested on its own and owned by a different team. A deeper multi-agent setup adds latency, cost and harder debugging, because one bad answer spans several traces. So I would split only when traces show tool confusion that better descriptions cannot fix.

Normally yes. Tracing is observability, not part of the answer path, so an outage in Langfuse or LangSmith should not fail user requests. I would check how the deployed Langflow version behaves when the tracing backend is unreachable, rather than assume, and test that in staging. An alert should fire when traces stop arriving. The real decision is policy. If the team requires every operations answer to be auditable, keep a fallback such as local structured logs of tool calls. If even that is unavailable, it may be acceptable to degrade the service rather than run with no audit trail.

The run API suits request-response use: a bot asks a question and needs the answer in the response to post back to chat. A webhook suits event-triggered work, for example an alert firing that should start a flow which gathers metrics and runbook guidance. The caller does not wait for an answer. With a webhook, the flow itself must deliver its output somewhere, and that delivery must stay within the read-only rule, such as posting a summary to a channel. For the interactive Q&A in this workflow, the run API is the better fit.

Possibly. Langflow can act as an MCP client, so the Agent can use those tools directly. The benefit is tools with schemas and descriptions maintained by the team that owns the system, which often means better tool choice and fewer integration bugs. The risk is that an MCP server may expose write operations alongside reads. I would connect only the read tools, confirm that the credentials are read-only, and require authentication. API Request gives me direct control over method and host. So the choice depends on how much I trust and can restrict what the MCP server exposes.

1. AIt is switched to tool mode and connected to the Agent's tools inputCorrect answer
2. BIt is placed anywhere on the canvas and the agent discovers it automatically
3. CIt must be rewritten as an MCP server first
4. DIt is pasted into the Agent's system prompt as text

Why: The Agent component takes tools, and ordinary components are exposed as tools by switching them to tool mode and connecting them to the agent; other flows and MCP tools can also be connected.

1. APUT /api/v1/run/<flow-id>
2. BGET /api/v1/flows/<flow-id>
3. CPOST /api/v1/run/<flow-id>Correct answer
4. DPOST /api/v1/agent/<flow-id>/invoke

Why: Langflow runs flows through POST /api/v1/run/<flow-id>, optionally with tweaks.

1. AAsking engineers not to request changes
2. BA system prompt that says 'read-only'
3. CA larger, more capable model
4. DGET-only API Request on allowlisted hosts with read-only credentials and no write tools connectedCorrect answer

Why: Enforcement outside the model holds even if the model is confused or manipulated, whereas prompts only influence behaviour.

1. APass tweaks in the request to override that component settingCorrect answer
2. BEdit the flow in the canvas before each call
3. CPut the URL in the chat message and hope the agent uses it
4. DDuplicate the flow for staging

Why: Tweaks in the run API request override component settings at run time, so one flow can serve several environments.

1. ATracing is enabled in Langfuse
2. BBoth conversations use the same session id for message historyCorrect answer
3. CThe Agent component has too many tools
4. DThe Calculator tool caches results

Why: Message history stores chat memory per session id, so a shared session id mixes the two conversations.

1. AA caller could override settings such as the API base URL or modelCorrect answer
2. BTweaks bypass the Playground test set automatically
3. CTweaks permanently delete the saved flow
4. DTweaks disable message history for all users

Why: Tweaks override component settings per request, so an untrusted caller could redirect a tool or swap in an untested model.

1. AThe runbook retrieval flow
2. BThe model's own reasoning
3. CCalculatorCorrect answer
4. DURL fetch

Why: The Calculator gives deterministic, traceable math, which satisfies the rule that figures come from tool calls.

1. AIt only proxies calls to OpenAI
2. BIt exposes flows as tools that MCP clients can callCorrect answer
3. CIt stores model weights for MCP clients
4. DIt replaces the Agent component

Why: As an MCP server, Langflow exposes flows as tools to MCP clients.

1. AReturn an empty response
2. BReport what it found with sources and state clearly what it could not determineCorrect answer
3. CEstimate the missing numbers from similar services
4. DSilently retry forever

Why: The cap prevents loops, and the no-guessing rule means gaps must be stated, not filled.

1. ANothing; the request should crash
2. BLast month's figure presented as current
3. CA statement that the metrics source failed, with the error, and no invented figureCorrect answer
4. DA figure estimated from the status page colour

Why: Numbers must come from tool calls, so a failed source has to be reported honestly rather than replaced with a guess.

1. AThe page uses HTTPS
2. BThe Calculator is connected
3. CNo write-capable tools or credentials are available to the agentCorrect answer
4. DThe answer is streamed

Why: If no tool can write, injected instructions cannot change production and can at most distort text.

1. ACall API Request with POST
2. BRefuse, explain it is read-only, and point to runbook steps and the proper processCorrect answer
3. CAsk for the engineer's password
4. DRetry until the restart succeeds

Why: The agent must never take write actions, but it can still help by sharing guidance with its source.

1. AThe model's name
2. BThe ordered tool calls with their inputs and outputsCorrect answer
3. CThe final answer text only
4. DTotal token count

Why: The sequence of tool calls and their inputs and outputs reveals which tool was chosen and with what parameters.

1. AComparing numbers in the answer against numbers in the traced tool outputsCorrect answer
2. BChecking the answer length
3. CCounting how many tools exist
4. DMeasuring latency

Why: Any figure with no matching tool output is a candidate invention and can be flagged or blocked.

1. AArithmetic errors
2. BThe agent being steered to call arbitrary internal or external URLsCorrect answer
3. CSlow model responses
4. DSession mixing

Why: An allowlist stops manipulated or mistaken calls to unintended hosts, limiting request forgery and data leakage.

1. AA new unique session id for that taskCorrect answer
2. BThe same fixed session id used by all bots
3. CThe on-call engineer's session id
4. DNo session id, so it shares global history

Why: A fresh, unique session id keeps unrelated tasks from inheriting earlier context through message history.

1. AA record of every flow version
2. BLong-term storage of runbooks
3. CA cache of tool outputs shared across sessions
4. DChat memory of earlier turns for a given session idCorrect answer

Why: Message history gives per-session chat memory so follow-ups keep their context.

1. AAs SQL views
2. BAs JavaScript plugins loaded in the browser
3. CAs YAML files with prompt templates
4. DAs Python classes with inputs and outputsCorrect answer

Why: Langflow custom components are Python classes that declare inputs and outputs.

1. ALangfuse and LangSmithCorrect answer
2. BPrometheus and Grafana only
3. CKafka and Spark
4. DJaeger only

Why: The workflow traces to Langfuse or LangSmith, recording inputs, outputs and latency of each call.

1. AOnly in database logs
2. BOnly in the API response headers
3. CNowhere until production
4. DThe PlaygroundCorrect answer

Why: The Playground shows every tool call and its output for review during testing.

1. AIn the agent's system instructions
2. BIn the tool description
3. CHardcoded in the exported flow JSON
4. DIn stored variables or environment variables, using a read-only accountCorrect answer

Why: Keeping secrets out of flow definitions and prompts prevents leaks through exports or model output.

1. AReport both with sources and as-of timesCorrect answer
2. BOmit both
3. CPick the more reassuring source
4. DAverage them

Why: Separate point-in-time reads are not consistent, so transparency about source and timing is the honest approach.

1. ATracing is disabled
2. BThe session id changes per message or message history is not connectedCorrect answer
3. CThe Calculator is disconnected
4. DThe URL fetch allowlist is too strict

Why: Context comes from message history keyed by session id, so a changing id or missing history drops earlier turns.

1. AWith a per-bot Langflow API keyCorrect answer
2. BBy sending the metrics API password
3. CWith the on-call engineer's session id
4. DNo authentication on an internal network

Why: Per-bot API keys allow access control, revocation and attribution of each call.

1. ARemove the Calculator
2. BRun the test set against the new model in staging and compare tracesCorrect answer
3. CChange it directly in production and watch for complaints
4. DIncrease the iteration cap

Why: Model changes can alter tool choice and instruction-following, so they need regression testing first.

1. AModel calls and tokens per question, taken from test tracesCorrect answer
2. BNumber of components on the canvas
3. CSize of the flow JSON
4. DNumber of engineers on the team only

Why: Cost scales with model calls and tokens per request, which traces measure directly.

1. AAverage all matching services
2. BAsk which service is meant before calling toolsCorrect answer
3. CRefuse to answer
4. DPick the first match

Why: Guessing parameters produces confident wrong answers, so the agent should clarify ambiguity that tools cannot resolve.

1. A1%
2. B99.9%
3. C0.01%
4. D0.1%Correct answer

Why: The error budget is one minus the SLO target, so 100% − 99.9% = 0.1% of requests may fail.

1. ALangflow deduplicates all calls
2. BThe model remembers previous results
3. CTools only perform GET reads, which do not change stateCorrect answer
4. DRetries are free

Why: Read-only GET calls do not change state, so repeating them causes no side effects.

1. AFail every request until it recovers
2. BDisable the Agent component
3. CKeep answering, alert on missing traces, and keep fallback logs if audit is mandatoryCorrect answer
4. DSwitch to a different model

Why: Tracing is observability, so its outage should not normally block answers, but audit requirements need a fallback.

1. AWhen a bot needs the answer back in the same response
2. BWhen starting the flow from an event such as an alert, without waiting for a replyCorrect answer
3. CWhen testing in the Playground
4. DWhen changing the model per request

Why: Webhooks fit event triggers, while the run API fits synchronous question-and-answer calls.

1. ABy Langflow acting as an MCP client, connecting only the read toolsCorrect answer
2. BThrough the Calculator
3. CIt cannot; only local components work
4. DOnly by copying their code into a prompt

Why: Langflow can act as an MCP client and pass MCP tools to the Agent, but write tools should be excluded.

1. ATo change the session id
2. BTo enable tool mode
3. CTo make the Calculator faster
4. DTo control token cost and stay within the model's context limitCorrect answer

Why: Long histories add tokens to every model call and can exceed context limits.