PDF Q&A flow served as an API

I treat isolation as a backend responsibility, not a flow responsibility. Each tenant gets its own collection or namespace in the vector store, and the ingest flow writes there using a tweak the backend sets from the authenticated session. On every chat call, the backend again sets the vector store component's scope through tweaks, and it ignores any tweak fields coming from the client. The Langflow server sits on a private network with an API key only the backend holds, so nobody can call the run API directly. I also add a test suite that asks tenant A's questions with tenant B's credentials and checks nothing leaks, and traces in Langfuse let me audit which chunks were retrieved for each run.

There are three parts: a web app with a backend, two Langflow flows, and shared storage. The user signs in, and the backend resolves the tenant and holds the Langflow API key. On upload, the backend saves the PDF to object storage and calls the ingest flow through POST /api/v1/run/<ingest-flow-id>. That flow loads, splits, embeds and upserts chunks into the tenant's namespace. For a question, the backend calls the chat flow with a session_id and tweaks that pin the tenant scope. The flow retrieves top-k chunks, runs a relevance gate, builds the prompt with history, and the model answers with citations. The backend parses and renders the answer, and Langfuse or LangSmith traces every run.

The run API takes tweaks that can override any component setting, including the vector store's namespace or collection. Anyone holding the key can call any flow with any tweaks. That means they could query another tenant's documents, run up model costs, or ingest junk. Keeping the key server-side means every call passes through code that has already authenticated the user and set the tenant scope itself. If the key leaks, I rotate it straight away and review the traces and logs for unusual flows, tenants or volumes. I also check that the Langflow instance is not reachable from the public internet, so that only the backend's network can call it.

It is a trade-off. Small chunks give precise retrieval, but they lose surrounding context and split tables or arguments in half. Large chunks keep context, but they dilute the embedding and use more prompt tokens per retrieved chunk. Overlap stops a key sentence being cut at a boundary, but it adds duplicated text and storage. I start with a moderate size and some overlap. Then I measure on a labelled set of real questions from the target PDFs: did the right chunk appear in the top-k, and was the answer correct? I tune from those results rather than from intuition. Any change means re-ingesting every document, so I version the settings in the chunk metadata.

Models are inclined to be helpful. Even when told to answer only from context, they sometimes fill gaps with general knowledge, and that text then looks as if it came from the PDF. The custom component checks retrieval scores and chunk count. If nothing passes, it returns a fixed 'not found in your documents' message and the model is never called. The behaviour is deterministic, it is cheap because no tokens are spent, and it is easy to test. I still keep the instruction to refuse in the prompt as a second layer. The gate's threshold has to be tuned per embedding model, and I log every gated question so I can find gaps in recall.

I use several layers. First, the relevance gate stops the model from running at all when retrieval is weak. Second, the system prompt restricts answers to the supplied context and requires a source label for each claim. Third, every chunk enters the prompt with a label carrying its document and page, so citations can be checked. Fourth, the backend validates the citations after generation: each cited label must match a chunk that was actually retrieved in that run. If an answer has no valid citations, I show it with a clear warning or replace it with the not-found message. I also sample traces and score faithfulness offline, so that drift shows up early.

I would reject it. Tweaks can override the vector store's namespace, the file path, the model or the prompt. If the client controls them, any user can point retrieval at another tenant's scope or read arbitrary file paths. The browser should send only the question, the conversation id and the document selection. The backend builds the tweaks from server-side state: the tenant from the authenticated session, and allowed document ids checked against the database. I would also keep an allowlist of which components and fields the backend may tweak. The backend work saved by the proposal is trivial, and the security risk is the exact failure the system must never have.

The run API lets the AI part evolve without redeploying the web app. Someone can change the chunk size, swap the model, or rewrite the prompt in the canvas, test it in the Playground, and the app keeps calling the same POST /api/v1/run/<flow-id> endpoint. Tweaks give the backend controlled per-request overrides such as tenant scope or temperature without forking the flow. The cost is a contract between app and flow: tweaks reference component ids, so I version flows, keep a staging flow id, and run a regression set before switching production. If the pipeline becomes very stable and latency-critical, moving parts into code is an option, but early on the iteration speed matters more.

First I split the failure into retrieval or generation. Using traces from Langfuse or LangSmith, I look at the exact chunks retrieved for the bad answer. If the right page is missing, it is a retrieval problem: chunking that cuts tables apart, scanned pages with no text, wrong top-k, or a query that needs rephrasing. If the right chunk was there but the answer is still wrong, it is the prompt or model: instructions too loose, too much context, or no rule to say 'not found'. I build a labelled set of questions with expected pages, measure hit rate and groundedness, change one thing at a time in the flow, and compare runs before promoting the new version.

A separate namespace or collection per tenant gives stronger isolation. A forgotten filter cannot leak data, deletion per tenant is simple, and noisy tenants are easier to reason about. The cost is operational: many tenants mean many collections, and some vector stores handle that less well. A shared index with a metadata filter scales to many small tenants more cheaply, but every query depends on the filter being present and correct. My default is a namespace per tenant, with tenant_id also stored in the metadata as defence in depth. I would only move to a shared index after confirming the vector store enforces filters efficiently, and I would add tests that try cross-tenant queries.

I track the document's status in the app database: uploaded, ingesting, ready or failed. Chat queries only include documents marked ready, by filtering on document ids in the metadata. Vector ids are deterministic, for example the document id, a chunk index and the splitter version. That makes retries upserts rather than duplicates. On failure, the backend either retries the whole ingest run, which is safe because upserts are idempotent, or deletes all vectors carrying that document id and marks the document failed. The user sees a clear status with a retry button. The trace records where the failure happened, such as PDF parsing, embedding rate limits or vector store errors.

Deletion has to cover every copy of the data, not just the file. That means the object in object storage, every vector and its chunk text in the tenant's namespace (deleted by document id metadata), and the database record. Two places are easy to miss. Message history may hold answers quoting the document, and traces in Langfuse or LangSmith may hold retrieved chunks and prompts. I need a retention policy, or a deletion path, for both. Afterwards I verify the erasure by running a retrieval for that document id and expecting zero results. I also make deletion an audited, asynchronous job with a status, because erasing from several stores can partly fail and must be retried.

I would not guess numbers. I would build the estimate from measured inputs: uploads per day per tenant, average pages per PDF, chunks per page at the chosen size and overlap, and therefore embeddings per day. Each chunk costs an embedding call and stores a vector plus its text and metadata. So vector storage grows with the total chunk count multiplied by the vector dimension and metadata size. I then check the embedding provider's rate limits and the vector store's write throughput against peak upload bursts, not daily averages. Finally, I load-test the ingest flow with representative PDFs to get real per-document latency, and size workers and queues from that.

For anything beyond small files, no. Parsing, splitting and embedding a long PDF can take longer than a typical HTTP timeout. Users will also retry when an upload looks stuck. Instead, the upload request stores the file, creates a document record with the status 'ingesting', and puts a job on a queue. A worker calls the ingest flow's run API, updates the status when it finishes, and retries with backoff on errors that may clear on their own. The web app polls or receives a notification when the document is ready. This also lets me cap concurrent ingest runs, which protects Langflow and the embedding provider's rate limits.

That is indirect prompt injection through retrieved content. The defences are layered. The system prompt marks retrieved chunks as untrusted data to quote, not instructions to follow, and the chunks sit inside clear delimiters. The chat flow has no tools or agent, so an injected instruction cannot trigger actions; at worst it changes the text of the answer. Tenant scope is set by the backend, so injection cannot widen retrieval. Output checks validate the citations and can flag answers that echo the system prompt. Because each tenant only retrieves its own documents, a malicious PDF mostly affects the tenant who uploaded it, which limits the blast radius.

I treat flows as deployable artefacts. I export the flow definitions and keep them in version control, and I run separate Langflow environments for development, staging and production. The backend reads the flow ids from configuration per environment and never hard-codes them. Before promoting a change, I run the evaluation set against the staging flow and compare the results in Langfuse or LangSmith with the current version. Changes to the ingest flow that affect chunking or embeddings need a re-ingest plan, so they ship as a migration. Rollback means pointing the configuration back at the previous flow id. That only works if the vector data is still compatible with it.

I keep an evaluation set of real questions per document type. Each question has the expected source pages and a reference answer, and some questions are deliberately unanswerable. For each candidate flow version I measure four things. Retrieval: did the expected chunks appear? Faithfulness: is every claim supported by the cited chunks? Correctness against the reference answer. Refusal: did unanswerable questions get the not-found message? Tracing lets me compare versions side by side on latency and token use too. I check a sample of scores by hand, because automatic judges make mistakes. A change ships only if it holds steady on faithfulness and cross-tenant safety tests, even if other metrics improve.

Traces are a second copy of customer data, so they need the same controls. I restrict access to the tracing project by role and set a retention period. I make sure traces are covered by the deletion process when a document or tenant is erased. If traces go to a hosted service, that has to match our data-processing agreements and data residency commitments. For sensitive tenants I might keep only chunk ids, scores and metadata instead of the full text. I would also mask obvious personal data in questions. The goal is to keep enough detail to debug bad answers without making the tracing tool a weak spot.

Without handling, retrieval finds nothing and the gate says 'not found in your documents'. That is technically true, but it misleads the user. The document's status lives in the app database, so the backend knows it is still ingesting. The UI should show that status, and either disable questions about that document or answer with a clear 'still processing' message. Chat queries filter on document ids that are marked ready, so a partly indexed document is never searched. This is eventual consistency between the database and the vector store, and the fix is to make that state visible to the user rather than hide it.

I treat it as a new document version with its own id. I ingest the new version fully first, then switch the database pointer so queries use it, and only then delete the old version's vectors. This avoids a window where neither version is searchable, and it avoids mixing chunks from both versions in one answer. Version or document ids go into the chunk metadata, so retrieval can filter to the current version. If users need history, I keep old versions selectable rather than deleting them. Citations should name the version, so a page reference stays accurate after the update.

The ingest flow would produce few or no chunks, and the document would look ready but answer nothing. So I add a check after loading: if the extracted text is empty or very short relative to the page count, I mark the document as needing OCR instead of 'ready'. Then I either route it through an OCR step, which could be a custom component or a separate service, or I tell the user that the file has no readable text. OCR output is noisier, so I flag those chunks in the metadata. That helps when debugging poor answers and lets me show users that the source text is OCR-derived.

A higher k raises recall: the right chunk is more likely to be included. But it costs more prompt tokens, adds latency, and gives the model more irrelevant text to be distracted by. A lower k is cheaper and more focused, but it misses answers that are spread across sections. I tune k on an evaluation set by checking whether the needed chunks appear within the top-k and whether answer quality holds. I set k together with the chunk size, because larger chunks need fewer of them. I also combine k with the score threshold in the gate, so weak matches inside the top-k can still be dropped.

The prompt includes only the most recent turns for the session_id, not the full history. That keeps tokens and latency bounded and leaves room for the retrieved chunks, which matter more for grounding. Older turns stay stored, so users can scroll back. If long conversations really need earlier context, I could add a summary of older turns. But a summary can drift, so it must never be treated as a source the answer can cite. I also set a retention period for stored history, since it can contain sensitive questions and quoted document text, and I delete it when a user or tenant asks.

The similarity search runs on the raw question, and 'the second one' carries almost no meaning without the earlier turns. So the retrieved chunks are weak and the gate returns 'not found'. The fix is to rewrite the question before retrieval. A small model call takes the recent turns and the new question and produces a standalone query, for example 'what is the second termination clause in the contract'. That rewritten query goes to the vector store. The trade-off is an extra model call per turn, which adds latency and cost. I log both the original and the rewritten query in the trace, so a bad rewrite can be debugged.

I start by measuring. Tracing records token use per run, so I can attribute costs by tenant and by flow version. Then I use several levers. The relevance gate skips model calls entirely when retrieval fails. A sensible top-k and limited history keep prompts short. A smaller model may be enough if the evaluation set shows quality holds. Ingest costs are mostly one-time per document, so I avoid needless re-ingests through idempotent vector ids. I also set quotas per tenant and rate limits in the backend, so one heavy user cannot drive costs for everyone, and I alert when spending trends look unusual.

I enforce limits in the backend before calling Langflow: rate limits per tenant for chat requests, and a separate queue with a concurrency limit for ingest jobs. That way a bulk upload cannot starve chat traffic. Chat and ingest can also run on separate Langflow workers or instances, because their load profiles differ: ingest is bursty and heavy, while chat needs low latency. Provider rate limits for the model and embeddings are shared, so I give tenants fair shares and back off with retries on throttling. Tracing and metrics per tenant show who is consuming capacity, and quotas can then be raised deliberately.

Not by default. This flow follows a fixed path: retrieve, gate, prompt, answer. A fixed path is predictable, cheap, and easy to test and trace. An Agent with retrieval as a tool decides for itself when and how often to search. That helps with multi-step questions, but it adds latency, adds variance, and makes it harder to guarantee that every answer is grounded. It also means the deterministic not-found gate no longer clearly controls whether the model speaks. If I needed multi-step reasoning across documents later, I would try an agent behind the same tenant-pinned retrieval and compare it against the fixed flow on the evaluation set.

The backend wraps the run calls with timeouts and limited retries with backoff for errors that may clear on their own. If chat still fails, the user gets a clear error saying answers are temporarily unavailable. The backend never falls back to an ungrounded model call. Uploads still succeed, because the file is stored and the ingest job stays queued until Langflow recovers. A circuit breaker stops a pile-up of requests against a failing dependency. I alert on error rates and latency, and I check the traces to tell whether Langflow itself, the vector store or the model provider is failing.

Scores depend on the embedding model and the distance metric, so a threshold copied from somewhere else is meaningless. I calibrate it on the evaluation set. I record the top scores for answerable questions and for deliberately unanswerable ones, then choose a threshold that rejects most unanswerable questions without blocking real ones. A strict threshold causes false not-found answers. A loose one lets weak context through to the model. Combining the threshold with a minimum chunk count helps. The threshold must be recalibrated whenever the embedding model or chunking changes, so I store it as configuration tied to the flow version.

The run response is nested JSON containing the outputs of the flow's output components. I would write one small adapter that pulls out the answer text and any source metadata, and validate the result against an expected shape. If the shape is wrong, the backend logs it and shows a safe error rather than crashing or showing raw JSON. Because flow edits can change the output structure, I pin the flow version per environment and test the adapter in the same pipeline as flow changes. Citations shown to the user should be checked against the retrieved chunks, not just taken from the model's text.

Langflow can act as an MCP server and expose flows as tools, so this is technically easy. The risk is the tenant boundary. Today the backend sets the tenant scope through tweaks after authenticating the user. An MCP client calling the flow directly would bypass that unless the tenant identity is enforced on the server side. I would not expose the raw flow. Instead I would put a thin, authenticated layer in front of it, either an MCP endpoint or a dedicated flow, that derives the tenant from the caller's credentials and never accepts the scope as a tool argument. Then I would test cross-tenant attempts specifically.

It is a Python class with inputs and outputs, so I can unit-test its logic outside the canvas. I feed it lists of chunks with chosen scores and counts and assert whether it passes them through or emits the fixed not-found message. The cases include an empty list, all scores just below the threshold, one strong chunk, and missing score metadata, which should fail safe by refusing. Then I test it in the Playground within the flow, and I include unanswerable questions in the evaluation set. Keeping the component small and free of side effects makes it easy to trust.

The model should not quietly pick one source. Each chunk carries a document and page label, so the prompt can tell the model to point out disagreements and cite both sources, for example 'Policy A, page 3 says X; Policy B, page 7 says Y'. If document versions or dates are in the metadata, I can tell the model which source is newer, but it should still show both. Letting users limit a question to selected documents, through document id filters set by the backend, also cuts down on conflicts. I add conflict cases to the evaluation set to check this behaviour holds.

1. APOST /api/v1/run/<flow-id>Correct answer
2. BGET /api/v1/flows/<flow-id>/execute
3. CPOST /api/v1/chat/<session-id>
4. DPUT /api/v1/run/<flow-id>

Why: Flows run through POST /api/v1/run/<flow-id>, with tweaks to override component settings.

1. AServer-side in the backend onlyCorrect answer
2. BEmbedded in the PDF metadata
3. CIn the flow's prompt component
4. DIn the browser's local storage

Why: Step 1 requires the key to be held server-side and never sent to the browser, because it can run any flow with any tweaks.

1. AThe vector store, chosen at random
2. BThe user, through a dropdown
3. CThe chat model, based on the question
4. DThe backend, from the authenticated sessionCorrect answer

Why: Only the backend's authenticated tenant resolution is trustworthy; values controlled by the client could point retrieval at another tenant.

1. ASearches all tenants' documents
2. BRetries with top-k set to its maximum
3. CCalls the model with no context
4. DReturns a fixed 'not found in your documents' message without calling the modelCorrect answer

Why: The gate skips the model so that ungrounded text is never presented as coming from the PDF.

1. AOverriding settings of specific components in the flow for that requestCorrect answer
2. BChanging the Langflow server's global configuration
3. CUploading new custom component code
4. DChoosing which tracing provider receives spans

Why: Tweaks are a per-request map keyed by component id that overrides component settings, such as a vector store's collection or a model's temperature, without editing the saved flow.

1. ASetting a higher temperature on the chat model
2. BRe-embedding the PDF before each question
3. CUsing different flow ids for each user
4. DPassing a distinct session_id per conversation so message history is scopedCorrect answer

Why: Message history in Langflow is keyed by session id, so each conversation needs its own session_id to keep turns from mixing.

1. Asession_id and flow_id
2. Btweaks and api_key
3. Ctop_k and threshold
4. Dinput_type and output_typeCorrect answer

Why: Step 4 sets input_value, with input_type and output_type set to chat.

1. ASelecting the tenant's namespace
2. BScoping message history to one conversationCorrect answer
3. CChoosing the embedding model
4. DAuthenticating the API call

Why: Message history provides chat memory per session id; tenant scope and authentication are handled separately.

1. ATo keep sentences that cross chunk boundaries retrievableCorrect answer
2. BTo encrypt the text
3. CTo make embeddings faster
4. DTo reduce storage

Why: Overlap stops important text from being split away from its context at a boundary, at the cost of some duplication.

1. ATenant id, document id and pageCorrect answer
2. BThe user's password hash
3. CThe model's temperature
4. DOnly the embedding vector

Why: Tenant and document ids support isolation and deletion, and page numbers support citations.

1. AOnly restarting Langflow
2. BClearing message history
3. CNothing; old chunks still match
4. DRe-ingesting the documents with the new settingsCorrect answer

Why: Existing vectors reflect the old chunking, so documents must be re-processed for consistent results.

1. ARun ingest asynchronously from a queue and track document statusCorrect answer
2. BSkip embedding for large files
3. CIngest only the first page
4. DRaise the browser timeout indefinitely

Why: A queued worker calling the ingest flow avoids HTTP timeouts and lets users see progress.

1. AUse deterministic vector ids based on document and chunk index, and upsertCorrect answer
2. BDisable retries
3. CStore vectors in the browser
4. DUse random vector ids

Why: Deterministic ids make repeated upserts overwrite the same records instead of adding duplicates.

1. AOnly LangSmith
2. BNone; Langflow has no tracing
3. CLangSmith and LangfuseCorrect answer
4. DOnly Langfuse

Why: The notes list LangSmith and Langfuse as tracing integrations.

1. ARelying on the prompt to ignore other tenants
2. BFiltering in the browser
3. CA shared index with no filter
4. DA separate namespace or collection per tenant, with tenant id also in metadataCorrect answer

Why: Physical separation, backed by metadata, means a single forgotten filter cannot leak data.

1. AThe database record
2. BThe vectors in the namespace
3. CThe object storage file
4. DTraces and message history that contain quoted chunksCorrect answer

Why: Traces and chat history hold copies of document text, so they need retention rules or deletion paths too.

1. ANothing, vectors from different models are compatible
2. BOnly the chat model needs to be changed to match
3. CRe-embed existing documents and use the same model for queries in the chat flowCorrect answer
4. DIncrease top-k to compensate

Why: Vectors from different embedding models live in different spaces, so the index must be rebuilt and the query side must use the same model for similarity search to be meaningful.

1. AMore tokens, latency and distracting contextCorrect answer
2. BIt disables citations
3. CIt breaks session memory
4. DLower recall

Why: More chunks raise prompt cost and latency, and can dilute the model's focus.

1. AUsers' passwords
2. BJust the final answer
3. COnly HTTP status codes
4. DRetrieved chunks, prompt, latency and token useCorrect answer

Why: Seeing the retrieved context and prompt shows whether a bad answer came from retrieval or from generation.

1. AA higher temperature
2. BA larger top-k
3. CLonger message history
4. DThe chat flow has no tools, and the backend fixes the tenant scopeCorrect answer

Why: With no tools and a scope set on the server, injected text can at worst change the answer's wording, not its actions or data access.

1. AEmbeddings cannot handle short text at all
2. BThe raw question lacks context, so it needs rewriting with recent turns before retrievalCorrect answer
3. CSession ids expire after each turn
4. DThe vector store blocks pronouns

Why: Similarity search runs on the query text, so a standalone rewrite of the question restores its meaning.

1. AA Python class with inputs and outputsCorrect answer
2. BA JavaScript plugin for the browser
3. CA SQL stored procedure
4. DA YAML-only config file

Why: The notes define custom components as Python classes with inputs and outputs.

1. AThe browser's developer tools only
2. BThe object storage bucket
3. CThe PlaygroundCorrect answer
4. DThe vector store console

Why: Langflow flows are tested in the Playground.

1. AAlways use the same fixed value
2. BLet the user set it
3. CUse a value copied from any blog post
4. DCalibrate it on answerable and unanswerable evaluation questions for the specific embedding modelCorrect answer

Why: Score scales differ by embedding model and distance metric, so the threshold must be calibrated.

1. APass them to the model anyway
2. BFail safe and return the not-found messageCorrect answer
3. CCrash the flow silently
4. DSearch other tenants

Why: Failing closed keeps the grounding guarantee when inputs are unexpected.

1. AAnswer from general model knowledge
2. BShow that the document is still processing and search only documents marked readyCorrect answer
3. CSearch the partial vectors anyway
4. DDelete the document

Why: Making the eventual-consistency state visible avoids misleading not-found answers and partial results.

1. AA fixed retrieve-gate-answer path is more predictable, testable and easier to keep groundedCorrect answer
2. BAgents do not support session ids
3. CAgents require webhooks
4. DAgents cannot use retrieval

Why: An agent decides tool use dynamically, which adds variance and weakens the deterministic gate.

1. AHard-coded in the frontend
2. BFrom configuration per environmentCorrect answer
3. CFrom the user's request body
4. DFrom the PDF filename

Why: Configuration per environment supports promotion and rollback without code changes or exposure to the client.

1. ARaising top-k
2. BSharing one synchronous thread for everything
3. CRunning ingest on a separate queue with concurrency limitsCorrect answer
4. DTurning off tracing

Why: Isolating heavy, bursty ingest from latency-sensitive chat avoids starving chat requests.

1. AAsking the user
2. BTrusting the model's text
3. CMatching each cited label to a chunk actually retrieved in that runCorrect answer
4. DChecking the answer's length

Why: Validating citations against the run's retrieved chunks catches invented sources.

1. AIt deletes message history
2. BMCP cannot transport text
3. CCallers could bypass the backend's tenant scoping unless it is enforced on the serverCorrect answer
4. DMCP makes embeddings invalid

Why: Tenant scope is set by the backend today, so a direct tool path must derive the tenant from the caller's credentials.

1. ARetry forever
2. BMark it ready anyway
3. CDetect the low text yield and route it to OCR or flag it to the userCorrect answer
4. DEmbed blank chunks

Why: Otherwise the document looks searchable but can never answer anything.

1. AHang until it recovers
2. BFall back to an ungrounded model with no context
3. CReturn a clear 'temporarily unavailable' error after bounded retriesCorrect answer
4. DAnswer from another tenant's cache

Why: Failing clearly preserves the grounding guarantee and avoids piling up requests.