We're building a multi-tenant RAG chatbot platform on .NET 10: an embeddable widget, SignalR streaming, pgvector retrieval, a text-to-SQL agent, MCP tools and episodic memory. It worked, but a data question took 30-40 seconds. I audited the whole request path to find out why.
#1. The Platform in One Picture
Next.js admin portal | embeddable JS widget (iframe)
│ REST + SignalR
▼
ASP.NET Core gateway (.NET 10) ─ Semantic Kernel agents
│ │ │ │ │
pgvector RabbitMQ Redis MongoDB mem0 (Qdrant)
(RAG) (ingestion) (cache + (chat + (long-term
backplane) audit) memory)Heavy work (document ingestion and audit logging) goes through RabbitMQ consumers, so the gateway stays non-blocking. Vectors are namespaced per tenant.
#2. Where 40 Seconds Went
For a SQL-style question, roughly 12-15 seconds is unavoidable LLM and database work. The other ~13 seconds was waste:
nested SQL LLM calls ~8000 ms ◀ biggest win
redundant intent re-parsing ~2000 ms
artificial streaming delay ~1500 ms
duplicate vector search ~300 ms
redundant kernel clones ~300 ms
other overhead ~500 ms#3. Finding #1: The LLM Call Cascade
The SQL path made its calls in sequence: classify intent, re-derive intent, discover tables, analyse the schema, generate SQL, validate SQL. That's 7-8 LLM calls per request where 2-3 would do. The step that re-derived intent was redundant, because the classifier had already decided the question was a SQL question.
A single generation prompt with the relevant schema inlined, which analyses, generates and validates in one pass. It's the simplest architecture, the easiest to debug, and worth an estimated 8-12 seconds. The risk is SQL accuracy, so it ships behind a feature flag and gets A/B tested on 20+ reference questions.
#4. Finding #2: Streaming That Wasn't
Responses were re-chunked into 20-character pieces with a 15 ms pause between each. A 2,000-character answer means 100 chunks, or 1.5 seconds of pure delay, added to a model that already streams natively. The fix is to forward tokens as they arrive.
#5. Finding #3: Doing the Same Work Twice
- Duplicate vector search: two services ran the same schema-context search on every request. Search once, pass the result down.
- Kernel cloning: the Semantic Kernel instance was cloned 3-4 times per request, each clone creating a new client connection (50-100 ms each). Reuse the base kernel and attach plugins per step.
- MCP connections: a new connection per request (300-800 ms). Cache them in a
ConcurrentDictionarykeyed by server and URL, and dispose them deterministically when config changes.
#6. The Reliability Checklist That Came With It
Performance audits find correctness problems too. These are the checks I now run on any multi-tenant AI backend:
- Every cache key includes the tenant ID. Anything else is a cross-tenant data leak waiting to happen.
- No fire-and-forget without retries. Background ingestion needs retries and visibility, or data goes missing silently.
- Invalidate caches per tenant. A global flush makes every tenant rebuild at once.
- Put a bound on in-process caches, and make disposal ownership explicit for pooled clients.
#7. The Plan
Week 1 quick wins: streaming, duplicate search, kernel reuse, MCP pooling
target: 30-40 s ─▶ 20-25 s
Week 2-3 single-shot SQL, cached intent, parallel pipeline steps
target: ─▶ 8-12 s
Week 3-4 tenant isolation, retries, per-tenant invalidation, latency dashboards#8. Takeaways
- In LLM apps, count the calls before tuning anything else. Sequential calls dominate latency.
- Don't simulate streaming on top of a model that already streams.
- Measure first: this audit turned "the chatbot is slow" into a ranked list of 12 fixes with estimated savings.