We're building a multi-tenant RAG chatbot platform on .NET 10: an embeddable widget, SignalR streaming, pgvector retrieval, a text-to-SQL agent, MCP tools and episodic memory. It worked, but a data question took 30-40 seconds. I audited the whole request path to find out why.

#1. The Platform in One Picture

Diagram
Next.js admin portal  |  embeddable JS widget (iframe)
              │  REST + SignalR
              ▼
   ASP.NET Core gateway (.NET 10) ─ Semantic Kernel agents
     │           │           │          │          │
 pgvector    RabbitMQ      Redis     MongoDB    mem0 (Qdrant)
 (RAG)     (ingestion)  (cache +    (chat +    (long-term
                        backplane)   audit)     memory)

Heavy work (document ingestion and audit logging) goes through RabbitMQ consumers, so the gateway stays non-blocking. Vectors are namespaced per tenant.

#2. Where 40 Seconds Went

For a SQL-style question, roughly 12-15 seconds is unavoidable LLM and database work. The other ~13 seconds was waste:

Diagram
nested SQL LLM calls           ~8000 ms   ◀ biggest win
redundant intent re-parsing    ~2000 ms
artificial streaming delay     ~1500 ms
duplicate vector search         ~300 ms
redundant kernel clones         ~300 ms
other overhead                  ~500 ms

#3. Finding #1: The LLM Call Cascade

The SQL path made its calls in sequence: classify intent, re-derive intent, discover tables, analyse the schema, generate SQL, validate SQL. That's 7-8 LLM calls per request where 2-3 would do. The step that re-derived intent was redundant, because the classifier had already decided the question was a SQL question.

Plan

A single generation prompt with the relevant schema inlined, which analyses, generates and validates in one pass. It's the simplest architecture, the easiest to debug, and worth an estimated 8-12 seconds. The risk is SQL accuracy, so it ships behind a feature flag and gets A/B tested on 20+ reference questions.

#4. Finding #2: Streaming That Wasn't

Responses were re-chunked into 20-character pieces with a 15 ms pause between each. A 2,000-character answer means 100 chunks, or 1.5 seconds of pure delay, added to a model that already streams natively. The fix is to forward tokens as they arrive.

#5. Finding #3: Doing the Same Work Twice

  • Duplicate vector search: two services ran the same schema-context search on every request. Search once, pass the result down.
  • Kernel cloning: the Semantic Kernel instance was cloned 3-4 times per request, each clone creating a new client connection (50-100 ms each). Reuse the base kernel and attach plugins per step.
  • MCP connections: a new connection per request (300-800 ms). Cache them in a ConcurrentDictionary keyed by server and URL, and dispose them deterministically when config changes.

#6. The Reliability Checklist That Came With It

Performance audits find correctness problems too. These are the checks I now run on any multi-tenant AI backend:

  • Every cache key includes the tenant ID. Anything else is a cross-tenant data leak waiting to happen.
  • No fire-and-forget without retries. Background ingestion needs retries and visibility, or data goes missing silently.
  • Invalidate caches per tenant. A global flush makes every tenant rebuild at once.
  • Put a bound on in-process caches, and make disposal ownership explicit for pooled clients.

#7. The Plan

Diagram
Week 1    quick wins: streaming, duplicate search, kernel reuse, MCP pooling
          target: 30-40 s ─▶ 20-25 s
Week 2-3  single-shot SQL, cached intent, parallel pipeline steps
          target: ─▶ 8-12 s
Week 3-4  tenant isolation, retries, per-tenant invalidation, latency dashboards

#8. Takeaways

  • In LLM apps, count the calls before tuning anything else. Sequential calls dominate latency.
  • Don't simulate streaming on top of a model that already streams.
  • Measure first: this audit turned "the chatbot is slow" into a ranked list of 12 fixes with estimated savings.