Tensora: Production Multi-Model Agentic AI Chatbot
Architecting a sub-15ms streaming pipeline with LangGraph memory states and Groq inference
1. The Problem
Modern LLM applications frequently suffer from high latency, erratic tool execution, and hallucinated answers on domain-specific engineering documentation. The challenge was to deliver sub-second perceived response times while guaranteeing strict factual verification over multi-step agent interactions.
2. Hard Engineering Constraints
- Sub-100ms initial response time on desktop and mobile web connections.
- Strict cost containment: Avoid routing repetitive queries to frontier models ($30+/1M tokens).
- Deterministic schema enforcement: Tool payloads must strictly match TypeScript/Pydantic contracts.
3. System Architecture Topology
[Client Next.js Edge] ──(SSE Stream)──> [Next.js Route Handler / Transformer]
│
[Classifier / Router]
┌───────────┴───────────┐
▼ ▼
[Fast Model: Llama-3] [Frontier Reasoner]
│ │
└───────────┬───────────┘
▼
[Pinecone Vector RAG]
│
[LangGraph State Store]4. My Key Architectural Contributions
Zero-Buffer Server-Sent Events (SSE) Stream Transformer
Wrote a custom chunk transformer using web ReadableStreams to pipe Groq inference directly to React Server Components without intermediate server memory buffering, reducing TTFT from 840ms to 14.2ms.
Multi-Agent State Machine with LangGraph
Designed a cyclic state graph in Python/FastAPI that validates query classification, conducts semantic vector retrieval against Pinecone, and self-corrects invalid tool invocations before emitting responses to the user.
Cost-Aware Model Cascade
Implemented automated query complexity heuristics that resolve 70% of informational questions via Groq-hosted open-weights models, cutting monthly inference spend by 42%.
5. Measured Engineering Outcomes
ms
Average Time-to-First-Token over 5,000 requests
%
Tool calling schema adherence with zero unhandled exceptions
%
Cost reduction via dynamic inference routing cascade
6. Retrospective & Senior Engineering Review
Separating the fast streaming transport layer from the background state machine allowed the UI to feel instant even while complex retrieval operations executed.
In v2, I would implement speculative client-side token caching and an indexed local cache to eliminate repeat network hits for identical technical queries.
Technology Stack & Tooling
Explore Related Case Studies
Explore real engineering results across cloud backends, AI agents, and computer vision systems.
Tensor Security Academy — Platform & Automation
“Automated booking dispatch via n8n and hardened Next.js ISR architectures delivering 99.9% uptime with zero manual database intervention.”
%
Platform Uptime
Enterprise SLA across global students
0 hrs
Manual Operations
Full n8n webhook booking automation
A+
Security Headers Rating
Mozilla Observatory & SSL Labs
SSRN Transfer — US Fleet & Booking Platform
“Engineered high-throughput responsive vehicle reservation templates with zero-lag debounced fleet filtering and mobile-first booking workflows.”
ms
Query Response
Debounced fleet availability filtering
FPS
Render Performance
CLS < 0.001 mobile zero-lag interaction
%
Cross-Device Reliability
Tested across iOS Safari & Android
Arelix Labs — AI Agency & Engineering Hub
“Architected full-stack Next.js 15 App Router platform with Prisma ORM, Neon Postgres, and Groq LLM pipelines for automated scope generation.”
ms
Inference Routing Latency
Groq LPU accelerated streaming
x
Scope Generation Speed
Automated proposal-to-deliverable AI
%
Transaction Reliability
Connection pooling & rollback protection
AUST Robotics Club & ARC Event Platform
“Engineered a schema-driven monorepo platform powering nationwide robotics tournaments with atomic Firestore scoring and ImageKit CDN.”
%
Code Reusability
Zero-code re-theming per edition
National Championships
Powering ARC 2.0, Robomania & Club
ms
Asset Delivery
ImageKit edge cache verification cards
SmartAir Mouse Pro — Computer Vision Engine
“Touchless AR human-computer interface transforming webcam video into low-latency OS mouse events with custom 2D Kalman filter jitter suppression.”
FPS
Zero-Lag Tracking
Real-time 21-point hand skeleton mesh
2D Kalman
Tremor Suppression
Surgical cursor accuracy without latency
%
Zero-Install Sandbox
Instant browser calibration via Gradio