SmartAir Mouse Pro: Zero-Latency Touchless OS Control
Overcoming sensor noise & tremor with a 2D Kalman filter on 21 MediaPipe hand landmarks
1. The Problem
Touchless computer vision mouse systems usually suffer from an unavoidable dilemma: without smoothing, hand tremors cause violent cursor jitter; with traditional moving-average filters, the cursor lags by 150-200ms, making accurate clicking virtually impossible.
2. Hard Engineering Constraints
- Must run smoothly at 60 FPS on integrated laptop webcams without requiring a dedicated GPU.
- Total pipeline latency from camera photon to OS cursor dispatch must remain under 30ms.
- Reliable pinch-to-click detection without false positives caused by accidental finger drift.
3. System Architecture Topology
[Webcam Stream (720p 60fps)] ──> [OpenCV Frame Preprocessor]
│
[MediaPipe 21-pt Hand Mesh]
│
[2D Kalman State Estimator]
│
┌─────────────────────┴─────────────────────┐
▼ ▼
[OS Cursor Dispatch (Win32)] [Neon Particle AR HUD]4. My Key Architectural Contributions
Dynamic 2D Kalman Filter Tuning
Implemented state-space prediction matrices calibrated specifically for human micro-tremors, predicting future coordinates to cancel sensor delay while aggressively dampening static jitter.
Euclidean Pinch Distance Calibration
Developed an adaptive depth-normalized distance metric between index tip (landmark 8) and thumb tip (landmark 4) that dynamically scales based on palm bounding box dimensions.
Kinetic Particle AR HUD & Gradio Sandbox
Designed a transparent overlay that explodes visual particle physics on click events, and deployed a browser-accessible Gradio demo for real-time hyperparameter tuning.
5. Measured Engineering Outcomes
FPS
Locked frame rate on modest CPU hardware (Core i5 integrated graphics)
<ms
Total pipeline latency (frame capture -> landmark -> Kalman -> dispatch)
%
Precision on intentional click gestures vs incidental finger movement
6. Retrospective & Senior Engineering Review
The Kalman filter completely solved the jitter vs lag problem that plagued simpler moving-average implementations.
In a future release, I would replace the Win32 mouse API with cross-platform virtual HID drivers to support Linux and macOS touchpads.
Technology Stack & Tooling
Explore Related Case Studies
Explore real engineering results across cloud backends, AI agents, and computer vision systems.
Tensor Security Academy — Platform & Automation
“Automated booking dispatch via n8n and hardened Next.js ISR architectures delivering 99.9% uptime with zero manual database intervention.”
%
Platform Uptime
Enterprise SLA across global students
0 hrs
Manual Operations
Full n8n webhook booking automation
A+
Security Headers Rating
Mozilla Observatory & SSL Labs
SSRN Transfer — US Fleet & Booking Platform
“Engineered high-throughput responsive vehicle reservation templates with zero-lag debounced fleet filtering and mobile-first booking workflows.”
ms
Query Response
Debounced fleet availability filtering
FPS
Render Performance
CLS < 0.001 mobile zero-lag interaction
%
Cross-Device Reliability
Tested across iOS Safari & Android
Arelix Labs — AI Agency & Engineering Hub
“Architected full-stack Next.js 15 App Router platform with Prisma ORM, Neon Postgres, and Groq LLM pipelines for automated scope generation.”
ms
Inference Routing Latency
Groq LPU accelerated streaming
x
Scope Generation Speed
Automated proposal-to-deliverable AI
%
Transaction Reliability
Connection pooling & rollback protection
AUST Robotics Club & ARC Event Platform
“Engineered a schema-driven monorepo platform powering nationwide robotics tournaments with atomic Firestore scoring and ImageKit CDN.”
%
Code Reusability
Zero-code re-theming per edition
National Championships
Powering ARC 2.0, Robomania & Club
ms
Asset Delivery
ImageKit edge cache verification cards
Tensora — Agentic AI Platform & Chatbot
“Multi-model reasoning agent system with token streaming over HTTP/2 SSE and dynamic cost guardrails cutting inference spend by 42%.”
ms
First-Token Latency
Streaming SSE transformer pipeline
%
Cost Optimization
Frontier-to-lightweight model routing
%
Tool Call Precision
Deterministic DB + vector search