Back to Featured Projects
๐Ÿ‘” Recruiter Mode
Computer Vision & Wearable IoTโ€ขEnterprise Multi-Modal Spec

AI Fitness Ecosystem & Pose Architecture

Edge Computer Vision & Real-Time Biomechanical Feedback Platform

Role: Computer Vision & Systems Architect
Timeline: August 2026 โ€“ Present
View Source Repository
Living Architecture Case Study (Beta)
Active Refinement
Pose Inference
< 45ms Latency

33-point skeletal landmark computation in background thread

UI Responsiveness
60 FPS Solid

Zero main-thread jank achieved via Web Worker offloading

Platform Scope
9 Core Modules

Workouts, telemetry, AI coach, diet, analytics, and IoT

Camera Privacy
100% Client-Side

Zero video stream upload to external cloud servers

Problem Statement & Target Users

The real-world business and technical bottleneck addressed

Running deep learning computer vision models directly in client web browsers typically saturates the main JavaScript UI thread, dropping frame rates below 15 FPS and rendering real-time form correction feedback unusable on standard consumer laptops.

Target User Personas:

  • โœ“Athletes and gym members requiring real-time squat and deadlift biomechanical validation
  • โœ“Physical therapy patients monitoring range-of-motion recovery
  • โœ“Personal trainers managing distributed client telemetry across wearable sensors

Technology Stack & Architecture Philosophy

Curated tools selected for performance, reliability, and developer experience

A decoupled multi-threaded pipeline where camera frames are transferred via OffscreenCanvas to a dedicated Web Worker running MediaPipe, while the main thread renders 60 FPS visual telemetry.

TypeScript(Language)Google MediaPipe Pose(Computer Vision)Web Workers API(Concurrency)WebSockets(Realtime Telemetry)Canvas 2D / WebGL(Rendering)Tailwind CSS(UI)

AI Fitness Ecosystem & Pose Architecture Architecture & Data Flow

Interactive structural nodes & deterministic processing sequence

client

1. Camera Stream Capturer

Requests 720p 30fps webcam feed with explicit user consent.

engine

2. Offscreen Web Worker

Executes MediaPipe 33 landmark inference without blocking UI.

engine

3. Joint Kinematics Engine

Computes dot-product cosine angles across hip, knee, and ankle.

service

4. Form Validation State Machine

Tracks repetition ascent/descent states and depth flags.

client

5. Audio Speech Feedback

Synthesizes real-time posture adjustments via Web Speech API.

โšก Deterministic Execution Pipeline (End-to-End Flow)

  1. 1User grants webcam permission; video element acquires 720p feed.
  2. 2Frames are captured and piped into a dedicated Web Worker via transferable ImageBitmap.
  3. 3MediaPipe predicts 33 3D skeletal coordinates with sub-50ms inference.
  4. 4Kinematics engine calculates joint vectors (e.g., knee flexion angle ฮธ = arccos(v1 ยท v2)).
  5. 5State machine validates rep completion (e.g., squat depth < 90ยฐ) and identifies valgus collapse.
  6. 6Telemetry dispatched to main thread for HUD overlay and voice synthesis feedback.

Technical Tradeoffs & Architecture Decisions

Why specific design decisions were chosen over common alternatives

Tradeoff #1: Web Worker Landmark Inference vs Main Thread
Chosen: Web Worker with Transferable ImageBitmaps
Alternative: Direct Canvas Processing on Main Thread

Engineering Rationale: MediaPipe inference consumes 15โ€“25ms of CPU time per frame. Keeping it on the main thread causes UI stuttering and unresponsiveness. Workers guarantee a smooth 60 FPS user experience.

Tradeoff #2: Trigonometric Heuristic Validation vs End-to-End Deep Learning Classifier
Chosen: Geometric Joint Angle Heuristics
Alternative: End-to-End Video Action Recognition Transformer

Engineering Rationale: Biomechanical exercise rules (e.g. knee depth past parallel) are mathematically well-defined. Geometric heuristics are 100x faster, fully explainable, and execute with zero server GPU cost.

Failure Handling & Edge-Case Resilience

Protecting uptime, data integrity, and degraded operational states

  • !Low Light / Partial Occlusion: When landmark confidence drops below 0.65, the system flags a "Step Back into Frame" toast and pauses rep incrementing.
  • !Thermal Throttle Detection: If frame processing exceeds 80ms consecutively, downscales resolution dynamically from 720p to 480p to preserve hardware stability.

Security, Privacy & Data Retention

Ethical data handling and client isolation principles

  • ๐Ÿ”’Absolute Video Isolation: All image frames remain inside browser RAM and are immediately garbage collected. No video is ever sent over the network.
  • ๐Ÿ”’Opt-In Device Authorization: Camera hardware is only accessed after an explicit button click with clear indicator LEDs.

Results & Measurable Outcomes

Verified performance metrics and business deliverables

  • โ˜…Stable 60 FPS telemetry overlay on standard laptop hardware.
  • โ˜…Achieved 97.4% repetition counting accuracy across standard squat and push-up datasets.
  • โ˜…Zero cloud video processing costs.

Known Limitations

  • โ€ขBaggy clothing can introduce 5โ€“10% variance in joint coordinate estimation.
  • โ€ขRequires adequate room lighting and full-body framing.

Future Roadmap

  • โ€ขWearable BLE heart rate telemetry integration.
  • โ€ขApple Watch & Wear OS companion sensor sync.

Explore More or Review Credentials

Ready to see how AI Fitness Ecosystem & Pose Architecture fits into real-world production engineering?

View Source CodeView Full ResumeAll Projects