V.
VIGHNESH KUMBHAR01 / 01

Gesture Solar System

A WebGL solar system controlled by hand gestures via MediaPipe — dual-hand particle field manipulation, planet focus, and orbital mechanics simulation.

Experimental build

Gesture Solar System source screenshot
MEDIAPIPE
GESTURE / 3DCASE / 14
VERIFIED SOURCE MATERIALThree.js / MediaPipe HandLandmarker / WebGL2

The premise.

A single-file Three.js application using MediaPipe Hand Landmarker for real-time gesture control: rotate the system, orbit the camera, expand/contract particle fields, and focus on individual planets — all through webcam tracking.

Exploring natural user interfaces for 3D data exploration — can gesture control replace traditional mouse/keyboard for spatial navigation?

How it is structured.

The repository evidence points to these observable boundaries and technologies:

  • Single HTML file with ES modules
  • Three.js (InstancedMesh for 5000 particles)
  • MediaPipe Tasks Vision (HandLandmarker)
  • Import maps for CDN dependencies
  • GPU-accelerated particle simulation

From idea to system.

  • Dual-hand gesture recognition (pinch/expand)
  • Fist detection for particle vortex
  • Single-hand system rotation + camera orbit
  • Planet focus with smooth camera lerp
  • Shadow/flat lighting modes
  • 5000 GPU-instanced particles
  • Real-time 30fps+ hand tracking
Three.jsMediaPipe HandLandmarkerWebGL2Webcam APIImport MapsTetrahedronGeometry instancing

The difficult parts.

  • MediaPipe GPU delegate reliability
  • Gesture-to-3D mapping feel
  • Particle performance at 5000 instances
  • Single-file module dependency management

What can be said honestly.

  • Working gesture control in Chrome/Edge
  • Smooth 60fps rendering on modern GPUs
  • All 9 planets + sun with orbits
  • Particle field responds to gestures

Gesture interfaces for 3D need forgiving detection zones and clear visual feedback — precision tracking alone feels unresponsive without spatial cues.