Gesture Three.js

gesture-threejs is a lightweight TypeScript library that bridges MediaPipe Hand Landmarker with Three.js scenes. Exposes a clean React-like hook API: useGestures(scene, camera) returns gesture events (pinch, grab, point, swipe) mapped to 3D interactions (rotate, scale, translate, select).

Adding gesture control to 3D web apps requires boilerplate MediaPipe setup, coordinate mapping, and gesture classification. This library abstracts all of it.

The repository evidence points to these observable boundaries and technologies:

• TypeScript library with Rollup bundling
• MediaPipe Tasks Vision (HandLandmarker) via CDN
• Three.js peer dependency
• React hooks + vanilla JS API
• Gesture classification: pinch/grab/point/swipe

• useGestures() hook for React
• createGestureController() for vanilla JS
• Automatic MediaPipe model loading from CDN
• Normalized screen → 3D world coordinate mapping
• Configurable gesture sensitivity and debouncing
• Tree-shakable: import only needed gestures

• MediaPipe GPU delegate inconsistency across browsers
• Normalized landmark → Three.js world space math
• Library size vs feature completeness tradeoff
• TypeScript declarations for MediaPipe types

• ~18KB gzipped (zero runtime deps beyond Three.js)
• Demo: orbit controls via pinch, object selection via point
• Works in Chrome/Edge/Firefox with webcam permission
• TypeScript definitions included

Gesture libraries succeed when they handle the messy coordinate math and leave interaction design to the app. MediaPipe CDN loading simplifies distribution.