HovrPad Gesture Engine: High-Fidelity Optical Gesture Recognition & Low-Latency Spatial Input Synthesis

1. System Architecture & Execution Pipeline

The HovrPad Gesture Engine processes optical sensor input through a decoupled, multi-threaded pipeline. Traditional webcam apps run image capture, model inference, and cursor updates in a single blocking thread, which introduces frame drops and noticeable input lag. In contrast, HovrPad isolates the ingestion and landmark inference loop from the OS input injector, dispatching multi-step actions asynchronously via lightweight daemon worker threads.

1. Stage 1: High-Throughput Sensor Ingestion — The camera stream is ingested via Windows DirectShow / Media Foundation, converting frames in memory with zero intermediate copy steps.
2. Stage 2: 21-Point Skeletal Graph Inference — MediaPipe's lightweight palm detector localizes the hand region, followed by a 2.5D regression model that outputs 21 distinct 3D joint landmarks (wrist, knuckles, phalanx joints, and fingertips).
3. Stage 3: Two-Layer Hierarchical Signal Conditioning — Landmark coordinates are first smoothed with an Exponential Moving Average (EMA, α=0.2) featuring dynamic jump catch-up (α=0.8) for fast hand repositions. Projected screen target coordinates then pass through a dynamic 1€ (One-Euro) filter (β=0.1, min_cutoff=1.0 Hz) to eliminate micro-tremors during fine targeting.
4. Stage 4: Finite State Machine (FSM) Classification — Hand topology (finger curl angles, knuckle spreads, and relative pinches) is evaluated against an explicit state machine with hysteresis thresholds and decoupled pointing/trigger mechanics.
5. Stage 5: Native OS Event Synthesis — Validated cursor coordinates and mouse clicks are injected into the Windows subsystem via native user-mode Win32 APIs (SetCursorPos, mouse_event, keybd_event) with sub-millisecond dispatch. Multi-step cadences (such as Double Click and Right Click hold-delays) run on dedicated background threads to prevent UI stalling.

2. Core Algorithmic Concepts & Signal Conditioning

2.1 The Two-Layer Hierarchical Smoothing Pipeline

Optical mouse control requires resolving two distinct noise profiles: raw landmark jitter in 3D camera space, and cursor trembling in 2D screen space during stationary targeting. HovrPad solves this using a two-layer architecture:

2.2 Decoupled Pointing Axis vs. Trigger Axis (Thumb-to-Middle Tap)

A classic flaw in vision-based virtual mice is using a thumb-to-index pinch to click. Pinching the thumb to the pointing finger collapses the fingertip's spatial position, causing severe cursor jitter and aim deflection at the exact moment of clicking.

2.3 Point-to-Segment Proximity & Schmitt-Trigger Pixel Hysteresis

The primary click detector calculates the geometric distance from the thumb tip landmark (lm[4]) to the middle finger skeletal segments (pt_to_segment and segment intersection tests). To prevent click chattering at the contact boundary, the engine enforces Schmitt-trigger pixel hysteresis:

• Click Press: The thumb-to-middle segment distance must drop below 50 pixels to trigger a click down event.
• Click Release: The distance must expand beyond 70 pixels before the click releases.

This 20-pixel buffer zone ensures crisp single clicks and prevents unintended drops during long click-and-drag operations.

3. Algorithmic Pipeline Implementation

The following listing illustrates the landmark EMA pre-filtering and dynamic One-Euro screen coordinate smoothing pipeline:

import math
import time
from dataclasses import dataclass

class OneEuroFilter:
    def __init__(self, min_cutoff: float = 1.0, beta: float = 0.1, d_cutoff: float = 1.0):
        self.min_cutoff = min_cutoff
        self.beta = beta
        self.d_cutoff = d_cutoff
        self.x_prev = 0.0
        self.dx_prev = 0.0
        self.t_prev = None

    def _smoothing_factor(self, dt: float, cutoff: float) -> float:
        r = 2.0 * math.pi * cutoff * dt
        return r / (r + 1.0)

    def filter(self, x: float, t: float) -> float:
        if self.t_prev is None:
            self.x_prev, self.t_prev = x, t
            return x
        
        dt = max(t - self.t_prev, 1e-4)
        
        # 1. Filter the discrete derivative (velocity)
        dx = (x - self.x_prev) / dt
        alpha_d = self._smoothing_factor(dt, self.d_cutoff)
        dx_hat = alpha_d * dx + (1.0 - alpha_d) * self.dx_prev
        self.dx_prev = dx_hat
        
        # 2. Modulate cutoff frequency dynamically based on velocity
        fc = self.min_cutoff + self.beta * abs(dx_hat)
        alpha = self._smoothing_factor(dt, fc)
        
        # 3. Apply position EMA with speed-adaptive alpha
        x_hat = alpha * x + (1.0 - alpha) * self.x_prev
        self.x_prev, self.t_prev = x_hat, t
        return x_hat

def smooth_landmark_ema(curr_pt: tuple, prev_pt: tuple, wrist_jump: bool) -> tuple:
    # Fast catchup (0.8) on wrist repositioning jump; baseline (0.2) otherwise
    alpha = 0.8 if wrist_jump else 0.2
    return (
        alpha * curr_pt[0] + (1.0 - alpha) * prev_pt[0],
        alpha * curr_pt[1] + (1.0 - alpha) * prev_pt[1]
    )

4. Core Libraries & Dependency Ecosystem

The HovrPad Gesture Engine is constructed using high-performance Python libraries and native Windows user32 APIs. All packages are statically bundled into the standalone zero-dependency portable executable.

5. Performance Benchmarks & Latency Profiling

Measurements were recorded across 5,000 consecutive interaction cycles on an Intel Core i7-11800H @ 2.30 GHz running Windows 11 x64 with a standard 1080p RGB USB sensor.

6. Gesture Lexicon & State Machine Transitions

The HovrPad Gesture Engine organizes gestures into fundamental spatial navigation primitives (unlocked permanently in the Free engine) and advanced productivity modifiers (Pro profile).

7. Hardware Prerequisites & Setup Tips

Because optical tracking detects finger contours from live RGB video, good ambient lighting helps the camera maintain a high shutter speed, preventing motion blur and ensuring accurate joint localization.

8. Citation & Artifact Verification

If you reference this software or evaluate the HovrPad Gesture Engine in your research or project, please cite this engineering release:

@software{hovrpad_gesture_engine_2026,
  author    = {Priyadarshi, C. and HovrPad HCI Systems Group},
  title     = {HovrPad Gesture Engine: High-Fidelity Optical Gesture Recognition and Low-Latency Spatial Input Synthesis},
  year      = {2026},
  version   = {7.4.2},
  publisher = {HovrPad.com},
  url       = {https://www.hovrpad.com/whitepaper.html}
}