Date of Award

5-2026

Document Type

Thesis

Degree Name

Master of Science (MS)

Department

Electrical Engineering and Computer Science

First Advisor

Neji Mensi

Second Advisor

Hemant Purohit

Third Advisor

Vivek Sharma

Fourth Advisor

Brian A. Lail

Abstract

Astronauts undergo extensive training to perform simple tasks like lifting and grabbing objects, because those same tasks become genuinely difficult in microgravity. Microgravity refers to the condition experienced aboard orbiting spacecraft such as the International Space Station, where the gravitational force is effectively canceled by continuous free fall and leaves objects and people in a state of near-weightlessness. Unlike on Earth where gravity keeps objects grounded and makes tasks like grabbing and placing things straightforward, microgravity removes that stability entirely and even the most basic manipulation tasks become challenging and require dedicated prior training. These challenges make astronaut training extra difficult, and real microgravity training remains extremely expensive and quite infrequent. To address these challenges, we developed a seven-stage human-in-the-loop 3D simulation framework. The system starts by capturing synchronized color and depth data in real time from the user using an RGB-D camera and aligns the data temporally. Then a neural network model detects the hand and locates 21 anatomical landmarks on it. Those landmarks are then reconstructed into a full 3D hand skeleton using the corresponding depth values already available with the system. Then, those 3D coordinates of the hand skeleton are fed into a virtual physics simulator that models free-floating object motion under microgravity. Since the objects in the simulator are completely virtual and the object's position is determined by the physics simulator, a Kalman filter is used which processes the noisy readings of the object as if it was a real object observed by the RGB-D camera. This produces a reliable estimate of the object's position and velocity at every frame. A Model Predictive Controller then uses this estimate to look ahead over a short future horizon, predict where the object is headed, and provide guidance to the user via a trajectory and a waypoint so the user can steer their hand toward the object. We conducted a within-subject experiment with 15 participants on a simple pick-and-place task under three conditions: Earth gravity, simulated zero gravity, and simulated zero gravity with predictive guidance. Task completion time and placement error remained lowest under Earth gravity and highest under zero gravity. Adding predictive guidance reduced both measures by a meaningful margin, placing the zero gravity with predictive guidance condition in the middle ground of all three conditions. The complete system ran on a laptop and an off-the-shelf depth camera, which makes it a cost-effective and accessible platform for microgravity manipulation research and astronaut training.

Download

Available for download on Tuesday, May 09, 2028

Share

COinS