Full-Stack Solution Engineer - Sensorized Human

Overview

• At NVIDIA, we are closing the "embodiment gap." We don’t just build robots; we build digital and physical nervous systems that allow humans to teach robots.
• You will lead the development of DexUMI (Dexterous Universal Manipulation Interface), a framework that leverages human-worn hardware and advanced computer vision to transfer complex skills from human hands to robotic actuators.
• This is a True Full-Stack role in Solutions Architecture Team: you will touch everything from the tactile sensor firmware on a wearable exoskeleton to the cloud-based data pipelines that train our diffusion policies.
• What you'll be doing: Hardware-Software Co-Design: Maintain and iterate on the DexUMI wearable exoskeleton.
• You will bridge the kinematics gap between human hands and diverse robot end-effectors (e.g., XHand, Inspire Hand).
• Sensor Fusion & Integration: Integrate high-fidelity tactile sensors and IMUs into wearable interfaces.
• Ensure low-latency data streaming.
• Vision & Perception Pipelines: Implement and optimize the "software adaptation" layer—using tools to segment human operators out of training data and robot embodiments.
• Data Engineering for AI: Build robust pipelines to collect, clean, and replay dexterous manipulation data for Imitation Learning and Diffusion Policies.
• Optimization: Solve bi-level optimization problems to parameterize exoskeleton designs that maximize human wearability while preserving robot-equivalent fingertip workspaces.
• What we need to see: A MS/PhD in Robotics, Machine Learning, Computer Science, Electrical Engineering, Mechanical Engineering, or a related field (or equivalent experience) with at least 1 years of research and engineering experience.
• The "Body" (Hardware/Embedded): Proficiency in C/C++ for embedded systems and ROS2, with hands-on experience in tactile sensing, force-feedback (haptics), and motor control.
• Mechatronics & Prototyping: Experience with CAD (SolidWorks/Fusion360) and rapid prototyping, including 3D printing and PCB design.
• The "Brain" (Software/AI): Expertise in Python and deep learning frameworks (PyTorch), with familiarity in computer vision (CV) models.
• Advanced AI Techniques (Plus): Understanding of Imitation Learning or Reinforcement Learning (RL) is a strong plus.
• The "Bridge" (Integration): Experience with Record3D or iPhone-based spatial tracking, enabling integration between perception and physical systems.
• Systems & Infrastructure: Experience working in Ubuntu/Linux environments with high-performance data serialization tools such as Protobuf and MQTT.