Xiwen Liang1*, Liang Ma1*, Shanshan Guo2, Jianhua Han3, Hang Xu3,
Shikui Ma4, Xiaodan Liang1↑
Complex Tasks for Grasping and Navigation
Recorded Human Demonstration Tracks by VR
It features real-time lighting, capable of realistically generating features that may appear in images captured by cameras in real scenarios, such as illuminated shadows, reflections and refractions from transparent mirrors, and bright spots caused by non-transparent mirror reflections.
The simulation environment incorporates humans who may partially obstruct the path in the corridor, or potentially walk into the robot's route. We support generating up to 50 unique 3D human models across gender, skin color, and age groups, each with smooth walking or running animations.Our walker control interface provides an intuitive way to manage the virtual pedestrians in your scene. It allows selecting the walker model to generate, specifying where the walkers should be placed, setting whether they move freely or follow preset paths, and controlling the overall speed of their movement.
The simulator supports multiple types of robots, all of which correspond to the actual parameters of real robots from the Dataa Robotics. For instance, the first robot shown in the picture has 21 adjustable parameters throughout its body, including the head, neck, waist, elbow, wrist, fingers, etc. These parameters allow it to perform complex actions. The head, chest and waist are equipped with RGB-D cameras
Simple Instruction： Locate the nearest monitor. Robot, move towards the knife. Approach the chess set nearby. Walk towards a closet in your vicinity. Seek out any visible book in the area.
Reasoning Instruction： I'm thinking about doing some screen-based work, find me a device that will help me with that. I'm about to make a sandwich and need a helpful utensial to get the job done efficiently. Feeling inspired to make a cozy, warm meal for tonight. Could you fetch me the key vessel we'll be using to cook it in? In the mood for some strategic fun, can you locate a board game with elegant pieces engraved with squares on it? Find me a place where I can enjoy my meal more comfortably.
Step-by-step Instruction with Movement： Robot, I need you to follow this path to reach the Tongs. Begin by moving forward towards the Teapot, continue going straight, passing the Drinking Machine and Alcoholic Drink. Keep moving forward until you reach the Cake and then take a slight left. Proceed forward next to more Cakes, then take a right turn as you keep forward. After that final turn, you will see the Tongs in front of you. Please stop upon reaching the Tongs.
Step-by-step Instruction without Movement： Robot, from the starting point, please move towards the monitor. As you get closer to it. Continue moving, keeping both the monitor within your view. Keep this trajectory until you find the plants, which is your target destination. Once you've reached the plants, stop and stay there.