Update
🚀🚀[8/17/2023]v0.2.0: More assets!2 new scenes,50 new walkers,954 new objects,1k+ new instructions
We have released 
of the MO-VLN benchmark simulator.
- Support for grabbing and navigation tasks.
- Added many different walker states, including 50 unique walkers across gender, skin color, and age groups, with smooth walking or running motions.
- Added walker control interface. This interface supports:
- Selecting the walker type to generate
- Specifying where walkers are generated
- Setting whether they move freely
- Controlling the speed of their movement
- Added 1k+ instructions to our four tasks.
- We modeled an additional 954 classes of models to construct the indoor scene.
- Two new scenes have been added, bringing the total to five:
- Café
- Restaurant
- Nursing Room
- Separate tables -- Multiple tables can provide a large and efficient grasping parallel training
- Home scene -- A home suite consisting of a living room, kitchen, dining room, and multiple bedrooms
[6/18/2023]v0.1.0: 3 scenes,2165 objects, real light, shadow characteristics, and support instruction tasks with four tasks
We have released 
of the MO-VLN benchmark simulator.
- Built on UE5.
- 3 scene types:
- Café -- Modelled on a 1:1 ratio to a Café
- Restaurant -- Modelled on a 1:1 ratio to a restaurant
- Nursing Room -- Modelled on a 1:1 ratio to a Nursing Room
- We handcrafted 2,165 classes of models at a 1:1 ratio to real-life scenarios to construct these three scenes. These three scenes were ultimately constructed from a total of 4,230 models.
- We selected 129 representative classes from the models built and supported navigation testing. Among them, 54 classes are fixed within the environment, while 73 classes support customization by users.
- With real light and shadow characteristics
- Support instruction tasks with four tasks:
- goal-conditioned navigation given a specific object category (e.g., "fork");
- goal-conditioned navigation given simple instructions (e.g., "Search for and move towards a tennis ball");
- step-by-step instructions following;
- finding abstract objects based on high-level instruction (e.g., "I am thirsty").
Todo List
- ☑Added more walker states.
- ☑Added walker control interface.
- ☐Provide more classes of generative objects.
- ☐Construct complex tasks involving combined navigation and grasping.
- ☐10+ scenes are under construction and will be updated successively in the future.
- ☐Generate high-quality instruction-ground truth pairs for the newly constructed scenes.
- ☐Continue to update the simulator's physics engine effects to achieve more realistic dexterous hand-grabbing effects
- ☐Adding more interactive properties to objects in the environment, such as a coffee machine that can be controlled to make coffee.
