Ai2 发布开源具身模型 MolmoAct 2

“I really shouted, ‘whoa is this actually working!’ when I saw it worked!”

MolmoAct 2 had been public for just ten days when South Park Commons’ embodied AI hackathon kicked off in San Francisco on May 15. By the end of the weekend, the winning entry was a MolmoAct 2-powered, voice-controlled robot built by Binh Pham, a robotics software engineer at LiveKit.

“One thing really caught my eye, which is everyone says this works out of the box—they just deploy on their robots, and it just works,” says Pham.

On Pham’s robot, MolmoAct 2 did what he needed it to do—he connected the model to a voice-controlled setup and gave the robot jobs it hadn’t been specifically trained to perform. What stood out for Pham was not only that the model could recognize objects in front of it, but that it could use those observations to guide actions.

“The model actually has very good spatial awareness,” says Pham. “MolmoAct 2 can identify objects really well.” In robot learning, often the hardest part is connecting an instruction to the scene in front of the robot, reasoning about where objects are, and producing actions that make sense for the hardware. “This is the first time I can actually see a model that wasn’t trained on the tasks that I had given, but can actually do it,” adds Pham.

Pham’s takeaway was that MolmoAct 2 showed the kind of general-purpose behavior roboticists have been trying to unlock for decades. What makes MolmoAct 2 “amazing,” in his words, is that the community now has access to a model like this.

“We really look for this kind of generalization [in open models],” he says. “I truly think it’s like the GPT-2 moment of robotics.”

MolmoAct 2 is available with the artifacts researchers and builders need to inspect, adapt, and customize it: model weights, data, training code, and fine-tuning scripts. For a field where strong robot learning systems can be difficult to reproduce or build on, that makes the release more useful than a single model checkpoint.

Watch our interview with Pham above. To build on MolmoAct 2 yourself, download the model weights from Hugging Face, read the release blog, and find the training code and fine-tuning scripts on GitHub.

Subscribe to receive monthly updates about the latest Ai2 news.