HKUST PhD Chronicle, Week 39, GR00T, Data and Exploration

Pre-training Does Help

This is the most valuable lesson I learned this week: pre-training DOES HELP even with synthetic data.

Here’s the experiment.

1. I collected 30 demonstrations of closing a blender by grasping the lid on top.
2. I perform SFT on the 📄GR00T N1: An Open Foundation Model for Generalist Humanoid Robots.
   1. Experiment A: I only fine-tuned the bare base model of GR00T.
   2. Experiment B: I fine-tuned GR00T using a pre-trained checkpoint from the RoboCasa365 synthetic dataset.

This finding is of tremendous importance to me. Since a single researcher cannot easily scale up the data collection through manual teleoperation, I can only hope that synthetic data will save my life!

Podcast with Jie Tan

I also really enjoyed the podcast episode featuring Xiaojun and Jie Tan, a research manager in Google DeepMind Robotics.

If I had to distill the podcast into one word, it would be “data”. That’s exactly what concerns me the most right now. If I want to deploy a policy even in a narrow domain, not to mention general-purpose robots, I must have a massive amount of data.

NVIDIA Developer Meetup

I took a day off to attend the NVIDIA Developer Meetup in Shenzhen. According to John Schulman’s blog, attending events like this can be considered the $\epsilon$ factor for exploration.

Here are a few thoughts after attending the event.

📌Opportunities and Competition

Judging by the number of guests at the hotel, the event was massive. This was just a local meetup in Shenzhen! It really makes you realize how many people around the world are currently working in the Physical AI industry.

📌DRY

As a former software engineer, I was trained by my mentors to follow the DRY principle. There are some wheels you simply don’t need to rebuild from scratch, especially when they are already provided by NVIDIA.

Remark

Of course, there is no free lunch in this world. Your usage contributes to NVIDIA’s moat. But as a researcher, I have to leverage whatever tools I can.

Here are a few tools I only learned about at this event.

• NVIDIA Cosmos: the cosmos transfer is definitely worth trying.
• Isaac Teleop: an easy-to-use teleoperation system.

Ultimately, these frameworks are incredibly helpful for generating and managing synthetic data.

I also had the good fortune to speak in private with Jiwei, the tech lead of Isaac Teleop. Sometimes, just a few words from the distinguished engineer can spark massive reflection.

After a long, long day, I treated myself to a nice buffet to reward myself for all my recent hard work. Yes, I am taking inspiration from reinforcement learning: a good reward is crucial for long term exploration : - )