Xingxin on Bug Hooooooooray! Between the beginning of February and this week, I managed to build the full pipeline from data collection on real robots to model training, evaluation, and deployment all by myself.
This is my first imitation learning demo. It might look clumsy, slow and a bit dumb, but it is my very first one. Today marks a special milestone for me!
Along the way, I found plenty of resources that were incredibly useful.
- Github Repo: Teleoperation of Franka FR3 with Spacemouse
- CRISP - Compliant ROS2 Controllers for Learning-Based Manipulation Policies: A really nice framework that reduces the complexity from ROS2 for Python users. I learned a lot from reading through this repository.
During this journey, I also had to self-teach quite a lot. Here are the learning materials I highly recommend.
- Introduction to Robotics: Mechanics and Control: An absolute bible. It is easy to understand and uses rigorous notation.
- Robotic Manipulation: Another great resource given by Russ Tedrake who incidentally also favors John Craig’s book.
TipFun Fact: John Craig actually wrote the Introduction to Robotics: Mechanics and Control while he was a PhD student! Huge respect!
To better understand how to teleoperate a robot, I also wrote down a few reflections of my own: