Xingxin on Bug Rotation Representation
The biggest lesson learned in this week was figuring out the optimal representation of rotation in machine learning.
Even though I once learned the math of using exponential map to express rotation:
from the book A Mathematical Introduction to Robotic Manipulation and learned about Euler angles in Introduction to Robotics: Mechanics and Control (as well as hearing about gimbal lock in MIT 6.421 Robotic Manipulation: Perception, Planning, and Control)… I still forgot to double-check how rotation was actually represented in my dataset!
Unsurprisingly, I was using Euler angles, which is the default for the CRISP - Compliant ROS2 Controllers for Learning-Based Manipulation Policies controller. This rotation representation is highly problematic for neural networks. Fortunately, I found an incredibly useful paper: 📄On the Continuity of Rotation Representations in Neural Networks.
The key idea is to use the first 2 rows as a 6-vector to represent this 3x3 rotation matrix
The missing last row can simply be deduced using Gram-Schmidt orthogonalization.
It turns out the action space in NVIDIA GR00T also prefers this 6D representation, and it is the canonical format used in PyTorch3D. Man, the research really is built upon the shoulder of the entire community!
Experiment as Instinct
Another issue I am currently struggling with is running experiments. It is incredibly tricky to debug your code just by observing a physical robot’s behavior.🫣
The Sea
As the son of the sea, this is a moment I want to remember forever. I am always full of gratitude for what I have and the blessings in my life.
