HKUST PhD Chronicle, Week 32, Franka Teleop and Co-Training

Action Space

I discovered that the action space is extremely important in teleoperation. The quality of recorded episodes varies significantly depending on whether I use a SpaceMouse, an Xbox controller or simply drag the gumball via viser.

Based on my testing, the incremental values recorded with SpaceMouse and Xbox controller work well. However, simply dragging the gumball introduces discontinuities in the action space, which makes the robot arm’s movement jerky.

Control Barrier Function

Another frequent obstacle is dealing with reflex errors thrown by libfranka: C++ library for Franka Robotics research robots. These usually fall into 2 categories:

• collision
• singularity

Because I am teleoperating the robot in task space using $\Delta$SE(3), it means I don’t have motion planning calculating the path ahead of time. As a result, the arm can easily collide with objects or hit a singularity.

Pausing and rerunning the pipeline every time is extremely tedious. To fix this, I am looking into Control Barrier Function. A paper titled 📄Safe, Task-Consistent Manipulation with Operational Space Control Barrier Functions caught my eye as a potential solution. However, since my background isn’t in control theory, I still have a knowledge gap to close before I can fully grasp it.

Co-Training

The next strategy I plan to try is co-training, which I first heard about it Yuke Zhu’s talk Toward Generalist Humanoid Robots.

My gut feeling is that it will work, but it might require a lot of fine-tuning. TRI also recently released a paper on co-training: 📄A Systematic Study of Data Modalities and Strategies for Co-training Large Behavior Models for Robot Manipulation. Therefore, I plan to spend some time reading up on that.

ROS2 Labs

Finally, I have to prepare the rest four ROS2 labs over the next four weeks for an undergraduate course I am TAing. That is going to take up quite a bit of my time!