Xingxin on Bug Real-Time Action Chunking
Finally, I finished writing A Visual Guide to Real-Time Action Chunking in Robotics! It feels great to learn something new. I remember when I first read about this topic… it was awkward trying to understand those concepts…
Another thing I found especially useful is drawing diagrams. Coming from a design background, I love illustrating ideas visually. When I draw a diagram, I am actually practicing and reinforcing the concept in my head. It’s a great way to learn!
Confounders
The biggest lesson I learned this week is about confounding variables and how they can affect experiments, leading to confusing outputs and scenarios.
For example, this week I tuned (the size of prediction chunk), (the size of execution chunk) and the Sim/Real data mixture all at the exact same time! Changing multiple variables at once created uncomfortable obstacles and made it impossible to tell which change actually caused the results.
Human Should Lead
Another big lesson I learned this week closely relates to what Mitchell Hashimoto (the author of Ghostty) wrote in this blog post, My AI Adoption Journey.
Recently, I kept hoping that an AI agent could just hand me the perfect solution. However, in my experience, the bottleneck is actually on the human side. Human throughput is much, much slower than an AI agent’s. If you constantly ask an agent for open-ended advice, you will quickly get swarmed with too much information to process.
Maybe it’s a good idea to apply Jacobi’s principle: “Invert, always Invert”. Instead of asking the AI agent, “What is the correct solution?”, I should propose my own solution and ask the agent, “Is my solution correct?” This way, the human leads the direction, and the AI works within your specific context.