Xingxin on Bug Happy Spring Festival! A long week off for the break, it is great to take time to recharge.
This week I finished Homework 1: Imitation Learning of the CS 185/285: Deep Reinforcement Learning course. Imitation learning is highly relevant to my current robot learning project. Having these concepts nailed down is extremely satisfying and builds essential foundational knowledge for my work this semester!
I also documented what I learned in two new notes:
Along the way, I discovered a few tools and language features that will be incredibly useful for my future research tech stack.
Tyro
I noticed the assignment uses tyro to generate CLI flags, which I found to be incredibly convenient!
For example, the following script:
from typing import Literal
def main(
name: str,
greet: Literal["Hello", "Hi"] = "Hi",
) -> None:
print(f"{greet}, {name}!")
if __name__ == "__main__":
import tyro
tyro.cli(main)can be directly consumed by tyro. You don’t need to explicitly write out the --help flag or use argparse; it is automatically deduced. The function parameters are automatically parsed as well:
$ python script.py --help
usage: script.py [-h] [OPTIONS]
Print a greeting.
╭─ options ──────────────────────────────╮
│ -h, --help show help message │
│ --name STR (required) │
│ --greet {Hello,Hi} (default: Hi) │
╰────────────────────────────────────────╯
$ python script.py --name World
Hi, World!As you can see, the --name flag is automatically generated straight from the function parameter name: str.
Keyword-Only Arguments
The keyword-only argument feature in Python is new to me. Take a look at the following function signature:
def parse_train_config(
args: list[str] | None = None,
*,
description: str = "Train a Push-T MLP policy.",
) -> TrainConfig:
passNotice the second parameter in this function: an isolated asterisk (*). Coming from a C++ background, seeing this syntax was completely new to me. After some research, I learned that the * here serves a special purpose. Any arguments placed after the * are forced to be keyword-only arguments.
This means if I want to pass ["--batch-size", "32"] as the args and provide a custom string for the description, I must explicitly use the keyword:
# ✅
parse_train_config(["--batch-size", "32"], description="custom string")
# ❌
parse_train_config(["--batch-size", "32"], "custom string")I feel like this feature is conceptually very close to designated initializers in modern C++. It forces you to be explicit, which makes the code much more readable!