110 lines
4.0 KiB
Markdown
110 lines
4.0 KiB
Markdown
# RSL-RL
|
|
|
|
A fast and simple implementation of learning algorithms for robotics. For an overview of the library please have a look at https://arxiv.org/pdf/2509.10771.
|
|
|
|
Environment repositories using the framework:
|
|
|
|
* **`Isaac Lab`** (built on top of NVIDIA Isaac Sim): https://github.com/isaac-sim/IsaacLab
|
|
* **`Legged Gym`** (built on top of NVIDIA Isaac Gym): https://leggedrobotics.github.io/legged_gym/
|
|
* **`MuJoCo Playground`** (built on top of MuJoCo MJX and Warp): https://github.com/google-deepmind/mujoco_playground/
|
|
* **`mjlab`** (built on top of MuJoCo Warp): https://github.com/mujocolab/mjlab
|
|
|
|
The library currently supports **PPO** and **Student-Teacher Distillation** with additional features from our research. These include:
|
|
|
|
* [Random Network Distillation (RND)](https://proceedings.mlr.press/v229/schwarke23a.html) - Encourages exploration by adding
|
|
a curiosity driven intrinsic reward.
|
|
* [Symmetry-based Augmentation](https://arxiv.org/abs/2403.04359) - Makes the learned behaviors more symmetrical.
|
|
|
|
We welcome contributions from the community. Please check our contribution guidelines for more
|
|
information.
|
|
|
|
**Maintainer**: Mayank Mittal and Clemens Schwarke <br/>
|
|
**Affiliation**: Robotic Systems Lab, ETH Zurich & NVIDIA <br/>
|
|
**Contact**: cschwarke@ethz.ch
|
|
|
|
|
|
## Setup
|
|
|
|
The package can be installed via PyPI with:
|
|
|
|
```bash
|
|
pip install rsl-rl-lib
|
|
```
|
|
|
|
or by cloning this repository and installing it with:
|
|
|
|
```bash
|
|
git clone https://github.com/leggedrobotics/rsl_rl
|
|
cd rsl_rl
|
|
pip install -e .
|
|
```
|
|
|
|
The package supports the following logging frameworks which can be configured through `logger`:
|
|
|
|
* Tensorboard: https://www.tensorflow.org/tensorboard/
|
|
* Weights & Biases: https://wandb.ai/site
|
|
* Neptune: https://docs.neptune.ai/
|
|
|
|
For a demo configuration of PPO, please check the [example_config.yaml](config/example_config.yaml) file.
|
|
|
|
|
|
## Contribution Guidelines
|
|
|
|
For documentation, we adopt the [Google Style Guide](https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html) for docstrings. Please make sure that your code is well-documented and follows the guidelines.
|
|
|
|
We use the following tools for maintaining code quality:
|
|
|
|
- [pre-commit](https://pre-commit.com/): Runs a list of formatters and linters over the codebase.
|
|
- [ruff](https://github.com/astral-sh/ruff): An extremely fast Python linter and code formatter, written in Rust.
|
|
|
|
Please check [here](https://pre-commit.com/#install) for instructions to set these up. To run over the entire repository, please execute the following command in the terminal:
|
|
|
|
```bash
|
|
# for installation (only once)
|
|
pre-commit install
|
|
# for running
|
|
pre-commit run --all-files
|
|
```
|
|
|
|
## Citing
|
|
|
|
If you use this library for your research, please cite the following work:
|
|
|
|
```text
|
|
@article{schwarke2025rslrl,
|
|
title={RSL-RL: A Learning Library for Robotics Research},
|
|
author={Schwarke, Clemens and Mittal, Mayank and Rudin, Nikita and Hoeller, David and Hutter, Marco},
|
|
journal={arXiv preprint arXiv:2509.10771},
|
|
year={2025}
|
|
}
|
|
```
|
|
|
|
If you use the library with curiosity-driven exploration (random network distillation), please cite:
|
|
|
|
```text
|
|
@InProceedings{schwarke2023curiosity,
|
|
title = {Curiosity-Driven Learning of Joint Locomotion and Manipulation Tasks},
|
|
author = {Schwarke, Clemens and Klemm, Victor and Boon, Matthijs van der and Bjelonic, Marko and Hutter, Marco},
|
|
booktitle = {Proceedings of The 7th Conference on Robot Learning},
|
|
pages = {2594--2610},
|
|
year = {2023},
|
|
volume = {229},
|
|
series = {Proceedings of Machine Learning Research},
|
|
publisher = {PMLR},
|
|
url = {https://proceedings.mlr.press/v229/schwarke23a.html},
|
|
}
|
|
```
|
|
|
|
If you use the library with symmetry augmentation, please cite:
|
|
|
|
```text
|
|
@InProceedings{mittal2024symmetry,
|
|
author={Mittal, Mayank and Rudin, Nikita and Klemm, Victor and Allshire, Arthur and Hutter, Marco},
|
|
booktitle={2024 IEEE International Conference on Robotics and Automation (ICRA)},
|
|
title={Symmetry Considerations for Learning Task Symmetric Robot Policies},
|
|
year={2024},
|
|
pages={7433-7439},
|
|
doi={10.1109/ICRA57147.2024.10611493}
|
|
}
|
|
```
|