K-Scale Manipulation Suite

Contents

Setup - Linux

https://github.com/kscalelabs/gym-kmanip

Clone and install dependencies

git clone https://github.com/kscalelabs/gym-kmanip.git && cd gym-kmanip
conda create -y -n gym-kmanip python=3.10 && conda activate gym-kmanip
pip install -e .

Run tests

pip install pytest
pytest tests/test_env.py

Setup - Jetson Orin AGX

No conda on ARM64, install on bare metal

sudo apt-get install libhdf5-dev
git clone https://github.com/kscalelabs/gym-kmanip.git && cd gym-kmanip
pip install -e .

Usage - Basic

Visualize the MuJoCo scene

python gym_kmanip/examples/1_view_env.py

Record a video of the MuJoCo scene

python gym_kmanip/examples/2_record_video.py

Usage - Recording Data

K-Scale HuggingFace Datasets Data is recorded via teleop, this requires additional dependencies

pip install opencv-python==4.9.0.80
pip install vuer==0.0.30
pip install rerun-sdk==0.16.0

Start the server on the robot computer

python gym_kmanip/examples/4_record_data_teleop.py

Start ngrok on the robot computer

ngrok http 8012

Open the browser app on the VR headset and go to the ngrok URL

Usage - Visualizing Data

Data is visualized using rerun

rerun gym_kmanip/data/test.rrd

Usage - MuJoCo Sim Visualizer

MuJoCo provides a nice visualizer where you can directly control the robot Download standalone MuJoCo

tar -xzf ~/Downloads/mujoco-3.1.5-linux-x86_64.tar.gz -C /path/to/mujoco-3.1.5

Run the simulator

/path/to/mujoco-3.1.5/bin/simulate gym_kmanip/assets/_env_solo_arm.xml

Citation

@misc{teleop-2024,
  title={gym-kmanip},
  author={Hugo Ponte},
  year={2024},
  url={https://github.com/kscalelabs/gym-kmanip}
}