465
edits
Changes
Created page with "= Expressive Whole-Body Control for Humanoid Robots = == Installation == <code> conda create -n humanoid python=3.8 conda activate humanoid cd pip3 install torch==1.10.0+cu11..."
= Expressive Whole-Body Control for Humanoid Robots =
== Installation ==
<code>
conda create -n humanoid python=3.8
conda activate humanoid
cd
pip3 install torch==1.10.0+cu113 torchvision==0.11.1+cu113 torchaudio==0.10.0+cu113 -f https://download.pytorch.org/whl/cu113/torch_stable.html
git clone git@github.com:chengxuxin/expressive_humanoid_covariant.git
cd expressive_humanoid_covariant
# Download the Isaac Gym binaries from https://developer.nvidia.com/isaac-gym
cd isaacgym/python && pip install -e .
cd ~/expressive_humanoid_covariant/rsl_rl && pip install -e .
cd ~/expressive_humanoid_covariant/legged_gym && pip install -e .
pip install "numpy<1.24" pydelatin wandb tqdm opencv-python ipdb pyfqmr flask dill gdown
</code>
Next install fbx. Follow the instructions here.
== Prepare dataset ==
Download from here and extract the zip file to ASE/ase/poselib/data/cmu_fbx_all that contains all .fbx files.
Generate .yaml file for the motions you want to use.
<code>
cd ASE/ase/poselib
python parse_cmu_mocap_all.py
</code>
This step is not mandatory because the .yaml file is already generated. But if you want to add more motions, you can use this script to generate the .yaml file.
== Import motions ==
<code>
cd ASE/ase/poselib
python fbx_importer_all.py
</code>
This will import all motions in CMU Mocap dataset into ASE/ase/poselib/data/npy.
== Retarget motions ==
<code>
cd ASE/ase/poselib
mkdir pkl retarget_npy
python retarget_motion_h1_all.py
</code>
This will retarget all motions in ASE/ase/poselib/data/npy to ASE/ase/poselib/data/retarget_npy.
== Generate keybody positions ==
This step will require running simulation to extract more precise key body positions.
<code>
cd legged_gym/legged_gym/scripts
python train.py debug --task h1_view --motion_name motions_debug.yaml --debug
</code>
Train for 1 iteration and kill the program to have a dummy model to load.
<code>
python play.py debug --task h1_view --motion_name motions_autogen_all.yaml
</code>
It is recommended to use motions_autogen_all.yaml at the first time, so that later if you have a subset it is not necessary to regenerate keybody positions. This will generate keybody positions to ASE/ase/poselib/data/retarget_npy. Set wandb asset:
== Usage ==
To train a new policy:
<code>
python train.py xxx-xx-some_descriptions_of_run --device cuda:0 --entity WANDB_ENTITY
</code>
xxx-xx is usually an id like 000-01. motion_type and motion_name are defined in legged_gym/legged_gym/envs/h1/h1_mimic_config.py. They can be also given as arguments. Can set default WANDB_ENTITY in legged_gym/legged_gym/utils/helpers.py.
To play a policy:
<code>
python play.py xxx-xx
</code>
No need to write the full experiment id. The parser will auto match runs with first 6 strings (xxx-xx). So better make sure you don't reuse xxx-xx. Delay is added after 8k iters. If you want to play after 8k, add --delay.
To play with example pretrained models:
<code>
python play.py 060-40 --delay --motion_name motions_debug.yaml
</code>
Try to press + or - to see different motions. The motion name will be printed on terminal. motions_debug.yaml is a small subset of motions for debugging and contains some representative motions.
Save models for deployment:
<code>
python save_jit.py --exptid xxx-xx
</code>
This will save the models in legged_gym/logs/parkour_new/xxx-xx/traced/.
== Viewer Usage ==
Can be used in both IsaacGym and web viewer.
* ALT + Mouse Left + Drag Mouse: move view.
* [ ]: switch to next/prev robot.
* Space: pause/unpause.
* F: switch between free camera and following camera.
=== IsaacGym viewer specific ===
* +: next motion in yaml.
* -: prev motion in yaml.
* r: reset the motion to the beginning.
* ]: camera focus on next env.
* [: camera focus on prev env.
== Arguments ==
* --exptid: string, can be xxx-xx-WHATEVER, xxx-xx is typically numbers only. WHATEVER is the description of the run.
* --device: can be cuda:0, cpu, etc.
* --delay: whether add delay or not.
* --checkpoint: the specific checkpoint you want to load. If not specified load the latest one.
* --resume: resume from another checkpoint, used together with --resumeid.
* --seed: random seed.
* --no_wandb: no wandb logging.
* --entity: specify wandb entity
* --web: used for playing on headless machines. It will forward a port with vscode and you can visualize seamlessly in vscode with your idle gpu or cpu. Live Preview vscode extension required, otherwise you can view it in any browser.
* --motion_name: e.g. 07_04 or motions_all.yaml. If motions_all.yaml is used, motion_type should be yaml.
* --motion_type: single or yaml
* --fix_base: fix the base of the robot.
For more arguments, refer legged_gym/utils/helpers.py.
== Installation ==
<code>
conda create -n humanoid python=3.8
conda activate humanoid
cd
pip3 install torch==1.10.0+cu113 torchvision==0.11.1+cu113 torchaudio==0.10.0+cu113 -f https://download.pytorch.org/whl/cu113/torch_stable.html
git clone git@github.com:chengxuxin/expressive_humanoid_covariant.git
cd expressive_humanoid_covariant
# Download the Isaac Gym binaries from https://developer.nvidia.com/isaac-gym
cd isaacgym/python && pip install -e .
cd ~/expressive_humanoid_covariant/rsl_rl && pip install -e .
cd ~/expressive_humanoid_covariant/legged_gym && pip install -e .
pip install "numpy<1.24" pydelatin wandb tqdm opencv-python ipdb pyfqmr flask dill gdown
</code>
Next install fbx. Follow the instructions here.
== Prepare dataset ==
Download from here and extract the zip file to ASE/ase/poselib/data/cmu_fbx_all that contains all .fbx files.
Generate .yaml file for the motions you want to use.
<code>
cd ASE/ase/poselib
python parse_cmu_mocap_all.py
</code>
This step is not mandatory because the .yaml file is already generated. But if you want to add more motions, you can use this script to generate the .yaml file.
== Import motions ==
<code>
cd ASE/ase/poselib
python fbx_importer_all.py
</code>
This will import all motions in CMU Mocap dataset into ASE/ase/poselib/data/npy.
== Retarget motions ==
<code>
cd ASE/ase/poselib
mkdir pkl retarget_npy
python retarget_motion_h1_all.py
</code>
This will retarget all motions in ASE/ase/poselib/data/npy to ASE/ase/poselib/data/retarget_npy.
== Generate keybody positions ==
This step will require running simulation to extract more precise key body positions.
<code>
cd legged_gym/legged_gym/scripts
python train.py debug --task h1_view --motion_name motions_debug.yaml --debug
</code>
Train for 1 iteration and kill the program to have a dummy model to load.
<code>
python play.py debug --task h1_view --motion_name motions_autogen_all.yaml
</code>
It is recommended to use motions_autogen_all.yaml at the first time, so that later if you have a subset it is not necessary to regenerate keybody positions. This will generate keybody positions to ASE/ase/poselib/data/retarget_npy. Set wandb asset:
== Usage ==
To train a new policy:
<code>
python train.py xxx-xx-some_descriptions_of_run --device cuda:0 --entity WANDB_ENTITY
</code>
xxx-xx is usually an id like 000-01. motion_type and motion_name are defined in legged_gym/legged_gym/envs/h1/h1_mimic_config.py. They can be also given as arguments. Can set default WANDB_ENTITY in legged_gym/legged_gym/utils/helpers.py.
To play a policy:
<code>
python play.py xxx-xx
</code>
No need to write the full experiment id. The parser will auto match runs with first 6 strings (xxx-xx). So better make sure you don't reuse xxx-xx. Delay is added after 8k iters. If you want to play after 8k, add --delay.
To play with example pretrained models:
<code>
python play.py 060-40 --delay --motion_name motions_debug.yaml
</code>
Try to press + or - to see different motions. The motion name will be printed on terminal. motions_debug.yaml is a small subset of motions for debugging and contains some representative motions.
Save models for deployment:
<code>
python save_jit.py --exptid xxx-xx
</code>
This will save the models in legged_gym/logs/parkour_new/xxx-xx/traced/.
== Viewer Usage ==
Can be used in both IsaacGym and web viewer.
* ALT + Mouse Left + Drag Mouse: move view.
* [ ]: switch to next/prev robot.
* Space: pause/unpause.
* F: switch between free camera and following camera.
=== IsaacGym viewer specific ===
* +: next motion in yaml.
* -: prev motion in yaml.
* r: reset the motion to the beginning.
* ]: camera focus on next env.
* [: camera focus on prev env.
== Arguments ==
* --exptid: string, can be xxx-xx-WHATEVER, xxx-xx is typically numbers only. WHATEVER is the description of the run.
* --device: can be cuda:0, cpu, etc.
* --delay: whether add delay or not.
* --checkpoint: the specific checkpoint you want to load. If not specified load the latest one.
* --resume: resume from another checkpoint, used together with --resumeid.
* --seed: random seed.
* --no_wandb: no wandb logging.
* --entity: specify wandb entity
* --web: used for playing on headless machines. It will forward a port with vscode and you can visualize seamlessly in vscode with your idle gpu or cpu. Live Preview vscode extension required, otherwise you can view it in any browser.
* --motion_name: e.g. 07_04 or motions_all.yaml. If motions_all.yaml is used, motion_type should be yaml.
* --motion_type: single or yaml
* --fix_base: fix the base of the robot.
For more arguments, refer legged_gym/utils/helpers.py.