K-Scale Teleop

From Humanoid Robots Wiki
Jump to: navigation, search

A minimal implementation of a bi-manual remote robotic teleoperation system using VR hand tracking and camera streaming.

Feature Status
VR and browser visualization ✔ Completed
Bi-manual hand gesture control ✔ Completed
Camera streaming (mono + stereo) ✔ Completed
Inverse kinematics ✔ Completed
Meta Quest Pro HMD + NVIDIA® Jetson AGX Orin™ Developer Kit ✔ Completed
.urdf robot model ✔ Completed
3dof end effector control ✔ Completed
Debug 6dof end effector control ⬜ Pending
Resets to various default poses ⬜ Pending
Tested on real world robot ⬜ Pending
Record & playback trajectories ⬜ Pending


Setup[edit]

git clone https://github.com/kscalelabs/teleop.git && cd teleop
conda create -y -n teleop python=3.8 && conda activate teleop
pip install -r requirements.txt

Usage[edit]

  • Start the server on the robot computer:
python demo_hands_stereo_ik3dof.py
  • Start ngrok on the robot computer:
ngrok http 8012
  • Open the browser app on the HMD and go to the ngrok URL.

Dependencies[edit]

  • Vuer is used for visualization
  • PyBullet is used for inverse kinematics
  • ngrok is used for networking

Citation[edit]

@misc{teleop-2024,
  title={Bi-Manual Remote Robotic Teleoperation},
  author={Hugo Ponte},
  year={2024},
  url={https://github.com/kscalelabs/teleop}
}