Epipolar Transformers

Screen Shot 2022-10-17 at 5.46.06 PM.png

Yihui He, Rui Yan, Katerina Fragkiadaki, Shoou-I Yu (Carnegie Mellon University, Facebook Reality Labs)

Oral presentation and human pose demo videos (playlist):

Models

config	MPJPE (mm)	model & log
ResNet-50 256×256, triangulate	45.3	https://github.com/yihui-he/epipolar-transformers/releases/download/outputs/outs.benchmark.keypoint_h36m_afterfix.zip
ResNet-50 256×256, triangulate, epipolar transformer	33.1	https://github.com/yihui-he/epipolar-transformers/releases/download/outputs/outs.epipolar.keypoint_h36m_fixed.zip
ResNet-50 256×256, triangulate, epipolar transformer (augmentation)	30.4	https://github.com/yihui-he/epipolar-transformers/releases/download/outputs/outs.epipolar.keypoint_h36m_fixed_aug.zip
ResNet-152 384×384, triangulate, epipolar transformer (extra data)	19

We also provide 2D to 3D lifting network implementations for these two papers:

3D Hand Shape and Pose from Images in the Wild, CVPR 2019
- configs/lifting/img_lifting_rot_h36m.yaml (Human 3.6M)
- configs/lifting/img_lifting_rot.yaml (RHD)
Learning to Estimate 3D Hand Pose from Single RGB Images, ICCV 2017
- configs/lifting/lifting_direct_h36m.yaml (Human 3.6M)
- configs/lifting/lifting_direct.yaml (RHD)

Python 3, pytorch > 1.2+ and pytorch < 1.4

pip install -r requirements.txtconda install pytorch cudatoolkit=10.0 -c pytorch

mkdir outscd datasets/bash get_pretrained_models.sh

Please follow the instructions in datasets/README.md for preparing the dataset