Sovereign 8s.is mirror of standardgalactic/gympose-pose-estimation
  • Python 97.8%
  • PowerShell 2.2%
Find a file
Repository files (latest commit first)
Filename Latest commit message Latest commit date
2026-09-01 21:47:39 +03:00
docs Add Camera 6 multi-view pose experiments 2026-09-01 21:47:39 +03:00
preprocessing Add Camera 6 multi-view pose experiments 2026-09-01 21:47:39 +03:00
scripts Add Camera 6 multi-view pose experiments 2026-09-01 21:47:39 +03:00
.gitignore Add Camera 6 multi-view pose experiments 2026-09-01 21:47:39 +03:00
README.md Add Camera 6 multi-view pose experiments 2026-09-01 21:47:39 +03:00
requirements.txt first commit 2025-12-17 22:34:49 +03:00

GymPose: A New Benchmark and Pose-based Framework for Fine-grained Action Classification

Python 3.8+

A pose-based framework for fine-grained classification of artistic gymnastics movements using keypoint annotations. This repository provides training scripts, preprocessing pipelines, and evaluation tools for movement recognition and temporal phase detection.

Overview

This project implements a hierarchical classification approach for gymnastics movements using human pose keypoints extracted from video frames. The framework supports:

  • Dual-granularity classification: 6-class movement recognition and 18-class phase-specific classification
  • Multiple pose estimators: ViTPose and RTMPose annotations
  • Four classification architectures: DenseNet121, ResNeXt50, FT-Transformer, and XGBoost
  • Multi-view support: ViTPose and RTMPose workflows for Camera 6 (3.Aci) with 6-class and 18-class outputs

Dataset

GymPose Dataset

The GymPose dataset contains 16,782 annotated frames of six artistic gymnastics movements:

  • TCF Flik Flak
  • TCF Geriye Toplu Salto (backward tucked somersault)
  • TCF Geyik Sicrama (deer jump)
  • TCF Makasli Geyik (scissor deer)
  • TCF Makasli Geyik Yarim (half scissor deer)
  • TCF Spagat (split)

Download (public): Kaggle – GymPose

Each movement is divided into three temporal phases:

  • Phase I: Starting position
  • Phase II: Mid-movement
  • Phase III: Ending position

Yoga-82 Dataset

The framework also supports the Yoga-82 dataset for pose-based yoga classification. Download the original dataset from the Yoga-82 Project Page.

Pose Estimation

This project uses pose keypoint annotations following the COCO-WholeBody format. Pose estimation can be performed using ViTPose or RTMPose frameworks.

ViTPose

easy_ViTPose provides a simplified implementation of ViTPose for human pose estimation. ViTPose-H extracts 25 body keypoints per person with high accuracy.

RTMPose

RTMPose is a real-time multi-person pose estimation framework from OpenMMLab. RTMPose extends the keypoint format to 26 keypoints, offering real-time inference capabilities.

Keypoint Format

Each annotation contains 25 keypoints (ViTPose) or 26 keypoints (RTMPose) with the following structure. For detailed keypoint indices and body part mappings, see docs/KEYPOINTS.md.

Each keypoint is represented by three values: (x, y, confidence), where (x, y) are pixel coordinates and confidence is a score in the range [0, 1].

CSV Data Format

Training scripts expect CSV files with the following structure:

class_id, class_name, image_path, kp0_x, kp0_y, kp0_conf, kp1_x, kp1_y, kp1_conf, ..., kp24_x, kp24_y, kp24_conf

For 18-class (phase-specific) configurations, additional columns are included:

..., phase_name, phase_id, combined_class_id

The classification models use only the (x, y) coordinate pairs, resulting in 50-dimensional feature vectors (25 keypoints × 2 coordinates).

Installation

Requirements

pip install -r requirements.txt

Dependencies

  • Python 3.8+
  • PyTorch 1.12+
  • torchvision
  • pandas
  • numpy
  • scikit-learn
  • xgboost
  • matplotlib
  • seaborn
  • tqdm

Pose Estimation Setup

ViTPose

For ViTPose-based pose estimation, install easy_ViTPose:

git clone https://github.com/JunkyByte/easy_ViTPose.git
cd easy_ViTPose
pip install -r requirements.txt

Refer to the easy_ViTPose repository for detailed installation and usage instructions.

RTMPose

For RTMPose-based pose estimation, install MMPose with RTMPose support:

pip install -U openmim
mim install mmengine
mim install "mmcv>=2.0.0"
mim install "mmpose>=1.0.0"

Refer to the RTMPose documentation for detailed setup and usage.

Usage

Camera 6 Pose Extraction

In the multi-angle dataset, Camera 6 is stored as 3.Aci. The pipeline scripts validate the input layout, select the configured frames, extract pose keypoints, and produce both movement-level (6-class) and phase-level (18-class) CSV files.

# ViTPose pipeline
.\scripts\run_tcf_camera6_vitpose_pipeline.ps1

# RTMPose pipeline; uses the same sampled frame indices
.\scripts\run_tcf_camera6_rtmpose_pipeline.ps1

Use -ValidateOnly to inspect inputs without loading a pose model, or -SkipTraining to stop after dataset preparation. Generated frames, pose annotations, CSV files, rendered videos, checkpoints, and training outputs are excluded by .gitignore.

Training Models

All training scripts are located in the scripts/ directory and support 5-run averaging with stratified cross-validation. Models are trained with early stopping based on validation accuracy.

GymPose Dataset (6-class and 18-class)

For the GymPose gymnastics dataset, use the Yoga-82 training scripts with your GymPose CSV files:

# 6-class movement classification
python scripts/train_yoga82_xgboost_classifier.py --data path/to/gympose_6class.csv
python scripts/train_yoga82_densenet121_classifier.py --data path/to/gympose_6class.csv

# 18-class phase-specific classification
python scripts/train_yoga82_densenet121_classifier.py --data path/to/gympose_18class.csv

Yoga-82 Dataset

Train on the Yoga-82 dataset with different granularity levels:

Yoga-82 (82 individual poses):

python scripts/train_yoga82_xgboost_classifier.py --data path/to/yoga_82class.csv
python scripts/train_yoga82_densenet121_classifier.py --data path/to/yoga_82class.csv
python scripts/train_yoga82_resnext50_classifier.py --data path/to/yoga_82class.csv
python scripts/train_yoga82_fttransformer_classifier.py --data path/to/yoga_82class.csv

Yoga-20 (20 semantic groups):

python scripts/train_yoga20_xgboost_classifier.py --data path/to/yoga_20class.csv
python scripts/train_yoga20_densenet121_classifier.py --data path/to/yoga_20class.csv
python scripts/train_yoga20_resnext50_classifier.py --data path/to/yoga_20class.csv
python scripts/train_yoga20_fttransformer_classifier.py --data path/to/yoga_20class.csv

Yoga-6 (6 broad categories):

python scripts/train_yoga6_xgboost_classifier.py --data path/to/yoga_6class.csv
python scripts/train_yoga6_densenet121_classifier.py --data path/to/yoga_6class.csv
python scripts/train_yoga6_resnext50_classifier.py --data path/to/yoga_6class.csv
python scripts/train_yoga6_fttransformer_classifier.py --data path/to/yoga_6class.csv

Output Structure

Training results are saved in the results/ directory:

results/
├── model_6class_5run_results/
│   ├── run_1/
│   │   ├── model.pth
│   │   ├── training_history.csv
│   │   └── confusion_matrix.png
│   ├── run_2/
│   ├── ...
│   ├── run_5/
│   └── model_5_run_summary.txt
└── model_18class_5run_results/
    └── ...

Summary files contain averaged metrics across 5 runs:

  • Test Accuracy (mean ± std)
  • Top-5 Accuracy
  • Precision (weighted)
  • Recall (weighted)
  • F1-Score (weighted)

Preprocessing Scripts

All preprocessing scripts are located in the preprocessing/ directory.

Dataset Conversion (Yoga-82)

Convert between different class granularities for the Yoga-82 dataset:

# Convert 82-class to 20-class
python preprocessing/convert_82_to_20_classes.py

# Convert 20-class to 6-class
python preprocessing/convert_20_to_6_classes.py

ViTPose Annotation Processing

Process raw ViTPose JSON annotations to CSV format:

# Convert JSON annotations to CSV
python preprocessing/convert_json_to_csv.py --input annotations.json --output dataset.csv

RTMPose Annotation Processing

Convert RTMPose annotations to different granularities:

# Convert RTMPose 82-class to 20-class (Yoga-82)
python preprocessing/convert_rtmpose_82_to_20.py

# Convert RTMPose 20-class to 6-class (Yoga-82)
python preprocessing/convert_rtmpose_20_to_6.py

GymPose Dataset Preparation

Prepare 18-class CSV for phase-specific training:

# Prepare GymPose 18-class training CSV
python preprocessing/prepare_18class_training_csv.py

This script converts the combined_class_id column to class_id for phase-specific classification.

Evaluation

Evaluate trained models on test sets:

python preprocessing/evaluate_model.py --model path/to/model.pth --data path/to/test_data.csv

Results

Camera 6 benchmark summaries are available for ViTPose and RTMPose.

Performance metrics for the GymPose dataset (ViTPose annotations):

Model TCF-6 Accuracy TCF-18 Accuracy
DenseNet121 95.15 ± 0.57% 90.04 ± 0.45%
ResNeXt50 91.50 ± 0.53% 85.81 ± 1.27%
FT-Transformer 94.83 ± 0.13% 89.07 ± 0.69%
XGBoost 93.75 ± 0.65% 88.13 ± 0.64%

For detailed preprocessing pipeline, see TCF Dataset Preprocessing.

Documentation

Acknowledgments

  • easy_ViTPose - Simplified ViTPose implementation for pose estimation
  • RTMPose - Real-time multi-person pose estimation framework
  • Yoga-82 Dataset - Yoga pose dataset for classification benchmarks
  • ViTPose and RTMPose teams - Original pose estimation architectures

Contact

For questions or issues, please open an issue on GitHub or contact the authors.