Sovereign 8s.is mirror of standardgalactic/3D-Magic-Mirror
Find a file
Repository files (latest commit first)
Filename Latest commit message Latest commit date
2026-02-13 12:35:23 +08:00
ablation-study test-hmr 2022-04-05 03:22:18 +10:00
data Create .gitignore 2025-11-16 14:33:09 +08:00
datasets update nolpl 2023-06-01 02:18:42 +08:00
doc Add files via upload 2023-03-25 10:20:59 +08:00
network add dense161 2023-01-01 00:12:59 +08:00
PCK add pck 2022-04-06 23:55:31 +10:00
rainbow update single demo 2023-05-28 03:58:48 +10:00
src update cub data 2022-04-08 19:13:41 +10:00
template big sphere 2022-01-05 23:46:06 +11:00
tool remove 2class 2022-09-27 02:29:13 +10:00
.gitignore add market 2022-09-23 15:14:15 +10:00
ATR_single_img.py update single image 2022-04-09 18:15:01 +10:00
convert_sphere2ellipsoid.py big sphere 2022-01-05 23:46:06 +11:00
CUB_single_img.py MKT 2023-05-30 04:25:30 +10:00
demo.py cub convert L 2022-03-27 20:01:57 +11:00
fid_score.py add cudnn 2022-07-15 14:09:28 +10:00
generate_market++.py update generation 2023-01-07 04:08:42 +11:00
generate_market_new_class9.py modify generation 2022-09-27 02:24:58 +10:00
inception.py init 2021-08-16 13:41:08 +08:00
INSTALL.md Update INSTALL.md 2025-11-14 13:57:36 +08:00
MKT_single_img.py MKT 2023-05-30 04:25:30 +10:00
networks.py add normal 2023-05-29 16:49:24 +10:00
poisson_image_editing.py modify generation 2022-09-27 02:24:58 +10:00
PREPARE-CUB.md Update PREPARE-CUB.md 2025-11-16 14:34:37 +08:00
prepare_ATR.py remove nouse lib 2023-02-19 04:40:32 +11:00
prepare_cub.py add range for cub and atr 2022-10-28 19:36:25 +11:00
prepare_cub_edge.py add lambda_contour 2022-08-13 21:05:07 +10:00
prepare_market.py update market 2022-10-28 21:42:17 +11:00
preproces_cub.py Update preproces_cub.py 2025-11-14 21:22:24 +08:00
README.md Update README.md 2026-02-13 12:35:23 +08:00
requirements.txt update install 2021-12-18 13:22:35 +11:00
show_camera.py update pretrained model config 2023-05-28 02:14:27 +10:00
show_rainbow2.py update rainbow 2023-05-28 01:50:24 +10:00
smr_utils.py move back 2022-10-24 03:25:28 +11:00
spec-file.txt update spec-file for 12.0 2022-09-09 02:35:26 +10:00
temp.txt glob sorted 2022-09-18 23:39:48 +10:00
template-change-animation.py Update template-change-animation.py 2025-11-16 15:40:24 +08:00
test.py update single 2023-05-29 23:02:29 +10:00
test_backface.py move back 2022-10-24 03:25:28 +11:00
test_CUB30.py add CUB30 2022-04-07 00:56:25 +10:00
test_THU.py update nolpl 2023-06-01 02:18:42 +08:00
THU_single_img.py update nolpl 2023-06-01 02:18:42 +08:00
train.py add smoother term/still testing 2023-03-31 17:39:21 +11:00
train_ATR.py add drop 2023-01-18 19:39:53 +11:00
train_ATR2.py add smoother term/still testing 2023-03-31 17:39:21 +11:00
train_ATR2_fix.py add seed 2023-02-23 23:00:19 +11:00
train_ATR2_fix_workers.py worker control 2023-02-24 14:43:33 +11:00
train_market.py add smoother term/still testing 2023-03-31 17:39:21 +11:00
trainer.py Update trainer.py 2025-11-14 15:17:34 +08:00

License: MIT Python 3.7

3D Magic Mirror: Clothing Reconstruction from a Single Image via a Causal Perspective

Cub Bird Market-HQ Exchange

[Project] [Code] [Paper]

3D Magic Mirror: Clothing Reconstruction from a Single Image via a Causal Perspective, arXiv preprint arXiv:2204.13096, 2022.
Zhedong Zheng, Jiayin Zhu, Wei Ji, Yi Yang, Tat-Seng Chua

Table of contents

News

  • We will release a new arXiv recently. We fix the stability problem. The results in paper are averaged with at least three runs.

Features

We have supported:

  • Train and test on 3 datasets: CUB, ATR, Market
  • Generate all figures in the paper

Prerequisites

  • Linux
  • Python >= 3.7
  • CUDA >= 11 (with nvcc installed)

If you use CUDA 10, please download the corresponding pytorch and kaolin 0.9 to match.

Getting Started

Installation

Please refer to INSTALL.md

Dataset Preparation

Download tool:

$ pip install gdown 
$ pip install --upgrade gdown #!!important!!

OR

Install gdrive for fast download the dataset from Google Driver. It is good for all command line users. (https://github.com/prasmussen/gdrive/releases/tag/2.1.1 )

wget https://github.com/prasmussen/gdrive/releases/download/2.1.1/gdrive_2.1.1_linux_386.tar.gz
tar -xzvf gdrive_2.1.1_linux_386.tar.gz

Download the dataset:

Download the processed data from Google Drive. Or use the gdrive tool to download:

gdown https://drive.google.com/uc?id=10gNi3n8Iny4O4MOZRs5dNFICfj8ri9eW
OR 
gdrive download 10gNi3n8Iny4O4MOZRs5dNFICfj8ri9eW
  • Market-Seg-HMR (We deploy smoothed HMR results as foreground marks. We found human parsing results are sensitive to input domain, such as illumination.)

Download the processed data from Google Drive. Or use the gdrive tool to download:

gdown https://drive.google.com/uc?id=1JwQTQA4F9WOqLAc7fhQN0DTYwnv6XrAR
OR 
gdrive download 1JwQTQA4F9WOqLAc7fhQN0DTYwnv6XrAR
  • CUB

Please refer to PREPARE-CUB.md

  • ATR

Download the processed data from Google Drive. Or use the gdrive tool to download:

gdown https://drive.google.com/uc?id=1kpsMDrbM4FQqtP7Y1nKslp4OlRKNvbaL
OR
gdrive download 1kpsMDrbM4FQqtP7Y1nKslp4OlRKNvbaL

Preparation:

Before prepare, the folder is like:

├── 3D-Magic-Mirror/
|   |-- data/
|       |-- CUB_Data/
├── kaolin/
├── Market/
│   ├── hq/
|       |-- seg_hmr/
|-- ATR/
|   |-- humanparsing/

Only Market dataset requires preparation, and other datasets are ready to run after download.
This code will calculate the ratio of foreground against background. During training, we will drop few wrong masks or ill-detected person.

Note to modify the dataset path to your own path.

python prepare_market.py         
python prepare_ATR.py
python prepare_cub.py

Testing

Download the trained model

  • Trained model

You may download it from GoogleDrive-Market, GoogleDrive-CUB and move it to the log. Or directly use the following code:

gdown 1-eqnnFt3D7-jUelJ5uj_4QcFv-TMok8c #Market model
gdown 1urxUeaULn2DNM-4XAcZ2_OmtGJuCSeYO #CUB model
├── log/
│   ├── CamN2_MKT_wgan_b48_lr0.5_em7_update-1_lpl_reg0.1_data2_m2_flat20_depthR0.15_drop220_gap2_beta0.95_clean67/
|       |-- ckpts/
│   ├──CUB_wgan_b48_ic1_hard_bg_L1_ganW0_lr0.7_em7_update-1_chf_lpl_reg0.1_data2_depthC0.1_flat10_drop220_gap2_beta0.95_bn_restart1_contour0.1/
|       |-- ckpts/
  • Visualization
python show_rainbow2.py --name CamN2_MKT_wgan_b48_lr0.5_em7_update-1_lpl_reg0.1_data2_m2_flat20_depthR0.15_drop220_gap2_beta0.95_clean67

It will generate the five gif animations in the log/your_model_name/. (We manually select some hard index to show the result.)

current_rainbow.gif: Swapping appearnce.

current_rotation.gif: Rotation via azumith.

current_rotation_ele.gif: Rotation via elevation.

current_rotation_dist.gif: Change distance to the camera.

current_rotation_XY.gif: Shift the camera in X-axis and Y-axis.

  • Test Flops, maskIoU and SSIM
python test.py --name CamN2_MKT_wgan_b48_lr0.5_em7_update-1_lpl_reg0.1_data2_m2_flat20_depthR0.15_drop220_gap2_beta0.95_clean67
or
python test.py --name ATR2_wgan_b48_ganW0_lr0.55_em7_update-1_chf_lpl_reg0.1_m2_recon2_flat10_depthR0.15_data2_drop222_gap2_beta0.95_s96_clean1826 
or 
python test.py --name CUB_wgan_b48_ic1_hard_bg_L1_ganW0_lr0.7_em7_update-1_chf_lpl_reg0.1_data2_depthC0.1_flat10_drop220_gap2_beta0.95_bn_restart1_contour0.1 

Please make sure the dataset name in your model. We use model name to set the test dataset.

Training

  • Training on Market (64*128)
python train_market.py --name CamN2_MKT_wgan_b48_lr0.5_em7_update-1_lpl_reg0.1_data2_m2_flat20_depthR0.15_drop220_gap2_beta0.95_clean67  --clean 0.36,0.49  --imageSize 64 --batch 48 --gan_type wgan --bg --L1 --ganw 0 --hard --lr 5e-5 --em 7 --update_shape -1  --lambda_data 2 --unmask 2  --lambda_flat 0.02 --lambda_depthR 0.15  --drop 0.2,0.2,0  --em_gap 2 --beta1 0.95   --pretrainc none
  • Training on CUB (128*128)
python train.py --name CUB_wgan_b48_ic1_hard_bg_L1_ganW0_lr0.7_em7_update-1_chf_lpl_reg0.1_data2_depthC0.1_flat10_drop220_gap2_beta0.95_bn_restart1_contour0.1  --drop 0.2,0.2,0 --imageSize 128 --batch 48 --gan_type wgan --bg --L1 --ganw 0 --hard --lr 7e-5 --em 7 --update_shape -1  --lambda_data 2 --lambda_depthC 0.1 --lambda_flat 0.01   --unmask 2   --em_gap 2 --beta1 0.95 --update_bn --gamma 0.1 --scheduler restart1 --lambda_contour 0.1
  • Training on ATR (96*160) We suggest to run this dataset on A100 or A5000, instead of P5000 or R5000. The result is more stable on A series GPUs. It may be due to the float accuracy.
python train_ATR2.py --name ATR2_wgan_b48_ganW0_lr0.55_em7_update-1_chf_lpl_reg0.1_m2_recon2_flat10_depthR0.15_data2_drop222_gap2_beta0.95_s96_clean1826  --imageSize 96 --batch 48 --gan_type wgan --bg --L1 --ganw 0 --hard --lr 5.5e-5 --em 7 --update_shape -1 --unmask 2 --lambda_data 2  --lambda_flat 0.01 --lambda_depthR 0.15  --drop 0.2,0.2,0.2  --em_gap 2 --beta1 0.95 --ratio 1.666666 --clean 0.18,0.26    --pretrainc none  

Visualize Template Changes

After you train the model, the code will automatically store the intermediate template. You can use the following code to generate the template change animation.

python template-change-animation.py

Illustrations.

  • em is for the accumulated template update. If we set smooth=6, the movement will be the neighbor move. If we set smooth=7, the movement will be the smooth over neighbors again. by default 7

  • clip is to truncate the movement. by default 0.05

  • em_step is the initial moving speed, which is decay by 0.99 during training. by default 0.1

Citation

Please cite this paper if it helps your research:

@article{zheng2026magic,
  title={3D Magic Mirror: Clothing Reconstruction from a Single Image via a Causal Perspective},
  author={Zheng, Zhedong and Zhu, Jiayin and Ji, Wei and Yang, Yi and Chua, Tat-Seng},
  journal={npj Artificial Intelligence},
  url = {https://zdzheng.xyz/files/3D_Recon.pdf}, 
  code = {https://github.com/layumi/3D-Magic-Mirror}, 
  year={2026}
}

We would like to thank to the great projects in SMR and UMR.

The person re-identification part is from Pytorch re-ID

License

Copyright (C) 2025 AIGC-DL Lab. All rights reserved. Licensed under the MIT. The code is released for academic research use only. For commercial use, please contact zhedongzheng@um.edu.mo.