Sovereign 8s.is mirror of standardgalactic/scPEFT
  • Jupyter Notebook 72.5%
  • Python 27.5%
Find a file
Repository files (latest commit first)
Filename Latest commit message Latest commit date
Fei He 9d1739bd20
Add DOI badge and enhance project description
Added DOI badge and updated project overview.
2025-12-01 13:30:33 -06:00
geneformer_peft Update README.md 2025-08-24 18:58:30 -05:00
img update readme 2025-08-25 15:36:06 -05:00
pipeline_scBERT Update README.md 2025-08-24 19:00:18 -05:00
reproduction Add files via upload 2025-03-29 06:36:22 -05:00
scfoundation Add files via upload 2025-08-24 23:38:50 -05:00
scgpt Add files via upload 2024-11-08 12:30:51 +08:00
tutorial_peft Add files via upload 2025-08-24 23:33:49 -05:00
LICENSE initial commit 2024-08-17 17:45:17 +08:00
README.md Add DOI badge and enhance project description 2025-12-01 13:30:33 -06:00
requirements.yaml Update requirements.yaml 2025-03-30 18:23:01 +08:00

scPEFT: Harnessing the Power of Single-Cell Large Language Models with Parameter-Efficient Fine-Tuning

This is the official repository for Harnessing the Power of Single Cell Large Language Models with Parameter Efficient Fine-Tuning using scPEFT. To reproduce the results from the paper, please visit scPEFT_reproduction.

❗ 9/25/2025 UPDATE We have released the reproduction for benchmarking the linear model on perturbation prediction. Please see here for more details.

🔥UPDATE: We have released several new reproductions for the revised manuscript. Please see scPEFT_reproduction for more details.

Preprint   License

DOI

Overview

we propose scPEFT, a framework that integrates Parameter-Efficient Fine-Tuning (PEFT) techniques into scLLMs to calibrate them for specialized use cases. Unlike traditional finetuning, which modifies the entire model, scPEFT employs low-dimensional, learnable, and pluggable adapters to customize scLLMs in a separate, reduced-dimensional subspace. The critical role of these proxy adapters is to estimate a ‘model delta’ (standing for changes in some model parameters) for context alignment under the guidance of task-specific objective functions and limited custom data. During the adaptation process, the original scLLM parameters are frozen to preserve pre-learned biological knowledge, while only the smaller adapter parameters are updated. This design reduces the complexity of domain adaptation, enabling higher performance with fewer resources than traditional finetuning strategies of scLLMs in out-of-context scenarios.

overview

Installation

scPEFT works with Python >= 3.7.13. scPEFT is available on PyPI. To install scPEFT, run the following command:

pip install scpeft

For developing, run the following command:

git clone https://github.com/coffee19850519/scPEFT
cd scPEFT

Note: scPEFT is currently built on top of scGPT, scBERT, scFoundation and Geneformer. Please follow their installation instructions to ensure all necessary versioned dependencies are installed. We provide a requirements. ymal file for the environment in which scPEFT was developed.

Get Started

  1. Download the backbone model, e.g., scGPT model checkpoint and place it at e.g., work_dir/scPEFT/save. We recommend using the whole-human model for most applications by default, which pretrained on 33 million normal human cells.

  2. The tutorials of scPEFT for downstream tasks in tutorial_peft. Here are the links to the downstream tasks and tutorials mentioned in our article

Backbone Downstream task Link
scGPT cell type identification Tutorial_Identification.ipynb
scGPT batch correction Tutorial_BatchCorrection.ipynb
scGPT perturbation Tutorial_Perturbation.ipynb
scGPT cell population discovery Tutorial_CellPopulationDiscovery.ipynb
scGPT marker gene detection Tutorial_MarkerGeneDetection.ipynb
scFoundation cell type identification Tutorial_identification_scFoundation.ipynb
scFoundation perturbation Tutorial_Perturbation_scFoundation.ipynb
Geneformer cell type identification Tutorial_identification_Geneformer.ipynb
scBERT cell type identification Tutorial_identification_scBERT.ipynb

To-do-list

  • Publish to pypi
  • Adapting scPEFT for native-attention
  • Release scripts for more downstream tasks
  • Release scripts for more scLLM backbones
  • Adapting scPEFT for flash-attention
  • Only retain PEFT-related parameters when saving peft-model weights.

Contributing

We greatly welcome contributions to scPEFT. Please submit a pull request if you have any ideas or bug fixes. We also welcome any issues you encounter while using scPEFT.

Built With

We sincerely thank the authors of following open-source projects:

Citing scPEFT

@article {He2025.04.21.649754,
	author = {He, Fei and Fei, Ruixin and Krull, Jordan E. and Zhang, Xinyu and Gao, Mingyue and Su, Li and Chen, Yibo and Yu, Yang and Li, Jinpu and Jin, Baichuan and Chang, Yuzhou and Ma, Anjun and Ma, Qin and Xu, Dong},
	title = {Harnessing the Power of Single-Cell Large Language Models with Parameter Efficient Fine-Tuning using scPEFT},
	year = {2025},
	doi = {10.1101/2025.04.21.649754},
	publisher = {Cold Spring Harbor Laboratory},
	URL = {https://www.biorxiv.org/content/early/2025/04/23/2025.04.21.649754},
	journal = {bioRxiv}
}