Sovereign 8s.is mirror of standardgalactic/tianmoucv
  • Python 73.6%
  • Jupyter Notebook 13.6%
  • C++ 12.1%
  • CMake 0.3%
  • Batchfile 0.2%
  • Other 0.2%
Find a file
Repository files (latest commit first)
Filename Latest commit message Latest commit date
2026-06-12 12:26:56 +08:00
.github/workflows change workflow 2025-11-24 17:00:42 +08:00
docs align with nature sensors 2026-04-17 01:19:18 +08:00
resources update to sync with 0.4.1.2 dev version 2025-11-24 16:34:08 +08:00
tianmoucv align with nature sensors 2026-04-17 01:17:51 +08:00
tianmoucv_example raft update 2026-04-19 14:15:19 +08:00
.gitignore update 2024-07-19 10:36:48 +08:00
install.bat update to sync with 0.4.1.2 dev version 2025-11-24 16:34:08 +08:00
install.sh update to sync with 0.4.1.2 dev version 2025-11-24 16:34:08 +08:00
LICENSE update to sync with 0.4.1.2 dev version 2025-11-24 16:34:08 +08:00
README.md Update README.md 2026-06-12 12:26:56 +08:00
requirements.txt seems good 2026-04-09 17:48:53 +08:00
setup.py align with nature sensors 2026-04-17 01:17:51 +08:00
update.bat update to sync with 0.4.1.2 dev version 2025-11-24 16:34:08 +08:00
update.sh update to sync with 0.4.1.2 dev version 2025-11-24 16:34:08 +08:00

TianmouCV

PyPI - Version PyPI - Wheel PyPI - License PyPI - Downloads

未标题-1

Introduction

It is algorithms library for the first complementary vision sensor (CVS) Tianmouc. The APIs of algorithms and various tools can be found in Tianmoucv Project Page More details of the project can be found in Tianmouc Sensor doc

Table of Contents

Background

Tianmouc is the world's first multi-pathway brain-inspired vision sensor, which can simultaneously realize high-speed (10000 fps), high dynamics (130 dB), high-sensitivity (72%@530nm, NIR), high accuracy (10bit RGB, 757 fps@8bit - 10,000fps@2bit), low power consumption, and low bandwidth (90% bandwidth reduction compared to traditional high-speed cameras) visual perception. It provides three different data modalities: Frame-based RGB, sparse Temporal Difference and sparse Spatial Differences.

TianmouCV is the official algorithm library of Tianmouc V1 (and soon will support Tianmouc V2..), including codec of the Tianmouc data (.tmdat files), deeplearning-friendly dataset reader, basic ISP tools, feature descriptions of TSD, classical optical flow estimator and gray-scale reconstructor, some basic but useful neural-network (reconstruction, optical flow, deblur, instance segmentation and so on).

Installation

Environment Prepare

(1) Unix

-install cmake, make, g++>9, build-essential

(2) Windows

install cmake, make, minGW/VSstudio, details please refer to doc(need permission), or you can use the pre-compiled dll/pyd for installation

(3) (optional)Tianmouc SDK

if you want to use TianmouCV to read tianmouc camera directly on your PC, please install tianomuc-SDK usb version first

cd tianmouc_sdk/usb/install/cyusb_linux_1.0.5
sh install

and input the absolute path of tianmoucsdk when you install TianmouCV by executing install.sh

Compile and Install

(1) from PyPI

pip install tianmoucv

(2) Install from source codes:

git clone git@github.com:Tianmouc/tianmoucv_dev.git
cd tianmoucv_dev
sh install.sh

For windows, please refer to the "install.bat", with the help of winGW

If you do not need to use the SDK, just press ENTER when inputing the SDK PATH

(3) Only update python codes (If you have already compile the lib manually), it will be helpful if you want to save your time modifying some python codes in TianmouCV.

sh update.sh

Prepare your TMDAT files

You can download a demo tianmouc data clip in Tianmouc File Server, You can refer to the tutorial for a trial.

A standard tianmouc data structure is like:

├── dataset
│   ├── matchkey
│   │   ├── cone
│   │       ├── info.txt
│   │       ├── xxx.tmdat
│   │   ├── rod
│   │       ├── info.txt
│   │       ├── xxx.tmdat

The {matchkey} is the clip name used as a fileter key in Tianmouc data reader.

Tianmouc Research paper list and public resources

For more, see Tianmoucv Group Page

Examples

For some of the function we've provided the example in tianmoucv_example.

Most examples can be directly run on jupyter notebook, while some of them may need SDK support

conda activate [your environment]
pip install jupyter lab
jupyter lab

The examples are listed below

├── Tianmoucv example
│   ├── introduction_to_tianmouc_data: Introduce how to read and visualize Tianmouc data.
│   ├── >>>camera<<< connect to camera (SDK installation required).
│   │   ├── open_camera: receive and visualize Tianmouc data (template code)
│   │   ├── qrcode_demo:  Decode QR code based on SD.
│   │   ├── calibration_OpenCV: CameraCalib tool
│   │   ├── deblur: (buggy) deblur rgb using tsd
│   │   ├── realtime_inf: Realtime instance segmentation
│   ├── >>>data<<<
│   │   ├── covert_to_tmdat_and_calculate_bandwidth: Encode np array into the tmdat and calculate the bandwidth.
│   │   ├── rotate_tsd: trick for rotate 2D vector filed -- SD
│   │   ├── convert_pcie_bin_to_tmdat: tool for fpga dev version
│   ├── >>>proc<<<
│   │   ├── segmentation: dual pathway fusion instance segmentation
│   │   ├── feature_tracking_gray_sd: feature tracking based only on SD
│   │   ├── reconstructor:
│   │   │   ├──reconstruct_fuse_net: best HDR-RGB video NN-based reconstructor
│   │   │   ├──reconstruct_gray: Gray-scale image reconstruction based on SD
│   │   │   ├──reconstruct_hdr_poisson_iter: simple HDR fusion method
│   │   │   ├──reconstruct_original_nature_paper: RGB video NN-based reconstructor in original  Nature paper
│   │   │   ├──reconstruct_recurrent: E2VID trained on TD
│   │   │   ├──reconstruct_tiny_unet: fastest HDR-RGB video NN-based reconstructor
│   │   ├── optical_flow:
│   │   │   ├──opticalflow_HS_method: HS OF estimator with TSD
│   │   │   ├──opticalflow_LK_method: LK OF estimator with TSD
│   │   │   ├──opticalflow_spynet: NN-based OF estimator in original Nature paper
│   │   │   ├──opticalflow_RAFT: best NN-based OF estimator
│   │   ├── denoise:
│   │   │   ├──denoise_tmdat_lvatf:denoise TSD using LVATF
│   │   ├── deblur
│   │   │   ├──deblur_stgdnet: deblur rgb using tsd
│   ├── >>>simulator<<<
│   │   │   ├──sim.ipynb: run simulator for rgb/td/sd sequence with video input

We provide a example multi-thread realtime processing python script template in tianmoucv/camera/open_camera.py

you can directly run it if all the prerequisite are installed

Maintainers

@lyh983012.

Contributors

This project exists thanks to all the people who contribute.

Thanks to: Taoyi Wang, Yuguo Chen, Yapeng Meng, Yuxiang Li, Lin Yang

License

GPLv3 © Yihan Lin

Please contact: linyh@xmu.edu.cn