- Jupyter Notebook 74.9%
- Python 25%
- Dockerfile 0.1%
| Filename | Latest commit message | Latest commit date |
|---|---|---|
|
|
||
| earli | ||
| outputs | ||
| photos | ||
| .gitignore | ||
| config.yaml | ||
| config_cuopt.yaml | ||
| config_initialization.yaml | ||
| config_test.yaml | ||
| config_train.yaml | ||
| constraints.txt | ||
| Dockerfile | ||
| environment.yaml | ||
| Example.ipynb | ||
| ExampleTrain.ipynb | ||
| license.txt | ||
| NVIDIA_CLA_v1.0.1.docx | ||
| pyproject.toml | ||
| README.md | ||
EARLI: Evolutionary Algorithm with RL Initialization
a cuOpt‑powered combinatorial optimization framework
EARLI is a hybrid framework for solving the Vehicle Routing Problem (VRP). It uses Reinforcement Learning (RL) to learn to quickly generate high-quality solutions; then uses these as the initial population of a Genetic Algorithm (GA). The RL initialization enables leveraging of data for generalization across problems and acceleration of solving new instances. The GA search leverages inference time to refine the solution iteratively. Together, EARLI accelerates the time to find state-of-the-art solutions to VRP instances. More details are available in the paper Accelerating Vehicle Routing via AI-Initialized Genetic Algorithms.
This repo integrates EARLI's framework with NVIDIA's cuOpt genetic algorithm solver. The repo includes:
- A parallelisable VRP environment compatible with both gym and stable-baselines.
- Training code from a given data file of problem instances (see training example notebook).
- Inference code that generates multiple solutions for a set of given problem instances.
- Integration to NVIDIA's cuOpt: the generated solutions are injected to the solver as its initial population (see inference example notebook).
- Several pre-trained RL models are provided, based on synthetic training data.
Contents
![]() |
|---|
| EARLI: a, During offline training, an RL agent interacts with a dataset of problems and learns to generate high-quality solutions. b, On inference, the trained RL agent faces a new problem instance and generates K solutions with quick decision making. c, The K solutions are used as the initial population of the cuOpt genetic algorithm, initiating its optimization loop. |
![]() |
|---|
| The solution of cuOpt, with and without EARLI, in a sample problem of 100 customers in Rio de Janeiro, given a time-budget of 1s. The RL agent was trained on Sao-Paulo problems and generalized to Rio. Note that the arrows are straight for visualization only: the actual traveling costs correspond to road-based driving time. |
Installation
Option I: pip install
- Clone this repo:
git clone https://github.com/NVlabs/EARLI.git
cd EARLI
- (Recommended) Create and activate a new virtual environment with Python>=3.10:
python3.11 -m venv earli_venv
source earli_venv/bin/activate
pip install --upgrade pip
- Install build tools (for compiling C/C++ extensions):
sudo apt-get update -y && sudo apt-get install -y git build-essential ninja-build
- Install EARLI and its dependencies:
pip install --upgrade --extra-index-url https://pypi.nvidia.com -c constraints.txt .
Option II: Download our prebuilt Docker
- Install Docker and NVIDIA Container Toolkit. Follow the instructions in the NVIDIA Container Toolkit.
- Download the docker image.
docker pull ido90/earli:1.00
- Run either a command-line session or a Jupyter Lab inside the docker:
Option A: Command-line
- Run:
docker run -it --rm --runtime=nvidia --network=host --gpus all -v <SOURCE_DIR_PATH>:/opt/source ido90/earli:1.00 /bin/bash
where <SOURCE_DIR_PATH> is the path to the folder containing the EARLI source code and data files on your host machine.
- Inside the docker:
cd /opt/source
Option B: Jupyter Lab
Run the command below and open the URL that it prints in the browser:
docker run --rm -it \
-p 127.0.0.1:8888:8888 \
-v "$PWD":/work -w /work --gpus all \
ido90/earli:1.00 \
bash -lc '
git config --global --add safe.directory /work || true
PY=/opt/conda/bin/python3
$PY -V
$PY -m pip install -q jupyterlab ipykernel
$PY -m jupyter lab --ip=0.0.0.0 --no-browser --port=8888 --allow-root
'
If port 8888 is occupied, replace it (e.g. -p 127.0.0.1:8889:8888) and adjust the URL accordingly when opening it in the browser.
Option III: Build the Docker yourself
Same steps as Option II, but instead of step 2, clone the repo and run:
docker build --network=host -t earli:latest -f Dockerfile .
Examples
Example for training is provided in ExampleTrain.ipynb.
Example for inference is provided in Example.ipynb. It can be used with our pretrained RL agents provided under earli/pretrained_models/, or with a new trained agent.
Paper's hyper-parameters
In the paper, the RL agent was trained via curriculum learning over problem sizes, starting from problem instances of 50 nodes and increasing gradaully up to 500, using the following curriculum schedule and hyper-parameters from config_train.yaml:
| Phase | Train problem size | epochs |
learning_rate |
data_steps_per_epoch |
unused_capacity_penalty |
pretrained_fname |
save_model_path |
|---|---|---|---|---|---|---|---|
| #1 | 50 | 4000 |
3e-4 |
1024 | 0 | null |
earli/pretrained_models/vrp_50.m |
| #2 | 100 | 1000 |
2.5e-4 |
1024 | 0 | earli/pretrained_models/vrp_50.m |
earli/pretrained_models/vrp_100.m |
| #3 | 200 | 800 |
1.5e-4 |
512 | 8 | earli/pretrained_models/vrp_100.m |
earli/pretrained_models/vrp_200.m |
| #4 | 500 | 300 |
0.5e-4 |
256 | 10 | earli/pretrained_models/vrp_200.m |
earli/pretrained_models/vrp_500.m |
Data
The data of problem instances - for either training or inference - should be stored in a pickle file that contains a python dictionary with the following fields:
'positions':numpy.ndarray, (n_problems, problem_size, 2)'distance_matrix':numpy.ndarray, (n_problems, problem_size, problem_size)'demands':numpy.ndarray, (n_problems, problem_size)'capacities':numpy.ndarray, (n_problems,)
Sample problems can be downloaded from the NVIDIA Labs olist-vrp-benchmark repository:
python download_data.py --cleanup
This downloads into ./datasets/ VRP problems based on real Brazilian e-commerce data with various sizes (50-500 nodes) for Rio de Janeiro and São Paulo.
Alternative Training APIs
The training example in the notebook uses Stable-Baselines VecEnv API, enabling both an efficient vectoric environment and the standard SB3 algorithms. To use this API, in config.yaml, compatibility_mode should be set to stable_baselines. We support two additional APIs:
gym: The standard non-vectoric Gym API (>0.27).null: The API used in our paper, enabling both our custom PPO implementation and K-beam solution generation. This mode is necessary for generatingK>1solutions on inference.
By default, we recommend mode stable_baselines for training and null for inference.
stable_baselines API
stepreturns a tuple of(state, reward, done, info), where:stateis a dictionary of numpy arrays, each of dimensions(n_problems, ...).rewardis a numpy array of lengthn_problems, containing the rewards for each problem.doneis a numpy array of lengthn_problems, indicating whether each problem is done.infois a dictionary containing additional information about the environment.
resetreturns a state.
gym API
stepreturns a tuple of(state, reward, done, truncated, info), where:stateis a dictionary of numpy arrays, each of dimensions(n_problems, n_beams, ...).rewardis a numpy array of length(n_problems, n_beams), containing the rewards for each problem.doneis a numpy array of length(n_problems, n_beams), indicating whether each problem is done.truncatedis a numpy array False of lengthn_problems, for compatibility with the latest Gym API.infois a dictionary containing additional information about the environment.
resetreturns a tuple (state, {}).
null API
stepreturns a tuple of(state, reward, done,info), where:stateis a TensorDict, each tensor is of of dimensions(n_problems, n_beams, ...).rewardis a torch array of length(n_problems, n_beams), containing the rewards for each problem and beam.doneis a numpy array of length(n_problems, n_beams), indicating whether each problem and beam is done.infois a dictionary containing additional information about the environment.
resetreturns astate.
Cite Us
This repo is published by NVIDIA as part of the paper Accelerating Vehicle Routing via AI-Initialized Genetic Algorithms. To cite:
@article{EARLI25,
title={Accelerating Vehicle Routing via AI-Initialized Genetic Algorithms},
author={Greenberg, Ido and Sielski, Piotr and Linsenmaier, Hugo and Gandham, Rajesh and Mannor, Shie and Fender, Alex and Chechik, Gal and Meirom, Eli},
journal={arXiv preprint},
year={2025}
}

