MAMUT-routing: curated VRPTW contracts and OSM-backed instances for solver testing

[0nyr]

We have just released the public website for MAMUT-routing. See this Reddit r/optimization announcement.

I am posting here first because part of the project may be relevant to PyVRP users and solver developers, second because this project is directly inspired by PyVRP / PyVRP/Instances and use it through the mamut-routing-lib CLI as a dependency.

MAMUT-routing is an open-source, open-science benchmark infrastructure for routing research. The current public snapshot contains 2 problem classes (CVRP and VRPTW with a focus on the later), 5 benchmark families, 1294 instances, and 1296 BKS/reference-solution entries. The repository combines curated benchmark artifacts, objective metadata, BKS/route files, provenance information, a website, and a workbench for visualizing or generating instances.

One motivation is the ambiguity around classical VRPTW benchmarks. The same Solomon / Gehring-Homberger customer data can be used under different contracts, which adds further work and difficulty on students, researchers and practitioners looking to compare their algorithms with the state-of-the-art (see this reddit post).

From a PyVRP perspective, the useful part is not only the public website. The repository is meant to expose reusable benchmark artifacts with enough metadata to let external tooling decide how to load or reject them. For example, a solver script should be able to distinguish a DIMACS-style integer mono-cost VRPTW instance from a SINTEF-style hierarchical objective instance before reporting a result. Likewise, generated Mamut2026 instances should carry enough context to explain whether a cost matrix comes from road-network travel time, road-network distance, or Euclidean distance.

We are also interested in whether the MAMUT-routing JSON structure can be made easier to consume from PyVRP examples or benchmark scripts. Have a look at mamut-routing-lib for that matter, directly inspired by PyVRP/Instances. If there are fields that would make conversion cleaner, or if some objective conventions should be represented differently, this is the right time to discuss it.

We would be very interested in feedback or direct involvement from the PyVRP community.

The source code is MIT. Some data has family-specific terms: ORTEC material is CC BY-NC 4.0, and OSM-derived artifacts are ODbL where applicable.

Issues and discussions are open here:

• MAMUT-routing Issues
• MAMUT-routing Discussions

[leonlan]

Hi @0nyr, congrats on the MAMUT-routing library! It touches upon a lot of things I've personally encountered while benchmarking PyVRP: the different conventions, BKS tracking, and hard-to-find instances. These types of initiatives have been on our minds on several occasions. I believe some students in the Netherlands have worked on a similar benchmarking platform together with ORTEC, though it was never published afterwards.

Here are our answers to your questions and comments:

From a PyVRP perspective, the useful part is not only the public website. The repository is meant to expose reusable benchmark artifacts with enough metadata to let external tooling decide how to load or reject them. For example, a solver script should be able to distinguish a DIMACS-style integer mono-cost VRPTW instance from a SINTEF-style hierarchical objective instance before reporting a result. Likewise, generated Mamut2026 instances should carry enough context to explain whether a cost matrix comes from road-network travel time, road-network distance, or Euclidean distance.
We are also interested in whether the MAMUT-routing JSON structure can be made easier to consume from PyVRP examples or benchmark scripts.

We've already covered most of these conventions through our VRPLIB work and are currently still OK with our VRPLIB-based examples. They are fairly simple and straightforward for our benchmarking purposes/testing. We don't feel much friction that would be solved from adopting different formats, and maintaining adapters for them is an effort we don't have capacity for at the moment. I don't mind occasionally reviewing an adapter to PyVRP, so feel free to ping me on GitHub to request a review.

We would be very interested in feedback or direct involvement from the PyVRP community.

Unfortunately, we currently don't have the capacity to provide feedback or get involved in MAMUT-routing, or other academic work. Instead, I'd highly recommend reaching out to the CVRPLIB maintainers to see if you can involve them - I think you both share the same goals.

Just an FYI regarding your Reddit post:

CVRPLib instances and BKS, which currently propose another variant over the classical Solomon/Gehring-Homberger data: mono-cost minimization with double-precision Euclidean arc costs.

This is not accurate: CVRPLIB instances and BKS for VRPTW follow the DIMACS convention, which we also use in our instances.

One thing doesn't sit well with me here: our discussions board is primarily used for our own development work, and announcements like this feel a bit like self-promotion. A heads-up email beforehand would have been more appropriate. I'm closing this discussion since I've touched upon our stance here, but feel free to continue the discussion if you've got other thoughts to share.

Wishing you the best of luck!

[0nyr]

Dear @leonlan,

This is my first real effort at open-sourcing a project like this, though I'm expecting other projects to come to this point in a near future. As such, I'm sorry if opening this discussion was not well-received on behalf of the PyVRP project. It seems that this was the best location to open such a discussion, and your precious answer just helped me collect relevant information.

Let me answer several of your points as well:

We've already covered most of these conventions through our VRPLIB work and are currently still OK with our VRPLIB-based examples. They are fairly simple and straightforward for our benchmarking purposes/testing. We don't feel much friction that would be solved from adopting different formats, and maintaining adapters for them is an effort we don't have capacity for at the moment. I don't mind occasionally reviewing an adapter to PyVRP, so feel free to ping me on GitHub to request a review.

Indeed, we did not try at proposing a PR to VRPLib. Its so-called format is an all-time classic in the community, and mamut-routing-lib offers conversions from our simple .json format to the VRPLib format where JSON keys are replaced to uppercased text sections. I do not believe any effort from your side is needed for now, though we expect to work and add Time-Dependent (TDVRP, TDVRPTW) benchmark families and formats that we will not be able to easily convert to VRPLib format.

Unfortunately, we currently don't have the capacity to provide feedback or get involved in MAMUT-routing, or other academic work. Instead, I'd highly recommend reaching out to the CVRPLIB maintainers to see if you can involve them - I think you both share the same goals.

Thanks for the clear message. Just a quick note, my initial post is not only intended at the maintainers of PyVRP but also other researchers and contributors of the PyVRP community to which my welcoming announcement is also aimed at. Of course, as an open-source initiative, I do not expect anyone, including myself, to commit to this project beyond what anyone is willing to go.

CVRPLib instances and BKS, which currently propose another variant over the classical Solomon/Gehring-Homberger data: mono-cost minimization with double-precision Euclidean arc costs.

Many thanks on this call ! I looked everywhere and just assumed its contract, because it is mentioned nowhere. I got a nice email from @EduardoQueiroga who provided some BKS files on behalf of CVRPlib when he was still the official maintainer. I used its instances to consolidate the other VRPTW benchmark instances. I will correct this claim asap.

The project is still on its early phase. We will contact CVRPlib members soon.

Many thanks again for the detailed reply.