Mixture of Raytraced Experts

This repository contains the code for the paper Mixture of Raytraced Experts

Repo description

• moe.py: implementation of the mixture of experts layer;
• routing_net.py: implementation of the routing network and the activation sequence logic;
• train.py: main training script, which runs experiments (and stores results) based on the input config file. Requires a Weight and Biases API key to log results online;
• baselines: contains code for running baseline tests;contains the code that defines and trains the baselines;
• configs: contains yaml configuration files with hyperparameters for the runs;
• plots: contains the code for generating relevant plots;
• utils: contains utils for handling data and backbones of our models.

Setup

Recommended setup

We provide a Dockerfile, with which to create an image containing all the necessary software. CUDA 12 is required to be installed in the machine where the image is run.

Alternative setup

If you prefer to have things running locally, you can run

pip install -r requirements.txt

to install all relevant packages. As usual, using a virtual environment is highly recommended.

Licensing

See the license.

See below for further patenting details:

• U.S. Patent No. 11,823,027 B1 issued on November 21, 2023;
• U.S. Patent No. 12,112,255 issued on October 8, 2024;
• U.S. Patent No. 12,175,355 issued on December 24, 2024;
• U.S. Patent No. US-12321860-B1 published on June 3, 2025;