ByteGrader is an open-source, modular autograder designed to evaluate programming assignments in embedded systems, IoT, and edge AI. Built to support a variety of programming languages, it uses containerized environments to reliably compile, run, and assess student submissions.
This project was created to streamline the grading process for technical coursework, offering:
- Flexible assignment configurations
- Support for custom test scripts and code pattern checks
- Reproducible Docker-based environments for isolation and consistency
While ByteGrader is optimized for embedded systems and hardware-centric courses, its architecture is general enough to be extended to other domains and languages.
- Language-agnostic grading engine (Python, C, etc.)
- Docker-based execution to isolate and reproduce environments
- Pluggable modules for different courses or assignments
- Code analysis tools to verify API usage, function calls, or design patterns
- Grading STM32 or ESP32 firmware assignments for embedded systems courses
- Evaluating IoT device data parsing and MQTT communication tasks
- Verifying Python scripts for edge AI inference pipelines
- Ensuring students follow proper function use and coding standards
If you want to test a grader locally, see the Developing a Grader section to see how to spin up a container with a grader. Setting up the full server will require a little more effort.
For the full server, you will need to buy a domain or configure a domain with a subdomain (e.g. esp32-iot.bytegrader.com) for SSL/TLS certificate signing. I recommend Namecheap.
With a domain, you can then set up the hosting service. This can be a self-maintained server from your home/office or a paid virtual private server (VPS). See the following guides for buying/configuring a server:
Make sure you can get root shell access to your server (e.g. through SSH).
We will assume that you want the server to handle grading for multiple courses. So, we will create a subdomain for each course. For example, if bytegrader.com
is the main domain for the grader, we will redirect bytegrader.com
to the ByteGrader GitHub repo (using nginx), but esp32-iot.bytegrader.com
will be the URL for the autograder.
Log in to your domain name provider and click to manage your domain for your grader. Add the following records (ask ChatGPT for specifics if you don't know how to do this for your particular domain name provider). Note that <SUBDOMAIN>
is your course tag (e.g. esp32-iot
) and <YOUR_SERVER_IP>
is the IP address we got for our server in the previous step.
Type | Host | Value | TTL |
---|---|---|---|
A | @ | <YOUR_SERVER_IP> | Automatic |
A | www | <YOUR_SERVER_IP> | Automatic |
A | <YOUR_SERVER_IP> | Automatic |
To start, you'll need to install Docker (from these instructions). As root, install dependencies and add Docker's official GPG key:
apt-get update
apt-get install ca-certificates curl
install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
chmod a+r /etc/apt/keyrings/docker.asc
Add the Docker repository to the Apt sources:
echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
$(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}") stable" | \
tee /etc/apt/sources.list.d/docker.list > /dev/null
apt-get update
Install Docker:
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
we'll create a new bytegrader user (so that we don't run everything as root), clone the repository, and configure the server. You should only need to do this once, and it should be done as root.
Make sure that you are SSH'd into your server (as root). Create the bytegrader user (this name is important, as the setup scripts assume you have such a user and home directory).
adduser --disabled-password --gecos "" bytegrader
usermod -aG docker bytegrader
You can optionally copy the SSH keys so you can remotely log into the server as either root or bytegrader.
mkdir -p /home/bytegrader/.ssh
cp /root/.ssh/authorized_keys /home/bytegrader/.ssh/
chown -R bytegrader:bytegrader /home/bytegrader/.ssh
chmod 700 /home/bytegrader/.ssh
chmod 600 /home/bytegrader/.ssh/authorized_keys
If you don't want to log in directly as bytegrader, you can log in as root and switch to bytegrader.
su - bytegrader
Make sure you are in the bytegrader user, and then clone the repo:
cd /home/bytegrader/
git clone https://github.com/ShawnHymel/bytegrader.git
cd bytegrader/
Feel free to check out a particular tag, version, or branch. (e.g. git checkout v1.2
).
Make sure the setup scripts are executable:
chmod +x deploy/*.sh
Log back in as root (or enter logout
to escape out of the su - bytegrader
shell). Then, run the server setup script as a superuser:
cd /home/bytegrader/bytegrader/
bash deploy/setup-server.sh
This will walk you through the process of assigning several important environment variables that are used throughout the setup process:
- Main domain - The main domain name you purchased earlier (e.g. bytegrader.com). Note that for now, this will redirect to
github.com/ShawnHymel/bytegrader
, as we only need the subdomain for our autograder endpoints. - Course subdomain - The server will set up a subdomain for your course's autograder endpoints. For example,
esp32-iot
will mean the full URL of the autograder ishttps://esp32-iot.bytegrader.com
. - Email - Your email address for SSL certificate notifications (from certbot)
- IP whitelist - List of IP addresses (comma separated) that are allowed to connect to the server. Leave empty to allow all connections. Ideally, this should be the IPv4 and IPv6 addresses of your course site (LMS) and your personal, public IP address (so you can test from home/office).
- API key - Secret key (password) used to authenticate clients connecting to the server. Ideally, only your LMS site should have the same key.
If openssh-server pops up asking you what to do with the existing sshd_config file, accept the default (keep the local version).
Log in as the bytegrader user (e.g. su - bytegrader
), make an app/ directory, and run the deploy app. The deploy.sh script will copy the relevant files from the repo to the app/ directory.
cd /home/bytegrader/
mkdir -p app/
cd bytegrader/
bash deploy/deploy.sh /home/bytegrader/app/
Once that runs, you can check to make sure that the grader container is reachable locally:
curl http://localhost:8080/health
This should show {"status":"ok"}
.
Then, you can check to make sure that you can lookup your subdomain's IP address with:
nslookup <SUBDOMAIN>.<DOMAIN>
You can't make any requests yet, as you need to enable SSL.
Even though our app is running, we need to generate SSL certificates and get them signed (through Let's Encrypt). It also sets up certbot to renew certificates automatically. We needed to wait until now to run setup-ssl.sh, as we only just set up our domain and subdomain with the deploy.sh script.
Switch to the root user (logout
or re-login via SSH) and run the setup-ssl.sh script:
cd /home/bytegrader/bytegrader
bash deploy/setup-ssl.sh
Hopefully, this completes successfully. You can check with:
curl https://<SUBDOMAIN>.<DOMAIN>/health
This should show {"status":"ok"}
.
With the server running, you should be able to send test submissions to the /submit
endpoint from one of your clients on the approved IP address whitelist. See Test Grading for more information.
See the API endpoints page for a full list of endpoints and supported HTTP methods.
A grader is a combination of Docker image and Python grading script. Every time a student submits a file (a .zip file) to a particular assignment, the API app spins up a container from an image built for that assignment. Start with the graders/test-stub grader as a boilerplate template.
Here is how you build and run the test-stub grader locally (with an example submission):
cd graders/
docker build -t bytegrader-test-stub -f test-stub/Dockerfile .
cd ..
mkdir -p test/results/
docker run --rm -v "$(pwd)/test/submission-make-c-hello.zip:/submission/submission.zip:ro" -v "$(pwd)/test/results/:/results" bytegrader-test-stub
When you run the image, it will read in the submission-make-c-hello.zip file, rename to submission.zip in the container, process it with grader.py, and store the results in test/results as output.json. Note that test-stub does not actually build or run any submitted code. It simply verifies that the file is a .zip archive and returns a constant score and feedback.
Once the server is running, you can update ByteGrader (with minimal downtime) by logging into the server as the bytegrader user, stopping the container, updating the repository, and then calling the deploy.sh script again. Don't forget to give the script the location of your app directory!
ssh bytegrader@<SUBDOMAIN>.<DOMAIN>
cd ~/app
docker compose down
cd ~/bytegrader
git pull
./deploy/deploy.sh ~/app
Verify that the server is running with:
curl http://localhost:8080/health
From your home/office computer (assuming you've whitelisted your public IP address), you can test submitting a dummy file for grading using the test-stub grader (which always returns a static grade/feedback so long as it receives a valid .zip file).
curl -X POST -H "X-API-Key: <API_KEY>" -H "X-Username: test-user" -F "file=@test/make-c-add/submission.zip" https://<SUBDOMAIN>.<DOMAIN>/submit?assignment=make-c-add
You should receive a "File submitted for grading" JSON message back from the server. Copy the job_id and check the status of the grading job:
curl -H "X-API-Key: <API_KEY>" -H "X-Username: test-user" https://<SUBDOMAIN>.<DOMAIN>/status/<JOB_ID>
You can watch the real-time logs of the server with:
cd /home/bytegrader/app
docker compose logs -f
- Never commit API keys, passwords, or certificates
- Use environment variables for sensitive configuration
- Keep dependencies updated
- Enable branch protection on main
Docker Compose keeps a running set of logs. You can view them by logging into the server, navigating to the app/ directory, and running:
cd /home/bytegrader/app/
docker compose logs
You can also watch logs in realtime with:
docker compose logs -f
You can view the queue from an approved IP address and with the correct API key:
curl -H "X-API-Key: <API_KEY>" https://./queue
There is a quick and dirty health check script you can run to get stats on the server (CPU, RAM, disk usage, etc.):
bash /home/bytegrader/app/health-check.sh
To update Go dependencies (i.e. if you import a new package in main.go or want to update package listings in go.mod and go.sum), run the following:
docker run --rm -v "$PWD/server":/app -w /app golang:1.24 go mod tidy
If you want to do a quick build of the Go server and throw away the build artifacts to check for basic syntax and build-time errros, just run a quick Go container:
docker run --rm -v "$PWD/server":/app -w /app golang:1.24 go build -o /dev/null .
If the server is already running and you'd like to update the white list (e.g. so you can add another client for testing), edit the environment variables:
nano /home/bytegrader/.bytegrader_env
Once you've added/removed the desired IP addresses, redploy (which will read in the saved environment variables from that file):
cd /home/bytegrader/bytegrader
bash deploy/deploy.sh ../app
Note that if you need to get the "local" IP address (what the App container sees when making calls from the host server), as 127.0.0.1
and localhost
won't often work, you can run docker compose logs | grep "Security check"
. This will likely be 172.18.0.0/16
so the App container can see the host.
Release notes are kept in CHANGELOG.md.
- Add full integration test with make/C grader example
- Make multi-stage Docker build for adding in environments (e.g. Arduino, ESP-IDF)
- Build API for hosting site and LearnDash
All code, unless otherwise specified, is subject to the 3-Clause BSD License. See the LICENSE file for more details.