Refined Dataset API Server

Overview

This API server manages the workflow of dataset refinement and time tracking in a decentralized mining environment. The owner organizes and splits the data, which is then processed by miners. The refined datasets are collected to form a union dataset. The server tracks miner processing times based on request and commit timestamps.

Running Server

sudo apt install uvicorn alembic

python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

cd app
alembic init alembic
alembic upgrade head
uvicorn main:app --host 0.0.0.0 --port 8081 --reload

If you're using PM2

pm2 start uvicorn --name fineweb-api --interpreter python3 -- main:app --host 0.0.0.0 --port 8081 --reload