Please use the official RM API: https://api.www.root-me.org/
This project might be restored if the official api restricts the flow of connections.
URL: https://root-me-api.hackademint.org
BOT Discord:
- link1 (bot does not have admin rights)
- link2 (bot has admin rights) (see "[Discord Bot]" subsection to see how it works.)
Root-Me is a french platform that is the fast, easy, and affordable way to train your hacking skills. This website provides hundreds of challenges to train yourself in different and not simulated environments, offering you a way to learn a lot of hacking technics.
It also provides dozens of virtual environments, accessible with a few clicks, to give you a realistic learning environment, without any limitation.
The purpose of this project is to provide an API to use Root-Me data, provide a discord bot fetching events and data from this API and to build a distributed system thanks to workers using redis streams and consumer groups.
Using different worker instances is necessary to increase performance because
Root-Me website is blocking the requests if it detects too much connections
incoming from a specific public IP (You get an HTTP error code: "429 Too Many Requests").
(See advanced configuration).
When a client is making a request on the API, the server is checking if there is existing data in an associated redis key. If not, the server is sending a task to workers through the appropriate redis stream. If there is existing data, the API is checking if the last update is not too old (depends on UPDATE_TIMEOUT in constants.py) and ask the workers for updates if necessary.
The form of the data is:
{
'body': data,
'last_update': timestamp_isoformat
}
The workers are permanently reading the streams, which are in the same consumer group. You might read this script: init.py.
When a worker receive a task, it makes HTTPS request to Root-Me website and parse the result into JSON format inside redis keys.
You need to create a discord bot here: https://discordapp.com/developers/applications/, get a token and replace it in ./bot_discord/bot/constants.py.
docker-compose up -d- You can expose your redis by adding to redis service
docker-compose.yml:
ports:
- 6379:6379
- If you want to deploy several workers you might need to modify worker/main.py, using the public ip where you deploy the redis container.
app.redis = await aioredis.create_redis_pool(('PUBLIC_IP', 6379))Then you can deploy a new worker separately using worker/Dockerfile
- If you have a physical interface which can be bind with different public IPv4 address:
- Use the
docker-compose.ymlfile fromadvanced_configurationfolder
mv advanced_configuration/docker-compose.yml ./docker-compose.ymlYou might need the attributes from the docker macvlan network.
- Details
This configuration avoids interfaces from backend network to get an access to internet, but we want containers from this network to communicate together (api/redis/workers).
Every worker is bridged with a specific physical interface so that Root-Me detects those workers with different public IP. \ (We choose not to rotate with proxies for data privacy issues).
Example (extract from advanced_configuration/docker-compose.yml):
bridge_worker:
driver: macvlan
driver_opts:
parent: eth1
macvlan_mode: bridge
ipam:
config:
- subnet: 157.159.191.0/24
gateway: 157.159.191.2
ip_range: 157.159.191.56/29
If you deploy 6 workers (see "Install" subsection), the workers will take
157.159.191.56-61 as IPv4 address thanks to ip_range attribute (those are
taken in ascending order in the subnet specified).
Communication with the Docker host over macvlan. When using macvlan, you cannot ping or communicate with the default namespace IP address. For example, if you create a container and try to ping the Docker host’s eth0, it will not work. That traffic is explicitly filtered by the kernel modules themselves to offer additional provider isolation and security.
- Additional information
For 6 workers:
docker-compose up -d --scale worker=6Alternatively, you can use docker-compose --compatibility up with:
deploy:
mode: replicated
replicas: 6
It makes docker accept deploy section without using swarm.
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
aa46c63b6d8f root-me-api_worker "/bin/sh -c 'python3…" 18 seconds ago Up 5 seconds root-me-api_worker_3
a68c6583cbc4 root-me-api_worker "/bin/sh -c 'python3…" 18 seconds ago Up 2 seconds root-me-api_worker_5
2b35a79fe8bc root-me-api_worker "/bin/sh -c 'python3…" 18 seconds ago Up 2 seconds root-me-api_worker_1
3acc7e2fce96 root-me-api_worker "/bin/sh -c 'python3…" 18 seconds ago Up 7 seconds root-me-api_worker_6
d63206e68591 root-me-api_worker "/bin/sh -c 'python3…" 18 seconds ago Up 7 seconds root-me-api_worker_4
a14e3e6918b3 root-me-api_worker "/bin/sh -c 'python3…" 18 seconds ago Up 3 seconds root-me-api_worker_2
0361d34552be root-me-api_discord "/bin/sh -c 'python3…" 25 seconds ago Up 17 seconds root-me-api_discord_1
6eab3694fbce root-me-api_api "/bin/sh -c 'python3…" 25 seconds ago Up 17 seconds 0.0.0.0:3000->3000/tcp root-me-api_api_1
f21c0ef802aa redis "docker-entrypoint.s…" 25 seconds ago Up 17 seconds root-me-api_redis_1
You can update UPDATE_TIMEOUT in api/api/constants.py. This field represents the delay of time (in seconds) before which the API will not send a request to the worker to update the data.
Some endpoints need a valid Root-Me username you can extract from the URL of your profile.
Here is an example with https://www.root-me.org/zTeeed-115405 --> zTeeed-115405
- https://root-me-api.hackademint.org/
- https://root-me-api.hackademint.org/en/challenges
- https://root-me-api.hackademint.org/en/zTeeed-115405
- https://root-me-api.hackademint.org/en/zTeeed-115405/profile
- https://root-me-api.hackademint.org/en/zTeeed-115405/contributions
- https://root-me-api.hackademint.org/en/zTeeed-115405/details
- https://root-me-api.hackademint.org/en/zTeeed-115405/ctf
- https://root-me-api.hackademint.org/en/zTeeed-115405/stats
On your discord server you need to create a channel named root-me-news if you keep default configuration. You can change this channel name by updating the bot_channel value in bot_discord/bot/constants.py.
A docker volume (local) is used for the bot database. You have a read/write access on the JSON file at
/var/lib/docker/volumes/root-me-api_discord_data/_data/data.json
Info: data.json is not tracked by git on the local install.
git update-index --assume-unchanged bot_discord/data/data.json
Display new challenges solved by users from team.
Display available commands.
Add a user to team / Remove a user from team.
Extract the username from the Root-Me profile link.
Example with https://www.root-me.org/zTeeed-115405?inc=info
Show team scoreboard.
Show list of categories.
Show list of challenges from a category.
Show challenges solved grouped by users for last day.
Show challenges solved grouped by users for last week.
Show who solved a specific challenge.
Show difference of solved challenges between two users.
Show difference of solved challenges between a user and all team.
Flush all data from bot channel excepted events