Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dockerise the whole thing. #15

Merged
merged 10 commits into from
Jun 1, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# HTTP Configuration
HTTP_PORT=2001
HTTP_ADDRESS=0.0.0.0
HTTP_THREAD_COUNT=4
HTTP_USING_CLOUDFLARE=true

# MySQL Database Configuration
MYSQL_HOST=localhost
MYSQL_PORT=2002
MYSQL_USER=
MYSQL_PASSWORD=
MYSQL_DATABASE=
MYSQL_POOL_SIZE=10

# Redis Configuration
REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_PASSWORD=
REDIS_DB=0

# Discord Webhook Configuration
DISCORD_RANKED_WEBHOOK_URL=

# osu! Private Server Configuration (PS)
PS_NAME=RealistikOsu
PS_DOMAIN=ussr.pl
PS_BOT_USERNAME=RealistikBot
PS_BOT_USER_ID=999
PS_MINIMUM_CLIENT_YEAR=2023
PS_ENABLE_PY_COMMAND=true
PS_PY_COMMAND_WHITELIST=1000,1180

# Data Directory Configuration
DATA_BEATMAP_DIRECTORY=/path/to/your/beatmap/directory
DATA_GEOLOCATION_PATH=/path/to/geoloc/db
DATA_BIBLE_PATH=/path/to/bible

# Misc Configuration.
SERVICE_READINESS_TIMEOUT=30
4 changes: 2 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
**/__pycache__
**/build
config.ini
filters.txt
.data
.idea
redistest.py
*.c
*.so
.pyenv
config.json
.vscode/
.env
Empty file removed .gitmodules
Empty file.
16 changes: 0 additions & 16 deletions .vscode/c_cpp_properties.json

This file was deleted.

24 changes: 24 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
FROM python:3.12

ENV PYTHONUNBUFFERED=1

WORKDIR /app

# NOTE: These are handled as image resources, and therefore moved into the image.
COPY ./resources/geolocation_database.mmdb /app/resources/geolocation_database.mmdb
COPY ./resources/bible.txt /app/resources/bible.txt

ENV DATA_GEOLOCATION_PATH=/app/resources/geolocation_database.mmdb
ENV DATA_BIBLE_PATH=/app/resources/bible.txt

# Requirements
COPY ./requirements/main.txt /app/requirements.txt
RUN python3.12 -m pip install -r /app/requirements.txt

# Scripts
COPY ./scripts /app/scripts

# Application.
COPY ./peppy /app/peppy

ENTRYPOINT [ "/app/scripts/bootstrap.sh" ]
8 changes: 2 additions & 6 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,7 +1,3 @@
all: build run

#!/usr/bin/make
build:
python3.9 setup.py build_ext --inplace

run:
python3.9 pep.py
docker build -t peppy:latest .
33 changes: 4 additions & 29 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,42 +14,17 @@ This portion of the RealistikOsu manages all of the real-time, packet related po
## Why is our fork better?
This fork of pep.py has been developed specifically to suit the need of RealistikOsu. With the rapid growth of the server, more and more demand has been placed on us in regards of features alongside performance. The original repo features a large quantity of fatal flaws alongside performance hogs, and through our usage of the software, we have solved a majority of those issues.

- Full Dockerisation
- Fixed multiplayer
- MASSIVE OPTIMISATIONS (your database will thank you)
- Major performance optimisations.
- Relax and Autopilot support
- Extended Redis API
- Extended 3rd party API support
- Customised HWID system
- Extended in-game bot commands
- Python 3.9 support!
- Python 3.9 support

## Requirements
To run pep.py, there is an list of requirements to ensure the server runs at all.
- Python >=3.6
- Python >=3.9
- RealistikOsu MySQL Database
- Cython + GCC
- Linux (preferably Ubuntu 18.04)

## Notes for potential users
If you are planning on using our fork of pep.py, there is a multitude of things to consider that are unique to our variant
- Low reliance on `userutils` for performance reasons.

The entire `userutils` module promotes inefficient use of the database. This is especially bad on established servers with large
databases, where the cost of each query becomes more and more expensive with every user. This appends unnecessary stress on the
database and as a consequence, the server as a whole.
- Tendency to hardcode things.

I, RealistikDash, have a bad habit of hardcoding things. While this is usually fine for the intended application of this, being used
on RealistikOsu, it may be a pain to scan through the code if you are attempting to run this on your server. In this scenario, I would
advise searching through `constants/rosuprivs.py` and `constants/serverPackets.py` for any references you would like to change.
- Private database

As expected, our variant uses our own database schema. A copy of our database schema (designed for use with [USSR](https://github.com/RealistikOsu/USSR)) can be found [here!](https://github.com/RealistikOsu/USSR/blob/master/extras/db.sql)

Due to the old nature of the origin code, the age of the modules is **quite large**. This means that we do not benefit from any improvements,
bugfixes or any other quality of life improvements of any new module updates. This is an issue with the whole Python based Ripple stack, and is
not an exclusive to RealistikOsu pep.py. **This issue is however planned on being addressed soon.**
- No IRC

Due to the lack of usage from the RealistikOsu community, the entire IRC server has essentially been nuked. This is because while not being used, it
still took up a thread and served as dead code in the repo. Not much else to say other than that it was pretty much never used.
1 change: 0 additions & 1 deletion build.sh

This file was deleted.

6 changes: 0 additions & 6 deletions common/.gitignore

This file was deleted.

79 changes: 0 additions & 79 deletions config.py

This file was deleted.

File renamed without changes.
File renamed without changes.
File renamed without changes.
5 changes: 2 additions & 3 deletions collection/tokens.py → peppy/collection/tokens.py
100755 → 100644
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,8 @@
from typing import Optional

import redis

import settings
from common.ripple import userUtils
from config import config
from constants import serverPackets
from constants.exceptions import periodicLoopException
from events import logoutEvent
Expand Down Expand Up @@ -169,7 +168,7 @@ def usersTimeoutCheckLoop(self) -> None:
# Check timeout (fokabot is ignored)
if (
value.pingTime < timeoutLimit
and value.userID != config.SRV_BOT_ID
and value.userID != settings.PS_BOT_USER_ID
and not value.irc
and not value.tournament
):
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
55 changes: 20 additions & 35 deletions common/db/dbConnector.py → peppy/common/db/dbConnector.py
100755 → 100644
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,8 @@
from typing import Optional

import MySQLdb
from MySQLdb.connections import Connection

from logger import log
from MySQLdb.connections import Connection


class Worker:
Expand Down Expand Up @@ -42,35 +41,21 @@ def __del__(self) -> None:


class ConnectionPool:
"""
A MySQL workers pool
"""

__slots__ = (
"config",
"maxSize",
"pool",
"consecutiveEmptyPool",
)

def __init__(
self,
host: str,
port: int,
username: str,
password: str,
database: str,
size: int = 128,
) -> None:
"""
Initialize a MySQL connections pool
self.host = host
self.port = port
self.username = username
self.password = password
self.database = database

:param host: MySQL host
:param username: MySQL username
:param password: MySQL password
:param database: MySQL database name
:param size: pool max size
"""
self.config = (host, username, password, database)
self.maxSize = size
self.pool = queue.Queue(self.maxSize)
self.consecutiveEmptyPool = 0
Expand All @@ -84,7 +69,14 @@ def newWorker(self, temporary: bool = False) -> Worker:
:return: instance of worker class
"""
db = MySQLdb.connect(
*self.config, autocommit=True, charset="utf8", use_unicode=True
host=self.host,
port=self.port,
user=self.username,
password=self.password,
database=self.database,
autocommit=True,
charset="utf8",
use_unicode=True,
)
conn = Worker(db, temporary)
return conn
Expand Down Expand Up @@ -118,7 +110,7 @@ def getWorker(self) -> Worker:
if self.pool.empty():
# The pool is empty. Spawn a new temporary worker
log.warning("MySQL connections pool is empty. Using temporary worker.")
worker = self.newWorker(True)
worker = self.newWorker(temporary=True)

# Increment saturation
self.consecutiveEmptyPool += 1
Expand Down Expand Up @@ -172,22 +164,15 @@ class DatabasePool:
def __init__(
self,
host: str,
port: int,
username: str,
password: str,
database: str,
initialSize: int,
) -> None:
"""
Initialize a new MySQL database helper with multiple workers.
This class is thread safe.

:param host: MySQL host
:param username: MySQL username
:param password: MySQL password
:param database: MySQL database name
:param initialSize: initial pool size
"""
self.pool = ConnectionPool(host, username, password, database, initialSize)
self.pool = ConnectionPool(
host, port, username, password, database, initialSize,
)

def execute(self, query: str, params: object = ()) -> int:
"""
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Loading
Loading