Skip to content

Commit

Permalink
Implement rate limiting (#54)
Browse files Browse the repository at this point in the history
  • Loading branch information
felipeall committed Jan 2, 2024
1 parent 80a8489 commit 64d3247
Show file tree
Hide file tree
Showing 8 changed files with 513 additions and 294 deletions.
2 changes: 2 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
RATE_LIMITING_ENABLE=false
RATE_LIMITING_FREQUENCY=2/3seconds
10 changes: 10 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ This project provides a lightweight and easy-to-use interface for extracting dat
by applying web scraping processes and offering a RESTful API service via FastAPI. With this service, developers can
seamlessly integrate Transfermarkt data into their applications, websites, or data analysis pipelines.

Please note that the deployed application is used only for testing purposes and has a rate limiting
feature enabled. If you'd like to customize it, consider hosting in your own cloud service.

### API Swagger
https://transfermarkt-api.vercel.app/

Expand Down Expand Up @@ -50,3 +53,10 @@ $ docker run -d -p 8000:8000 transfermarkt-api
# Access the API local page
$ open http://localhost:8000/
````

### Environment Variables

| Variable | Description | Default |
|---------------------------|--------------------------------------------|--------------|
| `RATE_LIMITING_ENABLE` | Enable rate limiting feature for API calls | `false` |
| `RATE_LIMITING_FREQUENCY` | Delay allowed between each API call | `2/3seconds` |
1 change: 0 additions & 1 deletion app/api/endpoints/players.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,6 @@ def get_player_stats(player_id: str):
return player_stats


# Define the endpoint for injuries
@router.get("/{player_id}/injuries")
def get_player_injuries(player_id: str, page_number: Optional[int] = 1):
tfmkt = TransfermarktPlayerInjuries(player_id=player_id, page_number=page_number)
Expand Down
13 changes: 13 additions & 0 deletions app/main.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,23 @@
import uvicorn
from fastapi import FastAPI
from slowapi import Limiter, _rate_limit_exceeded_handler
from slowapi.errors import RateLimitExceeded
from slowapi.middleware import SlowAPIMiddleware
from slowapi.util import get_remote_address
from starlette.responses import RedirectResponse

from app.api.api import api_router
from app.settings import settings

limiter = Limiter(
key_func=get_remote_address,
default_limits=[settings.RATE_LIMITING_FREQUENCY],
enabled=settings.RATE_LIMITING_ENABLE,
)
app = FastAPI(title="Transfermarkt API")
app.state.limiter = limiter
app.add_exception_handler(RateLimitExceeded, _rate_limit_exceeded_handler)
app.add_middleware(SlowAPIMiddleware)
app.include_router(api_router)


Expand Down
10 changes: 10 additions & 0 deletions app/settings.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
from pydantic_settings import BaseSettings, SettingsConfigDict


class Settings(BaseSettings):
model_config = SettingsConfigDict(env_file=".env")
RATE_LIMITING_ENABLE: bool = False
RATE_LIMITING_FREQUENCY: str = "2/3seconds"


settings = Settings()
757 changes: 466 additions & 291 deletions poetry.lock

Large diffs are not rendered by default.

2 changes: 2 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ beautifulsoup4 = "==4.12.2"
requests = "==2.31.0"
fastapi = "==0.104.0"
uvicorn = {extras = ["standard"], version = "==0.23.2"}
slowapi = "==0.1.8"
pydantic-settings = "==2.1.0"

[tool.poetry.group.dev.dependencies]
jupyter = "==1.0.0"
Expand Down
12 changes: 10 additions & 2 deletions requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -5,17 +5,23 @@ certifi==2023.11.17 ; python_version >= "3.9" and python_version < "4.0"
charset-normalizer==3.3.2 ; python_version >= "3.9" and python_version < "4.0"
click==8.1.7 ; python_version >= "3.9" and python_version < "4.0"
colorama==0.4.6 ; python_version >= "3.9" and python_version < "4.0" and (sys_platform == "win32" or platform_system == "Windows")
deprecated==1.2.14 ; python_version >= "3.9" and python_version < "4.0"
exceptiongroup==1.2.0 ; python_version >= "3.9" and python_version < "3.11"
fastapi==0.104.0 ; python_version >= "3.9" and python_version < "4.0"
h11==0.14.0 ; python_version >= "3.9" and python_version < "4.0"
httptools==0.6.1 ; python_version >= "3.9" and python_version < "4.0"
idna==3.6 ; python_version >= "3.9" and python_version < "4.0"
importlib-resources==6.1.1 ; python_version >= "3.9" and python_version < "4.0"
limits==3.7.0 ; python_version >= "3.9" and python_version < "4.0"
lxml==4.9.3 ; python_version >= "3.9" and python_version < "4.0"
pydantic-core==2.14.5 ; python_version >= "3.9" and python_version < "4.0"
pydantic==2.5.2 ; python_version >= "3.9" and python_version < "4.0"
packaging==23.2 ; python_version >= "3.9" and python_version < "4.0"
pydantic-core==2.14.6 ; python_version >= "3.9" and python_version < "4.0"
pydantic-settings==2.1.0 ; python_version >= "3.9" and python_version < "4.0"
pydantic==2.5.3 ; python_version >= "3.9" and python_version < "4.0"
python-dotenv==1.0.0 ; python_version >= "3.9" and python_version < "4.0"
pyyaml==6.0.1 ; python_version >= "3.9" and python_version < "4.0"
requests==2.31.0 ; python_version >= "3.9" and python_version < "4.0"
slowapi==0.1.8 ; python_version >= "3.9" and python_version < "4.0"
sniffio==1.3.0 ; python_version >= "3.9" and python_version < "4.0"
soupsieve==2.5 ; python_version >= "3.9" and python_version < "4.0"
starlette==0.27.0 ; python_version >= "3.9" and python_version < "4.0"
Expand All @@ -25,3 +31,5 @@ uvicorn[standard]==0.23.2 ; python_version >= "3.9" and python_version < "4.0"
uvloop==0.19.0 ; (sys_platform != "win32" and sys_platform != "cygwin") and platform_python_implementation != "PyPy" and python_version >= "3.9" and python_version < "4.0"
watchfiles==0.21.0 ; python_version >= "3.9" and python_version < "4.0"
websockets==12.0 ; python_version >= "3.9" and python_version < "4.0"
wrapt==1.16.0 ; python_version >= "3.9" and python_version < "4.0"
zipp==3.17.0 ; python_version >= "3.9" and python_version < "3.10"

0 comments on commit 64d3247

Please sign in to comment.