Obere - URL Shortening Service

Obere is url shortening service that takes any given url and returns a shorter version, using randomly generated shortcodes (6-8 chars) or a custom alias/name. The name Obere is an Igbo word that means small. This characterizes the most important feature of our service, turning long, confusing urls into very simple urls that can even be memorized quickly.

Features

• URL shortening
• Caching with redis
• Non-guessable shortened links
• Custom alias

Tech Stack

• Python (Fastapi)
• Redis
• DynamoDB
• PostgreSQL

Architecture Overview

• Client queries API with a post requests to shorten a url
• api fetches a unique shortened key from key-dispenser service
• maps the shortened key to the original url
• When another client queries the shortened key (eg. 889P2nXx), api redirects client to the original url

Services

Main API

Maps original urls from client to short keys

Key Generator & Dispenser

Generates unique 8-character short codes and dispenses via an API, atomically. This ensures that a given short code is not associated with multiple original urls

Redis

Caches any shortcode accesed to avoid high traffic to DynamoDB for mere url association lookup

DynamoDB

A partioned database storage for storing url mappings, associating short codes with original urls. Chosen for scalability and flexible columns (wide-column).

PostgreSQL

Stores the unique short codes generated by keygen_service.

Setup Instructions

Requirements

This setup requires running more than one service concurrently. So, consider opening more terminals for each command that needs to run live. Better still, use a terminal multiplexer such as Tmux.

DynamoDB

For Local development, set it up as instructed by AWS (Download DynamoDB local).

After downloading and unizipping the Dynamodb, start your server using a specified port (eg. 8001), in a new terminal or in the background.

java -Djava.library.path=./DynamoDBLocal_lib -jar DynamoDBLocal.jar -sharedDb -port 8001

Redis

Install and setup a Redis server or use Redis cloud. Configuration for Redis as in .env.api.example.

PostgreSQL

Install and startup a postgresql server. Create a user with password for your database named keygen_db. Construct the Database url in the format postgresql://{keygen_user}:{password}@{host}/keygen_db

Python

Install on your local machine.

Note

The setup instructions here are made for ubuntu 20.04 LTS. Ensure you translate to corresponding commands in your local machine OS.

Initial setup

1. Clone this repository and change into its directory.

git clone https://github.com/Ebuube/obere

cd obere    # Change into project directory

Keygen Service

1. In a new terminal, copy .env.keygen.example to .env.keygen. After, update it with the postgres database url you created above in PostgreSQL.

cp .env.keygen.example .env.keygen

2. Change into the service directory

cd services/

3. Create a virtual environment and activate it. Then install the requirements in keygen_service/requirements.txt using pip.

pip install -r keygen_service/requirements.txt

4. Start up the keygen service thus. Put a --reload flag if you are in development mode.

uvicorn keygen_service.api:app --host 0.0.0.0 --port 5000

5. You can access the Swagger UI docs automatically created by FastApi /docs.

URL Shortener Service

1. In a new terminal, copy .env.api.example to .env.api. After, update it to reflect you setup. Ensure keyclaim and keygen health endpoints are using the correct ports like in your setup above.

2. Change into the service directory

cd services/

3. Create another virtual environment and activate it. Then install the requirements in api/v1/requirements.txt using pip.

pip install -r api/v1/requirements.txt

4. Use this command to start up the api service. Put a --reload flag if you are in development mode.

uvicorn api.v1.app:app --host 0.0.0.0 --port 8000

5. You can access the Swagger UI docs automatically created by FastApi /docs.

Development Notes

• Ensure DynamoDB local is running before starting the API
• Ensure Redis is accessible
• Ensure PostgreSQL server is online
• Keygen service must be running before API service

Future Improvements

• Docker-based setup for easier environment provisioning
• Centralized configuration management
• Rate limiting and abuse protection
• Authentication layer for API service