Our plan is to build a url shortener service that shortens lengthy urls and simplifies resource sharing. It’s a potential marketing tool to help brands and individuals engage with their customers and audience. Although there are many implementations of url shortener’s our’s is customized to the african market needs.
We chose 5 metrics to gauge our design outcome, as follows:
- Usability - 80% of users should be able to copy and paste lengthy urls and get a shorter link.
- Customer Experience - Make lengthy urls shareable and easy to read. People dislike reading lengthy material.
- Visual Appeal - Create an aesthetic and authentic look that evokes emotion. Colors, Fonts and Overall visual balance of the design should stay consistent across the entire application
- Engagement - Track link success through the analysis of the user click rate. Shorter urls tend to increase click through rates across social media channels.
- Cost - Make the service affordable to any potential user of our application through free subscription and effective price plans.
Url - Uniform Resource Locator. Describes a path to a file stored somewhere on the server.
The intended users for our url shortening service are :
- Content Creators.
- Marketers.
- Authors and Bloggers
- Brands and Businesses.
- Internet users.
We took upon the challenge of building a web application that enables users to shorten lengthy urls. Existing services i.e Bitly, Tiny Url and Goo.gl are effective but are doubtful of whether they’ve put African business needs into perspective. Our solution aims to tackle this and solely focus on the African market.
The application design will include an entire process of pasting urls, custom branding urls, analysing urls, enabling fast redirects and generation of QR codes. In addition to making our service scalable, all our design goals should be met.
- Individual users are people in need of url shortening services. Registered users have a user account and a dashboard. Non registered users can still shorten their urls without logging in the system via the landing page.
- Businesses represent a single established entity using the service for brand related and engagement purposes. Businesses subscribed to the service can customise their links to fit their brands theme and purpose. I.e https://safaricom/mpesa
The main product functions are as follows:
-
Authentication Service - enables user authentication and authorisation. Each user is assigned a token while their data is persisted in the database.
-
API Gateway - routes traffic according to the given path.
-
Shortening Service - responsible for shortening the lengthy urls, redirecting to the original URL and keeping the count of urls shortened by the user. Generates a shortened url by the help of the hash service which maps a hash to its original url.
-
Hash Service - manages a collection of pre-created and used hashes.
-
Scheduler Service - generates new hashes when the number of existing hashes in the database is low. In charge of recycling new hashes on the expiration date.
- Microservice Architecture - The backend is split into different services which function independently thus enabling modularity. It also allows for the separation of logic and scaling of each service according to traffic needs.
- Hashes - Generate a unique hash tokens for each url to enable user customization.
- Token generation - Use of the Hashid library to generate unique tokens. Hashids create short, unique ids from numbers 0-9 and custom alphabets.
E.g
| HashId input | Token Generated | Test Text |
|---|---|---|
| 1 | l6o3k1n | https://twitter.com/l6o3k1n |
|
- Encoding the short URL Algorithms - Base 36 or Base 62 encoding. In base 62 encoding, we use a-z,A-Z and 0-9 in the characters. Storage Estimates - 400 million new urls per month * 12 * 4 years = 19.2 billion urls.
- Length of Token and the character it contains - Small characters a-z and numbers in the range of 0 and 9 are considered safe, which results in a character set of 36 in total.
The table below describes the number of tokens that can be generated using tokens of various lengths.
A typical short url takes this format: https://ts.bv/qp37hi. Ts.bv -represents an application, service or company. The last part is the token, a unique random identifier.
| Token Length | Unique Token range combo | Range |
|---|---|---|
| 6 | 2,176,782,336 | >2 billion |
| 7 | 78,364, 164,096 | > 78 billion |
| 8 | 2,821,109,907,456 | > 2 trillion |
| 9 | 101,559,956,668,416 | 100 trillion |
- Angular - Front end development of the application.
- Spring Boot - Development of the micro-services.
- Spring cloud gateway - For effective api routing, security and monitoring.
- Spring cloud openfeign - Enables cross-service communication.
- Spring Security - Secure the api service.
- Spring cloud kubernetes - Deploy the backend application on the cloud.
-
Service should be able to create unique shortened url/links against a long url a. One shortened url should be different from another
-
Click the short url, Redirect user to the longer url
-
Generation of short urls (in terms of length size) as possible
-
Branded link export: Creation of custom url with a maximum of 16 characters:
- Numbers 0-9 or alphabetical letter to a-z
-
Analytics:
- collect metrics like the number of clicks on the short url
- Most clicked short url
-
Original and Shortened url stays in the system for a period of 4 years
-
Shortened url expires after a period of 2 years of inactivity
-
Updation and deletion of a shortened url
-
Hash url stored in the form of key value pairs
-
Link search for the shortened links
-
Embed a shortened link in a QR code
- Service should be up and running all the time.
- Response time: Fast URl redirection and should not degrade at any point.
- Random short URls
- Unique short urls
- Availability of shortened url: 4 years since its creation
- New nodes added in the cluster talk to the zookeeper service.
- Each application node has to requests for a unique number range from the zookeeper.
- The load balancer receives the request to shorten the long url. Workload is placed on the shortening service which works on a first come first served basis, turn by turn. (round-robin scheduling)
- An entry for an existing short url that points to the long url is checked in the database. A response status code HTTP 200 OK is given for an existing entry that is not expired.
- For a Url not available in the database, the shortening service uses the next number range allocated by the zookeeper service and a new token is generated with the hashid library. The generated short url with a set expiry date is saved to the database. The short URL is return to the client with the status code of HTTP 201 CREATED.
- When the user eventually clicks the short url, the load balancer receives the request and passes it to one of the URL shortening application nodes.
- The shortening service communicates to the database to find an entry for the long url and if it exists. A redirection to the long url occurs with a status code of HTTP 302
- If no token for the long url is non existent, the application responds with a status of HTTP 404 NOT FOUND.
- ZZookeeper is a configuration management distributed system that manages different servers enrolled to it.
Zookeeper Use Cases
- A user may upload a long url which goes through a load balancer to determine which server to assign the payload to.
- Let’s say server 2 is free to accept a payload. It get’s a request to increment it’s counter, which ranges between 2 million and 3 million and assigns a value to the request. The request goes through the base 62 algorithm and the result is a 7 character key.
- Zookeper ensures that each server has a particular counter range, with no duplicate keys or database collisions.
- If Server 3 goes down, it’s entry is removed from the keeper service and set to unassigned. Server 4 becomes assigned to the system with a new range.This way, we are able to scale in case of down time.
- Service accepts a long URL as a HTTP post request.
- Service checks if the long url already exists in the database. If the long url already exists, a short url is constructed using the token and sent in the response.
- If the long url doesn’t exist a new token for the long url and saves the token and long url entry into the database.
- Service responds with status of 201 CREATED, else 200 OK, if the token already exists.
- Service receives a short URL GET request like https://meliora/as7du98
- Service checks if the token exists in the database. If it doesn’t exist, then a HTTP 404 not found is given as a response.
- For an existing token the corresponding entry is given. A response status of HTTP 498 is returned for an expired token
- Service responds with a long URL in the LOCATION HEADER, with a HTTP 302 redirect status.