Lightweight url shortner with key/value database
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
storage
.gitignore
LICENSE
Makefile
README.md
gurl.go

README.md

gurl

Hight perfomance and compact url shortening server.

Features

+---------------------+
|                     |
|         Gurl        |
|                     |
|  +---------------+  |    +-----------------+
|  |               |  |    |                 |
|  |      api      <-------+   You backend   |
|  |               |  |    |                 |
|  +---------------+  |    +--------^--------+
|                     |             |
|                     |             |
|  +---------------+  |    +--------+--------+      +-----------------+
|  |               |  |    |                 |      |                 |
|  |    redirect   <-------+  reverse proxy  <------+       User      |
|  |               |  |    |                 |      |                 |
|  +---------------+  |    +-----------------+      +-----------------+
|                     |
+---------------------+

Why gurl?

You can find lots of solution at github but:

  1. It requires remote storage (like mysql or MongoDB) or operate with low-performance storage system (like sqlite, or own file storate).

  2. It depdends on Mysql or PostgreSQL features (like AUTO_INCREMENT etc.)

  3. It Mix api server and redirect server (same server handle both requests)

  4. It marshal / unmarshal JSON

  5. It contain complex logic, validation, huge data structures

Gurl devoid these disadvantages.

Internals

Gurl start two HTTP servers:

  • api server - manage links (crete, find, update etc.)
  • redirect server - just find link, and send 301 (or 404)
$ gurl --help
Usage of gurl:
  -api-address string
    	Control server bind address (default ":7070")
  -database string
    	Database file path (default "links.db")
  -redirect-address string
    	Redirect server bind address (default ":8090")

Storage schema

Each short link represents a [12]byte array:

  1. 4 byte time preffix, with format "0601" (see time).
  2. 8 byte hash, created by xxhash and encoded in base36

Api server interface

POST link/add

Create one (or more) short urls and returns CSV with hashes. Arguments:

  • link - string (requred) - links for shortening, each in new line
curl -X POST -d $'link=http://1.x\nhttp://2.x' http://localhost:7070/link/add
http://1.x,17107z1g4rbg
http://2.x,17101wa6cbnd

GET link/byHash

Find full link by hash. Arguments:

  • hash - string (requred)
curl -X GET http://localhost:7070/link/byHash?hash=17101wa6cbnd
http://2.x

GET link/list

Find list of hashes (part of sorted list) by arguments:

  • start - string (optional)
  • end - string (optional)
curl -X GET 'http://localhost:7070/link/list?start=17101wa6cbnd&end=17102wa6cbnd'
17101wa6cbnd,http://2.x
171020y8fgzn,http://x2.x2
17102ewiejoz,http://xx.xx
17102msb5big,http://v.v

# OR only start
curl -X GET 'http://localhost:7070/link/list?start=17101wa6cbnd'

POST hash/remove

Remove hashes. Arguments:

  • hash - string (required) - one ore more hashes, each in new line
curl -X POST -d 'hash=17101wa6cbnd' 'http://localhost:7070/hash/remove'
17101wa6cbnd

POST hash/cleanup

Remove hashes by range (probably day range). Arguments:

  • start (optional)
  • end (optional)
curl -X POST -d 'start=171020y8fgzn&end=17102msb5big' 'http://localhost:7070/hash/cleanup'
total,3

GET backup

Build consistent storage backup.

curl -X GET 'http://localhost:7070/backup'
...cut

Test redirect server

# Create short url for http://example.com  
curl -X POST -d 'link=http://example.com' 'http://localhost:7070/link/add'
http://example.com,17105tai0tvo

# Found hash "17105tai0tvo", send request to redirect server
curl -v -X GET 'http://localhost:8090/17105tai0tvo'
* Hostname was NOT found in DNS cache
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8090 (#0)
> GET /17105tai0tvo HTTP/1.1
> User-Agent: curl/7.35.0
> Host: localhost:8090
> Accept: */*
>
< HTTP/1.1 302 Found
< Location: http://example.com
< Date: Mon, 16 Oct 2017 12:05:01 GMT
< Content-Length: 41
< Content-Type: text/html; charset=utf-8
<
<a href="http://example.com">Found</a>.

* Connection #0 to host localhost left intact

Pitfalls

  • Gurl is not scalable.

Related projects

  • xxhash - high-quality hashing algorithm that is much faster than anything in the Go standard library
  • bolt - pure Go key/value store
  • gurl-cli - command line util for system admin