A toy URL datastore.
This is a toy project which can be used to store any amount of data in the URL of a URL-shortening service. It exploits the fact that most URL shortening services do not check that a URL points to a real website, and as such, any data can be URL-shortened.
How it works:
Takes a file pathname as input.
Compresses the file with the DEFLATE compression scheme.
Encodes the compressed file using the Base64 encoding scheme.
Splits the data into chunks of typically a few kilobytes long, depending on the URL shortening service used.
Creates a short URL for each chunk.
Concatenates the path components of all short URLs into 1 long string, separated by a forward slash
Again, creates a short URL using the concatenated path components as input.
The result is a single short URL that has an arbitrary amount of data stored in the URL.
To store a file in a URL:
To write a file to disk given a URL:
(shorty:unshorten-file "http://short-url" :file "/path/to/out-file")
There are 2 main issues that will likely never be fixed, as this is just meant to be a proof of concept toy.
Local limit: Shortening and unshortening are done completely in memory, so you need enough heap space.
Remote limit: During the concatenation of URL path components to create the single short URL at the end of the shortening process, the resulting string must also fit into a chunk size supported by the specified URL-shortening service. The path component length depends on the service. Do not expect to store more than a few megabytes of data at best.
Copyright © 2017-2018 Michael Fiano.
Licensed under the MIT License.