To start the ShortLinks Phoenix Server:
- Run
mix setupto install and setup dependencies - Start Phoenix endpoint with
mix phx.serveror inside IEx withiex -S mix phx.server
Now you can visit localhost:4000 from your browser.
Pretty much a standard Elixir application here.
- Run
mix testfrom the project root directory.
I have some benchmarking code setup using Benchee. I recommend running the benchmarks using MIX_ENV=benchmark both for performance reasons and to avoid spamming your dev database.
MIX_ENV=benchmark mix setupMIX_ENV=benchmark mix phx.server- Then in a separate terminal
mix run benchmark.exs- You can also run
mix run benchmark.exs -- 10to change the parallel count for Benchee. - The benchmark itself runs in dev right now. Just for less changes to dependencies and where they run.
- Eventually an alias should be made for this
mix benchmark - Likewise,
mix bechmark.server, etc would be nice but needs some environment setup to make the command run in the right environment.
- You can also run
I chose to use SQLite in this project mainly for simplicity of setup and hosting, and because it's my default for small Elixir projects these days.
For a small project like this most of SQLite's drawbacks could be addressed as needed:
- LiteFS for some amount of read replication and backups
- LiteStream for backups.
- Performance is quite adequate, but obviously could become a concern in a large project with lots of traffic. See benchmarks below.
- Columns comparisons are case insensitive by default, but that doesn't matter since the slugs should also be treated as case insensitive since they're transmitted via URLs.
- Tests can't run async. If we had lots of database tests this could become a concern but this project is bounded in features, so not a big deal.
I'm storing this directly on the links table and incrementing it synchronously every time a user visits a link. This would quickly become a bottleneck as the traffic to this application grows, although benchmarks suggest it could scale to hundreds or thousands of users pretty easily (depending on traffic patterns).
Keeping the visits on the links table makes the code way simpler, by letting me avoid joining when querying the data, and not having to track asynchronous code on the incrementer.
- SQLite already limits us to a single write at a time, so splitting the links out to a different table wouldn't benefit our write throughput.
- I tested spinning the increment function off into a Task, but that also didn't have a major benefit to throughput.
- This app doesn't support updating links so row level locking wouldn't really matter anyway (counting links is the only write we do on rows after creation). In a real application we'd probably anticipate more CRUD features so row level locking would probably be desireable even though we don't use it right now.
- If we were using Postgres or another RDBMS splitting to a separate table could benefit since we could then update the main link record indepdently of the view tracking.
If we expected to hit Google/Nextflix scale we'd need to reconsider this decision. Postgres would take us much farther and give us scaling options, but we might also consider using something like Kafka or other event based systems at that point.
I partially implemented this as a streaming download. it would be good to go back and update it to stream links from the database, which would take a bit of pagination work.
I didn't build this into the context, controllers, or UI for the stats table. It would be easy enough, but if you add enoucn links performance will be bad.
Right now I'm generating these in the controller via a function on the context. It doesn't feel great, but the flexibility is nice for testing the unique constraints and other slug-related tests.
Doing a Map.put(params, "slug", generate_slug()) in the controller so it should still keep users
from submitting their own. I'm using Nanoid to generate the slugs, no particular reason other than
it's fairly easy to use.
The UI is pretty basic, mainly just the default Phoenix UI with some modifications where needed. There's plenty of room for more polish, especially around navigation. Also it's only as responsive as the defaults generated by Phoenix which isn't great.
My benchmarks are far from scientific. Thus far I've been running the bunchmarks on the same machine as the server. I also have added overhead parsing out the CSRF token with Floki which probably hurts speeds slightly on the create test.
Based on my benchmarking, this solution meets the required performance characterstics up to 500 parallel connections on my machine (M1 Macbook Pro, 16gb of RAM).
I suspect that the higher parallel count benchmarks are stealing more CPU time from the server resulting in the slower speeds. I'd need to benchmark across two machines to verify, which also means I'd need to update the benchmark config to listen on a locally addressable interface (i.e. not just 127.0.0.1)
| Name | Description |
|---|---|
| execute contentious | Tests the redirect on a smaller number of links (10). Trying to test performance when visits (and the corresponding writes) are focused on a few links. |
| execute short link | Picks a random slug out of a list of 100 that are preinserted, and hits the redirect endpoint for it (/:slug) |
| mixed use | Mixes redirect execution and create. Trying to see if the visit count slows creation or vice versa |
| create link | Tests the speed of creating links. Does a GET to the new/form page first to get a CSRF token, then a POST to create a link. |
Name ips average deviation median 99th %
execute contentious (/:slug) 5.40 K 185.05 μs ±20.15% 177.42 μs 344.81 μs
execute short link (/:slug) 5.18 K 193.15 μs ±21.27% 184.13 μs 358.06 μs
mixed use 1.77 K 563.89 μs ±109.01% 265.19 μs 2523.68 μs
create link (post /) 0.30 K 3342.23 μs ±40.04% 3081.17 μs 7454.18 μs
Name ips average deviation median 99th %
execute contentious (/:slug) 161.51 6.19 ms ±25.14% 5.96 ms 13.66 ms
execute short link (/:slug) 158.16 6.32 ms ±19.94% 6.12 ms 12.47 ms
mixed use 97.48 10.26 ms ±60.89% 7.49 ms 30.47 ms
create link (post /) 33.87 29.53 ms ±24.51% 27.17 ms 58.26 ms
Name ips average deviation median 99th %
execute contentious (/:slug) 34.14 29.29 ms ±14.15% 28.18 ms 47.52 ms
execute short link (/:slug) 30.12 33.20 ms ±25.73% 30.86 ms 75.27 ms
mixed use 16.16 61.89 ms ±58.98% 45.59 ms 165.74 ms
create link (post /) 6.83 146.33 ms ±9.92% 141.32 ms 195.00 ms
Taxon is a similar app that I've written implementing Go Links, that's been running for almost a year now. The test coverage is worse though 😁 the focus there was getting the features I needed done as quickly as possible.
Go links are basically named short links you use to access common links (go/board -> JIRA board, go/prs -> Team PRs, etc)