Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
add alternative light weight ruby backend without dependencies #10
Hey there @bilde2910 thanks for building Hauk! Really like the idea of having a self hosted real time location share service! However since I'm only using it rarely and don't have PHP & Memcached on my server I felt like quickly hacking together a drop in replacement backend that consists of a single Ruby file and has almost no dependencies (except http://sinatrarb.com/). It stores sessions in memory and saves/loads them to/from a json file when the server gets restarted.
It runs standalone with or without SSL or can run behind an existing webserver such as nginx.
So was wondering if you would like to merge this somehow in.
Hi @pachacamac, thanks a lot for your contribution! I've been meaning to figure out a way to avoid using PHP as a backend, or at least have an alternative to it that scales better. PHP is easy to work with and has wide compatibility, but it's not without issues, so your Ruby implementation is a welcome one.
I took a brief look of the code and it looks good, but I have a few suggestions/recommendations.
The original PHP code generates a random session ID and has a separate function to convert that ID to a link ID which is used in the public share URL. I see you chose to generate nicer, readable IDs for the link ID, which is fine, but would it be possible to make it so that instead of
Additionally, some minor things:
Finally, I'd like to ask what your thoughts are on dropping the sessions.json file and keeping all sessions entirely in memory. The reason I use memcached in the PHP backend is that not only are shares meant to be temporary, but updates to the backend could change the structure of data that is stored there; if the data is transient then there is less chance that a version conflict could cause issues with the session data, and makes it easier to completely flush all data. If you think keeping sessions on disk when not running is better, could there be an option to disable that behavior, e.g. if the file name in the config is an empty string, data is only kept in memory and not read from/written to file?
I appreciate your contribution and look forward to hearing your thoughts.
Hey @bilde2910 thanks for the nice feedback! Let me go through your points one by one:
I'm not so sure this version scales better. Would be interesting to do some load testing and compare. Usually Webrick (built in ruby webserver) isn't the best choice for scaling but I wanted to keep dependencies at a minimum so that it's an alternative that requires minimum setup and can just be dropped in quickly for when setting up php/memcached seems like hassle.
Makes sense. Thought at first this would be prevented by the password but I get what you mean. Fixed that by creating a session id out of a friendly id by hashing it together with a secret salt. Lines 110 and 124.
Sure can and now is.
Did that, also updated dockerfile.
I added a new config that makes it possible to select if sessions should be stored or not. Furthermore there is a before hook that takes care of session expiration before every request to emulate the memcached session expiration feature.
PS: Right now I copied your frontend files to serve them from the ruby backend. Might be better if we move the frontend into its own folder outside of the backend so that the same frontend code is used by both backends?
Thanks for the quick patches!
Consider an example where there is one active share, with one other person watching that share in real-time. The sharing interval is set to 1 second, meaning we get 2 HTTP requests per second; one from each user.
Edit, Sept 10: I did load testing that would indicate that I am mistaken regarding the above paragraph. Results here.
Your Ruby implementation starts and runs its own web server, and when a request is received from a client, it is handled entirely in memory. There is no reads/writes to/from disk involved. That should mean that your implementation is mostly CPU-bound, the cap for which I'd assume is generally much higher than I/O load.
PHP is admittedly a poor choice for the backend for this project. The ideal solution would be to have some kind of solid backend (a native one in Rust, for example), with websocket support. With websockets, each client watching the map would only result in a single connection being maintained instead of a new request being fired every second. This also cuts down on data usage quite dramatically. I want to make such a backend, but the only language I'm skilled enough in to pull that off in is Java (which honestly doesn't seem like a very good alternative). I've never worked with Rust before, but it would be nice if I can find some examples to base it on and then try my best at implementing a solid backend. That would probably make your Ruby backend obsolete, though.
Definitely agree on this one. The frontend code could be in its own frontend folder in the repository root. Would need some small changes to the Dockerfile, but should be quite straight-forward.
The rest of your changes look good. Aside from testing your backend, there is one remaining problem that I would like to resolve if this is merged. There is currently another issue open, whose resolution will result in some significant changes to the backend. When #7 is implemented, the backend must be capable of handling multi-user shares. The protocol and associated backend code for that change is not finalized yet, but when it's merged, it would need to be covered in the Ruby implementation as well. I'm new to Ruby, so I might need your help to implement that part once it's done.
But regarding the "old" backend I'm also happy to help you make the needed changes to get #7 working in the Ruby code once the spec is fix.
As for the Android app, I need to do more consideration first. It's possible that I can return a WS URL as part of the create.php handshake, but I need to do more research into how WS works in Android, how to implement it, whether it can be done without a lot of libraries, and how to e.g. gracefully handle disconnects.
I've completed the backend on #7, renamed the
As for the changes to this patch - the structure is a bit reworked. I suppose the most important part is the changes to the API. All responses must return a header
The interal data structure has also changed significantly. I can provide an in-depth explanation here if you wish.
I did some load testing on the PHP backend. Running the
To verify this, I ran the same flood against a server running both its OS and Hauk from traditional, spinning rust HDDs, using the Hauk Docker image. There was an initial spike of disk read activity, but this settled down to 0 B/s within a few seconds. So there is definitely caching going on, which is good.
Seems I was mistaken about PHP not caching and reading a lot of files per second. This thread is getting some attention from elsewhere on the web, so I'll be updating my previous comment with the result of this test.