Are you looking to try this out?
It's almost there, but still not quite yet ready.
Are you a developer wanting to run this yourself?
You should start by setting up the Vagrant virtual machine. It's not required, but it will make things go a lot smoother.
This is a LAMP web application, written for PHP 5.6, MySQL 5.6 and Apache 2.4. It inherits its back-end approach from Flamework (see: philosophy and style guide) and uses Bootstrap for the front-end implementation.
is will be a federated service
The software is being designed with a federated hosting model in mind. Something like: you plug a Raspberry Pi into your home internet router, it connects to the Smol Data VPN, and joins a kind of mini-cloud to host your data on your own hardware. The idea being that at any time you could unplug the thumb drive from the Raspberry Pi and back it up somewhere of your choosing.
The software is just not there yet. It's aspriational, and self-hosting won't be required of all users. But it's an important design constraint going forward.
How to install
mkdir /usr/local/smoldata cd /usr/local/smoldata git clone firstname.lastname@example.org:smoldata/thingmonger.git cd thingmonger make setup
See also: Raspberry Pi install instructions
Things you may be asked during setup
- Choose a root MySQL password (choose something you will remember later)
Load it up in a browser
If everything goes as planned, you should be able to load up http://localhost:4700/ in your browser.
Right now there is support for archiving Twitter and mlkshk accounts (the latter got a priority boost because it is shutting down soon). Instagram is next in line, then maybe Wikipedia? Jeez, maybe YouTube should come sooner than later if they can't get their shit together.
Each service you add will need API keys and secrets. Here are some links for you to set up your own applications:
Then add the following to your
# Twitter API $GLOBALS['cfg']['twitter_api_consumer_key'] = '...'; $GLOBALS['cfg']['twitter_api_consumer_secret'] = '...'; # mlkshk API $GLOBALS['cfg']['mlkshk_api_key'] = '...'; $GLOBALS['cfg']['mlkshk_api_secret'] = '...';
I OAuth'd my account but ... nothing is happening?
The script you want to run is
bin/download.php. Here is how you can test it out to make sure things are working as expected.
cd /usr/local/smoldata/thingmonger sudo -u www-data php bin/download.php --verbose
Here is an entry for your
/etc/crontab to run
bin/download.php every 1 minute.
* * * * * www-data /usr/bin/php /usr/local/smoldata/thingmonger/bin/download.php
And don't forget the thing about crontabs needing a line break at the end of the file.