Set-Up Documentation

[Mikowmer]

While I have some coding experience, I am a complete noob when it comes to setting up SQL databases and the like. I have attempted to use PostGreSQL to set up a database for hermes, but have struggled to get it to run. In particular, I keep running into issues with permissions.

Would it be possible to set up some documentation on how to install and set up hermes with an SQL database?

[iridescent-beacon]

What operating system/distribution are you attempting to do this on? At which point do you receive permission errors, and what do those permission errors say?

It's pretty hard to give general advice here, but the Arch Wiki page for PostgreSQL and this quickstart guide on DigitalOcean community are good places to start.

It really boils down to:

1. install postgres
2. if required, do initial configuration
3. if required, start postgres
4. create a role
5. create a db owned by that role (currently the codebase assumes it's named hermes but it's configurable)

Then do the hermes specific db initialization:

1. tell hermes to use that db and role (export OIL_DB_DBNAME and OIL_DB_USER environment variables)
2. generate store_bases.py by running schema.py
3. follow the instructions in doc/db_setup -- the easiest which is to run schema.py --init

The documentation in doc/ can definitely be improved. Let's use this issue to hash out what all needs added to make it more beginner friendly.

If you're familiar with docker at all, the FanFicDev/alexandria_api_infra-dev project is an option to get the api portion of this project running locally so that you can test changes, but isn't the most seamless way to run hermes itself if you're trying to use the ncurses interface.

[n33tfr33k]

How about some guidance on what is needed in the priv module if using the ncurses interface.

[iridescent-beacon]

I just pushed some initial documentation improvements, including the new doc/setup file that outlines the steps required and expanded doc/db_setup.

priv.py is imported in more than just the cui, it's used to get default cookies when doing most crawling. How to create it is described in doc/setup.

Please let me know if there still seem to be any glaring omissions and I'll give it another pass myself. This ticket might evolve into a general documentation ticket, since the READMEs after you do initial setup aren't very great either :)

[n33tfr33k]

After several hours of spinning my wheels tracing errors, I would also like to suggest that a call out be made not to test the initial hermes setup with ffNet. AO3 worked right out of the box, without a skitter or alexandria docker running. Only found this out as a Naruto fic on AO3 would show up randomly in the database.

[iridescent-beacon]

Good idea! Added, thanks. Probably every time I mention something like this in chat I should make sure it's written down somewhere as well; I'll have to read back through to see if there's anything else.

There probably needs to be clarification about what the distinction between hermes and alexandria are, since running alexandria would not have helped you in this situation.

More broadly, changes need to be made so all the adapters use a common interface and skitter itself should have some local mode so this is eventually moot. I'll have to extend the skitter docs as well in case someone actually tries to run that locally for ffn etc.