Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[www] Make www scalable #545

Closed
5 tasks done
marcopeereboom opened this issue Oct 18, 2018 · 3 comments
Closed
5 tasks done

[www] Make www scalable #545

marcopeereboom opened this issue Oct 18, 2018 · 3 comments
Labels
enhancement The issue enhances an existing feature.

Comments

@marcopeereboom
Copy link
Member

marcopeereboom commented Oct 18, 2018

Politeia www layer currently uses a leveldb database for user data. In order to make www scale that database needs to become distributed. In addition we need to move some values that we currently store on disk also in the new database layer (session keys, cookies, and some other secret material). These values are sensitive so they must be stored encrypted.

  • Pick scalable database (RoachDB, Reddis, Mongo, ?)
  • Add a new database to www user interface.
  • Create migration tools to move users to the new database.
  • Determine how to move secret bits into the database (or if we want some sort of agent that provides secrets in a different way)
  • Create migration tools to move secrets to the new database/agent.
@marcopeereboom marcopeereboom added enhancement The issue enhances an existing feature. Large change labels Oct 18, 2018
@lukebp
Copy link
Member

lukebp commented Dec 5, 2018

I think any of those database options would be fine for the user database, with marginal benefits over one another. It may be best to use the same database that we choose for #546 in order to keep complexity to a minimum.

Side note: the contractor management system has swapped out leveldb for CockroachDB.

@marcopeereboom
Copy link
Member Author

We agree on that. Whatever #546 determines will become the same target. I think at this point that we have determined cockraochdb is the way to go and it is fortunately compatible with postgres.

So what we will need is to migrate all on disk bits that currently are secret into the database but encrypted.

The gorilla session package can have multiple backends. We should probably fork https://github.com/antonlindstrom/pgstore and add the encryption bits to that. Similarly we need to modify the api to encrypt stuff on the fly that is secret.

When everything is said and done www should ONLY have a configuration file and logs stored on disk. We also should debate offloading logs to some sort of repo but that is a different issue/pr.

@marcopeereboom
Copy link
Member Author

This has been achieved with various commits.

vibros68 pushed a commit to vibros68/politeia that referenced this issue Aug 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement The issue enhances an existing feature.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants