create a new user after creating a fresh db

[Ahmad-Wahid]

Changes proposed

• Whenever, a user wants to create a fresh database and a user, then run the following command make clean-db db_name=flexmeasures-db db_user=flexmeasures.
• Here, db_name is required, while db_user is optional.
• If db_user is provided then after creating this user will be connected to the db for granting basic privileges.

[nhoening]

This looks good!

However, it got me thinking that a) I believe I was partly wrong and an existing user will survive a db drop and b) we have never (also not in documentation/host/data.rst ) told developers who should own the database.

If it's okay for the database to be owned by the postgres user and that every user on the server can connect to the databse, then leaving away the user is fine.
See https://stackoverflow.com/questions/40745559/user-can-always-connect-to-the-database-postgres

I am sorry I didn't know this exactly earlier.

I would:

• make the user parameter optional, it's still a neat way to add a user for connecting, but not required
• add a comment in data.rst to the effect described above (db owned by postgres user, all users can access, which might not be what you'd do in production)

Another thought :
If we add the flexmeasures db upgrade command, the new database is actually ready to be used.
We could even tell devs in the documentation to just use this make target to create a database?

[Ahmad-Wahid]

I have already used flexmeasures db upgrade after creating the database and adding extensions.
I will make the user an optional param.

Is it a good idea, if we provide a user then only this user will have access to the new DB? see this https://medium.com/@mohammedhammoud/postgresql-create-user-create-database-grant-privileges-access-aabb2507c0aa

[nhoening]

I have already used flexmeasures db upgrade after creating the database and adding extensions. I will make the user an optional param.

Yes, please.

Is it a good idea, if we provide a user then only this user will have access to the new DB? see this https://medium.com/@mohammedhammoud/postgresql-create-user-create-database-grant-privileges-access-aabb2507c0aa

That looks good. What is the difference to making that user the owner of the new database in the createdb call?

[Ahmad-Wahid]

well, we create databases by using the default postgres user and don't give the access to the user we have created. Let me know, If I have understood it wrong.

[nhoening]

I haven't actually said this anywhere in this PR, true. My idea is to make the user the owner of the database like described here:

Normally, the database user who executes this command becomes the owner of the new database. However, a different owner can be specified via the -O option, if the executing user has appropriate privileges.

But just adding privileges for this new user is fine as well. Maybe we can not do all, but keep it to the ones needed for using it (e.g. read, update, enter, delete ...), but not the privilege to drop the database.

[Ahmad-Wahid]

After creating DB and the user, I would take these steps.

1. First connect user to the DB.
   GRANT CONNECT ON DATABASE database TO user
2. Then grant the needed privileges as,
   GRANT USAGE, SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA public TO user
3. Lastly, the user would not be allowed to drop the DB.
   REVOKE DROP ON DATABASE database FROM user

Let me know, if something is missing. @nhoening

[nhoening]

Makes sense.
Not sure why you are revoking DROP, as it has not been granted. Just to make sure?

[Ahmad-Wahid]

I thought, user can DROP the db by default. I will remove this step.

[Ahmad-Wahid]

@nhoening Please review these new changes.

[nhoening]

Thanks, after I tested a bit I have some more ideas to make this work well.

[Ahmad-Wahid]

Right now, the extensions are created by the default postgres user. Should we create them by the newly created user.

[nhoening]

Great step forward.
I tried it out and ran into a small issue that should be easy to fix.

[nhoening]

Right now, the extensions are created by the default postgres user. Should we create them by the newly created user.

I don't think that matters.

[nhoening]

It looks really good, but I ran into a small problem. The database was dropped, but not created. Only when I ran the script once more. See trace below, but I already indicated the potential cause in the code.

(fm-3.10) nicolas at pop-os in ~/workspace/seita/flexmeasures on docs-tutorial-improvement (2 untracked)
± make clean-db db_name=flexmeasures_test db_user=flexmeasures_test
./flexmeasures/data/scripts/clean_database.sh flexmeasures_test flexmeasures_test
[sudo] password for nicolas: 
flexmeasures_test database exists
Make a backup first? [y/N] 
This will drop your database and re-create a clean one. Continue?[y/N] y
Dropping database ...
flexmeasures_test database is dropped
(fm-3.10) nicolas at pop-os in ~/workspace/seita/flexmeasures on docs-tutorial-improvement (2 untracked)
± make clean-db db_name=flexmeasures_test db_user=flexmeasures_test
./flexmeasures/data/scripts/clean_database.sh flexmeasures_test flexmeasures_test
flexmeasures_test database does not exist
Creating a new database ...
flexmeasures_test database is created
User flexmeasures_test is already available.
Connect flexmeasures_test to flexmeasures_test 
GRANT
Grant required privileges
GRANT
Creating cube extension in flexmeasures_test ...
You are now connected to database "flexmeasures_test" as user "postgres".
CREATE EXTENSION
Creating earthdistance extension in flexmeasures_test ...
You are now connected to database "flexmeasures_test" as user "postgres".
CREATE EXTENSION
Updating database structure ...

[nhoening]

Looks good!

I think this is a good example for bash scripting, which we can look at if we write more of them - very structured :)