Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added Basic Charm #1

Merged
merged 16 commits into from Feb 17, 2022
Merged

Added Basic Charm #1

merged 16 commits into from Feb 17, 2022

Conversation

WRFitch
Copy link
Contributor

@WRFitch WRFitch commented Dec 17, 2021

  • Added basic pgbouncer charm based on pgbouncer container.
    • Since this container hasn't been pushed anywhere, the integration tests are failing. These should run fine once this is rectified, but in the meantime they pass locally. To run them locally, there should be instructions in contributing.md.
  • Added unit tests, coverage is now at 97%
  • added basic config file management.
    • These files are necessary for pgbouncer to run, but especially in the case of pgbouncer.ini, they can get very complex. Generating these more complex files will be updated in future PRs.
    • More encryption options for userlist.txt will be added in a future PR.
  • added user management actions.
  • Added some small amount of docs.

Apologies for the size of this PR; future ones should be smaller and more incremental, and shouldn't have a bunch of user management stuff grafted on.

WRFitch added 5 commits December 17, 2021 17:36
Added basic pgbouncer charm based on pgbouncer container found at https://github.com/canonical/pgbouncer-container (currently available in a PR)
fixed/removed tests as necessary - coverage will be added back in future commits
added some basic action stubs I'm still working on
added some simple default config files to ensure pgbouncer behaves as expected on startup
install and config-changed hooks
updated readme
added basic action stubs
added bare minimum config options to get container running
documented build process, including installing, building, and adding container to microk8s container registry
tested and removed peer replication
removed default config files
added some documentation to readme
implemented config/user management
cleaned up integration tests
added unit tests, coverage is now 93%
Added and tested an action each to add and remove users, update passwords, and get the list of existing usernames.
fixed contributing.md docs
updated links in readme.md
added hooks and unit tests for each action
streamlined config management and added a function stub to restart pgbouncer, once we start work on ensuring the health of the pgbouncer process
added checks to not update config unnecessarily
added better user management - a user can now explicitly set the user list through the config.
increased randomly generated password length
removed unnecessary integration test stubs
Users stored in a local dict were being overwritten, so now they're parsed in from the existing userlist.txt, modified where necessary, and then rewritten. This should be updated to be more efficient.
Updated tests accordingly
restructured charm.py
updated some method names to make more sense to a reviewer
Updated docstrings
Some minor updates to tests, including testing that changes in config propagate to local config files
@WRFitch WRFitch self-assigned this Feb 4, 2022
@WRFitch WRFitch marked this pull request as ready for review February 4, 2022 16:00
CONTRIBUTING.md Outdated Show resolved Hide resolved
requirements.txt Outdated Show resolved Hide resolved
…r action mandatory

Also renamed change-password action
updated a template variable in contributing.md
Copy link
Member

@marceloneppel marceloneppel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing PR, Will! I left one comment about the possibility of the users list to be empty when deploying, and some other possible improvements.

updated ops version in requirements.txt
Handled empty userlist.txt files
Documented user management integration test
updated on_install unit test to verify that it runs on.install, as configured in the charm's constructor.
tested handling of empty userlist.txt files.
CONTRIBUTING.md Show resolved Hide resolved
README.md Show resolved Hide resolved
WRFitch added 2 commits February 8, 2022 12:28
added contributor agreement docs to contributing.md
added license and security sections to readme
ensured CI only runs on push to branch
Passwords are now hashed using md5 by default
tests updated to match updated passwords
cleaned up integration tests
@marceloneppel
Copy link
Member

LGTM! Thanks Will!

Copy link

@MiaAltieri MiaAltieri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good so far, mostly for README.md and CONTRIBUTING.md I'll review the the charm code and tests tomorrow.

CONTRIBUTING.md Show resolved Hide resolved
CONTRIBUTING.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
actions.yaml Outdated Show resolved Hide resolved
src/charm.py Outdated Show resolved Hide resolved
src/charm.py Outdated Show resolved Hide resolved
src/charm.py Outdated Show resolved Hide resolved
src/charm.py Show resolved Hide resolved
src/charm.py Outdated Show resolved Hide resolved
src/charm.py Outdated Show resolved Hide resolved
README.md Show resolved Hide resolved
tests/integration/test_charm.py Outdated Show resolved Hide resolved
tests/integration/test_charm.py Show resolved Hide resolved
tests/unit/test_charm.py Outdated Show resolved Hide resolved
@MiaAltieri
Copy link

MiaAltieri commented Feb 10, 2022

Hey! lots of nice work here, its great to see all of this, sorry I took so long in the review. Overall I am quite impressed and things are looking good 😎 , but I just have some concerns.

I was a little confused about the TODOs, are the functions with TODOs finished? If not maybe they should be removed/saved for a future PR. (or alternatively you could add "in future patch" after them). Also some of the TODOs are long, it might be useful to link them to a JIRA or issue. But I am not sure what the best practice is here. I think it would be good to get the opinion of someone more senior than me since I am new here :) (because after all this might be a fine thing to do, but I don't really know what the call here is 🤷 )

(Side note these todos show that you're really thinking through things, some of them might make useful spec documents)
(Side note also curious for what the answers are for your questions in_on_remove_user_action , keep me updated on what you find)

test_reload_pgbouncer seems to not be implemented or tested, I assume this is for the next PR? Maybe you can just remove this for now? Or maybe its fine, I don't know here maybe ask someone more senior.

Also some small things that need to be done:

  • Need type hinting for functions
  • Specify returns of functions in docstrings

WRFitch added 3 commits February 11, 2022 17:46
formatting
removing unnecessary TODO comments
added environment setup docs
documented actions in readme
type hinting
better docstrings

Still haven't fixed the integration test yet - for one reason or another, accessing files from machines created during an integration test is proving difficult.
src/charm.py Outdated Show resolved Hide resolved
src/charm.py Outdated Show resolved Hide resolved
src/charm.py Outdated Show resolved Hide resolved
Copy link

@MiaAltieri MiaAltieri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks amazing! 🤩 just some knits and comments about adding some comments to make your code more clear (Feel free to ignore those if you disagree thought)


https://discourse.charmhub.io/t/4208
"""
"""Charmed PgBouncer connection pooler."""

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some parts of the code are a little confusing to me (since this a lot of this is new to me). I think some of it could use some comments before a code block when it isn't obvious whats going on. (IMHO)

But if you disagree then that is also fine. I can leave comments where I got confused if you want to add comments (feel free to add all/some/none).

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes Mia! Clarifying comments in code is one of the hallmarks of good coding. Also, please do not fall into the trap of translating to English what is written in code.

src/charm.py Show resolved Hide resolved
src/charm.py Show resolved Hide resolved
src/charm.py Outdated
}
)
return
password = hashlib.md5(event.params["password"].encode())

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comment

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've extracted hashing into a separate function, which is independently documented. Does this adequately explain the function of the hashing, or shall I elaborate?

src/charm.py Show resolved Hide resolved
src/charm.py Outdated
if current not in self._stored.things:
logger.debug("found a new thing: %r", current)
self._stored.things.append(current)
users = self._get_userlist_from_container()

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comment

src/charm.py Outdated Show resolved Hide resolved
src/charm.py Show resolved Hide resolved
Copy link

@quiet-ranger quiet-ranger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well done!

@MiaAltieri MiaAltieri self-requested a review February 17, 2022 09:44
Copy link

@MiaAltieri MiaAltieri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wunderbar ⭐

@WRFitch WRFitch merged commit adc92a4 into canonical:main Feb 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants