Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clearer logs and small refactoring #73

Merged
merged 7 commits into from
Sep 15, 2021

Conversation

jankatins
Copy link
Contributor

No description provided.

@jankatins jankatins requested a review from a team as a code owner September 14, 2021 20:46
@jankatins jankatins marked this pull request as draft September 14, 2021 20:56
@jankatins
Copy link
Contributor Author

Needs converting to github actions first. Also at least one pytype fix conflicts with flake8 :-(

@jankatins jankatins force-pushed the jankatins/clearer_logs_and_small_refactoring branch from dc4f07b to 2d7c246 Compare September 14, 2021 21:15
@jankatins
Copy link
Contributor Author

Ok, now it passes locally:

λ  make test
python3 -m flake8 --ignore E722 --max-line-len=125 pglookout/ test/
/usr/bin/pylint-3 --rcfile .pylintrc pglookout/ test/

--------------------------------------------------------------------
Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)

python3 -m pytest -vv test/
=========================================== test session starts ===========================================
platform linux -- Python 3.9.7, pytest-6.0.2, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/jan.katins/work/pglookout/.hypothesis/examples')
rootdir: /home/jan.katins/work/pglookout
plugins: timeout-1.4.2, forked-1.3.0, xdist-1.34.0, cov-2.11.1, hypothesis-5.43.9, mock-3.3.1, Faker-4.1.3
collected 34 items

test/test_cluster_monitor.py::test_replication_lag PASSED                                           [  2%]
test/test_cluster_monitor.py::test_main_loop PASSED                                                 [  5%]
test/test_common.py::test_convert_xlog_location_to_offset PASSED                                    [  8%]
test/test_common.py::test_parse_iso_datetime PASSED                                                 [ 11%]
test/test_common.py::test_get_iso_timestamp PASSED                                                  [ 14%]
test/test_lookout.py::test_connect_to_cluster_nodes_and_cleanup_old_nodes PASSED                    [ 17%]
test/test_lookout.py::test_state_file_write PASSED                                                  [ 20%]
test/test_lookout.py::test_load_config PASSED                                                       [ 23%]
test/test_lookout.py::test_check_cluster_state_warning PASSED                                       [ 26%]
test/test_lookout.py::test_check_cluster_do_failover_one_standby PASSED                             [ 29%]
test/test_lookout.py::test_check_cluster_master_gone_one_standby_one_observer PASSED                [ 32%]
test/test_lookout.py::test_check_cluster_do_failover_one_standby_one_observer PASSED                [ 35%]
test/test_lookout.py::test_check_cluster_do_failover_with_a_node_which_is_is_maintenance PASSED     [ 38%]
test/test_lookout.py::test_check_cluster_do_failover_with_a_node_which_should_never_be_promoted PASSED [ 41%]
test/test_lookout.py::test_check_cluster_do_failover_two_standbys PASSED                            [ 44%]
test/test_lookout.py::test_check_cluster_do_failover_two_standbys_when_the_one_ahead_can_never_be_promotedPASSED [ 47%]
test/test_lookout.py::test_failover_with_no_master_anymore PASSED                                   [ 50%]
test/test_lookout.py::test_failover_over_replication_lag_when_still_connected_to_master PASSED      [ 52%]
test/test_lookout.py::test_failover_over_replication_lag_with_one_observer_one_standby_no_connections PASSED [ 55%]
test/test_lookout.py::test_cluster_state_when_observer_has_also_non_members_of_our_current_cluster PASSED [ 58%]
test/test_lookout.py::test_failover_no_connections PASSED                                           [ 61%]
test/test_lookout.py::test_failover_master_two_standbys_one_observer_no_connection_between_standbys PASSED[ 64%]
test/test_lookout.py::test_failover_master_one_standby_one_observer_no_connections PASSED           [ 67%]
test/test_lookout.py::test_find_current_master PASSED                                               [ 70%]
test/test_lookout.py::test_two_standby_failover_and_autofollow PASSED                               [ 73%]
test/test_lookout.py::test_replication_positions PASSED                                             [ 76%]
test/test_lookout.py::test_node_map PASSED                                                          [ 79%]
test/test_lookout.py::test_node_map_disconnected_current_master PASSED                              [ 82%]
test/test_lookout.py::test_standbys_failover_equal_replication_positions PASSED                     [ 85%]
test/test_lookout.py::test_node_map_when_only_observer_sees_master PASSED                           [ 88%]
test/test_lookout.py::test_poll_observers_on_warning_only PASSED                                    [ 91%]
test/test_pgutil.py::test_connection_info PASSED                                                    [ 94%]
test/test_pgutil.py::test_mask_connection_info PASSED                                               [ 97%]
test/test_webserver.py::test_webserver PASSED                                                       [100%]

=========================================== 34 passed in 5.03s ============================================

@jankatins jankatins force-pushed the jankatins/clearer_logs_and_small_refactoring branch 5 times, most recently from 17e60b1 to d4396ea Compare September 14, 2021 21:48
@jankatins jankatins marked this pull request as ready for review September 14, 2021 22:07
.github/workflows/build.yml Show resolved Hide resolved
.github/workflows/build.yml Outdated Show resolved Hide resolved
Makefile Show resolved Hide resolved
pglookout/cluster_monitor.py Outdated Show resolved Hide resolved
pglookout/pglookout.py Outdated Show resolved Hide resolved
f_result = None
result = {"fetch_time": get_iso_timestamp(), "connection": False}
if not db_conn:
db_conn = self._connect_to_db(instance, self.config["remote_conns"].get(instance))
if not db_conn:
return result
phase = None
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit. I would instead move the next phase = "querying status from" out of the try, since the assignment won't fail (and if it did, it wouldn't fail with an exception type that is part of the except clause).

pglookout/pglookout.py Outdated Show resolved Hide resolved
The logs were always showing "We still have some connected masters: {}, not failing over"
which is confusing to the user as the real reason (not yet expired timeout) was hidden.
The names were misleading as they implied that only the standbys were queries, but actually all cluster members were queried
Nothing was every reading from it and it is confusing to rely on dict references to get everything updated
@jankatins jankatins force-pushed the jankatins/clearer_logs_and_small_refactoring branch from d4396ea to f21173b Compare September 15, 2021 12:42
@jankatins jankatins force-pushed the jankatins/clearer_logs_and_small_refactoring branch from f21173b to d6f32ab Compare September 15, 2021 12:44
@jankatins jankatins force-pushed the jankatins/clearer_logs_and_small_refactoring branch from d6f32ab to defb2ce Compare September 15, 2021 14:20
@kmichel-aiven kmichel-aiven merged commit 6924e3f into master Sep 15, 2021
@kmichel-aiven kmichel-aiven deleted the jankatins/clearer_logs_and_small_refactoring branch September 15, 2021 14:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants