Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cartridge role reload resets metrics data #334

Closed
DifferentialOrange opened this issue Dec 22, 2021 · 4 comments · Fixed by #341
Closed

Cartridge role reload resets metrics data #334

DifferentialOrange opened this issue Dec 22, 2021 · 4 comments · Fixed by #341
Assignees
Labels
bug Something isn't working

Comments

@DifferentialOrange
Copy link
Member

metrics package should support role reload, while it resets all observations and collectors.

Steps to reproduce

  1. Create template application

    cartridge create --name test-app
  2. Change init.lua

    local ok, err = cartridge.cfg({
        roles_reload_allowed = true,
        roles = {
            'cartridge.roles.vshard-storage',
            'cartridge.roles.vshard-router',
            'cartridge.roles.metrics',
            'app.roles.custom',
        },
    })
  3. Start an application

    cartridge build
    cartridge start -d
    cartridge replicasets setup --bootstrap-vshard
  4. Connect to instance

    cartridge enter router
  5. Execute reload

    test-reload.router> metrics = require('metrics')
    ---
    ...
    
    test-reload.router> metrics.counter('c'):inc()
    ---
    ...
    
    test-reload.router> metrics.collect()
    ---
    - - label_pairs:
         alias: router
       timestamp: 1640183661554368
       value: 1
       metric_name: c
    ...
    
    test-reload.router> require("cartridge.roles").reload()
    ---
    - true
    ...
    
    test-reload.router> metrics = require('metrics')
    ---
    ...
    
    test-reload.router> metrics.collect()
    ---
    - []
    ...
    
    test-reload.router> metrics.collectors()
    ---
    - []
    ...
    
@DifferentialOrange DifferentialOrange added the bug Something isn't working label Dec 22, 2021
@Totktonada
Copy link
Member

Why do you think that this is a problem?

@DifferentialOrange
Copy link
Member Author

Why do you think that this is a problem?

I've discussed it with @yngvar-antonsson and he said this behavior is unexpected.

@yngvar-antonsson
Copy link
Collaborator

Reload of metrics package (without Cartridge) doesn't reset collectors, so cartridge hot reload doesn't have to reset them too.

@DifferentialOrange
Copy link
Member Author

I think

package.loaded['cartridge.hotreload']
if hotreload ~= nil then
    hotreload.whitelist_globals({'__metrics_registry'})`
end

is the simplest and satisfying solution.

DifferentialOrange added a commit to tarantool/crud that referenced this issue Dec 24, 2021
After this patch, statistics module state (enabled/disabled, driver)
preserved between package reloads and Tarantool Cartridge role
reloads [1]. Since `metrics` package do not fully support
preserving registry between role reloads [2] now, this feature doesn't
work for `metrics` driver. Corresponding tests are marked with xfail
until issue is resolved for metrics.

1. https://www.tarantool.io/en/doc/latest/book/cartridge/cartridge_api/modules/cartridge.roles/#reload
2. tarantool/metrics#334

Follows up #224
DifferentialOrange added a commit to tarantool/crud that referenced this issue Jan 18, 2022
After this patch, statistics module state (enabled/disabled, driver)
preserved between package reloads and Tarantool Cartridge role
reloads [1]. Since `metrics` package do not fully support
preserving registry between role reloads [2] now, this feature doesn't
work for `metrics` driver. Corresponding tests are marked with xfail
until issue is resolved for metrics.

1. https://www.tarantool.io/en/doc/latest/book/cartridge/cartridge_api/modules/cartridge.roles/#reload
2. tarantool/metrics#334

Follows up #224
DifferentialOrange added a commit to tarantool/crud that referenced this issue Feb 3, 2023
Before this patch, tests were marked with xfail since there was a bug in
metrics module [1]. This bug is fixes in newer versions, so xfail is
replaced with skip based on metrics version.

1. tarantool/metrics#334

Follows #244
DifferentialOrange added a commit to tarantool/crud that referenced this issue Feb 3, 2023
Before this patch, tests were marked with xfail since there was a bug in
metrics module [1]. This bug is fixes in newer versions, so xfail is
replaced with skip based on metrics version.

1. tarantool/metrics#334

Follows #244
DifferentialOrange added a commit to tarantool/crud that referenced this issue Feb 13, 2023
Before this patch, tests were marked with xfail since there was a bug in
metrics module [1]. This bug is fixes in newer versions, so xfail is
replaced with skip based on metrics version.

1. tarantool/metrics#334

Follows #244
DifferentialOrange added a commit to tarantool/crud that referenced this issue Feb 13, 2023
Before this patch, tests were marked with xfail since there was a bug in
metrics module [1]. This bug is fixes in newer versions, so xfail is
replaced with skip based on metrics version.

1. tarantool/metrics#334

Follows #244
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants