Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add metadata for vhosts in global definitions exported via UI #11454

Closed

Conversation

anhanhnguyen
Copy link
Contributor

Proposed Changes

Fix the issue reported at #10515.

Types of Changes

What types of changes does your code introduce to this project?
Put an x in the boxes that apply

  • Bug fix (non-breaking change which fixes issue Definitions export from management UI does not include vhost metadata #10515)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause an observable behavior change in existing systems)
  • Documentation improvements (corrections, new content, etc)
  • Cosmetic change (whitespace, formatting, etc)
  • Build system and/or CI

Checklist

Put an x in the boxes that apply.
You can also fill these out after creating the PR.
If you're unsure about any of them, don't hesitate to ask on the mailing list.
We're here to help!
This is simply a reminder of what we are going to look for before merging your code.

  • I have read the CONTRIBUTING.md document
  • I have signed the CA (see https://cla.pivotal.io/sign/rabbitmq)
  • I have added tests that prove my fix is effective or that my feature works
  • All tests pass locally with my changes
  • If relevant, I have added necessary documentation to https://github.com/rabbitmq/rabbitmq-website
  • If relevant, I have added this change to the first version(s) in release-notes that I expect to introduce it

Further Comments

The change aims to synchronise the output of GET /api/definitions and rabbitmqctl export_definitions

  1. The management plugin code paths are removed and replaced by rabbit_definitions:all_definitions() as rabbitmqctl export_definitions
  2. definition_import_SUITE and its input cases are covered
  3. Update test cases to confirm and import definitions via Management API
  4. Manually exporting and importing are tested

@anhanhnguyen
Copy link
Contributor Author

The limit code part remains untouched.

  • Limits are always returned in vhosts via rabbitmqctl export_definitions
  • [] is returned if there is no limit set.
  • Limits in vhost is informative. They are not imported. The value in parameters is imported instead.

The behaviours of GET /api/definitions/vhost and POST /api/definitions/vhost also remain the same.

@michaelklishin Please confirm if limits and /api/definitions/vhost behaviours need to be changed. I'll update other commits for it.

@michaelklishin
Copy link
Member

The forced push was a rebase.

@michaelklishin
Copy link
Member

@anhanhnguyen I will have to take a look at what the behavior of the virtual host-specific exporting endpoint is, and how it compares to CLI tools. I will get back to you next week.

Copy link
Member

@michaelklishin michaelklishin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This breaks HTTP API tests:

gmake ct-rabbit_mgmt_http

or, with Bazel

bt //deps/rabbitmq_management:rabbit_mgmt_http_SUITE
bt //deps/rabbitmq_management:rabbit_mgmt_http_SUITE-mixed

@michaelklishin
Copy link
Member

{{expected,200,got,500,type,"GET",path,"/definitions",body,[]},

in the definition import test immediately reveals an exception. Looks like this is the relevant part:

2024-06-23 12:59:54.687659-04:00 [error] <0.7151.0>   crasher:
2024-06-23 12:59:54.687659-04:00 [error] <0.7151.0>     initial call: cowboy_stream_h:request_process/3
2024-06-23 12:59:54.687659-04:00 [error] <0.7151.0>     pid: <0.7151.0>
2024-06-23 12:59:54.687659-04:00 [error] <0.7151.0>     registered_name: []
2024-06-23 12:59:54.687659-04:00 [error] <0.7151.0>     exception error: bad argument
2024-06-23 12:59:54.687659-04:00 [error] <0.7151.0>       in function  maps:from_list/1
2024-06-23 12:59:54.687659-04:00 [error] <0.7151.0>          called as maps:from_list(<<>>)
2024-06-23 12:59:54.687659-04:00 [error] <0.7151.0>          *** argument 1: not a list
2024-06-23 12:59:54.687659-04:00 [error] <0.7151.0>       in call from rabbit_definitions:runtime_parameter_definition/1 (rabbit_definitions.erl, line 1084)
2024-06-23 12:59:54.687659-04:00 [error] <0.7151.0>       in call from rabbit_definitions:'-list_runtime_parameters/0-lc$^0/1-0-'/1 (rabbit_definitions.erl, line 1077)
2024-06-23 12:59:54.687659-04:00 [error] <0.7151.0>       in call from rabbit_definitions:all_definitions/0 (rabbit_definitions.erl, line 275)
2024-06-23 12:59:54.687659-04:00 [error] <0.7151.0>       in call from rabbit_mgmt_wm_definitions:all_definitions/2 (rabbit_mgmt_wm_definitions.erl, line 52)
2024-06-23 12:59:54.687659-04:00 [error] <0.7151.0>       in call from cowboy_rest:call/3 (src/cowboy_rest.erl, line 1590)
2024-06-23 12:59:54.687659-04:00 [error] <0.7151.0>       in call from cowboy_rest:set_resp_body/2 (src/cowboy_rest.erl, line 1473)
2024-06-23 12:59:54.687659-04:00 [error] <0.7151.0>       in call from cowboy_rest:upgrade/4 (src/cowboy_rest.erl, line 284)
2024-06-23 12:59:54.687659-04:00 [error] <0.7151.0>     ancestors: [<0.7150.0>,<0.743.0>,<0.736.0>,<0.735.0>,<0.733.0>,
2024-06-23 12:59:54.687659-04:00 [error] <0.7151.0>                   rabbit_web_dispatch_sup,<0.685.0>]
2024-06-23 12:59:54.687659-04:00 [error] <0.7151.0>     message_queue_len: 0
2024-06-23 12:59:54.687659-04:00 [error] <0.7151.0>     messages: []
2024-06-23 12:59:54.687659-04:00 [error] <0.7151.0>     links: [<0.7150.0>]
2024-06-23 12:59:54.687659-04:00 [error] <0.7151.0>     dictionary: []
2024-06-23 12:59:54.687659-04:00 [error] <0.7151.0>     trap_exit: false
2024-06-23 12:59:54.687659-04:00 [error] <0.7151.0>     status: running
2024-06-23 12:59:54.687659-04:00 [error] <0.7151.0>     heap_size: 4185
2024-06-23 12:59:54.687659-04:00 [error] <0.7151.0>     stack_size: 28
2024-06-23 12:59:54.687659-04:00 [error] <0.7151.0>     reductions: 4345
2024-06-23 12:59:54.687659-04:00 [error] <0.7151.0>   neighbours:

binary_to_list(Filename), ReqData)
end,
Context).
rabbit_definitions:all_definitions(),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function can return keys that the HTTP API cannot serialize to JSON as is.

Which is why this filtering exists in the same module:

rw_state() ->
    [{users,              [name, password_hash, hashing_algorithm, tags, limits]},
     {vhosts,             [name]},
     {permissions,        [user, vhost, configure, write, read]},
     {topic_permissions,  [user, vhost, exchange, write, read]},
     {parameters,         [vhost, component, name, value]},
     {global_parameters,  [name, value]},
     {policies,           [vhost, name, pattern, definition, priority, 'apply-to']},
     {queues,             [name, vhost, durable, auto_delete, arguments]},
     {exchanges,          [name, vhost, type, durable, auto_delete, internal,
                           arguments]},
     {bindings,           [source, vhost, destination, destination_type, routing_key,
                           arguments]}].

filter(Items) ->
    [filter_items(N, V, proplists:get_value(N, rw_state())) || {N, V} <- Items].

@michaelklishin
Copy link
Member

I'm afraid I cannot accept this pull request as is.

It breaks the HTTP API by removing the filtering and assuming that anything returned by rabbit_definitions:all_definition/0 can be formatted as JSON, which is not the case.

It hijacks an existing test to test something different then uses dummy values such as an empty list of tags and an empty description string. Using an empty list of tags specifically won't test much.

The implementation part should have been

diff --git a/deps/rabbitmq_management/src/rabbit_mgmt_wm_definitions.erl b/deps/rabbitmq_management/src/rabbit_mgmt_wm_definitions.erl
index 06f98b1408..65c833450f 100644
--- a/deps/rabbitmq_management/src/rabbit_mgmt_wm_definitions.erl
+++ b/deps/rabbitmq_management/src/rabbit_mgmt_wm_definitions.erl
@@ -254,7 +254,7 @@ export_name(_Name)                -> true.

 rw_state() ->
     [{users,              [name, password_hash, hashing_algorithm, tags, limits]},
-     {vhosts,             [name]},
+     {vhosts,             [name, description, tags, default_queue_type, metadata]},
      {permissions,        [user, vhost, configure, write, read]},
      {topic_permissions,  [user, vhost, exchange, write, read]},
      {parameters,         [vhost, component, name, value]},
diff --git a/deps/rabbitmq_management/src/rabbit_mgmt_wm_vhosts.erl b/deps/rabbitmq_management/src/rabbit_mgmt_wm_vhosts.erl
index 2decdfc9ed..a0779a2af3 100644
--- a/deps/rabbitmq_management/src/rabbit_mgmt_wm_vhosts.erl
+++ b/deps/rabbitmq_management/src/rabbit_mgmt_wm_vhosts.erl
@@ -55,7 +55,7 @@ augment(Basic, ReqData) ->

 augmented(ReqData, #context{user = User}) ->
     case rabbit_mgmt_util:disable_stats(ReqData) of
-        false ->
+        false ->
             rabbit_mgmt_db:augment_vhosts(
               [rabbit_vhost:info(V) || V <- rabbit_mgmt_util:list_visible_vhosts(User)],
               rabbit_mgmt_util:range(ReqData));
@@ -64,4 +64,4 @@ augmented(ReqData, #context{user = User}) ->
     end.

 basic() ->
-    rabbit_vhost:info_all([name]).
+    rabbit_vhost:info_all([name, description, tags, default_queue_type, metadata]).

and a new test should have been added to test the metadata specifically.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants