Add "Note" in the CLI if API has default query "limit=50" and update rule endpoint help #3514

humblearner · 2017-06-23T20:57:30Z

Changes:

$ st2 rule list
+---------------------------------+--------+---------------------------------+---------+
| ref                             | pack   | description                     | enabled |
+---------------------------------+--------+---------------------------------+---------+
| github.deploy_pack_on_deploymen | github | Trigger an ST2 pack.install     | False   |
| t_event                         |        | when receiving a deployment     |         |
                ~~~~~~~~~~~~~~~~~~~~~~~
                ~~~~~~~~~~~~~~~~~~~~~~~
| st2cd.st2_pkg_test_stable_u16_e | st2cd  | Test stable packages            | True    |
| nterprise                       |        |                                 |         |
+---------------------------------+--------+---------------------------------+---------+
+-----------------------------------------------------------------------------------------+
| Note: Only first 50 results are displayed. Use -n/--last flag for more results, if any. |
+-----------------------------------------------------------------------------------------+

$ st2 rule list -h
usage: st2 rule list [-h] [-t TOKEN] [--api-key API_KEY] [-j] [-y]
                     [-a ATTR [ATTR ...]] [-w WIDTH [WIDTH ...]] [-n LAST]
                     [--iftt] [-p PACK] [-c ACTION] [-g TRIGGER]
                     [--enabled | --disabled]

Get the list of the 50 most recent rules.

optional arguments:
                ~~~~~~~~~~~~~~~~~~~~~~~
                ~~~~~~~~~~~~~~~~~~~~~~~
  -n LAST, --last LAST  List N most recent rules. Default is 50.

CLI Updated:

Functionality:

Dynamically display note (in stderr) if # of entries are greater than limit (default or set by -n flag).
Only in (default) table format, not in json or yaml.

Addresses: #3488

Kami · 2017-06-26T09:14:16Z

LGTM, but we should also do the change @armab suggested aka printing out "there are more results" when using CLI in user-friendly (table) mode (we don't want to print that when using --json or --yaml flag).

Kami · 2017-06-26T09:15:30Z

st2client/st2client/commands/rule.py

                                       resource.get_plural_display_name().lower()))
        self.parser.add_argument('--iftt', action='store_true',
                                 help='Show trigger and action in display list.')
+        self.parser.add_argument('-p', '--pack', type=str,


Perhaps we should add changelog entry for this change.

Yes, once all the changes are ok, I'll update the changelog.

Kami · 2017-06-26T09:19:41Z

st2client/st2client/commands/rule.py

@@ -34,20 +34,26 @@ def __init__(self, description, app, subparsers, parent_parser=None):
        self.commands['disable'] = RuleDisableCommand(self.resource, self.app, self.subparsers)


-class RuleListCommand(resource.ContentPackResourceListCommand):
+class RuleListCommand(resource.ResourceTableCommand):


Just curious why the inherit class change was needed.

armab · 2017-06-26T15:36:07Z

True, without explicit note that only first 50 results are showed up, - it's not user friendly and doesn't solve the confusion.
As said in #3488 to be consistent we need to check if all CLI commands have this feature as well.

humblearner · 2017-06-27T08:55:22Z

Besides rules, the default limit=50 is set for: rule_enforcement, execution, triggerinstance and trace. If above Note change is ok, then I'll add it to the above CLIs as well.

Kami · 2017-06-27T09:02:13Z

Yeah, that note works for me, but maybe we could also make it more dynamic and only display the note if there are in fact more results.

We could do that by utilizing X- headers returned by the API, bit it could also make it a bit more complex and require more refactoring, because headers and raw response objects are not available in those methods right now.

Kami · 2017-06-28T17:02:16Z

st2client/st2client/commands/trace.py

-                              attributes=args.attr, widths=args.width,
-                              json=args.json, yaml=args.yaml,
-                              attribute_transform_functions=self.attribute_transform_functions)
+            if args.json or args.yaml:


It looks like this is repeated in quite some places so eventually it will need some refactor, but it should be fine for now.

Kami · 2017-06-28T17:03:52Z

st2client/st2client/models/core.py

@@ -274,11 +274,14 @@ def query(self, **kwargs):
            self.handle_error(response)
        items = response.json()
        instances = [self.resource.deserialize(item) for item in items]
-        return instances
+        if 'X-Total-Count' in response.headers:


It looks like query is used in quite some places, were all the affected places updated?

https://gist.github.com/Kami/365f58a69f90594ce970dcd2c2069a8e

Yes, I have updated all the places, where query was called. However, unit tests are not happy, as we are mocking the response w/o headers.

Kami · 2017-06-28T17:05:48Z

st2client/st2client/commands/action.py

-                          json=args.json,
-                          yaml=args.yaml,
-                          attribute_transform_functions=self.attribute_transform_functions)
+            if args.last >= self.default_limit and int(count) > args.last:


It looks like query method returns Noneif response doesn't contain X-Total header so thisint(count)` would fail.

So probably need to do something along the lines of and count is not None and int(count)...

Agree, however, is it safe to assume if args.last is present X-Total-Count will be present in the header?

I believe so, but would need to confirm.

And perhaps better to be on the safe side :)

Kami · 2017-06-28T17:07:26Z

st2client/st2client/formatters/table.py

+        note = PrettyTable([""])
+        note.header = False
+        note.add_row([message])
+        print note


Minor API consistency thing - all other formatter class methods return a value instead of printing it so it would be better for this one to do that as well.

And perhaps, as @armab said earlier, maybe even print this note to stderr instead of stdout.

…_st2client

Kami · 2017-06-29T09:44:41Z

st2client/st2client/formatters/table.py

+        note = PrettyTable([""])
+        note.header = False
+        note.add_row([message])
+        return sys.stderr.write(str(note))


Missing line break at the end.

It would also be good if you could align note box table with actual content table (so it's the same number of columns / width).

Kami · 2017-06-29T09:48:04Z

st2client/st2client/commands/triggerinstance.py

+            self.print_output(reversed(instances), table.MultiColumnTable,
+                              attributes=args.attr, widths=args.width,
+                              attribute_transform_functions=self.attribute_transform_functions)
+            if args.last >= self.default_limit and count and int(count) > args.last:


It looks like it doesn't correctly handle a scenario where you pass in -n 1 for example but there are more results.

So this logic needs to be changed to handle that scenario as well.

…_st2client

armab · 2017-07-01T09:36:16Z

CHANGELOG.rst

+  ``execution``, ``rule-enforcment``, ``trace`` and ``trigger-instance`` as they are not displayed.
+  Default limit is 50. (improvement)
+
+  Reported by Eugen C. #3488


Just #3514 #3488 is enough there, without "Reported by".

I think it's good idea to include "reported by" only if there was an external contribution.

Kami · 2017-07-03T07:36:10Z

st2client/st2client/formatters/table.py

                    value = transform_function(value=value)
                    value = strutil.unescape(value)
                    values.append(value)
            table.add_row(values)
+
+        # width for the note
+        global table_width


I will test it and go ahead and merge it if everything look OK so it will end up in v2.3.1, but we really need to refactor and fix this global variable thing since it's not robust and could cause issues.

Kami · 2017-07-03T11:58:38Z

I'll go ahead and merge this so we can start testing changes for v2.3.1 release, but please look into getting rid of the global variable as soon as you get a chance.

Kami · 2017-07-05T17:18:00Z

@humblearner It would be great if you can also work on automated robot tests for this CLI functionality - so testing all the common scenarios + edge cases.

CLI is one of the places where we lack tests and regressions are easy to introduce so we should invest some time in more automated tests.

result. In StackStorm v2.3.0, breaking client change has been introduced for query method as part of StackStorm/st2#3514.

Add default rule list limit to 50 message in help

d51eb77

Kami approved these changes Jun 26, 2017

View reviewed changes

Kami reviewed Jun 26, 2017

View reviewed changes

Anirudh Rekhi added 2 commits June 27, 2017 00:52

Add note for more than 50(default) rules

64f6af4

Hide note if flag for action or trigger is used

6981123

Kami added this to the 2.3.1 milestone Jun 27, 2017

Update above mentioned endpoints and dynamic note based on X-Total-Count

42df7da

humblearner changed the title ~~Add default rule list limit to 50 message in help~~ Add default rule list limit to 50 messages in help Jun 28, 2017

Kami reviewed Jun 28, 2017

View reviewed changes

Anirudh Rekhi added 3 commits June 28, 2017 19:41

Note in stderr, update tests

c439e3a

Merge branch 'master' of github.com:StackStorm/st2 into rule_50_limit…

bba6119

…_st2client

Fix test_formatters

1a3fb5b

humblearner changed the title ~~Add default rule list limit to 50 messages in help~~ Add "Note" in the CLI if API has default query "limit=50" and update rule endpoint help Jun 29, 2017

Kami reviewed Jun 29, 2017

View reviewed changes

Anirudh Rekhi added 4 commits June 29, 2017 06:01

Address the line break after the note

c75aa6d

Merge branch 'master' of github.com:StackStorm/st2 into rule_50_limit…

7025104

…_st2client

Note row size same as the table size

f9eecad

Add changelog entry

eb414ba

armab reviewed Jul 1, 2017

View reviewed changes

Kami reviewed Jul 3, 2017

View reviewed changes

Make it more robust if there are no new lines.

b63c85d

Re-word changelog entry.

cd3bb60

Merge branch 'master' into rule_50_limit_st2client

69be3f5

Kami merged commit 62256ef into master Jul 3, 2017

armab deleted the rule_50_limit_st2client branch July 3, 2017 12:35

armab mentioned this pull request Jul 3, 2017

'st2 rule list' is limited to 50 results #3488

Closed

Kami mentioned this pull request Jul 25, 2017

Fix all actions which use client.query() method StackStorm-Exchange/stackstorm-st2#4

Closed

humblearner mentioned this pull request Sep 1, 2017

v2.3 Running the "st2 key list" command returns 100 entries vs all #3708

Closed

Add "Note" in the CLI if API has default query "limit=50" and update rule endpoint help #3514

Add "Note" in the CLI if API has default query "limit=50" and update rule endpoint help #3514

humblearner commented Jun 23, 2017 •

edited

Kami commented Jun 26, 2017

Kami Jun 26, 2017

humblearner Jun 27, 2017

Kami Jun 26, 2017

armab commented Jun 26, 2017

humblearner commented Jun 27, 2017 •

edited

Kami commented Jun 27, 2017

Kami Jun 28, 2017

Kami Jun 28, 2017 •

edited

humblearner Jun 28, 2017

Kami Jun 28, 2017

humblearner Jun 28, 2017

Kami Jun 28, 2017

Kami Jun 28, 2017

Kami Jun 28, 2017

Kami Jun 29, 2017

Kami Jun 29, 2017

humblearner Jun 29, 2017

armab Jul 1, 2017 •

edited

Kami Jul 3, 2017

Kami commented Jul 3, 2017

Kami commented Jul 5, 2017

Add "Note" in the CLI if API has default query "limit=50" and update rule endpoint help #3514

Add "Note" in the CLI if API has default query "limit=50" and update rule endpoint help #3514

Conversation

humblearner commented Jun 23, 2017 • edited

Kami commented Jun 26, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

armab commented Jun 26, 2017

humblearner commented Jun 27, 2017 • edited

Kami commented Jun 27, 2017

Choose a reason for hiding this comment

Kami Jun 28, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

armab Jul 1, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Kami commented Jul 3, 2017

Kami commented Jul 5, 2017

humblearner commented Jun 23, 2017 •

edited

humblearner commented Jun 27, 2017 •

edited

Kami Jun 28, 2017 •

edited

armab Jul 1, 2017 •

edited