Rework the packs pack (more packs-related ChatOps!) #2982

emedvedev · 2016-10-31T11:42:21Z

Aliases for the packs pack:
— pack install for installing a pack or multiple packs (just a minor rework here);
— pack search to query the index;
— pack show to display information about an installed pack, including HEAD location in the git tree (N commits ahead/behind);
— pack show available to fetch information about a pack from the index.
Most of packs.deploy is now obsolete, so I'm removing it and the helper actions for now, but I would appreciate @jjm finding some time to talk it over with me. A thing or two that was possible with packs.deploy might not be covered by our new pack management, and if that's the case, I'm fine with keeping the deploy workflow or breaking it down into multiple actions.
There was a linux pack directory and an action. Inside the packs pack. Apparently, for quite some time, too (I remember @enykeev noticing it). I removed it. Sorry, not sorry.
Some minor cleanups (this PR started as a minor cleanup, actually, but then things just got more and more interesting).

emedvedev · 2016-10-31T11:42:47Z

WIP! Like, really WIP. Don't test it yet, there's still some work to be done.

emedvedev · 2016-10-31T11:49:51Z

Btw, pack show available <pack> shows content count if it's present in the index, but pack show <pack> doesn't have content count for installed packs, because we don't have helper functions for content counting yet. @Kami promised to get it done :)

codecov-io · 2016-10-31T12:44:28Z

Current coverage is 77.11% (diff: 100%)

Merging #2982 into master will increase coverage by 0.22%

@@             master      #2982   diff @@
==========================================
  Files           430        430          
  Lines         21997      21997          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          16914      16962    +48   
+ Misses         5083       5035    -48   
  Partials          0          0

Powered by Codecov. Last update 02e3c95...4ddbe99

jjm · 2016-10-31T12:50:31Z

@emedvedev Yes, I think packs.deploy and it's associated actions can be removed. In fact GitHub pack deployment_event workflow that I added earlier in October does not call packs.deploy and calls packs.install directly.

I think with the release of Stackstorm-exchange the bitbucket rule post_receive_webhook.yaml should be removed.

emedvedev · 2016-11-01T11:23:39Z

In fact GitHub pack deployment_event workflow that I added earlier in October does not call packs.deploy and calls packs.install directly.

Once we make the move, that rule and workflow will become a bit simpler, too!

action: packs.install
input:
  packs: [  "<% $.ssh_url %>=<% $.deploy_ref %>" ]

I'll skim through all the packs to make sure all the packs.install and packs.download references are updated when the packs are moved.

arm4b · 2016-11-23T21:00:07Z

contrib/packs/actions/pack_mgmt/lookup_remote.py

+        pack_formatted = None
+        # Try to get the original yaml file because we know
+        # how it's stored in the Exchange, otherwise fall back
+        # to transforming json.


@emedvedev
As I see, we already getting pack meta from index from endpoint: https://index.stackstorm.org/v1/index.json
And then we're downloading https://index.stackstorm.org/v1/packs/github.yaml again with the same meta.

What was initial idea here? Because it seems a bit dirty with all the hardcoding, additional request, regex.

I'm thinking about removing this second pack.yaml part and rely only on get_pack_from_index(pack) meta.

I'm thinking about removing this second pack.yaml part and rely only on get_pack_from_index(pack) meta.

👍

Problem with json->yaml is that it's not ordered, but that doesn't really matter much.

- `!pack get` - get info about installed pack - `!pack show` - show remote StackStorm Exchange pack - polished & working `!pack show`

arm4b · 2016-11-24T21:28:12Z

So it's fixed & Polished & Acceptance tested in Slack.
The new list of ChatOps Action Aliases is adjusted to conform with CLI commands for consistency.

!pack install (screenshot) - Install pack(s)
!pack get (screenshot) - Get info about locally installed pack
!pack search (screenshot) - Search for pack(s) in StackStorm Exchange
!pack show (screenshot) - Show pack info from StackStorm Exchange

The Aliases automated tests as well as Integration Tests (e2e, robo) are in my TODO list and will be added, but It's good to merge now to see things in pre-production (packages).

Additionally, I'll need staging-unstable packages to make sure that it works in EL6 (outdated git).

emedvedev · 2016-11-24T22:02:17Z

❤️ ❤️ ❤️ ❤️

LGTM.

enykeev · 2016-11-25T04:28:22Z

enykeev	[10:30] 
!pack show st2

stuffBOT	[10:30]  
@enykeev: Getting back to you about the `st2` pack:
Here's the full index entry at StackStorm Exchange:
{author: st2-dev, description: StackStorm pack management, email: info@stackstorm.com,
    name: st2, repo_url: 'https://github.com/StackStorm-Exchange/stackstorm-st2',
    version: 0.1.2}

To install the pack: `pack install st2`
If the pack is already installed, I will upgrade it to the latest version, keeping the config.

I'm kinda missing hubot alias before the command, though I'm not sure we can get it now in any sane way.

Also, #3051 to prevent different kinds of output formatting for different packs.

Other than that, looks good to me.

enykeev · 2016-11-25T04:31:19Z

contrib/packs/actions/install.meta.yaml

+    upgrade:
+      type: "boolean"
+      default: false
+      description: "Upgrade the pack if already installed."


Wait a second. I don't see us using it in any way. I also don't think we should have something like that to begin with. Can settle to default: true, though.

I'm not sure what that flag is doing here 😛 Should just remove.

Don't know where it comes from too. Will remove.

Hm, but this is interesting.

In future it would be good to have pack install really idempotent.
So if pack is already installed - it won't be reinstalled again, unless some special flag like that specified. Re-downloading/re-installing by default takes time and it's in fact "upgrade" or "reinstall".

This makes sense especially for configuration management tools, when you apply playbook/cookbook, which apart of deploying st2 installs a bunch of st2 packs on the same machine. And it's normal to run same playbook on the same machine several times.

This is a bigger discussion that applies to package management in general. It will be idempotent if you specify a precise version, and we already support that, but a repeated install is normally regarded as upgrade, we're certainly not alone here. :)

Installing + Downloading over and over again (even if it reaches the same state) is not considered really "Idempotent" in configuration management.
If resources is in desired state - we shouldn't repeat the same operation again.

Fair. They do stay in that state though 😛

Let's make sure we get to that issue after the release, it's pretty important.

arm4b · 2016-11-25T16:43:37Z

I'm kinda missing hubot alias before the command, though I'm not sure we can get it now in any sane way.

Yeah, @enykeev I just verified, and there is no hubot alias variable in context. I bet none could think that it might be useful to have in context. Might be something good to adjust in future.

Kami · 2016-11-26T10:05:24Z

Nice work everyone!

emedvedev added 8 commits October 30, 2016 11:02

Remake packs.info and its alias

0cbd95d

Packs.install alias

a78bb42

Publish installed packs in packs.install

04f6fa5

Remove packs.deploy and the associated actions

7efb676

Somehow there's an action from the linux pack here. Oh well.

f603daf

Lookups

2f6a734

Getting yaml from the index

b71db46

Add action and alias for index search

cc598c6

Rewrite the descriptions

adac442

emedvedev mentioned this pull request Nov 11, 2016

Remove packs.info action #3019

Merged

arm4b self-assigned this Nov 22, 2016

arm4b added the refactor label Nov 22, 2016

emedvedev and others added 6 commits November 23, 2016 21:16

Merge branch 'master' into packs-cleanup

9d5f4b9

Fix pack_show.yaml alias Jinja formatting error

7a5c113

Merge remote-tracking branch 'origin/master' into packs-cleanup

d0e504c

Remove obsolete tests first

cd7203c

Bump pack version

77aefa6

Touch pack descriptions

f4db2cd

arm4b reviewed Nov 23, 2016

View reviewed changes

Simplify packs.lookup_remote action

6cc8fd0

arm4b added chatops WIP labels Nov 24, 2016

armab added 3 commits November 24, 2016 18:31

Follow the CLI naming for consistency

f96beda

- `!pack get` - get info about installed pack - `!pack show` - show remote StackStorm Exchange pack - polished & working `!pack show`

Fix & Polish !pack search Action Alias

c9745b8

Fix & Polish !pack install Action Alias

17eeffb

armab added 2 commits November 24, 2016 21:46

Fix force=true parameter for packs.download

60a88a1

Fix & Polish !pack get Action Alias

5cd04e4

arm4b added RFR and removed WIP labels Nov 24, 2016

Remove obsolete config settings for packs pack

16f5249

enykeev changed the title ~~[WIP] Rework the packs pack (more packs-related ChatOps!)~~ Rework the packs pack (more packs-related ChatOps!) Nov 25, 2016

enykeev reviewed Nov 25, 2016

View reviewed changes

arm4b mentioned this pull request Nov 25, 2016

Pack Install should be Idempotent #3052

Open

Remove upgrade flag for packs.install action

4ddbe99

arm4b merged commit 94bc537 into master Nov 25, 2016

arm4b deleted the packs-cleanup branch November 25, 2016 16:45

This was referenced Nov 28, 2016

ChatOps Aliases Unit Tests #3056

Closed

Unit Tests for Actions in packs pack #3057

Open

emedvedev mentioned this pull request Nov 29, 2016

[RFR] StackStorm 2.1 compatibility StackStorm-Exchange/stackstorm-github#1

Closed

cognifloyd removed the RFR label Aug 9, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rework the packs pack (more packs-related ChatOps!) #2982

Rework the packs pack (more packs-related ChatOps!) #2982

emedvedev commented Oct 31, 2016 •

edited by arm4b

emedvedev commented Oct 31, 2016

emedvedev commented Oct 31, 2016 •

edited

codecov-io commented Oct 31, 2016 •

edited

jjm commented Oct 31, 2016 •

edited

emedvedev commented Nov 1, 2016 •

edited

arm4b Nov 23, 2016 •

edited

emedvedev Nov 23, 2016

arm4b commented Nov 24, 2016

emedvedev commented Nov 24, 2016

enykeev commented Nov 25, 2016

enykeev Nov 25, 2016 •

edited

emedvedev Nov 25, 2016

arm4b Nov 25, 2016

arm4b Nov 25, 2016

emedvedev Nov 25, 2016

arm4b Nov 25, 2016

arm4b Nov 25, 2016

emedvedev Nov 25, 2016

arm4b commented Nov 25, 2016

Kami commented Nov 26, 2016

Rework the packs pack (more packs-related ChatOps!) #2982

Rework the packs pack (more packs-related ChatOps!) #2982

Conversation

emedvedev commented Oct 31, 2016 • edited by arm4b

emedvedev commented Oct 31, 2016

emedvedev commented Oct 31, 2016 • edited

codecov-io commented Oct 31, 2016 • edited

Current coverage is 77.11% (diff: 100%)

jjm commented Oct 31, 2016 • edited

emedvedev commented Nov 1, 2016 • edited

arm4b Nov 23, 2016 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

arm4b commented Nov 24, 2016

emedvedev commented Nov 24, 2016

enykeev commented Nov 25, 2016

enykeev Nov 25, 2016 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

arm4b commented Nov 25, 2016

Kami commented Nov 26, 2016

emedvedev commented Oct 31, 2016 •

edited by arm4b

emedvedev commented Oct 31, 2016 •

edited

codecov-io commented Oct 31, 2016 •

edited

jjm commented Oct 31, 2016 •

edited

emedvedev commented Nov 1, 2016 •

edited

arm4b Nov 23, 2016 •

edited

enykeev Nov 25, 2016 •

edited