Switch to a resource-progress oriented view for pulumi preview/update/destroy #1116

CyrusNajmabadi · 2018-04-04T21:17:32Z

No description provided.

…th diagnostics.

ellismg

I think at a minimum this is going to give the old behavior for stacks managed by the service and we'll want to address that before we land. I left a lot of other feedback, but it was mainly style things.

ellismg · 2018-04-09T23:17:31Z

pkg/backend/local/stream.go

+)
+
+// copied from: https://github.com/docker/cli/blob/master/cli/command/out.go
+// replace with usage of that library when we can figure out hte right version story


nit: typo, "teh" should be "the"

ellismg · 2018-04-09T23:22:30Z

cmd/config.go

@@ -206,7 +206,7 @@ func newConfigSetCmd(stack *string) *cobra.Command {
 			// If we saved a plaintext configuration value, and --plaintext was not passed, warn the user.
 			if !secret && !plaintext {
 				cmdutil.Diag().Warningf(
-					diag.Message(
+					diag.Message("", /*urn*/


Do you think it is worthwhile to have diag.Message and diag.URNMessage, so we can centralize the empty URN dance to a single location instead of throwing it in every call site?

@ellismg I'm on the fence. One of the nice things about this approach was that it forced me to go look at every usage of Message to ensure that we at least think about what we're trying to pass along. However, now that we have examined all usages, we can now try to make this easier to use in teh future. However, part of me still likes making the caller have to think about this sort of thing.

ellismg · 2018-04-09T23:27:16Z

pkg/backend/cloud/state.go

@@ -184,10 +186,9 @@ func (u *cloudUpdate) RecordAndDisplayEvents(action string,

 	// Start the local display processor.
 	displayEvents := make(chan engine.Event)
-	go local.DisplayEvents(action, displayEvents, done, debug, opts)
+	go local.DisplayDiffEvents(action, displayEvents, done, debug, opts)


I think you want to change this line so that it defaults to DisplayProgressEvents or DisplayDiffEvents depending on the command line. This call here is the thing that is showing messages locally, so as written I think we'll continue to use the old style for hosted stacks, which is not what we want.

Gotcha. Will fix.

ellismg · 2018-04-09T23:42:18Z

pkg/backend/local/progress.go

+var (
+	// simple regex to take our names like "aws:function:Function" and convert to
+	// "aws:Function"
+	typeNameRegex = regexp.MustCompile("^(.*):(.*):(.*)$")


I think because of greedy matching, if presented with something like:

one:two:three:four

We'll end up giving back one:two:three:four, we'll end up giving back one:two:four which is unlikely what we want.

We can either use non-greedy matching (e.g: regexp.MustCompile("^(.*?):(.*?):(.*?)$")) or perhaps use the methods on the tokens.Type object?

If it's not possible to have a tokens.Type that has more than two colons, I guess we are safe. I'm not sure if such a thing is possible today.

Using the methods may still be nicer anyway, since we don't have to convert to a string and then regexp.

ellismg · 2018-04-09T23:42:58Z

pkg/backend/local/progress.go

+		return payload.URN
+	}
+
+	return ""


Given the above comment, it sounds like we should never reach this case? If so, can we add an assert?

fixed comment. it was wrong.

ellismg · 2018-04-09T23:47:39Z

pkg/backend/local/progress.go

+
+		// Once we've written everything to the pipe, we're done with it can let it go.
+		err := pipeWriter.Close()
+		if err != nil {


NIT: if we are ignoring the error anyway, don't have this if condition. contract.IgnoreError does the right thing when the error is nil.

done. thanks!

ellismg · 2018-04-09T23:49:48Z

pkg/backend/local/progress.go

+	}
+
+	printStatusForTopLevelResource := func(status Status) {
+		writeAction := func(msg string) {


Can we restructure this such that we don't have a local method that itself has a local function? e.g. could we move the code that follows this up to before it and have it just set a msg local that the rest of the function uses?

ellismg · 2018-04-09T23:53:08Z

pkg/backend/local/progress.go

+
+func renderProgressEventWorker(
+	event engine.Event, seen map[resource.URN]engine.StepEventMetadata,
+	debug bool, opts backend.DisplayOptions, isPreview bool) string {


Should we try to push the debug flag into the backend.DisplayOptions bag?

yes. i would like that. i'm not a fan that in most of the engine it seems like we have 2-3 places to get options (often several parameters, and sometimes off of the 'this' object).

ellismg · 2018-04-09T23:55:29Z

pkg/backend/local/progress.go

+	isPreview bool, isComplete bool) string {
+
+	out := &bytes.Buffer{}
+	summary := getMetadataSummaryWorker(metadata, isPreview, isComplete)


FWIW, I think the Worker suffix here is confusing (I would expect the worker to actually do some long running operation). Can we just inline the worker into getMetadataSummary? Similar feedback for the other Worker methods you've introduced. They only ever look like they are called from one place and I'd personally find it clearer if the code was inline.

ellismg · 2018-04-09T23:57:36Z

pkg/backend/local/progress.go

+	return op.Prefix() + getStepCompleteDescriptionNoColor(op) + colors.Reset
+}
+
+func getStepDescription(op deploy.StepOp, isPreview bool) string {


Again, my preference would be to inline this into the above (assuming it's the only place we call this, which it looks like is the case)

CyrusNajmabadi · 2018-04-10T19:03:04Z

Merging in. Let me know if there are any changes you want and let me know how things feel. Note: if you want the old experience, just pass --diff to "pulumi update/preview/destroy", othewise you'll get the new expeirence.

11122: [SDK/Python] Respect --parallel flag r=RobbieMcKinstry a=RobbieMcKinstry  # Description This PR sets the default futures executor to use the `--parallel` flag to determine thread count.  Fixes #1116 ## Checklist  - [x] I have added tests that prove my fix is effective or that my feature works. **(Not unit tests, but I ran a thorough experiment.)**  - [x] I have run `make changelog` and committed the `changelog/pending/<file>` documenting my change  - [ ] Yes, there are changes in this PR that warrants bumping the Pulumi Service API version  Co-authored-by: Robbie McKinstry <robbie@pulumi.com>

Cyrus Najmabadi added 28 commits March 30, 2018 18:37

require a resource urn when making diag events.

696f3e2

Reduce duplication.

fb2a0f6

Allow information about the associated resource to be passed along wi…

5727e71

…th diagnostics.

Runtime error probably can't be parented.

1540758

Include diag URN info in diag event.

0e760ed

Getting there.

64b0293

Lipstick.

e58c87a

Liptstick all the things.

e2408cf

Formatting.

ca2b994

Formatting.

4a3a3e9

Summary view.

f9e73d8

Revert this.

f2ce175

Simplify.

a155dfc

More tweaking.

89f3814

Align output.:

7087820

Ensure unique id names.

339f584

Feedback.

01210c2

Cleanup

708ffdd

Getting closer.

d882d17

Do not serialize these functions.

df4145f

Simplify code.

e00e2ca

Linting.

efed00d

Linting.

956e9a2

Break out helpers into separate file.

5673cda

Simplify.

d6553c6

Comments.

ec43433

Comments.

f441510

Cleanup

45634fa

CyrusNajmabadi changed the title ~~WIP: do not review~~ Switch to a resource-progress oriented view for pulumi preview/update/destroy Apr 7, 2018

Merge remote-tracking branch 'origin/master' into commandOutput2

02d52c1

CyrusNajmabadi requested a review from ellismg April 9, 2018 19:17

Cyrus Najmabadi added 6 commits April 9, 2018 12:22

Merge branch 'master' of github.com:pulumi/pulumi

33aea8c

Merge branch 'master' into commandOutput2

723c00e

Simplify logging api.

28cac50

Merge remote-tracking branch 'origin/master' into commandOutput2

f9e879d

Remove line.

16d5c3b

Linter.

732237a

CyrusNajmabadi requested a review from joeduffy April 9, 2018 20:55

Cyrus Najmabadi added 2 commits April 9, 2018 13:56

move code back.

3027137

Move helper.

b6db73a

ellismg suggested changes Apr 10, 2018

View reviewed changes

Cyrus Najmabadi added 11 commits April 10, 2018 10:38

Merge remote-tracking branch 'origin/master' into commandOutput2

f7cd584

Fix spelling.

ee37171

Local display for the cloud backend should follow user options as well.

7ac0682

Fix comment.

3014829

remove dead code.

242b23c

Remove unnecessary if.

7dc90ea

Move out local function.

f75fc75

Inline method.

8afa77a

inline functions.

27a2218

Centralize where we colorize.

08cab0d

Make 'debug' a display flag as well.

eca9ba1

CyrusNajmabadi merged commit a759f2e into master Apr 10, 2018

CyrusNajmabadi deleted the commandOutput2 branch April 10, 2018 19:03

lindydonna added the impact/changelog label Apr 14, 2018

mmdriley mentioned this pull request Jun 5, 2018

Pulumi crashes if the same error is encountered twice #1464

Closed

RobbieMcKinstry mentioned this pull request Oct 22, 2022

[SDK/Python] Respect --parallel flag #11122

Merged

3 tasks

RobbieMcKinstry mentioned this pull request Oct 24, 2022

[AutomationAPI/Python] Confirm Parallelism Flag Respected #11123

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Switch to a resource-progress oriented view for pulumi preview/update/destroy #1116

Switch to a resource-progress oriented view for pulumi preview/update/destroy #1116

CyrusNajmabadi commented Apr 4, 2018

ellismg left a comment

ellismg Apr 9, 2018

ellismg Apr 9, 2018

CyrusNajmabadi Apr 10, 2018

ellismg Apr 9, 2018

CyrusNajmabadi Apr 10, 2018

ellismg Apr 9, 2018

ellismg Apr 9, 2018

CyrusNajmabadi Apr 10, 2018

ellismg Apr 9, 2018

CyrusNajmabadi Apr 10, 2018

ellismg Apr 9, 2018

CyrusNajmabadi Apr 10, 2018

ellismg Apr 9, 2018

CyrusNajmabadi Apr 10, 2018

ellismg Apr 9, 2018

CyrusNajmabadi Apr 10, 2018

ellismg Apr 9, 2018

CyrusNajmabadi Apr 10, 2018

CyrusNajmabadi commented Apr 10, 2018

Switch to a resource-progress oriented view for pulumi preview/update/destroy #1116

Switch to a resource-progress oriented view for pulumi preview/update/destroy #1116

Conversation

CyrusNajmabadi commented Apr 4, 2018

ellismg left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

CyrusNajmabadi commented Apr 10, 2018