Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rewrite annotated to apply annotations in certain order #983

Merged
merged 14 commits into from
Nov 16, 2022

Conversation

tchung1118
Copy link
Contributor

@tchung1118 tchung1118 commented Oct 28, 2022

Currently all annotations that are passed to fx.Annotate are applied at once when annotated.Build() is called.
This has led to an issue where lifecycle hook annotations (fx.OnStart and fx.OnStop) do not work properly when
used with other parameter/result annotations. (#937)

This PR splits up existing code that applies annotations and makes annotated.Build() apply each annotation
one by one, mostly in the order that they're passed in. In the process of applying individual annotations in order,
it puts lifecycle hook annotations aside and applies them after all the other annotations are applied.

Also, lifecycle hook annotations have been changed so that it can now take zero or more of the following as their arguments:

  • parameters of the annotated function
  • results of the annotated function
  • an fx.Lifecycle object
  • a context.Context object

In other words, lifecycle hook annotations can no longer pull in extra dependencies outside of things on which
the annotated constructor is dependent, results that the annotated constructor provides, context.Context
object which is injected by Lifecycle, and the Lifecycle object itself.

@codecov
Copy link

codecov bot commented Oct 28, 2022

Codecov Report

Merging #983 (97ecbb9) into master (b2c7f3c) will decrease coverage by 0.25%.
The diff coverage is 98.83%.

@@            Coverage Diff             @@
##           master     #983      +/-   ##
==========================================
- Coverage   98.81%   98.56%   -0.26%     
==========================================
  Files          39       39              
  Lines        1608     1816     +208     
==========================================
+ Hits         1589     1790     +201     
- Misses         14       18       +4     
- Partials        5        8       +3     
Impacted Files Coverage Δ
annotated.go 98.88% <98.83%> (-0.81%) ⬇️
app.go 96.23% <0.00%> (-1.08%) ⬇️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

annotated.go Outdated
return
}
if err := la.scope.Invoke(invokeFn.Interface()); err != nil {
results[0] = reflect.ValueOf(err)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wrap this error for nicer format

Copy link
Contributor

@sywhang sywhang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Took a first pass

annotated.go Outdated Show resolved Hide resolved
annotated.go Outdated Show resolved Hide resolved
annotated.go Outdated Show resolved Hide resolved
annotated.go Outdated Show resolved Hide resolved
annotated.go Outdated Show resolved Hide resolved
annotated.go Outdated Show resolved Hide resolved
annotated.go Outdated Show resolved Hide resolved
annotated.go Outdated Show resolved Hide resolved
annotated.go Outdated Show resolved Hide resolved
annotated.go Outdated Show resolved Hide resolved
annotated.go Outdated Show resolved Hide resolved
annotated.go Outdated Show resolved Hide resolved
annotated.go Outdated Show resolved Hide resolved
annotated.go Outdated Show resolved Hide resolved
annotated.go Outdated
}
if err := scope.Invoke(invokeFn.Interface()); err != nil {
results[0] = reflect.ValueOf(fmt.Errorf("Error invoking hook installer: %w", err))
return
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This return isn't technically necessary. Also, I don't think this reads as well as how you have it currently, but if you combine the ifs into an if / else if, neither return is necessary, can just rely on the one under these two ifs. Not really sure what the preference for this kind of thing is.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can't combine these ifs. While they look like conditionals, they're not conditionals.

These two things are in plain English:

  1. If you try providing the constructor to the scope, and it fails, set the result to an error and return
  2. Then try Invoking this function, and if it fails, set the result to another error and return.

2 is dependent on 1 succeeding so it's not an if-else clause.

Copy link
Contributor

@JacobOaks JacobOaks Nov 2, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand wanting to keep it the way it is. But isn't this technically the same behavior:

if err := scope.Provide(ctor); err != nil {
    results[0] = reflect.ValueOf(fmt.Errorf("Error providing possible parameters for hook installer: %w", err))
} else if err := scope.Invoke(invokeFn.Interface()); err != nil {
    results[0] = reflect.ValueOf(fmt.Errorf("Error invoking hook installer: %w", err))
}
return

Currently:

  1. If providing the constructor to the scope fails, we set result to an error and return.
  2. Otherwise, we try to invoke the function. If this fails, we set the result to an error and return.
  3. Finally, if both of those succeed, we return.

With an if / else if clause:

  1. If providing the constructor to the scope fails, we set the result to an error. We do not interact with the else if, and jump straight over it to the return statement and return
  2. Otherwise, we try invoking the function. If that fails, we set the result to another error. We exit the else if, and return
  3. If both succeed, we return.

Aren't these the same?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, I see what you mean. You're right, it is possible to remove the returns by changing this to else if.

That being said, I don't think this necessarily helps the readability of the code.

Here are some thoughts behind that:

  1. if-else statements are usually mutually exclusive statements. i.e.
if A { 
  // something 
} else if B {
  // another thing
} else {
  // something else 
}

In the code above, we take "A/something", "B/another thing", and "C/something else" are all mutually exclusive statements that don't seem correlated. So if we grouped this together, it makes it seem like Provide and Invoke are mutually exclusive.

  1. That being said, you could argue that the thing that corresponds to A/B/C above in this code is failure cases , which technically are mutually exclusive - you can't fail in both cases.

  2. With these in mind, the code here is doing a sequence of actions, while checking for failures in each action. It's more natural to not group them under an if-else.

You can see examples of code like this in the standard library as well where they check for errors and return early if an error is encountered using series of if statements instead of an if-else statement. Example: https://github.com/golang/go/blob/master/src/internal/poll/fd_unix.go#L369.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah okay. Yeah, I agree it's more readable how it is now. Cool, thanks for the standard library example.

annotated.go Outdated Show resolved Hide resolved
annotated.go Outdated Show resolved Hide resolved
annotated.go Outdated Show resolved Hide resolved
Copy link
Contributor

@sywhang sywhang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM other than some outlying things re: testing and some updated docstring. We can do some more post cleanups in smaller chunks.

annotated.go Outdated Show resolved Hide resolved
annotated.go Show resolved Hide resolved
annotated_test.go Show resolved Hide resolved
annotated_test.go Outdated Show resolved Hide resolved
@tchung1118
Copy link
Contributor Author

Also I just tested, and I think this PR also implemented suggestion mentioned in #936.

@sywhang sywhang mentioned this pull request Nov 16, 2022
@tchung1118 tchung1118 merged commit 5517d2c into uber-go:master Nov 16, 2022
jasonmills pushed a commit to jasonmills/fx that referenced this pull request Nov 17, 2022
Currently all annotations that are passed to `fx.Annotate` are applied
at once when `annotated.Build()` is called.
This has led to an issue where lifecycle hook annotations (`fx.OnStart`
and `fx.OnStop`) do not work properly when
used with other parameter/result annotations. (uber-go#937)

This PR splits up existing code that applies annotations and makes
`annotated.Build()` apply each annotation
one by one, mostly in the order that they're passed in. In the process
of applying individual annotations in order,
it puts lifecycle hook annotations aside and applies them after all the
other annotations are applied.

Also, lifecycle hook annotations have been changed so that it can now
take zero or more of the following as their arguments:
- parameters of the annotated function
- results of the annotated function
- an `fx.Lifecycle` object
- a `context.Context` object

In other words, lifecycle hook annotations can no longer pull in extra
dependencies outside of things on which
the annotated constructor is dependent, results that the annotated
constructor provides, `context.Context`
object which is injected by Lifecycle, and the Lifecycle object itself.
// Only one OnStop annotation may be applied to a given function at a time,
// however functions may be annotated with other types of lifecylce Hooks, such
// as OnStart.
// as OnStop. The hook function passed into OnStop cannot take any arguments
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Misspelled lifecylce.

Also this diff replaces OnStart with OnStop so OnStop is used twice in the sentence. The words "other types" makes me think it should be OnStop followed by OnStart as before this diff.

"Only one OnStop annotation ... however functions may be annotated with other types of lifecylce Hooks, such as OnStop"

Wondertan added a commit to celestiaorg/celestia-node that referenced this pull request Apr 18, 2023
This PR solves two problems:
* Updates FX and fixes issues we observed in [the
PR](#1801)
* Fixes #2041

Surprisingly, those two problems were related, so I decided to fix them
once and for all. The issue with FX happens after [this
change](uber-go/fx#983). The outcome of this
change is summarized:
> In other words, lifecycle hook annotations can no longer pull in extra
> dependencies outside of things on which
> the annotated constructor is dependent, results that the annotated
> constructor provides, context.Context
> object which is injected by Lifecycle, and the Lifecycle object itself.

Our current code does pull extra dependencies in the service having
`modfraud.Lifecycle` on them, like
[DASer](https://github.com/celestiaorg/celestia-node/blob/main/nodebuilder/das/module.go#L47).
Specifically, it pulls FraudService, and this is no longer allowed. This
forces us to rewrite the fraud lifecycling and here is the solution,
which additionally satisfies #2041.

This also unblocks
#2040, which is now
implemented in celestiaorg/go-fraud#1

I tried to split FX update and the refactor into two diff PRs. However,
the solution does not work with the old FX version, so we have to couple
those.

The chain here is that I needed a new version of FX that fixes
`OnStart/OnStop` hooks, and updating created the whole story. The PR
that does clean-ups basing on new version of FX will com right after.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

4 participants