Constructor injection (was: Annotating Kotlin primary constructor properties)

[rgoldberg]

Support annotating Kotlin primary constructor properties, like:

@Command(…)
class Example(@Parameters(…) private val param)

This would presumably require supporting applying annotations to PARAMETER element types. If implemented, this support should probably support annotating parameters in other use cases, like Java & other language constructor parameters for classes annotated as @Command, parameters for subcommand methods / functions, etc.

This request can be retitled / redescribed as general support for annotating JVM parameters, if the Kotlin primary constrictor property support must indeed be implemented as I outlined above.

[remkop]

@rgoldberg Thank you for raising this.

This topic came up earlier in #1107, #1281 and #1358 and while exploring some ideas, we encountered two major issues:

• dependency injection
• implementation complexity

Dependency injection

Picocli currently supports integration with Dependency Injection containers via the IFactory interface. This allows DI containers to inject services and beans into commands, subcommands, type converters, and any other component.

I would like to be able to use the IFactory for all object instantiation done in picocli.

However, it is not clear to me how a factory implementation (like the the PicocliSpringFactory) would implement constructor injection. To go into some detail, the IFactory interface currently has this signature:

interface IFactory {
  <K> K create(Class<K> clazz) throws Exception;
}

One problem is binary compatibility: this interface does not allow any parameters to be passed other than the class. Would we need to change this interface? That would be bad, but also, what would that new interface look like?

Alternatively, perhaps we can avoid changing the IFactory interface, if we have another interface that allows picocli to somehow "register" the constructor parameters in the DI container context, and the DI container will then "pick" these parameters when it calls the constructor. I am very vague on how that would work, and how, for example, the DI container would be able to determine the correct parameter order when there are multiple parameters of the same type.

Note also that this means changes not just to picocli, but also to factory implementations, like the Spring Boot one, and the Micronaut, and Quarkus ones. These are not insurmountable blockers, but just to clarify the scope.

Potential edge case: what about "hybrid" constructors, that have some picocli-annotated constructor parameters like @Options, but also some other constructor parameters that are services coming from the DI container? So some constructor values would be supplied by picocli and other by the DI container.

Without Dependency Injection

Another alternative is to implement constructor injection of @Options, @Parameters, @Mixins and @ArgGroups only with reflection within picocli, so without invoking the IFactory interface.

Note that any object that is constructed by picocli via reflection cannot have any services injected by the DI container. So that would be the trade-off.

Implementation complexity

Another consideration is that user object creation is already one of the most complex areas of the library. Picocli instantiates subcommands, mixins, argument group classes and various other objects. For reference, some related tickets are [#690] (delayed instantiation), and [#962][#961][#990], about how default values are applied. I am not sure if the complexity is because of how the implementation has grown or whether it is because of the nature of the problem. It is possible that a refactoring can simplify things.

However, any new work in this area will need to deal with this and at the very least not cause any regressions.
This does not mean it cannot be done, it just means that it will require a fairly significant time commitment.

To be honest, at the moment I don't see myself working on this in the near future.

[rgoldberg]

Thanks for the info. I might look into the DI integrations you mentioned in the future (I can't look at it right now).

[rgoldberg]

If it proves difficult to implement the hybrid edge case of some constructor / function arguments being supplied by picocli, and others being supplied directly by DI, you could go with a simple MVP that requires all arguments to be supplied by picocli, if any are.

I imagine that uses of this hybrid edge case would be exceptionally rare.

[remkop]

Yes, the hybrid edge case was more for completeness.
The real problem to solve is how to pass parameters from picocli to the DI container, and then how to control that the DI container would inject constructor parameters in the correct order.

The minimum version would probably be what I mentioned under "Without Dependency Injection" in my previous comment.
This has a trade-off. The negative side of the trade-off is that construction injection would essentially "break" dependency injection integration, and it would not be immediately obvious to picocli users that this is the case...
So, I am not yet convinced that the positive side (the benefits of constructor injection) are worth this potential confusion. 😅

[rgoldberg]

From a naive perspective, without having looked at the picocli code, if picocli can require Java 8 or newer, could you add the following to IFactory?

default <K> K create(Class<K> clazz, Object... args) throws Exception {
    throw new UnsupportedOperationException();
}

default boolean supportsConstructorArguments() {
    return false;
}

The latter could be used by the annotation processor to determine if the current IFactory supports constructor arguments; if it doesn't, then it can output an error.

The arguments will be passed in the order of the args array.

Is there anything that this approach is missing?

Would picocli be unable to determine which argument corresponds with each parameter? Shouldn't that be able to be determined via reflection?

If you must support Java older than 8, then you could just add the overload to a subinterface of IFactory, then check for implementation of that subinterface to determine if constructor args are supported.

Or, if all IFactory implementations are controlled by picocli, just change the arguments to the original create method

[remkop]

Thanks for helping me think through this.
Yes that makes sense. This approach may be a feasible design to solve the dependency injection issue.

[rgoldberg]

Cool. Glad my spitballing was useful. Thanks for making picocli.

[rgoldberg]

If it makes sense to also allow picocli annotations on method parameters, not just constructor parameters, I don't know if you'd still use IFactory, since it wouldn't be creating anything, just calling a method. But, if you do use the same code to support method calls & constructions, then obviously remove "constructor" from any names I suggested, and rename other methods as advisable.

[rgoldberg]

I might look into implementing constructor injection.

A few questions:

1. Does picocli own all the implementations of IFactory? Or is it OK to break third-party implementations of IFactory by changing:

interface IFactory {
  <K> K create(Class<K> clazz) throws Exception;
}

to:

interface IFactory {
  <K> K create(Class<K> clazz, Object... args) throws Exception;
}

2. If not, can picocli require java 8+ so that we can add the 2 default methods suggested in my comment above?

3. Or, should another way be used to implement constructor injection?

[remkop]

1a Does picocli own all the implementations of IFactory?

No, this is a public interface that is an extension point, meant to be implemented by 3rd parties.

1b Or is it OK to break third-party implementations of IFactory

No, we cannot break existing applications. Publishing an extension point and then breaking applications that use it would be a betrayal of trust.

2. can picocli require java 8+

No, I prefer to remain Java 5 compatible. 😬

3. should another way be used to implement constructor injection?

One idea is to introduce a new IFactory2 extends IFactory interface, and do instanceof checking in picocli, there may be other ways.

---

Disclosure:

I have to warn you that I won't be able to spend much, if any, time on this.

Other commitments than picocli are taking more and more of my time recently, so frankly I hesitate to committing to this feature. I worry that it will be a source of follow-up work, while I am personally not yet convinced of the value of this feature.

So any PR would have to be picture-perfect, with docs, tests, etc., and the implementation would need to be simple enough that I am confident that I can maintain it, extend it and fix bugs in the future.

To be perfectly honest, I would rather not do this. 🤔

[rgoldberg]

Thanks for info. I'll put it on the back burner, then. Good luck with other projects.

[remkop]

Thank you for the discussion.
Doing the analysis together was helpful!

I will close this ticket for now.

[rgoldberg]

Do you have a backlog issue label?

That would be useful to remember what issues weren't actually resolved, and that could possibly be revived later when resources are available.

I think this is a good idea that there just aren't resources for right now, rather than being an intrinsically bad idea or a non-issue, which is what I typically associate with wontfix.

[remkop]

You are probably right. I removed the wontfix label.

[rgoldberg]

If I implement this nicely with tests, docs, etc., would you have the bandwidth to review and accept it?

May I call the new interface IFactoryWithArguments instead of IFactory2?

[remkop]

@rgoldberg Thanks for looking at this again.

I have to say, I still have the same reservations as before:

• I am not convinced of the value of this proposal. It seems more a matter of aesthetics than functionality: true, user code may look more elegant when using Java 16-style records or in general immutable objects for commands/argument groups, but is there a functional difference - what becomes possible with this change that is currently not possible?
• I worry that this will increase the complexity of picocli's most fragile functionality (the area where command and argument group objects are instantiated), and that this will increase the maintenance burden in the future.
• I continue to have very little time to spend on picocli.

So, while I don't want to be disrespectful of your time, I cannot guarantee when I will review this or whether your PR will ever get merged; upon seeing it I may decide that I still don't like the trade-off between functionality and complexity and choose not to merge this.

I realize that this is not very encouraging, but I just want to be honest and clear before you invest a lot of time.

[rgoldberg]

Thanks for the info.

I completely understand about limited bandwidth, and I agree about trying to limit complexity.

The functional difference of this feature is to allow Pico CLI users to manage the complexity of their programs without Pico CLI forcing them to use constructs that increase the complexity of the user's codebase.

So it's sorta addressing for Pico CLI users the same concerns you have for Pico CLI itself.

I'll look into this in some spare time to see if it can be managed without adding much complexity to Pico CLI.

I'll let you know if it looks simple, medium, or complex, along with whatever ideas I have.

Probably not amazingly soon, but sometime in the future.

Then, if & when you have the time to look it over, you can assess if it looks acceptable.

[remkop]

Great, it sounds like we are on the same wavelength then.
Thank you for offering to spend time on this.

[rgoldberg]

While looking through the code, I've found some minor issues like:

• javadoc typos (e.g., "whether whether" in javadocs)
• code that could break if libraries return unmodifiable sets (library specs don't specify that returned sets must be modifiable) (e.g., AbstractCommandSpecProcessor#isSubcommand using roundEnv.getElementsAnnotatedWith(Option.class).removeAll(elements))
• code that can easily be simplified making it both more understandable & more performant (e.g., AbstractCommandSpecProcessor#isSubcommand)
• static loggers that aren't final
• etc.

All changes would be localized, simple, Java 6 compatible & pass all tests.

Do you want me to submit PRs?

How should I break them down?

[remkop]

@rgoldberg yes please! Improvements are welcome!

How to break them down: the things you mentioned can all go into one single 'improvements' PR. (Feel free to break them up if you want to, either way is fine.)

That said, if there are any changes that add to or modify the public API, then let's have a separate PR for those. Anything that impacts the public API needs to go into the next minor release (4.7.0), while things that don't impact the API can go into the next bugfix release (4.6.3), so can be merged sooner. I will probably do a 4.6.3 release first.

[rgoldberg]

I noticed that annotating a method with @Command will pass positional parameters in order as the method's arguments.

Could @Command be allowed to be applied to constructors, and use pretty much the same code as is used for methods?

The main differences that I see are that a class can also be annotated with @Command (but a constructor can't be a sub command, unlike a method), and instance fields can be initialized outside a constructor.

The simplest solution would be for annotating a class with @Command to be mutually exclusive with annotating any of its constructors with @Command. If a class is not annotated with @Command, one or more of its constructors may be annotated with @Command, as long as each constructor has a unique command name (possibly taken in conjunction with a parameter count, if picocli can already choose between @Command methods with the same name but a different parameter count; I assume parameter types can't be taken into account, because how can it tell if a command-line 123 is an int, long, String, etc., without making possibly incorrect assumptions?).

The simplest solution could disallow picocli injection annotations on fields if any of the constructors are annotated with @Command. If it isn't too difficult to implement, though, picocli could allow injection annotations on fields even if one or more constructors are annotated with @Command.

@Command constructors could start off with a simpler interface, then eventually allow more complex combos in the future.

[remkop]

Could @Command be allowed to be applied to constructors, and use pretty much the same code as is used for methods?

I had not thought of that point until now: currently, a @Command-annotated method means the method defines a subcommand of the command defined by the enclosing class. Constructors would obviously be different. The implementation needs to be careful because java.lang.reflect.Constructor is a subclass of java.lang.reflect.Executable - the reflection class for methods. Some of the existing logic for methods may not apply to constructors.

By the way, it is not yet clear to me what the user object of the CommandSpec for a @Command-annotated constructor would be. I guess a java.lang.reflect.Constructor instance, but we would need to track whether Constructor.newInstance has been called or not, and any created instances would need to be cached somewhere in the user object.

If a class is not annotated with @Command, one or more of its constructors may be annotated with @Command (...)

I think it would be confusing to allow multiple constructors to be annotated with @Command. Also, there should be only one @Command annotated element, so I would not allow both the class and a constructor being annotated with @Command.

(...) disallow picocli injection annotations on fields if any of the constructors are annotated with @Command.

Yes, I think it would also be confusing to allow @Option, @Parameters or @Mixin-annotated fields in combination with a @Command-annotated constructor.

[rgoldberg]

Sorry. Wasn't able to look at this until now.

It sounds like you're ok with the general principle of allowing @Command on a constructor, as long as all potential conflicts & confusing situations are handled. Correct?

If so, I'll look into this after modifying my existing PR for DefaultFactory#create.

[remkop]

Sorry, I have been extremely busy with Log4j the last few days.

I did not want to leave your question unanswered, but perhaps in answering them I sounded more eager than intended. 😅

My stance essentially has not changed.
To re-iterate my position: I suspect that implementing this will come with a high cost (complexity, maintenance, future enhancements), while the benefits are not clear to me.