Should Jakarta Config Specify A Programmatic API?

[Emily-Jiang]

Do we want a programmtic API defined by this spec?

[radcortez]

Yes.

[ljnelson]

I see "configuration" {waves hands} as a spectrum (Github will probably make you click on this diagram to zoom in):

The Jakarta Config specification can choose to operate anywhere in this spectrum.

This discussion asks the question: "Should the specification operate in the yellow/left end of the spectrum?" I think the answer is yes.

More details:

• On the "left" side of the spectrum (yellow), you have Java-centric retrieval concerns. A component developer says "get me some Java object" (which may be as simple as a String or as complicated as an object graph; that's not up for discussion here). (That's the programmatic API, or "loading", end of the spectrum. Note that "programmatic API" could also encompass injection depending on how you read this; I'm staying high-level here so would include that as well on this end of the spectrum.)
• On the "right" side of the spectrum (cyan), you have file- and syntax-centric authoring concerns. A configuration author bakes some settings into a file somewhere. (This is the non-programmatic, not-really-Java-centric end of the spectrum; think MIME-typed files.)
• In the middle (orange) are various concerns related loosely to object binding ("OK, a component asked for some Java object, and ah, I see the configuration author created a file in the right place, so now I, the infrastructure, have to marry them together after normalization"). (This is the "normalization and binding" squishy middle of the spectrum.)

So:

I personally think this specification does need to define a "skinny" programmatic API (the "yellow" end of the spectrum) that satisfies various goals (to be decided/corralled somewhere else). So yes.

[dmlloyd]

For readers, I think this vision and my comments below are not mutually exclusive. A "programmatic API" which is limited to a means to acquire the fully-formed configuration object mapping I described (without ad-hoc access to the "raw" configuration model "DOM") would be the limit for what I would consider to be an acceptable solution - however, I would also ask that there be a specific use case justification for having such an API in the specification. One example that we discussed on a call was usage within a CDI portable extension.

[ljnelson]

I agree. I see your comments as primarily "playing" in the cyan area of the spectrum above.

[radcortez]

An API to retrieve a configuration object mapping is necessary. Maybe we should open this up as a topic as well, but do we see Config only to be used in the context of a Jakarta runtime or platform, or do we see it as a standalone specification so that it can be used without the need of a Jakarta runtime?

Even in a Jakarta Runtime or some Jakarta specifications, CDI may not be available, so we cannot rely on injection to retrieve the object mapping. To name a few:

• JBatch can be used in a Java SE only environment. It even defined its own configuration annotation with @BatchProperty
• Bean Validation can be used standalone without CDI. Should a user be able to retrieve configuration from a custom validator for instance?
• While unlikely, JAX-RS does not require CDI either (but most people use them together=
• JAX-RS Client and MP REST Client

So yes, I think we need an API to retrieve the mapping programmatically.

[dmlloyd]

No.

(N.B. I'm understanding "programmatic API" to mean an API responsible for getting "raw" configuration properties, and potentially building up configurations as well. This does not include concepts such as MP Config's converters or configuration sources, which I would think of as "SPI".)

Let's examine our use cases, imagining a world without a programmatic API, instead relying on configuration object mapping for user interaction.

Users who want to configure their Jakarta applications

The user has a single view of what their configuration is. By only allowing users to express their configuration as a bound object graph (aka "configuration object model" or just "object model") which reflects that view - strongly typed, strongly schema'd, we get several benefits:

• We can validate the configuration not only for well-formedness, but also for:
  • Type safety - by ensuring that each of the provided configuration values can be converted into their corresponding single Java type
  • Semantic validity - by allowing the user to declare additional validation rules and constraints on each property
  • Completeness - by allowing the user to express that some configuration properties are required for correct application function
  • Misnamed keys - by detecting keys which do not correspond to the validated configuration
• We can provide a complete and validated configuration to the user as an atomic unit; if the configuration is not valid, then the application does not start
• We prevent the user from introducing inconsistent behaviors; for example, with a programmatic API the user can read the same property using different type conversions or validation rules, even swallowing or contriving exceptions, which creates a confusing and inconsistent experience for the person in the "configurator" role who can no longer presume to know the exact type and semantics of a given configuration property based on a single source of truth
• We can provide a single source of configuration documentation - and potentially tooling - using existing syntax and technologies such as JavaDoc. Quarkus and others have already proven the feasibility of this.

Implementors of this specification

The landscape of Jakarta implementations is very diverse today, with different philosophies around program execution and lifecycle as well as deployment and reconfigurability. A programmatic API which works for one vendor is not necessarily going to work for another. I do not believe the currently proposed programmatic API is a clean fit for Quarkus for example. Therefore I do not think it's likely that we would reach anything near a consensus (by vote) on any one programmatic API - and if we did, I think it would likely end up being the most inconsistent, contentious, and problematic aspect of the specification after release.

There will almost certainly be vendor-specific APIs for programmatic configuration access. These APIs would commonly be used as the foundation for supporting the production of injectable configuration object models. However, I feel confident in light of the last year's work that the APIs for each vendor implementation may be quite different from one another.

Sibling Jakarta specification configuration consumers

Most (if not all) of the arguments put forth in the "Users who want to configure their Jakarta applications" heading above will apply here as well, with some additional benefits.

• We could, in the future, potentially make it a standard practice for sibling specifications to provide, as a part of the specification delivery, the configuration object model classes for that specification, consequently providing the corresponding documentation as well.
• This could conceivably open a road towards phasing out specification-specific descriptors in favor of standardized configuration blocks in a more modern, single-configuration arrangement. A step along this road could include describing how existing descriptor components map to the standardized configuration properties. We could also, for example, standardize the priority relationship between the unified Jakarta configuration file(s) and the would-be-legacy existing descriptors, perhaps even folding those descriptors into the unified parsing model as specialized configuration sources in some uniform way.
• As sibling specifications move towards this unified configuration model, things like diagnostic reporting of deployment problems become more uniform instead of being implemented by each specification implementation on an ad-hoc basis.

Conclusion

Many of these ambitions can only ever be fully realized if configuration has a strict schema-driven underlying document model. Most of these validations can only be achieved if there are no "back doors" to reinterpreting configuration in an ad hoc manner. A programmatic API undermines the potential of this specification and reduces it to little more than shared access to a HashMap.

[ljnelson]

(Of note/prior art/examples in the wild/etc.: Dropwizard does something close to this sort of thing (i.e. modeling configuration, ultimately, as a "bound object graph", as you put it). You must create a configuration class for your application, and your application doesn't start unless such an object can be built and delivered to its run method. That's all the access you, as an application developer, get, so you are forced to fully model your configuration as a Java object.)

[radcortez]

I'm a big fan of this model, and I think this is the way to go, but we need at least one API to retrieve the object.

[ljnelson]

Right. Dropwizard is not a standard so it can just define how its applications start, i.e. it can just mandate that its applications get handed configuration objects as part of the startup sequence. With Jakarta EE, there is no such sequence. (In the old days, you maybe could do something with EJB startup, ServletContextListener, etc. etc. but certainly with the Core Profile there's nothing.) So yes, I think we need a way to get a/an/the configuration object(s), whatever shape they take (which is not the subject of this discussion) programmatically.

The moment such a "pull-based" API, no matter how simple, enters the picture, you must answer the question: what happens if someone calls it twice in a row with the same arguments? What, if anything, is guaranteed of the two invocations? That is not the subject of this discussion, but might be part of the subject of #109.

[radcortez]

The way I see it, the pull API is just retrieval of an instance already created, so you will get the same instance on every invocation (using the same arguments).

These mappings instances are populated during Config init, where configuration is retrieved from multiple sources, validated, etc. The question here is, when does that happen in the startup sequence?

[ljnelson]

(These are excellent subjects for discussion, but not here (#110) as I understand it. Just trying to keep the discussion managed, which is why I pointed at #109.)

[hantsy]

Like the Helidon Config, a programmatic Configuration builder is useful for the case in a Jakarta EE client application.

[m0mus]

My understanding is that there are two understandings of programmatic API features:

1. Programmatic API gives users abilities to access configuration DOM model. It's similar to what MicroProfile Config does.
2. Programmatic API is used only to get configuration object if injection is not available. It doesn't allow access to the internal DOM model. In fact, the DOM model is implementation specific.

If my understanding is correct, I am voting for Option 2 for MVP.

[radcortez]

Agree

[ljnelson]

Number 2 for sure.

[ljnelson]

From our last meeting, there was a general desire to start "doing PRs" in this area. I came up with three mutually exclusive options/suggestions: #131, #132, and #133.

I still think it is too early to do this sort of thing but since the consensus was to start writing code early, I guess, I've sketched them out.

[m0mus]

Thanks @ljnelson for doing great job and creating PRs. IMO options 2 and 3 are created more for spec developers (us) convenience than for users. Getting a configuration objects becomes too wordy.

Option 1 (#131):

MyConfig config = Loader.load(MyConfig.class);

Option 2 (#132):

MyConfig config = Loader.load(Request.builder(MyConfig.class).build());

Option 3 (#133):

MyConfig config = Loader.load(Request.builder(MyConfig.class).build()).get();

I see option 1 as the simplest and the clearest.

[ljnelson]

If we remove TypeToken from Option 1 and the load(TypeToken) method...would that constitute something appropriately minimal? Or does "far more" mean something else? (I introduced TypeToken because Roberto said that a configuration-related object could be of any (interface) type. It sounds like instead perhaps you would claim that a configuration-related object must be of a non-generic interface type.)

[dmlloyd]

It would, IMO. I think that clarifying that the interface must be non-generic is appropriate.

[ljnelson]

@m0mus Of course, option 2 and option 3 can have convenience methods; I just didn't add them. I focused on the "real" methods.

For example:

// option 2
public default <T> T load(Class<T> c) {
  return load(Request.builder(c));
}

// option 3
public default <T> T load(Class<T> c) {
  return load(Request.builder(c)).get();
}

[ljnelson]

It would, IMO. I think that clarifying that the interface must be non-generic is appropriate.

OK, I'll start a new discussion (just so it's linkable). See #134.

[radcortez]

because Roberto said that a configuration-related object could be of any (interface) type.

Sorry, I was not clear enough about generics there (not even remember about them), but yes I non-generic interfaces :)

My preference here is Option 1, without the TypeToken. Or I at least would like to have the use cases defined for that.