Define Concurrency Resources Portably

[glassfishrobot]

It is currently not possible to define concurrency resources (context services, managed thread factories, managed executor services, managed scheduled executor services, etc) via standard annotations or XML. This is a potential usability issue, especially in cloud environments.

It may be very helpful to have portable annotations or XML for this, just like we now have for data sources and JMS resources.

Do let me know if anything needs to be explained further - I am happy to help.

Please note that these are purely my personal views and certainly not of Oracle's as a company.

[glassfishrobot]

@glassfishrobot Commented
Reported by reza_rahman

[glassfishrobot]

@glassfishrobot Commented
arjan_t said:
Very much in favor of this!

Maybe the Java EE spec should even go one step further and set as a general rule that for every administered object introduced by a sub-spec there should be an equivalent portable solution (annotation and element in deployment descriptor). If such a solution is not provided there must be a really good (technical) reason given.

Would such a rule be an option?

[glassfishrobot]

@glassfishrobot Commented
f.rowe said:
We attempted to address this issue by adding the ability to use annotations to inject the Default instances of the four services. Are you looking to configure the propagation performed by the ContextService or something else? Will you provide some examples with associated use cases?

[glassfishrobot]

@glassfishrobot Commented
arungupta said:
Section 3.1.4.2 of the specification define several properties that can be used to configure ManagedExecutorService. It'll be great if these property names can be standardized, and supported in both annotation and web.xml.

[glassfishrobot]

@glassfishrobot Commented
arjan_t said:
What I think people are generally looking for is the ability to define and configure additional instances of the four services beyond the default ones and to configure the default ones in a portable way.

There are several use cases for this. The following holds for just the ability to define and configure the services from within an application archive:

• Portability; definition and configuration of the four services is standardized. It now relies on vendor specific configuration.
• Facilitating cloud deployments. In such scenarios the user may not have access to the container configuration and the only option is deploying an ear or war.

The ability to define more than one instance of any of the four services has as use case:

• Preventing dead locks in a producer / consumer system. Here you often need a separate thread pool for the consumer and a separate one for the producer. I elaborated on this in my comment here: EJB_SPEC-9#345825

And example for the XML version in web.xml or application.xml could be:

<managed-executor-service
      name="java:app/producerExecutorService"
      min-threads="10"
      max-threads="50"
      hung-task-threshold="50000"
      keepalive-time="10000"
   />

(an annotation variant would basically mirror this exactly)

And then inject this application defined instance via:

@Resource(lookup="java:app/producerExecutorService")
private ManagedScheduledExecutorService managedScheduledExecutorService;

[glassfishrobot]

@glassfishrobot Commented
nathan_r said:
The configuration examples in the EE Concurrency spec are just examples of some possible ways a Java EE Provider Provider might go about providing configuration. If a configuration will be standardized, it will need to be more fully thought out.

What is appropriate for configuration at the application level ought to differ from what is appropriate for the server administrator to configure. Specifically, I would point out it's a bad practice for applications to define their own thread pools. Management of server resources like threads should not be the role of the application. While it is already possible for applications to get a thread pool by pooling threads obtained from a ManagedThreadFactory, for example, by supplying one any of the various methods of java.util.concurrent.Executors that create thread pools, we should avoid encouraging the practice any further by putting thread pool settings on application defined resources.

I certainly agree there is some value in applications being able to define managed executors in ways that are particular to the needs of the application.

• The application knows better than anyone else which types of thread context it expects to use on tasks that it schedules. Fred proposed one way of addressing this under https://java.net/jira/browse/CONCURRENCY_EE_SPEC-36 which was to allow thread context propagation settings to be specified as execution properties (on a per task basis). This could alternately be accomplished at the level of the application by allowing an application to define a managed executor with specific thread context types.
• While the application should not own pools of threads, it makes sense that an application might want to limit its use of threads. A "MaxConcurrency" setting would allow for that without actually forcing the creation of a new thread pool.
• A hung task threshold could make sense for the application to define on a managed executor, or alternately as an execution property where it would be granular to the task itself.

[glassfishrobot]

@glassfishrobot Commented
arjan_t said:

Specifically, I would point out it's a bad practice for applications to define their own thread pools.

I would like to respectively disagree with this. It highly depends on the application type, the deployment scenario and the team setup.

In one team where I participated we developed a highly concurrent datafeeds processing system, internally using a consumer/producer pattern. We were in a small devops team and deployed a single application to a single AS. In that case it would not have made sense to hand over this job to any kind of server administrator (which incidentally wasn't there as a separate person). The separate thread pools that we used were all highly tuned and had relations such as one thread pool not allowed to have less threads than an other pool.

I know that you said to agree that there is value in applications being able to define executors, but in general if the platform limits developers out of an idea of enforcing best practices, then developers in practice will pretty much always find a way around it; use non-managed pools, embed an (other) application server with the archive, skip Java EE altogether, etc. I think none of these options are what we should strive for.

At most individual server products could put a limit on what the application is allowed to do, but I strongly belief it's not the Java EE platform's role to enforce what is a best practice for each and every kind of situation.

[glassfishrobot]

@glassfishrobot Commented
reza_rahman said:
I am afraid I think Arjan is very much correct on this. While I understand and acknowledge the traditional J2EE ideas around application development vs. deployment vs. administration roles we've already crossed this bridge in Java EE 6 and Java EE 7 with @DataSourceDefinition, @JmsDestinationDefinition, @JmsConnectionFactoryDefinition and others in the same vein.
Indeed the Java EE 8 survey makes it clear that the community strongly desires being able to define even things like security providers within the application. Part of the equation is that other successful platforms like Spring, Ruby, Play, etc have been doing this sort of thing well for years. Another factor at play here is that movements like DevOps have largely blurred the lines between traditional boundaries of development and administration for many more progressive/agile organizations.

Getting this addressed will very likely be extremely well received by the community and help address what could otherwise be a continued portability, usability and flexibility gap in Java EE
application servers.

[glassfishrobot]

@glassfishrobot Commented
aheusingfeld said:
reza_rahman I absolutely agree with you and Arjan! That's basically where I was heading with my mail to the users list. IMHO we certainly need to find a convenient, non-error-prone way for application developers to define the (semantic) resources their applications need including meaningful configuration defaults. Those configuration values will then have to be exposed via JMX e.g. in a ManagedExecutorService definition to make them changeable and checkable in production.

The main reason for me is that in the future I expect to see more of the scenarios where applications need to run in different environments e.g. on a local developer machine in a docker container, on a bare metal Q&A system and a fully virtualized pre-production or production environment. For sure those environments require different values for concurrency related configuration properties like our ExecutorServices. I'm talking of

• number of incoming requests to handle
• number of parallel calls to backend systems
• number of available CPUs/ processors

all of which differ in the above mentioned environments.

but I strongly belief it's not the Java EE platform's role to enforce what is a best practice for each and every kind of situation.

Same for me. Probably none of us can predict which environments Java EE customers will have in 5-8 years and what constraints they bring along but it'd be great if the customers could still run Java EE 8 servers on them, wouldn't it?

[glassfishrobot]

@glassfishrobot Commented
nathan_r said:
I'm clearly outnumbered, so I will try to go along with the proposal. But it needs more elaboration and it needs to be flexible enough for scenarios where the application wants the Java EE Product Provider to auto tune thread pooling, meaning it must avoid defaulting specific values for max and min.

We also need to consider that not every task submitted to an executor necessarily runs on a thread pool. For example, in the case of invokeAll(task1, task2), task1 might run on a pooled thread and task2 might run on the submitter's thread. Or maybe a task that hasn't started yet runs on the thread from which future.get() is attempted rather than a pooled thread. This is why establishing a PoolInfo:MaximumSize (section 3.1.4.2 example) doesn't address the usage pattern, whereas a Max Concurrency does.

Here are some more concrete examples of annotatively defining managed executors/thread factories/context service along these lines,

@ContextServiceDefinition(
name="java:comp/env/concurrent/MyContext",
context =

{ ContextServiceDefinition.CLASSLOADER_CONTEXT, ContextServiceDefinition.SECURITY_CONTEXT }

)

@ManagedExecutorDefinition(
name = "java:comp/env/concurrent/MyExecutor1",
contextService = "java:comp/env/concurrent/MyContext",
maxConcurrency = 20,
minConcurrency = 5
)

@ManagedScheduledExecutorDefinition(
name = "java:comp/env/concurrent/MyExecutor2",
contextService = "java:comp/env/concurrent/MyContext",
maxConcurrency = 10,
minConcurrency = 0
)

@ManagedThreadFactoryDefinition(
name="java:comp/env/concurrent/MyThreadFactory",
contextService="java:comp/env/concurrent/MyContext",
priority=5
)

We need more discussion about the proper granularity of hung-task-threshold. Is it better defined as a single value for all usage of an executor, or on a per task basis (via an execution property like ManagedTask.LONGRUNNING_HINT). Or should it be defined in both places, in which case the more granular setting takes precedence if specified.

The keepalive-time attribute seems like an implementation detail that would preclude otherwise valid implementations of EE Concurrency that choose not to use thread pools, or that choose to have multiple managed executors sharing thread pools. Maybe we could include a 'properties' attribute along the lines of what @DataSourceDefinition does for implementation specific properties.

I'm confused about "number of available CPUs/ processors" as a configurable setting. It would make more sense to me if we provided this information to the application rather than received it from the application. Was that the intention?

Are there other attributes that ought to be included?

[glassfishrobot]

@glassfishrobot Commented
This issue was imported from java.net JIRA CONCURRENCY_EE_SPEC-38

[glassfishrobot]

• Issue Imported From: https://github.com/javaee/concurrency-ee-spec/issues/38
• Original Issue Raised By:@glassfishrobot
• Original Issue Assigned To: @alai8

[njr-11]

I agree this would be a nice addition to the spec, but when adding it please remember to take into account that the EE Concurrency Utilities spec intentionally avoids making any requirement for the implementation to use separate thread pools for each managed executor. Some implementations, for example, OpenLiberty, have a single shared thread pool for the entire server and instead allow for the configuration of concurrency constraints for managed executors. Any configuration that is added by the spec should take care to be compatible with the various implementations in existence to ensure that it does not force any onerous changes upon them.

[aguibert]

@arjantijms @m-reza-rahman In MicroProfile Concurrency we have introduced a way to configure ManagedExecutor instances (essentially a ManagedExecutorService) via annotation with CDI, for example:

@Inject
@ManagedExecutorConfig(maxAsync = 5, maxQueued = 5)
ManagedExecutor exec;

Our intent with MicroProfile Concurrency has been to design it in a way that is compatible with EE Concurrency, in the hope that we can eventually fold the MP Concurrency functionality into a figure version of the EE Concurrency spec.

We encourage your feedback at this stage before the MP Concurrency spec is finalized! You can find the API/doc on GitHub here: https://github.com/eclipse/microprofile-concurrency

[arjantijms]

@aguibert I haven't looked deeply into the MP proposals at all, but just a very quick comment about the above:

Doesn't that create a single use executor?

Almost by definition executors (and their pools) are often defined first at a single place, and then re-used at many other places.

[njr-11]

@arjantijms I can help answer that. The MP Concurrency spec doesn't assume a one-to-one relation between executors and thread pools (and the EE Concurrency spec doesn't currently state a requirement like this either). So, even though executors can be created to be used in a more limited scope, it doesn't imply that a thread pool backing it is limited to that usage. The same thread pool could be backing any number of executors and can still be defined in a single place.

[njr-11]

What was mentioned in earlier comments about what MicroProfile configuration is something that was being considered but was never actually adopted. A simpler, and more flexible, approach with builders was taken by MicroProfile instead. It is documented here:
https://download.eclipse.org/microprofile/microprofile-context-propagation-1.0/microprofile-context-propagation.html#builders

The use of builders allows the application to compute fields that it configures, allows the user to set up overrides via MicroProfile Config if they so wish, and fits in really well with the CDI producer pattern for controlling life cycle and sharing with other beans.

@ApplicationScoped
public class SomeBean {
    @Produces @ApplicationScoped @MyQualifier
    ManagedExecutor executor = ManagedExecutor.builder()
        .maxAsync(10)
        .propagated(ThreadContext.SECURITY, ThreadContext.APPLICATION)
        .build();

    void disposeExecutor(@Disposes @MyQualifier ManagedExecutor exec) {
        exec.shutdownNow();
    }
}

In MicroProfile, the disposer ends up being optional for ApplicationScoped because MicroProfile requires the server to shut down instances that were created by the application upon application stop, but is shown here anyway for a more complete example.

MicroProfile standardized only a small number of configuration attributes (hopefully the most common and useful) so as to not get too far ahead of Java/Jakarta EE and have the best chance of remaining compatible. These deal with listing the types of context to propagate, clear from the executing thread, or ignore (leave unchanged), optionally capping the number of async tasks running at any given point for an executor, and optionally capping the number of async requests that can queue up waiting to start.

[smillidge]

I'd like to pick this up for Jakarta EE 10

[njr-11]

Here are some links to the source for the fluent builder pattern that is used by MicroProfile to accomplish this:

• the managed executor builder

executor = ManagedExecutorService.builder()
          .maxAsync(5)
          .propagated(ThreadContext.SECURITY)
          .build();

• the context service builder equivalent

contextSvc = ContextService.builder()
            .propagated(ThreadContext.SECURITY, ThreadContext.CDI)
            .build();

It should be noted that whether or not this pattern is adopted in Jakarta EE Concurrency is not a prerequisite of Jakarta EE Concurrency being able to adopt other MicroProfile capabilities. I'm mentioning it here to give an overview of what was done in MicroProfile in case Jakarta would like to reuse it.

[njr-11]

Resource definition annotations for ContextService, ManagedExecutorService, ManagedScheduledExecutorService, and ManagedThreadFactory were added to Concurrency 3.0 in the pulls referenced by this issue, including common configuration attributes. If additional configuration attributes are needed, issues can be opened for a follow on release of the Concurrency specification.