New Morphia config feedback (Discord continuation)

[UnfamousThomas]

Is your feature request related to a problem? Please describe.
This is the continuation of the discussion in the Morphia discord (yes, this thing has a discord) with evanchooly related to some scenarios that are now broken because of the new configuration system. I have been asked to make the discussion into an official issue, so this is it.

As of Morphia v2.4, Morphia moved away completely from programmatic configuration, which is somewhat of an issue for bigger projects. I have tried to gather some stories related to it, but if anyone has any extra to add, please do comment on it.
Layering
Consider this scenario: You have a “DatabaseManager” type abstract class setup in a “commons” module, and you want to expand it in the “core” module. Now, this becomes quite difficult. While it is relatively fine if the objects are the same in each module, they often are not.

In my case, the DatabaseManager has an abstract method that we use to add more classes to map to Morphia as needed by each module. This is basically impossible with the current META-INF type configuration.

Difficulty of setup
Honestly speaking, I’d like Morphia to remain beginner friendly - I do not think the new setup is that. For a beginner, having to figure out how to set up the build system so the file is included in a jar and what values to set in the configuration file, is a lot more difficult than the previous system. The documentation is also not great yet. Morphia should remain beginner-friendly, since it seems a lot of uses are related to Minecraft plugins, and a lot of the developers there are not the most experienced.

Proper updating guidelines
I feel like it might be a good idea to write some proper guidelines for updating from the pre-config era, and what to keep in mind. What errors to expect, what could solve them and so on. Right now, going from the programmatical approach to config approach is quite a big jump for some people.

Describe the solution you'd like
As discussed in the Morphia discord, we agreed that we should keep the principle of immutability, but we should include a programmatic way to set up configuration, or even revert to the previous system.

Describe alternatives you've considered
There might be some way to find some middle-path, but for now I have no idea. For now, evanchooly is planning to deprecate MapperOptions, but this is not a permanent solution.

Additional context
I’m including some of the important bits of the conversation that took place in the Morphia discord, if anyone wants more context, feel free to join the Discord and read it more a bit.

evanchooly: “hrm. curious. i might have to rethink all that then. in the tests i have a MutableMorphiaConfig to build various configs on the fly. i might have to expose that in core. i thought it'd be an easy win but it seems that runtime config is used in ways i had no idea on.”

UnfamousThomas: “Something like MutableMorphiaConfig would be really good.

I am currently also really not liking the really forcing of non-programmatical solutions, e.g in my case I have all my Mongo basic stuff setup in commons and then other modules, e.g core can expand functionality, including mapping more classes than defined in commons and such. With the new non-programmatical approach I am not sure if (at least currently) I can even do something like that.”

evanchooly: “yeah, that's a case i need to explore more. microprofile-config doesn't seem to support layering like that. At least, not that I can find. I need to rethink the goals/benefits of that config file and see what can be salvaged.
layering gets hard because how do tell between a defaulted value an an empty entry? but with a custom config datasource i might be able to fake that.
i'm not sure the complexity is worth it, honestly.
(vs. just sticking with programmatic configs)”

evanchooly: “i do like the MorphiaConfig interface better than MapperOptions for various reasons, though, so perhaps that mutable version is the path forward. (I like that the config is fundamentally immutable, e.g., once it's built. and the api is nicer, imo.)”

UnfamousThomas: “Right, the immutable idea is fine, but I still feel like there should be a programmatical approach to it as well, something like not being able to change stuff after calling some build()-type method perhaps? Not sure. I have not touched the API yet, but from what I could see, did look nicer overall, even if it will be a pain updating my SimpleDAO class haha.”

I got some feedback about this being accurate from a few industry friends who use it, but if you have something to add, ask feel free to do so here or on Discord. If anyone has any more feedback related to the new config-system, please post it here, so that it is easier to see in one place.

[evanchooly]

For those not on discord but following along at home, I've done a significant bit of work rethinking the config stuff in master/3.0 where I can break things without too much worry. :) the new docs can be found here with links to the javadoc for MorphiaConfig. How is that looking for all you dynamic/external config types?

[davidgubler]

Hi @evanchooly, not sure if this provides any new insighs for you, but here's some input about my use case:

I'm working on a GTFS (public transport timetable) handling tool which needs to be able to import large data sets into new databases on-the-fly (without restarting/reconfiguring the application). This comes with some unique challenges when it comes to database handling:

• There may be dozens or even hundreds of databases (user may switch between them), and I need to be able to connect to them dynamically. The list of available databases changes while the application runs.
• Import process uses different write concern than running application
• Import process wants to insert entities with indexes disabled, and only after the data is imported I want to instruct MongoDB to create the indexes (Datastore.ensureIndexes(), Datastore.ensureCaps())
• Not all entities are mapped to all databases (single, static database for users, dynamic databases for all entities related to timetable data)

All of this works just fine with Morphia 2.4, and I hope the changes in 3.0 will not make any of this functionality unavailable (I can't really tell for sure from the links in your previous comment).

I hope this provides some useful insights. Thanks for your work on Morphia!

David