refactor: Add settings and plugins attributes to Project

[WillDaSilva]

These changes seem to simplify quite a few things, and also decreased the amount of time it takes to run the tests by approximately 40%, which suggests that these changes improves Meltano's performance.

In a nutshell, since the project instance had to be passed around everywhere, and because of Project.find caching it was basically a singleton, now we make use of it by doing stuff like project.settings instead of ProjectSettingsService(project). This saves us from repeatedly instantiating many "service" classes over and over again (hundreds of times per executions in some cases), which was rather slow. We do need to re-instantiate them to reflect changes to the outside world (e.g. changes to env vars, changes to config files, etc.), so a refresh method has been added to Project which causes it to replace itself with a fresh instance, keeping the same object so that any existing references to it get the new "service" instances it carries.

I've also removed several unused params that would allow us to provide pre-instantiated "service" classes. Each such param multiplies the complexity of the object because it provides a whole new set of behaviors, almost none of which were tested. If one really needs to pass in a class with one of these parameters, we can add it back in later. Until then, I think it should be removed. You aren't gonna need it.

[netlify]

✅ Deploy Preview for meltano canceled.

Name	Link
🔨 Latest commit	5219eaf
🔍 Latest deploy log	https://app.netlify.com/sites/meltano/deploys/63e6786140c3480008ba24d5

[cjohnhanson]

@WillDaSilva Giving a tentative 👍 on the overall approach, pending a more thorough review in the next couple days. I think it makes sense and reduces a lot of unnecessary complexity.

[codecov]

Codecov Report

Merging #7273 (5219eaf) into main (dc04f5e) will decrease coverage by 0.37%.
The diff coverage is 93.45%.

@@            Coverage Diff             @@
##             main    #7273      +/-   ##
==========================================
- Coverage   89.57%   89.20%   -0.37%     
==========================================
  Files         288      287       -1     
  Lines       21497    21326     -171     
  Branches     2373     2365       -8     
==========================================
- Hits        19255    19023     -232     
- Misses       1875     1935      +60     
- Partials      367      368       +1     

Impacted Files	Coverage Δ
src/meltano/api/controllers/root.py	77.33% <ø> (-0.88%)	⬇️
src/meltano/api/events/notification_events.py	0.00% <0.00%> (ø)
src/meltano/api/workers/api_worker.py	37.77% <0.00%> (-1.81%)	⬇️
src/meltano/api/workers/ui_available_worker.py	57.14% <0.00%> (-2.86%)	⬇️
src/meltano/api/wsgi.py	0.00% <0.00%> (ø)
src/meltano/cli/interactive/config.py	28.57% <ø> (ø)
src/meltano/cli/run.py	86.07% <ø> (-0.18%)	⬇️
src/meltano/core/plugin_location_remove.py	38.14% <0.00%> (-52.68%)	⬇️
src/meltano/core/state_service.py	95.71% <ø> (-0.12%)	⬇️
src/meltano/core/validation_service.py	87.75% <ø> (-0.49%)	⬇️
... and 103 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[kgpayne]

@WillDaSilva this is epic 🙌 I've had a look through and overall this is a huge improvement. Kudos! The .refresh() change strikes me as the only minor risk to users - I wonder if our constant refreshing of the settings service might have masked issues with read-write consistency in async or parallel use cases, or in tools/scripts that wrap Meltano (ala Visch). However, as a potential side-effect rather than a feature, I still think it a significant improvement to be more deliberate about when refresh() happens. We should just keep an eye out for potential integration issues in Slack 👍

[cjohnhanson]

One question for my own edification, but looks good. 🚢

[WillDaSilva]

This PR should not be merged until #7294 has been merged, since we'll want to release Meltano 2.15.4 with the changes from #7294, and including these changes in that release would be a risk.

[edgarrmondragon]

@WillDaSilva This is an incredible refactor!

It also feels faster on some meltano config ... commands, which makes sense if we're instantiating fewer things and is a good progression towards addressing our Ravioli.

This LGTM, I just have a few questions.