Extract loaders from ipv8_module_catalog.py

[drew2a]

One step closer to the atomic community: introduction of launcher.py

I've extracted IPv8CommunityLauncher for each Tribler's community from ipv8_module_catalog.py and put them into launcher.py next to corresponding communities.

Preconditions

Preconditions have been separated from Launcher definition to Loader implementation:

community_loader.py

def create_default_loader(config: TriblerConfig, tunnel_testnet: bool = False,
                          bandwidth_testnet: bool = False, chant_testnet: bool = False):
    loader = IPv8CommunityLoader()

    loader.set_launcher(IPv8DiscoveryCommunityLauncher())
    loader.set_launcher(DHTCommunityLauncher())

    if bandwidth_testnet:
        from tribler_core.modules.bandwidth_accounting.launcher import BandwidthTestnetCommunityLauncher
        loader.set_launcher(BandwidthTestnetCommunityLauncher())
    else:
        from tribler_core.modules.bandwidth_accounting.launcher import BandwidthCommunityLauncher
        loader.set_launcher(BandwidthCommunityLauncher())
   ...

This allowed the use of top imports inside each loader (which should lead to easier navigation and more convenient work inside IDE).

launcher.py

from ipv8.loader import overlay, set_in_session, walk_strategy
from ipv8.peerdiscovery.discovery import RandomWalk
from tribler_core.modules.community_loader import INFINITE, TriblerCommunityLauncher

from tribler_core.modules.metadata_store.community.sync_strategy import RemovePeers
from tribler_core.modules.popularity.community import PopularityCommunity


@set_in_session('popularity_community')
@overlay(PopularityCommunity)
@walk_strategy(RandomWalk, target_peers=30)
@walk_strategy(RemovePeers, target_peers=INFINITE)
class PopularityCommunityLauncher(TriblerCommunityLauncher):
    def get_kwargs(self, session):
        return {
            'settings': session.config.popularity_community,
            'rqc_settings': session.config.remote_query_community,

            'metadata_store': session.mds,
            'torrent_checker': session.torrent_checker,
        }

Community loader

Community loader can be passed to Session constructor now:

session.py

    def __init__(self, config: TriblerConfig, core_test_mode: bool = False,
                 community_loader: IPv8CommunityLoader = None):
        if not community_loader:
            community_loader = create_default_loader(config)

Therefore you don't need to cheat session to specify your own set of community launchers:

run_bandwith_crawler.py

async def start_crawler(tribler_config):
    # We use our own community loader
    loader = IPv8CommunityLoader()
    loader.set_launcher(BandwidthCommunityCrawlerLauncher())
    session = Session(tribler_config, community_loader=loader)

[qstokkink]

I believe that moving module (/community) definitions to their own files is a good idea. However, I observe that this PR creates more problems than it solves.

I'd strongly suggest closing this PR and drafting a more formal design in an issue first.

[ichorid]

@qstokkink , I hoped to first discuss it with @drew2a , but you guys discussing the problem of modularisation here...
Dependency Injector framework is just what you're looking for. I propose converting the IPv8 modules catalog to it first, then gradually convert all of the Tribler code to it, finishing with the Session itself.

[qstokkink]

@ichorid I agree that this warrants more in-depth discussion and a formal design. This is why I called to move this PR to an issue first, instead of turning this PR into a long discussion.

I have not checked out the Dependency Injector framework. However, from my experience with Python plug-ins, we need to make doubly-sure that whatever we use (e.g., Dependency Injector) is compatible with a frozen environment (as created by PyInstaller). Ideally, we should have a list of all plug-in frameworks and weigh the pro's and con's. We'll be stuck with this for years probably, so this is not something we should decide on a whim.

[qstokkink]

With the shift in issue #1, dropping plugins, none of the points I raised matter anymore.

[qstokkink]

Oops, I was too fast. This one still holds: https://github.com/Tribler/tribler/pull/6182/files#r659146652
I've set the comments that were related to plug-ins to "resolved".

[qstokkink]

Nice 👍 Thanks for sticking with us through an entire lab vision revision.

Some future work notes:

• This breaks Gumby, specifically https://github.com/Tribler/gumby/blob/devel/gumby/modules/gumby_session.py
• The launcher/loader classes are made for plug-ins. Now that we're not doing plug-ins anymore, they can be removed.

[drew2a]

@qstokkink 🙏

I may need to do another PR with Dependency Injection as a result of today's meeting, which will remove the use of launcher/loader from Tribler.

After that, the Gumby can be fixed.

[ichorid]

Continuation of the DependencyInjector discussion: #4953

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
No Duplication information