Blacklist not working in 3.2.0

[RWoodring79]

I set up a PyPI mirror using bandersnatch v3.2.0 and a blacklist generated by pypistats. I let bandersnatch mirror download all weekend while the office traffic on the network was minimal. My local mirror is now over a terabyte, but the downloads were only into packages starting with the letter k. When I search my mirror folder, I am seeing blacklisted packages that have been downloaded. I restarted bandersnatch so I could see any plugin initialization messages and I see the whitelist plugin being initialized (even though I do not have one in my .conf file) but not the blacklist.

Here are the first 20 lines that bandersnatch spits out.
2019-02-12 07:48:37,171 INFO: bandersnatch/3.2.0 (cpython 3.6.7-final0, Linux x86_64)
2019-02-12 07:48:37,238 INFO: Status file /mnt/mirror/pypi/status missing. Starting over.
2019-02-12 07:48:37,238 INFO: Syncing with https://pypi.org.
2019-02-12 07:48:37,238 INFO: Current mirror serial: 0
2019-02-12 07:48:37,238 INFO: Resuming interrupted sync from local todo list.
2019-02-12 07:48:37,586 INFO: Initialized project plugin 'whitelist_project', filtering []
2019-02-12 07:48:37,589 INFO: No project filters are enabled. Skipping filtering
2019-02-12 07:48:37,589 INFO: Trying to reach serial: 4796058
2019-02-12 07:48:37,589 INFO: 85976 packages to sync.
2019-02-12 07:48:38,151 INFO: Syncing package: 2 (serial 1386393)
2019-02-12 07:48:38,151 INFO: Syncing package: 2dfly-manbanzhen (serial 4165025)
2019-02-12 07:48:38,155 INFO: Syncing package: 3d_bin_container_packing (serial 4047427)
2019-02-12 07:48:38,499 INFO: 3d_bin_container_packing no longer exists on PyPI
2019-02-12 07:48:38,505 INFO: Syncing package: ACPI (serial 4097070)
2019-02-12 07:48:38,527 INFO: 2dfly-manbanzhen no longer exists on PyPI
2019-02-12 07:48:38,528 INFO: 2 no longer exists on PyPI
2019-02-12 07:48:38,528 INFO: Syncing package: AMON-py (serial 4188130)
2019-02-12 07:48:38,529 INFO: Syncing package: Abaqus-RunINPFiles (serial 4490341)
2019-02-12 07:48:38,545 INFO: ACPI no longer exists on PyPI
2019-02-12 07:48:38,551 INFO: Syncing package: Accengage (serial 4006324)

I also see a large number of the following message throughout my log.
2019-02-12 07:48:51,073 INFO: Initialized prerelease plugin with [re.compile('.+rc\d$'), re.compile('.+a(lpha)?\d$'), re.compile('.+b(eta)?\d$')]

I have attached my .conf file for reference
bandersnatch.conf.txt

[cooperlees]

I believe backlist is working, and this is probably a documentation fail. Sorry.

You need to add a line to your conf - #142 made it no longer implicitly enabled and you have to name the plugins you want. So you need:

[blacklist]
plugins =
    blacklist_project
packages =
    CodeIntel
    ...

In regards to

2019-02-12 07:48:51,073 INFO: Initialized prerelease plugin with [re.compile('.+rc\d$'), re.compile('.+a(lpha)?\d$'), re.compile('.+b(eta)?\d$')]

• I've opened 3.2.0 Still see prerelease_name log messages #147 for this - But have not got to it. PR's welcome

[RWoodring79]

Thank you very much. I still do not see any mention of initializing the Blacklist filter, but I definitely see that it is working now.

2019-02-12 12:53:06,458 INFO: bandersnatch/3.2.0 (cpython 3.6.7-final0, Linux x86_64)
2019-02-12 12:53:06,472 INFO: Syncing with https://pypi.org.
2019-02-12 12:53:06,473 INFO: Current mirror serial: 4810763
2019-02-12 12:53:06,473 INFO: Syncing based on changelog.
2019-02-12 12:53:07,172 INFO: Initialized project plugin 'whitelist_project', filtering []
2019-02-12 12:53:07,175 INFO: Package 'bohrium-api' is blacklisted
2019-02-12 12:53:07,175 INFO: Package 'bohrium' is blacklisted
2019-02-12 12:53:07,176 INFO: Package 'tf-nightly' is blacklisted
2019-02-12 12:53:07,176 INFO: Package 'tf-nightly-2.0-preview' is blacklisted
2019-02-12 12:53:07,176 INFO: Package 'tf-nightly-gpu' is blacklisted
2019-02-12 12:53:07,176 INFO: Package 'tf-nightly-gpu-2.0-preview' is blacklisted
2019-02-12 12:53:07,176 INFO: Package 'ccxt' is blacklisted
2019-02-12 12:53:07,176 INFO: Package 'tendenci' is blacklisted

[cooperlees]

https://github.com/pypa/bandersnatch/blob/master/src/bandersnatch_filter_plugins/blacklist_name.py#L22

• It's debug. I'm tempted to make whitelist the same. I don't know why it's being initialized tho.

How the plugins are designed, I'm not a fan of and I wish I put more thought into it when I accepted the PRs. One day if I get time I'll clean this all up. Once again. PRs welcome.

[cooperlees]

Documentation is clearer now. Thanks for hi-lighting and glad it's working now.