Per-key priorities for dictionary-like settings #1135
I'm new, so here's a quick intro: Hi! I'm Jakob :) I came upon this while working on my GSoC proposal.
During Julia's GSoC, settings priorities were introduced to allow updating settings from different places without paying attention to order. They work great for 'simple' (non-compound) settings, but there are two issues with the dictionary-like settings (e.g.
This should prove especially problematic for the proposed add-on structure (SEP-021, discussed in #591). To resolve this, every key could have its own priority associated with it. @curita suggested making the compound setting variables an instance of the
There are currently three places where the dict-like settings are written to:
Scrapy's code could be updated in the following fashion with full backwards compatibility, even for non-intended uses (such as users working directly on a
The text was updated successfully, but these errors were encountered:
Great work Jakob, thanks! I really like your implementation ideas.
Another place where you can update settings is in the class attribute
I don’t particularly like using Settings objects directly in
I particularly like the idea of deprecating _BASE settings, one of the purposes of using settings priorities was to avoid these kinds of patterns. Not sure if I would keep those _BASE settings with an empty dict for backward compatibility, we’re modifying their value so users using them probably won’t get what they were expecting. I think supporting to set _BASE settings in
There are some missing things to consider about the old setting and getting dictionaries usage. It makes sense that
Another thing we have to make sure is that every dictionary setting allows to disable keys, otherwise we won’t be able to ‘turn off’ keys that were already set (For example, ‘scrapy.contrib.downloadermiddleware.retry.RetryMiddleware', a default downloader middleware, is added as key to DOWNLOADER_MIDDLEWARES with a default value of 500, there should be a value to indicate we want to disable it in the middleware manager). I think most dictionary settings allow to set
Also we should evaluate what to do with user defined settings (those not mentioned in
Right! I missed that. I think with that in mind the check whether a
I agree. As the default settings are loaded with a
Sounds reasonable. As
I also think we should return the
We should definitely check whether all builtin settings allow disabling via
On a side note, I think it would be nice if we could completely decouple