[MRG+1] Use w3lib.url.canonicalize_url() from w3lib 1.15.0 #2168
Conversation
Thanks! It seems a few other places needs to be changed:
|
Current coverage is 83.40% (diff: 100%)
|
Thanks, incorporated your feedback. Did a rebase, as not sure whether an additional commit would be acceptable (might be a good idea to incorporate what is recommended as a part of |
@ashkulz: it is good to have a readable commits history, thanks for taking time to do the rebase! Currently we are not too strict on rebase vs extra commits - sometimes we ask for a rebase when a commit history gets messy, but not always. Probably that's why it is not in contributing.rst - we haven't agreed on a policy, it is just a common sense now. |
|
||
# scrapy.utils.url was moved to w3lib.url and import * ensures this | ||
# move doesn't break old code | ||
from w3lib.url import * | ||
from w3lib.url import _safe_chars |
kmike
Aug 9, 2016
Member
I believe this import is needed for backwards compatibility
I believe this import is needed for backwards compatibility
return urlunparse((scheme, netloc.lower(), path, params, query, fragment)) | ||
|
||
|
||
def _unquotepath(path): |
kmike
Aug 9, 2016
Member
hmm, this function was here for a long time; I wonder if we should import it from w3lib.url to avoid breaking user code
hmm, this function was here for a long time; I wonder if we should import it from w3lib.url to avoid breaking user code
kmike
Aug 9, 2016
Member
other removed functions were here for much less time; I think it is fine to remove them as it is unlikely removal will break anything
other removed functions were here for much less time; I think it is fine to remove them as it is unlikely removal will break anything
@kmike: sorry for the delay. Is there a necessity to keep |
Also remove code/imports which are now unused due to this change. fixes #2157
@ashkulz , we sometimes break this guideline to be on the safe side with what users import. |
@redapple: any other feedback? I've already pushed a rebased commit which includes this change, unless you want me to include an entry in |
@ashkulz , no other feedback. Looks good to me! |
Thanks @ashkulz! |
fixes #2157