[MRG+1] Improve document about functions as processors

[lucywang000]

I came across this earlier today, and after quite some (result-less) debugging, google led me to #2735 . Hopefully this improvement can help others avoid this pitfall.

[codecov]

Codecov Report

Merging #3224 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #3224   +/-   ##
=======================================
  Coverage   82.12%   82.12%           
=======================================
  Files         228      228           
  Lines        9593     9593           
  Branches     1385     1385           
=======================================
  Hits         7878     7878           
  Misses       1456     1456           
  Partials      259      259

[kmike]

Looks good, thanks @lucywang000!

I think in a long term we should support plain functions somehow. There was also a discussion about moving item loaders to a separate Python package.

[lopuhin]

Thanks @lucywang000 !

[lucywang000]

Thanks for the review @kmike @lopuhin !

[IAlwaysBeCoding]

@kmike I think we should seriously have a third-party library that would have several processors that are highly needed. I've came up with several that I use on my own Scrapy projects.

Some processors that I have coded myself are:

• Processor that will output a unique value for true and/or false if an element is found using an xpath/css.

• Processor that will choose an n value from the extracted list, it's basically like a TakeFirst() but you can decide what n's index you want.

• A TakeLast() processor as well.

• Processor that will replace values

• Processor that will split the output into values from a given delimeter.

and many more...

You can't really fully utlize the ItemLoader if you don't have a library of Processors, the default Processors that come with Scrapy left me no choice than to cook my own.