Fixed Extraction when Meta tag has an empty content

[felipehertzer]

Hey @adbar,

I had a few cases that the below meta tags are empty, and when it happen the extraction stops to work.

   <meta name="title" content=" " />
   <meta property="og:title" content=" " />
   <meta name="twitter:title" content=" " />

I checked and the line below is checking for None before trying to get the correct title, but it never happens, because the title is ' ' in this line.

trafilatura/trafilatura/metadata.py

Line 507 in fb3e174

if metadata.title is None:

Evan on json parse it is checking for None but the current title is ' ':

trafilatura/trafilatura/json_metadata.py

Line 109 in fb3e174

if metadata.title is None:

And then this function convert the the title ' ' to none:

trafilatura/trafilatura/metadata.py

Line 573 in fb3e174

metadata.clean_and_trim()

And then it will fail here:

trafilatura/trafilatura/core.py

Line 929 in fb3e174

if only_with_metadata is True and any(

This is an example of the problem:

Example

I run the the tests and comparison_small.py and it appears to be the same.

Thanks.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.26%. Comparing base (fb3e174) to head (b4f037a).

Additional details and impacted files

@@           Coverage Diff           @@
##           master     #545   +/-   ##
=======================================
  Coverage   97.26%   97.26%           
=======================================
  Files          22       22           
  Lines        3438     3438           
=======================================
  Hits         3344     3344           
  Misses         94       94           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[adbar]

Instead of trimming maybe element.get("content").isspace() ? This would be more light-weight.

[felipehertzer]

Oh, that makes sense, I updated it. Thanks.

[adbar]

Thanks!