Full text feature broken

[mistermtu]

• [x ] I have read the CONTRIBUTING.md and followed the provided tips

Explain the Problem

Full text feature broken. Only short articles are shown. Full article feature does not have any effect.

Steps to Reproduce

Explain what you did to encounter the issue

1. Disable full text article in feed if already activated
2. Enable full text article in feed
3. Reload feed
4. Same content is shown (no full text)

System Information

• News app version: 13.1.1
• Nextcloud version: 15.0.5
• PHP version: 7.0
• Database and version: MariaDB 5.6
• Browser and version: Latest Chrome, Latest Firefox
• Distribution and version:

No related errors in logs

[matdb]

It shows

Internal server error! Please check your data/nextcloud.log file for additional information!

when I try to use the full text feature

The log contains the following:

{"reqId":"SLmus4uTDJYntUURtZhp","level":3,"time":"2019-03-14T08:16:07+00:00","remoteAddr":"**********","user":"********","app":"PHP","method":"PATCH","url":"\/index.php\/feeds\/49","message":"DateTime::__construct(): Failed to parse time string (0) at position 0 (0): Unexpected character at \/var\/www\/html\/custom_apps\/news\/lib\/Fetcher\/FeedFetcher.php#75","userAgent":"Mozilla\/5.0 (X11; Linux x86_64; rv:65.0) Gecko\/20100101 Firefox\/65.0","version":"15.0.5.3"} {"reqId":"e3lG8VlfxonktLpPwU6D","level":3,"time":"2019-03-14T08:16:15+00:00","remoteAddr":"*******","user":"******","app":"index","method":"PATCH","url":"\/index.php\/feeds\/99","message":{"Exception":"Exception","Message":"DateTime::__construct(): Failed to parse time string (0) at position 0 (0): Unexpected character","Code":0,"Trace":[{"file":"\/var\/www\/html\/custom_apps\/news\/lib\/Fetcher\/FeedFetcher.php","line":75,"function":"__construct","class":"DateTime","type":"->","args":["0"]},{"file":"\/var\/www\/html\/custom_apps\/news\/lib\/Fetcher\/Fetcher.php","line":68,"function":"fetch","class":"OCA\\News\\Fetcher\\FeedFetcher","type":"->","args":["https:\/\/sciencebasedmedicine.org\/feed\/",false,"0",null,null]},{"file":"\/var\/www\/html\/custom_apps\/news\/lib\/Service\/FeedService.php","line":228,"function":"fetch","class":"OCA\\News\\Fetcher\\Fetcher","type":"->","args":["https:\/\/sciencebasedmedicine.org\/feed\/",false,"0",null,null]},{"file":"\/var\/www\/html\/custom_apps\/news\/lib\/Service\/FeedService.php","line":490,"function":"update","class":"OCA\\News\\Service\\FeedService","type":"->","args":[99,"*******",true]},{"file":"\/var\/www\/html\/custom_apps\/news\/lib\/Controller\/FeedController.php","line":322,"function":"patch","class":"OCA\\News\\Service\\FeedService","type":"->","args":[99,"*******",{"fullTextEnabled":true}]},{"file":"\/var\/www\/html\/lib\/private\/AppFramework\/Http\/Dispatcher.php","line":166,"function":"patch","class":"OCA\\News\\Controller\\FeedController","type":"->","args":[99,null,true,null,null,null,null]},{"file":"\/var\/www\/html\/lib\/private\/AppFramework\/Http\/Dispatcher.php","line":99,"function":"executeController","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->","args":[{"__class__":"OCA\\News\\Controller\\FeedController"},"patch"]},{"file":"\/var\/www\/html\/lib\/private\/AppFramework\/App.php","line":118,"function":"dispatch","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->","args":[{"__class__":"OCA\\News\\Controller\\FeedController"},"patch"]},{"file":"\/var\/www\/html\/lib\/private\/AppFramework\/Routing\/RouteActionHandler.php","line":47,"function":"main","class":"OC\\AppFramework\\App","type":"::","args":["OCA\\News\\Controller\\FeedController","patch",{"__class__":"OC\\AppFramework\\DependencyInjection\\DIContainer"},{"feedId":"99","_route":"news.feed.patch"}]},{"function":"__invoke","class":"OC\\AppFramework\\Routing\\RouteActionHandler","type":"->","args":[{"feedId":"99","_route":"news.feed.patch"}]},{"file":"\/var\/www\/html\/lib\/private\/Route\/Router.php","line":297,"function":"call_user_func","args":[{"__class__":"OC\\AppFramework\\Routing\\RouteActionHandler"},{"feedId":"99","_route":"news.feed.patch"}]},{"file":"\/var\/www\/html\/lib\/base.php","line":987,"function":"match","class":"OC\\Route\\Router","type":"->","args":["\/apps\/news\/feeds\/99"]},{"file":"\/var\/www\/html\/index.php","line":42,"function":"handleRequest","class":"OC","type":"::","args":[]}],"File":"\/var\/www\/html\/custom_apps\/news\/lib\/Fetcher\/FeedFetcher.php","Line":75,"CustomMessage":"--"},"userAgent":"Mozilla\/5.0 (X11; Linux x86_64; rv:65.0) Gecko\/20100101 Firefox\/65.0","version":"15.0.5.3"} {"reqId":"e3lG8VlfxonktLpPwU6D","level":3,"time":"2019-03-14T08:16:15+00:00","remoteAddr":"*******","user":"*******","app":"PHP","method":"PATCH","url":"\/index.php\/feeds\/99","message":"DateTime::__construct(): Failed to parse time string (0) at position 0 (0): Unexpected character at \/var\/www\/html\/custom_apps\/news\/lib\/Fetcher\/FeedFetcher.php#75","userAgent":"Mozilla\/5.0 (X11; Linux x86_64; rv:65.0) Gecko\/20100101 Firefox\/65.0","version":"15.0.5.3"}

[SMillerDev]

@matdb it'll just be ignored. Your error is from a feed returning an invalid date as far as I can tell.

[megamaced]

+1 Many of my feeds are no longer displaying full text.

{"reqId":"eS0R1LFno8L54wOB73vM","level":3,"time":"2019-03-15T09:14:33+00:00","remoteAddr":"*******","user":"*********","app":"index","method":"PATCH","url":"\/index.php\/apps\/news\/feeds\/70","message":{"Exception":"Exception","Message":"DateTime::__construct(): Failed to parse time string (0) at position 0 (0): Unexpected character","Code":0,"Trace":[{"file":"\/var\/www\/html\/apps\/news\/lib\/Fetcher\/FeedFetcher.php","line":75,"function":"__construct","class":"DateTime","type":"->","args":["0"]},{"file":"\/var\/www\/html\/apps\/news\/lib\/Fetcher\/Fetcher.php","line":68,"function":"fetch","class":"OCA\\News\\Fetcher\\FeedFetcher","type":"->","args":["https:\/\/www.theregister.co.uk\/headlines.atom",false,"0",null,null]},{"file":"\/var\/www\/html\/apps\/news\/lib\/Service\/FeedService.php","line":228,"function":"fetch","class":"OCA\\News\\Fetcher\\Fetcher","type":"->","args":["https:\/\/www.theregister.co.uk\/headlines.atom",false,"0",null,null]},{"file":"\/var\/www\/html\/apps\/news\/lib\/Service\/FeedService.php","line":490,"function":"update","class":"OCA\\News\\Service\\FeedService","type":"->","args":[70,"********",true]},{"file":"\/var\/www\/html\/apps\/news\/lib\/Controller\/FeedController.php","line":322,"function":"patch","class":"OCA\\News\\Service\\FeedService","type":"->","args":[70,"*********",{"fullTextEnabled":true}]},{"file":"\/var\/www\/html\/lib\/private\/AppFramework\/Http\/Dispatcher.php","line":166,"function":"patch","class":"OCA\\News\\Controller\\FeedController","type":"->","args":[70,null,true,null,null,null,null]},{"file":"\/var\/www\/html\/lib\/private\/AppFramework\/Http\/Dispatcher.php","line":99,"function":"executeController","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->","args":[{"__class__":"OCA\\News\\Controller\\FeedController"},"patch"]},{"file":"\/var\/www\/html\/lib\/private\/AppFramework\/App.php","line":118,"function":"dispatch","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->","args":[{"__class__":"OCA\\News\\Controller\\FeedController"},"patch"]},{"file":"\/var\/www\/html\/lib\/private\/AppFramework\/Routing\/RouteActionHandler.php","line":47,"function":"main","class":"OC\\AppFramework\\App","type":"::","args":["OCA\\News\\Controller\\FeedController","patch",{"__class__":"OC\\AppFramework\\DependencyInjection\\DIContainer"},{"feedId":"70","_route":"news.feed.patch"}]},{"function":"__invoke","class":"OC\\AppFramework\\Routing\\RouteActionHandler","type":"->","args":[{"feedId":"70","_route":"news.feed.patch"}]},{"file":"\/var\/www\/html\/lib\/private\/Route\/Router.php","line":297,"function":"call_user_func","args":[{"__class__":"OC\\AppFramework\\Routing\\RouteActionHandler"},{"feedId":"70","_route":"news.feed.patch"}]},{"file":"\/var\/www\/html\/lib\/base.php","line":987,"function":"match","class":"OC\\Route\\Router","type":"->","args":["\/apps\/news\/feeds\/70"]},{"file":"\/var\/www\/html\/index.php","line":42,"function":"handleRequest","class":"OC","type":"::","args":[]}],"File":"\/var\/www\/html\/apps\/news\/lib\/Fetcher\/FeedFetcher.php","Line":75,"CustomMessage":"--"},"userAgent":"Mozilla\/5.0 (X11; Linux x86_64; rv:60.0) Gecko\/20100101 Firefox\/60.0","version":"15.0.5.3"}

[wwwindisch]

I can confirm that: Same issue with the following two error messages:

{"reqId":"XIycF38AAAEAAEs-cuwAAAAA","level":3,"time":"2019-03-16T06:47:52+00:00","remoteAddr":"","user":"","app":"PHP","method":"PATCH","url":"/index.php/apps/news/feeds/6","message":"DateTime::__construct(): Failed to parse time string (0) at position 0 (0): Unexpected character at /var/www/nextcloud/apps/news/lib/Fetcher/FeedFetcher.php#75","userAgent":"Mozilla/5.0 (Windows NT 6.1; WOW64; rv:60.0) Gecko/20100101 Firefox/60.0","version":"15.0.5.3","id":"5c8c9ce8da42b"}

{"reqId":"XIycF38AAAEAAEs-cuwAAAAA","level":3,"time":"2019-03-16T06:47:52+00:00","remoteAddr":"","user":"","app":"index","method":"PATCH","url":"/index.php/apps/news/feeds/6","message":{"Exception":"Exception","Message":"DateTime::__construct(): Failed to parse time string (0) at position 0 (0): Unexpected character","Code":0,"Trace":[{"file":"/var/www/nextcloud/apps/news/lib/Fetcher/FeedFetcher.php","line":75,"function":"__construct","class":"DateTime","type":"->","args":["0"]},{"file":"/var/www/nextcloud/apps/news/lib/Fetcher/Fetcher.php","line":68,"function":"fetch","class":"OCA\News\Fetcher\FeedFetcher","type":"->","args":["http://feeds2.feedburner.com/stadt-bremerhaven/dqXM",false,"0",null,null]},{"file":"/var/www/nextcloud/apps/news/lib/Service/FeedService.php","line":228,"function":"fetch","class":"OCA\News\Fetcher\Fetcher","type":"->","args":["http://feeds2.feedburner.com/stadt-bremerhaven/dqXM",false,"0",null,null]},{"file":"/var/www/nextcloud/apps/news/lib/Service/FeedService.php","line":490,"function":"update","class":"OCA\News\Service\FeedService","type":"->","args":[6,"markus",true]},{"file":"/var/www/nextcloud/apps/news/lib/Controller/FeedController.php","line":322,"function":"patch","class":"OCA\News\Service\FeedService","type":"->","args":[6,"markus",{"fullTextEnabled":true}]},{"file":"/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php","line":166,"function":"patch","class":"OCA\News\Controller\FeedController","type":"->","args":[6,null,true,null,null,null,null]},{"file":"/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php","line":99,"function":"executeController","class":"OC\AppFramework\Http\Dispatcher","type":"->","args":[{"class":"OCA\News\Controller\FeedController"},"patch"]},{"file":"/var/www/nextcloud/lib/private/AppFramework/App.php","line":118,"function":"dispatch","class":"OC\AppFramework\Http\Dispatcher","type":"->","args":[{"class":"OCA\News\Controller\FeedController"},"patch"]},{"file":"/var/www/nextcloud/lib/private/AppFramework/Routing/RouteActionHandler.php","line":47,"function":"main","class":"OC\AppFramework\App","type":"::","args":["OCA\News\Controller\FeedController","patch",{"class":"OC\AppFramework\DependencyInjection\DIContainer"},{"feedId":"6","_route":"news.feed.patch"}]},{"function":"__invoke","class":"OC\AppFramework\Routing\RouteActionHandler","type":"->","args":[{"feedId":"6","_route":"news.feed.patch"}]},{"file":"/var/www/nextcloud/lib/private/Route/Router.php","line":297,"function":"call_user_func","args":[{"class":"OC\AppFramework\Routing\RouteActionHandler"},{"feedId":"6","_route":"news.feed.patch"}]},{"file":"/var/www/nextcloud/lib/base.php","line":987,"function":"match","class":"OC\Route\Router","type":"->","args":["/apps/news/feeds/6"]},{"file":"/var/www/nextcloud/index.php","line":42,"function":"handleRequest","class":"OC","type":"::","args":[]}],"File":"/var/www/nextcloud/apps/news/lib/Fetcher/FeedFetcher.php","Line":75,"CustomMessage":"--"},"userAgent":"Mozilla/5.0 (Windows NT 6.1; WOW64; rv:60.0) Gecko/20100101 Firefox/60.0","version":"15.0.5.3","id":"5c8c9ce8daac4"}

[SMillerDev]

That's more than enough confirmation, thanks everyone. The most useful thing to do if you have the same issue is making a pull request. Otherwise you can point out the offending line or feed. If you just want people to know you have the same issue, give it a thumbs up but please don't post the same error log 25 times saying you can confirm the issue. It only makes it increasingly difficult to find the right information.

[enaut]

Not really the same error - or maybe same but different location: on manual update of the feed with:

$ sudo -u nginx php occ news:updater:update-feed 124 user
Could not update feed with id 124 and user user: DateTime::__construct(): Failed to parse time string (0) at position 0 (0): Unexpected character

The issue can not be fixed by manually disabling the fulltext collumn in the database entry - the issue persists. But before having clicked on the fulltext feature the feed worked well... and with another user that same feed works too until this user clicks the fulltext feature.

Feeds that do this are:

https://www.heise.de/rss/heise-atom.xml
https://www.computerbase.de/rss/news.xml

I'm not into php so no pr from me :(

[megamaced]

Does the latest 13.1.4 release fix this?

Nextcloud news is a killer app for me, I use it all the time and I had to roll back to 13.0.3. Is 13.1.4 good to update?

Keep up the awesome work!

[enaut]

I can confirm that at least for me the feeds that did not work with previous versions do work now... so it might be worth a shot to test...
However I do not see the full text of the article. its still just the summary. I do not know if I should see the full article. - I never understood the full text feature really.

[SMillerDev]

Full text isn't implemented yet. Scraping feeds is difficult so we're trying to get just reading them right first. After I'll evaluate if I want to remove fulltext or try writing a scraper.

[qezzo]

what do you mean by scraping? Do you mean scraping the whole rss content or an extra feature other readers have that scrapes the whole article on the main website. I don't miss this extra feature that you would need for feeds like heise.de oder spiegel.de.
If i compare what the news app shows with the original feed it only shows the part of the description tag. The content tag of feeds that put their whole article in rss is not 'scraped'. For example deskmodder.de:
https://www.deskmodder.de/blog/feed/

[SMillerDev]

The fulltext feature means that it used a scraper to fetch the page the item referred to.

[qezzo]

so not showing the full article of the feed is another bug?