Memory issue with OPML import

[hannes3120]

Since my import pretty much failed each time now (at first at 2/156, then I deleted an abandoned feed and got to 3/156, figured out that my 'complex' folder-structure might be a problem and simplified it and got to 14/156 (1 failed))

I think it's just a pain to not have some sort of feedback from the server other than the number of the feed - and if there is an issue (there sure is one since the server seems to crash every time and I have to kill the process to be able to launch it again).

And I feel like it shouldn't be able for a user to crash the server just with importing feeds...

From what I've seen the parser runs in an endless loop if there is an error in the parsed feed-url - at least it seems like that from the log...

Maybe ask the user after the 3rd failed try to import the feed?

In general more feedback would be pretty awesome since the design is pretty awesome already this would make this already great client more user-friendly...

[jendib]

Please provide us the reader.log file. Its location depends on your OS.
Also, the OPML file would be great.

[hannes3120]

These are the unregular log-entries:

17 Dec 2013 18:29:15,219 INFO com.sismics.reader.core.listener.async.SubscriptionImportAsyncListener.onSubscriptionImport(SubscriptionImportAsyncListener.java:76) OPML import requested event: SubscriptionImportedEvent{user=User{id=8fabbfbf-d9ab-4b3f-b13b-192b46f7f606, username=hannes3120}, importFile=C:\Users\Hannes\AppData\Local\Temp\reader_opml_import6235763280753652676.tmp}
17 Dec 2013 18:29:15,362 INFO com.sismics.reader.core.listener.async.SubscriptionImportAsyncListener.importOutline(SubscriptionImportAsyncListener.java:371) Importing outline 1/152
17 Dec 2013 18:29:15,365 INFO com.sismics.reader.core.service.FeedService.synchronize(FeedService.java:110) Synchronizing feed at URL: http://www.reddit.com/.rss?feed=5f12ebf731602974777da1562c33ff8a8006fe6f&user=hannes3120
17 Dec 2013 18:29:17,867 INFO com.sismics.reader.core.service.FeedService.synchronize(FeedService.java:248) Synchronization done in 2,502ms
17 Dec 2013 18:29:17,867 INFO com.sismics.reader.core.listener.async.ArticleUpdatedAsyncListener.onArticleUpdated(ArticleUpdatedAsyncListener.java:32) Updated article event: ArticleUpdatedAsyncEvent{articles=15 articles}
17 Dec 2013 18:29:17,908 INFO com.sismics.reader.core.listener.async.SubscriptionImportAsyncListener.importOutline(SubscriptionImportAsyncListener.java:371) Importing outline 2/152
17 Dec 2013 18:29:17,916 INFO com.sismics.reader.core.service.FeedService.synchronize(FeedService.java:110) Synchronizing feed at URL: http://www.ccc.de/de/rss/updates.rdf
17 Dec 2013 18:29:18,261 INFO com.sismics.reader.core.service.FeedService.synchronize(FeedService.java:248) Synchronization done in 345ms
17 Dec 2013 18:29:18,280 INFO com.sismics.reader.core.listener.async.SubscriptionImportAsyncListener.importOutline(SubscriptionImportAsyncListener.java:371) Importing outline 3/152
17 Dec 2013 18:29:18,283 INFO com.sismics.reader.core.service.FeedService.synchronize(FeedService.java:110) Synchronizing feed at URL: http://www.heise.de/newsticker/heise-top-atom.xml
17 Dec 2013 18:29:18,897 INFO com.sismics.reader.core.service.FeedService.synchronize(FeedService.java:248) Synchronization done in 614ms
17 Dec 2013 18:29:18,978 INFO com.sismics.reader.core.listener.async.SubscriptionImportAsyncListener.importOutline(SubscriptionImportAsyncListener.java:371) Importing outline 4/152
17 Dec 2013 18:29:18,981 INFO com.sismics.reader.core.service.FeedService.synchronize(FeedService.java:110) Synchronizing feed at URL: http://feeds.feedburner.com/Winrootkit
17 Dec 2013 18:29:19,486 INFO com.sismics.reader.core.service.FeedService.synchronize(FeedService.java:248) Synchronization done in 505ms
17 Dec 2013 18:29:19,575 INFO com.sismics.reader.core.listener.async.SubscriptionImportAsyncListener.importOutline(SubscriptionImportAsyncListener.java:371) Importing outline 5/152
17 Dec 2013 18:29:19,578 INFO com.sismics.reader.core.service.FeedService.synchronize(FeedService.java:110) Synchronizing feed at URL: http://gutjahr.biz/feed/
17 Dec 2013 18:29:24,471 INFO com.sismics.reader.core.listener.async.FaviconUpdateRequestedAsyncListener.onFaviconUpdateRequested(FaviconUpdateRequestedAsyncListener.java:35) Favicon update requested event: FaviconUpdateRequestedEvent{feedId=f557d6e2-5631-4ecd-9969-f915211b328e}
17 Dec 2013 18:29:24,742 INFO com.sismics.reader.core.service.FeedService.synchronize(FeedService.java:248) Synchronization done in 5,164ms
17 Dec 2013 18:29:24,850 INFO com.sismics.reader.core.listener.async.SubscriptionImportAsyncListener.importOutline(SubscriptionImportAsyncListener.java:371) Importing outline 6/152
17 Dec 2013 18:29:24,854 INFO com.sismics.reader.core.service.FeedService.synchronize(FeedService.java:110) Synchronizing feed at URL: http://www.xda-developers.com/feed/
17 Dec 2013 18:29:26,196 INFO com.sismics.reader.core.service.FeedService.synchronize(FeedService.java:248) Synchronization done in 1,341ms
17 Dec 2013 18:29:26,271 INFO com.sismics.reader.core.dao.file.html.FaviconDownloader.downloadFavicon(FaviconDownloader.java:147) Favicon file not found at URL http://gutjahr.biz/favicon.png
17 Dec 2013 18:29:26,288 INFO com.sismics.reader.core.listener.async.SubscriptionImportAsyncListener.importOutline(SubscriptionImportAsyncListener.java:371) Importing outline 7/152

...

17 Dec 2013 18:29:26,616 INFO com.sismics.reader.core.service.FeedService.synchronize(FeedService.java:248) Synchronization done in 324ms
17 Dec 2013 18:29:26,632 INFO com.sismics.reader.core.listener.async.SubscriptionImportAsyncListener.importOutline(SubscriptionImportAsyncListener.java:371) Importing outline 8/152
17 Dec 2013 18:29:26,710 INFO com.sismics.reader.core.service.FeedService.synchronize(FeedService.java:110) Synchronizing feed at URL: http://www.surroundscreengaming.com/feed/
17 Dec 2013 18:29:26,822 INFO com.sismics.reader.core.dao.file.html.FaviconDownloader.downloadFavicon(FaviconDownloader.java:147) Favicon file not found at URL http://gutjahr.biz/favicon.gif
17 Dec 2013 18:29:26,989 INFO com.sismics.reader.core.dao.file.html.FaviconDownloader.downloadFaviconFromPage(FaviconDownloader.java:90) Favicon successfully downloaded to C:\Users\Hannes\AppData\Roaming\Sismics\Reader\favicon\f557d6e2-5631-4ecd-9969-f915211b328e.ico
17 Dec 2013 18:29:26,990 INFO com.sismics.reader.core.listener.async.ArticleCreatedAsyncListener.onArticleCreated(ArticleCreatedAsyncListener.java:34) Articles created event: ArticleCreatedAsyncEvent{articles=10 articles}
17 Dec 2013 18:29:27,067 INFO com.sismics.reader.core.listener.async.FaviconUpdateRequestedAsyncListener.onFaviconUpdateRequested(FaviconUpdateRequestedAsyncListener.java:35) Favicon update requested event: FaviconUpdateRequestedEvent{feedId=65613ac4-2c3c-4279-b758-a878bce73678}
17 Dec 2013 18:29:28,294 INFO com.sismics.reader.core.dao.file.html.FaviconDownloader.downloadFaviconFromPage(FaviconDownloader.java:90) Favicon successfully downloaded to C:\Users\Hannes\AppData\Roaming\Sismics\Reader\favicon\65613ac4-2c3c-4279-b758-a878bce73678.ico
17 Dec 2013 18:29:28,294 INFO com.sismics.reader.core.listener.async.ArticleCreatedAsyncListener.onArticleCreated(ArticleCreatedAsyncListener.java:34) Articles created event: ArticleCreatedAsyncEvent{articles=10 articles}
17 Dec 2013 18:29:40,707 INFO com.sismics.reader.core.service.FeedService.synchronize(FeedService.java:248) Synchronization done in 13,995ms

...

17 Dec 2013 18:30:17,675 INFO com.sismics.reader.core.listener.async.SubscriptionImportAsyncListener.importOutline(SubscriptionImportAsyncListener.java:371) Importing outline 20/152
17 Dec 2013 18:30:17,755 INFO com.sismics.reader.core.service.FeedService.synchronize(FeedService.java:110) Synchronizing feed at URL: http://www.riseagainst.com/news/feed/*
17 Dec 2013 18:30:18,819 WARN com.sismics.reader.core.dao.file.rss.RssReader.parseDate(RssReader.java:509) Error parsing comment date: 2013-11-26T19:05:43+00:00
17 Dec 2013 18:30:19,189 WARN com.sismics.reader.core.dao.file.rss.RssReader.parseDate(RssReader.java:509) Error parsing comment date: 2013-09-20T16:23:01+00:00
17 Dec 2013 18:30:26,882 WARN com.sismics.reader.core.dao.file.rss.RssReader.parseDate(RssReader.java:509) Error parsing comment date: 2013-09-11T17:49:26+00:00
17 Dec 2013 18:30:26,951 WARN com.sismics.reader.core.dao.file.rss.RssReader.parseDate(RssReader.java:509) Error parsing comment date: 2013-09-05T22:24:07+00:00
17 Dec 2013 18:30:27,946 WARN com.sismics.reader.core.dao.file.rss.RssReader.parseDate(RssReader.java:509) Error parsing comment date: 2013-08-14T00:00:00+00:00
17 Dec 2013 18:30:28,895 WARN com.sismics.reader.core.dao.file.rss.RssReader.parseDate(RssReader.java:509) Error parsing comment date: 2013-08-01T00:00:00+00:00
17 Dec 2013 18:30:30,263 WARN com.sismics.reader.core.dao.file.rss.RssReader.parseDate(RssReader.java:509) Error parsing comment date: 2013-07-30T00:00:00+00:00
17 Dec 2013 18:30:30,571 WARN com.sismics.reader.core.dao.file.rss.RssReader.parseDate(RssReader.java:509) Error parsing comment date: 2013-07-16T00:00:00+00:00
17 Dec 2013 18:30:30,657 WARN com.sismics.reader.core.dao.file.rss.RssReader.parseDate(RssReader.java:509) Error parsing comment date: 2013-06-12T16:48:00+00:00
17 Dec 2013 18:30:30,657 WARN com.sismics.reader.core.dao.file.rss.RssReader.parseDate(RssReader.java:509) Error parsing comment date: 2013-05-02T22:37:00+00:00
17 Dec 2013 18:30:30,730 WARN com.sismics.reader.core.dao.file.rss.RssReader.parseDate(RssReader.java:509) Error parsing comment date: 2013-04-18T13:18:00+00:00
17 Dec 2013 18:30:31,843 WARN com.sismics.reader.core.dao.file.rss.RssReader.parseDate(RssReader.java:509) Error parsing comment date: 2013-04-01T20:11:00+00:00
17 Dec 2013 18:30:31,979 WARN com.sismics.reader.core.dao.file.rss.RssReader.parseDate(RssReader.java:509) Error parsing comment date: 2013-03-27T17:35:00+00:00
17 Dec 2013 18:30:32,123 WARN com.sismics.reader.core.dao.file.rss.RssReader.parseDate(RssReader.java:509) Error parsing comment date: 2013-03-22T14:10:00+00:00
17 Dec 2013 18:30:32,340 WARN com.sismics.reader.core.dao.file.rss.RssReader.parseDate(RssReader.java:509) Error parsing comment date: 2013-03-14T21:45:00+00:00
17 Dec 2013 18:30:32,566 WARN com.sismics.reader.core.dao.file.rss.RssReader.parseDate(RssReader.java:509) Error parsing comment date: 2013-03-04T12:52:00+00:00
17 Dec 2013 18:30:32,868 WARN com.sismics.reader.core.dao.file.rss.RssReader.parseDate(RssReader.java:509) Error parsing comment date: 2013-02-22T18:49:00+00:00
17 Dec 2013 18:30:33,013 WARN com.sismics.reader.core.dao.file.rss.RssReader.parseDate(RssReader.java:509) Error parsing comment date: 2013-01-30T18:30:00+00:00
17 Dec 2013 18:30:33,670 WARN com.sismics.reader.core.dao.file.rss.RssReader.parseDate(RssReader.java:509) Error parsing comment date: 2012-12-14T21:58:00+00:00
17 Dec 2013 18:30:37,413 WARN com.sismics.reader.core.dao.file.rss.RssReader.parseDate(RssReader.java:509) Error parsing comment date: 2012-12-13T14:12:00+00:00
17 Dec 2013 18:30:37,843 WARN com.sismics.reader.core.dao.file.rss.RssReader.parseDate(RssReader.java:509) Error parsing comment date: 2012-12-12T13:28:00+00:00
17 Dec 2013 18:30:38,060 WARN com.sismics.reader.core.dao.file.rss.RssReader.parseDate(RssReader.java:509) Error parsing comment date: 2012-11-26T13:04:00+00:00
17 Dec 2013 18:30:38,215 WARN com.sismics.reader.core.dao.file.rss.RssReader.parseDate(RssReader.java:509) Error parsing comment date: 2012-11-13T13:06:00+00:00
17 Dec 2013 18:30:39,240 WARN com.sismics.reader.core.dao.file.rss.RssReader.parseDate(RssReader.java:509) Error parsing comment date: 2012-11-05T19:26:00+00:00
17 Dec 2013 18:30:39,372 WARN com.sismics.reader.core.dao.file.rss.RssReader.parseDate(RssReader.java:509) Error parsing comment date: 2012-10-01T15:46:00+00:00
17 Dec 2013 18:30:39,669 INFO com.sismics.reader.core.listener.async.FaviconUpdateRequestedAsyncListener.onFaviconUpdateRequested(FaviconUpdateRequestedAsyncListener.java:35) Favicon update requested event: FaviconUpdateRequestedEvent{feedId=a4faac47-4f5d-4f4e-a721-9a67f8d60101}
17 Dec 2013 18:30:47,336 WARN org.hibernate.engine.jdbc.spi.SqlExceptionHelper.logExceptions(SqlExceptionHelper.java:143) SQL Error: -460, SQLState: S1000
17 Dec 2013 18:30:47,408 ERROR org.hibernate.engine.jdbc.spi.SqlExceptionHelper.logExceptions(SqlExceptionHelper.java:144) java.lang.OutOfMemoryError: GC overhead limit exceeded
17 Dec 2013 18:30:53,069 ERROR com.sismics.util.filter.RequestContextFilter.doFilter(RequestContextFilter.java:100) An exception occured, rolling back current transaction
javax.persistence.PersistenceException: org.hibernate.exception.GenericJDBCException: java.lang.OutOfMemoryError: GC overhead limit exceeded
at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1360)
at org.hibernate.ejb.AbstractEntityManagerImpl.find(AbstractEntityManagerImpl.java:817)
at org.hibernate.ejb.AbstractEntityManagerImpl.find(AbstractEntityManagerImpl.java:771)
at com.sismics.reader.core.dao.jpa.AuthenticationTokenDao.get(AuthenticationTokenDao.java:28)
at com.sismics.util.filter.TokenBasedSecurityFilter.doFilter(TokenBasedSecurityFilter.java:93)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
at com.sismics.util.filter.RequestContextFilter.doFilter(RequestContextFilter.java:96)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:368)
at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Unknown Source)
Caused by: org.hibernate.exception.GenericJDBCException: java.lang.OutOfMemoryError: GC overhead limit exceeded
at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:54)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:125)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:110)
at org.hibernate.engine.jdbc.internal.proxy.ConnectionProxyHandler.continueInvocation(ConnectionProxyHandler.java:146)
at org.hibernate.engine.jdbc.internal.proxy.AbstractProxyHandler.invoke(AbstractProxyHandler.java:81)
at com.sun.proxy.$Proxy11.prepareStatement(Unknown Source)
at org.hibernate.engine.jdbc.internal.StatementPreparerImpl$5.doPrepare(StatementPreparerImpl.java:147)
at org.hibernate.engine.jdbc.internal.StatementPreparerImpl$StatementPreparationTemplate.prepareStatement(StatementPreparerImpl.java:166)
at org.hibernate.engine.jdbc.internal.StatementPreparerImpl.prepareQueryStatement(StatementPreparerImpl.java:145)
at org.hibernate.loader.Loader.prepareQueryStatement(Loader.java:1720)
at org.hibernate.loader.Loader.doQuery(Loader.java:828)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:289)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:259)
at org.hibernate.loader.Loader.loadEntity(Loader.java:2042)
at org.hibernate.loader.entity.AbstractEntityLoader.load(AbstractEntityLoader.java:82)
at org.hibernate.loader.entity.AbstractEntityLoader.load(AbstractEntityLoader.java:72)
at org.hibernate.persister.entity.AbstractEntityPersister.load(AbstractEntityPersister.java:3710)
at org.hibernate.event.internal.DefaultLoadEventListener.loadFromDatasource(DefaultLoadEventListener.java:439)
at org.hibernate.event.internal.DefaultLoadEventListener.doLoad(DefaultLoadEventListener.java:420)
at org.hibernate.event.internal.DefaultLoadEventListener.load(DefaultLoadEventListener.java:204)
at org.hibernate.event.internal.DefaultLoadEventListener.proxyOrLoad(DefaultLoadEventListener.java:251)
at org.hibernate.event.internal.DefaultLoadEventListener.onLoad(DefaultLoadEventListener.java:148)
at org.hibernate.internal.SessionImpl.fireLoad(SessionImpl.java:992)
at org.hibernate.internal.SessionImpl.access$1900(SessionImpl.java:170)
at org.hibernate.internal.SessionImpl$IdentifierLoadAccessImpl.load(SessionImpl.java:2336)
at org.hibernate.internal.SessionImpl.get(SessionImpl.java:888)
at org.hibernate.ejb.AbstractEntityManagerImpl.find(AbstractEntityManagerImpl.java:796)
... 28 more
Caused by: java.sql.SQLException: java.lang.OutOfMemoryError: GC overhead limit exceeded
at org.hsqldb.jdbc.JDBCUtil.sqlException(Unknown Source)
at org.hsqldb.jdbc.JDBCUtil.sqlException(Unknown Source)
at org.hsqldb.jdbc.JDBCPreparedStatement.(Unknown Source)
at org.hsqldb.jdbc.JDBCConnection.prepareStatement(Unknown Source)
at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.hibernate.engine.jdbc.internal.proxy.ConnectionProxyHandler.continueInvocation(ConnectionProxyHandler.java:138)
... 51 more
Caused by: org.hsqldb.HsqlException: java.lang.OutOfMemoryError: GC overhead limit exceeded
at org.hsqldb.error.Error.error(Unknown Source)
at org.hsqldb.result.Result.newErrorResult(Unknown Source)
at org.hsqldb.Session.execute(Unknown Source)
... 57 more
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded

...

and after that it goes on and on and on like:

17 Dec 2013 19:05:25,614 INFO com.sismics.reader.core.service.FeedService.synchronize(FeedService.java:110) Synchronizing feed at URL: http://notepad-plus-plus.org/feed.rss
17 Dec 2013 19:05:25,959 INFO com.sismics.reader.core.service.FeedService.synchronize(FeedService.java:248) Synchronization done in 345ms

and then the log just stops without any further message regarding any error...

[jendib]

It seems to be a memory problem.
The Windows version is configured with only 150MB of RAM, which seems to be too low for your import.
If you know how to do, increase the memory here : https://github.com/sismics/reader/blob/master/reader-distribution-windows/src/main/launch4j/reader-agent.xml
<cmdLine>-Xmx512M -XX:MaxPermSize=128M</cmdLine>

recompile and try again. Feel free to submit a pull request if it works for you.

[jendib]

The bug is fixed, and a new version has been released (1.3).

[jendib]

About your initial issue (more feedback), note that you can access the logs from within the application (Cog icon > About), with a filter (on errors for example).