View Access via listener stuck indefinitely in updateIndexes.

[NitzDKoder]

Hi Folks,

1. App design couchbase lite java listener is thread safe? #82 and CBL 1.4 used.

2)@line 20848: [09/06 14:13:14:586] [2608] Listener: [REQUEST] (Acme.Utils.ThreadPool(1)-PooledThread: Acme.Serve.Serve$ServeConnection@475cf1a) GET: https://localdomain:31398/data/_design%2FgetHistory/_view/getHistory?key=%22tel%3A%2B919888880008%22&reduce=false&group=false

Request made.

[09/06 14:13:14:612] [2608] Listener: RouterCallbackBlock.onResponseReady() START
[09/06 14:13:14:612] [3016] Batcher: com.couchbase.lite.support.Batcher@1282833: scheduleWithDelay 250 ms, ignored
[09/06 14:13:14:612] [2608] Listener: RouterCallbackBlock.onResponseReady() END
[09/06 14:13:14:612] [3034] Batcher: com.couchbase.lite.support.Batcher@1282833: waitForPendingFutures, inbox size: 24
[09/06 14:13:14:612] [2608] Listener: CountDownLatch.await() START
[09/06 14:13:14:612] [2608] Listener: CountDownLatch.await() END

Responded.

3)@[09/06 14:13:14:636] [2608] Listener: [REQUEST] (Acme.Utils.ThreadPool(1)-PooledThread: Acme.Serve.Serve$ServeConnection@475cf1a) GET: https://localdomain:31398/data/_design%2FgetDr/_view/getDr?key=%22tel%3A%2B919888880008%22

Request made..

@ [09/06 14:13:14:640] [2608] Router: queryDesignDoc Before updateIndex enteringThreadId : 2608 enteringThreadName :Acme.Utils.ThreadPool(1)-PooledThread: Acme.Serve.Serve$ServeConnection@475cf1a
[09/06 14:13:14:640] [2608] Router: queryDesignDoc Entry updateIndex enteringThreadId : 2608 enteringThreadName :Acme.Utils.ThreadPool(1)-PooledThread: Acme.Serve.Serve$ServeConnection@475cf1a
[09/06 14:13:14:641] [2608] View: Re-indexing view: getDr/getDr

Stuck in updateIndexes indefinitely

https://github.com/couchbase/couchbase-lite-java-core/blob/1ae4c0d1873d1287c725406ceaf8035014fb4927/src/main/java/com/couchbase/lite/store/SQLiteViewStore.java#L246

4. @[09/06 14:15:45:681] [2610] Listener: [REQUEST] (Acme.Utils.ThreadPool(2)-PooledThread: Acme.Serve.Serve$ServeConnection@76b603b) POST: https://localdomain:31398/data/_design%2FgetContent/_view/getContent

Request made..

@[09/06 14:15:45:683] [2610] Router: queryDesignDoc Entry updateIndex enteringThreadId : 2610 enteringThreadName :Acme.Utils.ThreadPool(2)-PooledThread: Acme.Serve.Serve$ServeConnection@76b603b
[09/06 14:15:45:684] [2610] View: Re-indexing view: getContent/getContent

Stuck in updateIndexes indefinitely

@hideki please share your thoughts..

Detail CBL logs:

https://gist.github.com/NitzDKoder/6e87dba4b65272745352e730f315c5b6

Thanks
Nithin

[hideki]

Hi @NitzDKoder,
Do you think that Listener causes the deadlock?
Don't you mind to capture thread-dump when you encounter this problem again?
Thanks!

[NitzDKoder]

@hideki 2 different views can be accessed in parrallel..

Are asking for traces.txt /data/anr/traces.txt ??

[hideki]

@NitzDKoder

You could capture a thread-dump from Android Studio.

[NitzDKoder]

@hideki as the issue is random and happens when client has lot of new documents added.

Did you get check logs to see where issue can happen?

New use case:

1. T1 entered to get view stuck here.
   https://github.com/couchbase/couchbase-lite-java-core/blob/1ae4c0d1873d1287c725406ceaf8035014fb4927/src/main/java/com/couchbase/lite/store/SQLiteViewStore.java#L246

2)next view query with T2 are stuck in here.
https://github.com/couchbase/couchbase-lite-java-core/blob/1.4.0/src/main/java/com/couchbase/lite/router/Router.java#L2524

When are these synchronised added in queryDesignDoc function? Dont see in the branch code.

https://github.com/couchbase/couchbase-lite-java-core/blob/feature/issue_855_fetchRemoteCheckpointDoc/src/main/java/com/couchbase/lite/router/Router.java#L2035

We tried taking thread dump but failed. Will try again.

PS: yet to check whats causing deadlock.

Thanks
Nithin

[NitzDKoder]

@hideki raised #85

Please check detailed logs given..

[hideki]

Hi @NitzDKoder,

If one or more methods are hanging, it is hard to identify from the log file. Usually, the hannging state is not logged. I looked into it, but I am not sure which point hanging occurs.

In general, thread-dump helps us to identify it.

[hideki]

Ref: #85

[hideki]

Hi @NitzDKoder,
Do you have any additional information about this issue?

[djpongh]

Without more info, we can't tell what's happening.