Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error on wave import: Can't split further; too many results (300) between 14932 and 14933 #1

Closed
GoogleCodeExporter opened this issue Aug 6, 2015 · 2 comments
Closed

Error on wave import: Can't split further; too many results (300) between 14932 and 14933 #1

GoogleCodeExporter opened this issue Aug 6, 2015 · 2 comments
Labels
auto-migrated Priority-Medium Type-Defect

Comments

@GoogleCodeExporter
Copy link 

What steps will reproduce the problem?
1. Go to /import and try to import some waves.
2. Pick some wave from the list of found waves and choose "import".
Expected: The wave is imported to Walkaround instance.
Actual: Server error.

0.1.0.2 - - [05/Nov/2011:01:08:27 -0700] "POST /taskqueue/import HTTP/1.1" 500 
3195 "http://wavereactor.appspot.com/taskqueue/import" "AppEngine-Google; 
(+http://code.google.com/appengine)" "wavereactor.appspot.com" ms=1402 
cpu_ms=2867 api_cpu_ms=2517 cpm_usd=0.080832 queue_name=import 
task_name=1436821488731394914 instance=00c61b117c0604d75ef529c32c41907a29c46b
I 2011-11-05 01:08:26.072
com.google.walkaround.wave.server.util.HandlerServlet getHandler: 
getHandler(/taskqueue/import) = 
com.google.walkaround.wave.server.googleimport.ImportTaskHandler@134ea95
I 2011-11-05 01:08:26.072
com.google.walkaround.wave.server.googleimport.ImportTaskHandler doPost: 
Received task: queue import, task name 1436821488731394914, retry count 10
I 2011-11-05 01:08:26.073
com.google.walkaround.wave.server.googleimport.ImportTaskHandler doPost: 
userId=StableUserId(101674296113439515074), taskId=6007
I 2011-11-05 01:08:26.073
com.google.walkaround.wave.server.auth.AccountStore get: Fetching user 
credentials for user StableUserId(101674296113439515074)
I 2011-11-05 01:08:26.073
com.google.walkaround.util.server.RetryHelper runBodyOnce: Running body 
com.google.walkaround.wave.server.auth.AccountStore$3@11507e7
I 2011-11-05 01:08:26.075
com.google.walkaround.util.server.appengine.CheckedDatastore$1 run: Begun 
transaction CheckedTransactionImpl(Txn [wavereactor.7221239195565100975, BEGUN] 
(class com.google.appengine.api.datastore.TransactionImpl@14049d6))
I 2011-11-05 01:08:26.080
com.google.walkaround.wave.server.auth.AccountStore$3 run: Got 
AccountRecord2("u101674296113439515074")
I 2011-11-05 01:08:26.083
com.google.walkaround.util.server.RetryHelper runBodyOnce: Body exited 
normally, run time: 10ms
I 2011-11-05 01:08:26.083
com.google.walkaround.wave.server.auth.ServletAuthHelper populateContext: 
Populating UserContext(null, null, null) from record 
Record(StableUserId(101674296113439515074), vega113@gmail.com, 
OAuthCredentials(1/zdBA9Q615shX9-9N9LTWuV0C6gji5Q6G78naqgFFhv0, 
ya29.AHES6ZQrmXrbXSIcjF-bqBcUfrXwmEeZNAc2-uRpF3DNDZKCpSVOvM8))
I 2011-11-05 01:08:26.083
com.google.walkaround.wave.server.auth.ServletAuthHelper populateContext: User 
context is now UserContext(StableUserId(101674296113439515074), 
vega113@gmail.com, 
OAuthCredentials(1/zdBA9Q615shX9-9N9LTWuV0C6gji5Q6G78naqgFFhv0, 
ya29.AHES6ZQrmXrbXSIcjF-bqBcUfrXwmEeZNAc2-uRpF3DNDZKCpSVOvM8))
I 2011-11-05 01:08:26.085
com.google.walkaround.util.server.appengine.CheckedDatastore$1 run: Begun 
transaction CheckedTransactionImpl(Txn [wavereactor.8142946747478850324, BEGUN] 
(class com.google.appengine.api.datastore.TransactionImpl@997c11))
I 2011-11-05 01:08:26.085
com.google.walkaround.wave.server.googleimport.PerUserTable getTask: 
getTask(StableUserId(101674296113439515074), 6007)
I 2011-11-05 01:08:26.090
com.google.walkaround.wave.server.googleimport.PerUserTable getTask: got 
ImportTask(StableUserId(101674296113439515074), 6007, 1320480280121, 
com.google.walkaround.proto.gson.ImportTaskPayloadGsonImpl@ac10f2ef)
I 2011-11-05 01:08:26.093
com.google.walkaround.wave.server.googleimport.ImportTaskHandler handleTask: 
Task to process: ImportTask(StableUserId(101674296113439515074), 6007, 
1320480280121, 
com.google.walkaround.proto.gson.ImportTaskPayloadGsonImpl@ac10f2ef)
I 2011-11-05 01:08:26.093
com.google.walkaround.wave.server.googleimport.RobotApi search: 
search(after:2010/11/19 before:2010/11/19, 0, 300)
I 2011-11-05 01:08:26.093
com.google.walkaround.wave.server.googleimport.RobotApi callRobotApi: 
payload=[{"id":"op_id","method":"wave.robot.search","params":{"index":0,"query":
"after:2010/11/19 before:2010/11/19","numResults":300}}]
I 2011-11-05 01:08:26.093
com.google.walkaround.wave.server.auth.OAuthedFetchService fetch1: Sending 
request (token just refreshed: false): POST 
https://www-opensocial.googleusercontent.com/api/rpc
Content-Type: application/json; charset=UTF-8
I 2011-11-05 01:08:26.334
com.google.walkaround.wave.server.auth.OAuthedFetchService fetch1: response: 
200 with 136089 bytes of content
content-type: application/json; charset=UTF-8
date: Sat, 05 Nov 2011 08:08:26 GMT
expires: Sat, 05 Nov 2011 08:08:26 GMT
cache-control: private, max-age=0
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
x-xss-protection: 1; mode=block
server: GSE
x-google-cache-control: remote-fetch
via: HTTP/1.1 GWA
<content elided>
I 2011-11-05 01:08:27.390
com.google.walkaround.wave.server.googleimport.RobotApi callRobotApi: 
result={"id":"op_id","data":{"searchResults":{"digests":[{"title":"Changelog: 
Wave Robot Java Client 
Library","lastModified":1290203012321,"unreadCount":1,"waveId":"googlewave.com!w
+3djWsKA4A","snippet":"Friday, November 
19th:http://code.google.com/p/wave-robot-java-client/downloads/detail?name=wave-
robot-api-v2-20101119.zip&can=2&q=http://code.google.com/p","participants":["pam
ela.fox@googlewave.com","forumbotty+waveapi@appspot.com","public@a.gwave.com","m
prasetya@googlewave.com","exporty-bot@appspot.
I 2011-11-05 01:08:27.391
com.google.walkaround.util.server.RetryHelper runBodyOnce: Running body 
VoidBodyWrapper(com.google.walkaround.wave.server.googleimport.FindRemoteWavesPr
ocessor$2@d6ad74)
I 2011-11-05 01:08:27.394
com.google.walkaround.util.server.appengine.CheckedDatastore$1 run: Begun 
transaction CheckedTransactionImpl(Txn [wavereactor.-2823777095337069436, 
BEGUN] (class com.google.appengine.api.datastore.TransactionImpl@9fae74))
I 2011-11-05 01:08:27.394
com.google.walkaround.wave.server.googleimport.PerUserTable addRemoteWaves: 
Adding 300 digests
W 2011-11-05 01:08:27.461
com.google.walkaround.wave.server.googleimport.PerUserTable addRemoteWaves: 
Ignoring 300 existing entities; 0 remaining
I 2011-11-05 01:08:27.465
com.google.walkaround.util.server.RetryHelper runBodyOnce: Body exited 
normally, run time: 73ms
I 2011-11-05 01:08:27.465
com.google.walkaround.wave.server.googleimport.FindRemoteWavesProcessor 
storeResults: Successfully added 300 remote waves
I 2011-11-05 01:08:27.465
com.google.walkaround.wave.server.googleimport.FindRemoteWavesProcessor 
findWaves: Got 300 results between 14932 and 14933, splitting
E 2011-11-05 01:08:27.465
com.google.walkaround.wave.server.servlet.ServerExceptionFilter sendError: 
RuntimeException; sending 500: Internal server error
java.lang.RuntimeException: Can't split further; too many results (300) between 
14932 and 14933
    at com.google.walkaround.wave.server.googleimport.FindRemoteWavesProcessor.findWaves(FindRemoteWavesProcessor.java:237)
    at com.google.walkaround.wave.server.googleimport.TaskDispatcher.processTask(TaskDispatcher.java:101)
    at com.google.walkaround.wave.server.googleimport.ImportTaskHandler.handleTask(ImportTaskHandler.java:106)
    at com.google.walkaround.wave.server.googleimport.ImportTaskHandler.access$000(ImportTaskHandler.java:48)
    at com.google.walkaround.wave.server.googleimport.ImportTaskHandler$2.run(ImportTaskHandler.java:137)
    at com.google.walkaround.wave.server.auth.ServletAuthHelper.serve(ServletAuthHelper.java:184)
    at com.google.walkaround.wave.server.googleimport.ImportTaskHandler.doPost(ImportTaskHandler.java:134)
    at com.google.walkaround.wave.server.util.HandlerServlet.doPost(HandlerServlet.java:93)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
    at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)
    at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178)
    at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62)
    at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:168)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
    at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:168)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
    at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:168)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
    at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:168)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
    at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:168)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
    at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:168)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
    at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:168)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
    at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:168)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
    at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:168)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
    at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:168)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
    at com.google.walkaround.util.server.servlet.RequestStatsFilter.doFilter(RequestStatsFilter.java:95)
    at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
    at com.google.walkaround.wave.server.servlet.ServerExceptionFilter.doFilter(ServerExceptionFilter.java:108)
    at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
    at com.google.appengine.tools.appstats.AppstatsFilter.doFilter(AppstatsFilter.java:141)
    at com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
    at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
    at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
    at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at com.google.apphosting.utils.servlet.ParseBlobUploadFilter.doFilter(ParseBlobUploadFilter.java:97)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionFilter.java:35)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
    at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
    at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
    at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
    at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
    at com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionHandlerMap.java:249)
    at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
    at org.mortbay.jetty.Server.handle(Server.java:326)
    at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
    at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923)
    at com.google.apphosting.runtime.jetty.RpcRequestParser.parseAvailable(RpcRequestParser.java:76)
    at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
    at com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:135)
    at com.google.apphosting.runtime.JavaRuntime$RequestRunnable.run(JavaRuntime.java:392)
    at com.google.tracing.TraceContext$TraceContextRunnable.runInContext(TraceContext.java:449)
    at com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:455)
    at com.google.tracing.TraceContext.runInContext(TraceContext.java:695)
    at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:333)
    at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:325)
    at com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:453)
    at com.google.apphosting.runtime.ThreadGroupPool$PoolEntry.run(ThreadGroupPool.java:162)
    at java.lang.Thread.run(Thread.java:679)
I 2011-11-05 01:08:27.465
com.google.walkaround.wave.server.servlet.ServerExceptionFilter sendError: 
Trust status: UNTRUSTED
I 2011-11-05 01:08:27.466
com.google.walkaround.wave.server.servlet.ServerExceptionFilter sendError: 
Successfully sent GXP error page

Original issue reported on code.google.com by vega113 on 5 Nov 2011 at 8:12
@GoogleCodeExporter
Copy link
Author 

Sadly, this due to a limitation in wave.google.com's search
implementation: searches will only return up to 300 results.  This
affects both interactive searches and search RPCs through the robot
API.  Walkaround uses the API to search for waves by date, so if you
have 300 or more waves that have been modified on the same date,
walkaround aborts in the way you observed because it doesn't know
whether it found them all.  (I believe it still lets you import the
300 that the RPC returned.)

Can you search for after:2010/11/19 before:2010/11/19 on
wave.google.com and check whether the results are "normal" waves, or
were e.g. created by a runaway robot experiment that generated many
waves on one day?

Solutions to this problem that I can think of:
* You could add a private reply to some of the waves to move them out
of the 2010/11/19 search results.  This is obviously only a good
approach if you have just slightly over 300 waves on that day, not if
you have thousands.
* We could try to split the search space further in the hope that this
will leave us with buckets of size <300.  We could search for with:me
followed by -with:me and similar things.
* I could modify the wave.google.com servers to increase the limit
from 300 to a higher number.  This would be good to do but it's not as
easy as it sounds.

Original comment by oh...@google.com on 8 Nov 2011 at 5:34

@GoogleCodeExporter
Copy link
Author 

I see now that your problem description and the exception that you pasted are 
two different problems.

My previous response explains the exception, and I fixed that (increased limit 
to 10000).

Walkaround should be able to import individual waves except if they are too 
big.  I will work on big wave import soon.  Was the wave that you tried to 
import relatively big?  Does it work for small waves?  If you still get errors 
with the new code, please file a new bug.

Original comment by oh...@google.com on 25 Nov 2011 at 7:47

  • Changed title: Error on wave import: Can't split further; too many results (300) between 14932 and 14933
  • Changed state: Fixed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
auto-migrated Priority-Medium Type-Defect
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@GoogleCodeExporter