Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Worker does not report loaded network to broker #293

Closed
abyrd opened this issue Jun 30, 2017 · 3 comments
Closed

Worker does not report loaded network to broker #293

abyrd opened this issue Jun 30, 2017 · 3 comments

Comments

@abyrd
Copy link
Member

abyrd commented Jun 30, 2017

On analysis.conveyal.com, switching into analysis mode on a project using R5 v2.3.0. The worker completes pre-building the network and sits idle. The frontend says it's still waiting for the cluster to start up. The broker sees the worker, but does not associate it with any network or scenario IDs. Perhaps the worker is not sending them:

{
  "workerId": "59545a26fe874f35b6cb69a200e1a0ec",
  "architecture": "amd64",
  "processors": 4,
  "loadAverage": 0,
  "osName": "Linux",
  "osVersion": "4.9.17-8.31.amzn1.x86_64",
  "memoryMax": 7151288320,
  "memoryTotal": 3699900416,
  "memoryFree": 1084794592,
  "workerName": "R5",
  "workerVersion": "v2.3.0",
  "networks": [],
  "scenarios": [],
  "tasksPerMinute": 0,
  "ec2region": "eu-west-1",
  "ec2instanceId": "i-0ec8fa20a299bd1e4",
  "ec2instanceType": "r3.xlarge",
  "ec2machineImage": "ami-e5083683",
  "jvmStartTime": 1498795189,
  "jvmUptime": 2047,
  "jvmName": "OpenJDK 64-Bit Server VM",
  "jvmVendor": "Oracle Corporation",
  "jvmVersion": "25.131-b11",
  "seenSecondsAgo": 7
}
@abyrd
Copy link
Member Author

abyrd commented Jul 10, 2017

@mattwigway have you run into this issue with recent workers? I'm wondering if this is happening on all networks or there's something specific about this Swiss network. When the worker reports to the broker, the networks and scenarios fields are zero-length arrays.

@abyrd
Copy link
Member Author

abyrd commented Jul 11, 2017

I now suspect this is because the network is never being fully built:

06:04:43
[main] WARN com.conveyal.r5.labeling.LevelOfTrafficStressLabeler - Unable to parse maxspeed tag FR:urban
06:04:43
[main] WARN com.conveyal.r5.labeling.LevelOfTrafficStressLabeler - Unable to parse maxspeed tag AT:urban
06:04:43
[main] WARN com.conveyal.r5.labeling.LevelOfTrafficStressLabeler - Unable to parse maxspeed tag AT:urban
06:04:43
[main] WARN com.conveyal.r5.labeling.LevelOfTrafficStressLabeler - Unable to parse maxspeed tag AT:urban
06:04:43
[main] WARN com.conveyal.r5.labeling.LevelOfTrafficStressLabeler - Unable to parse maxspeed tag AT:urban
06:04:43
[main] INFO com.conveyal.r5.analyst.cluster.AnalystWorker - Done pre-loading network 77cc368d631747b9a64fa2ebc6e2cbe4

Note that it goes straight from reporting problems with the maxspeed tags to saying it's done pre-loading the network. That shouldn't happen.

@abyrd
Copy link
Member Author

abyrd commented Jul 11, 2017

There is in fact no problem with reporting loaded networks. The cache was silently consuming an exception and returning null, so there were in fact no networks loaded. See #301

@abyrd abyrd closed this as completed Jul 11, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant