Unable to access data in the Data_exploration_template notebook (404 error)

[darlev]

Hello,
I've tried to use the notebook here and got a 404 error in the 6th cell while trying to access the following URL:
http://cardshark.cs.berkeley.edu/datastreams/find_entries/timestamp
Is there something wrong with the credentials given in the input? Is there another way to access the public data? Thank you!

[shankari]

@darlev thanks for filing this issue!

Alas I had to move off cardshark after I graduated from Berkeley. We are exploring other options for hosting data. @singhish is also working to cleanup the notebooks before we launch a formal challenge around this dataset.

I should be able to have an alternate server up in a week or so.

[shankari]

@darlev if you need to access the dataset in the interim, I have a mongodump of all the data at:
https://damajash.org/~shankari/mobilitynet_mongodump.zip

So temporary instructions would be:

• Download the mobilitynet dataset (https://damajash.org/~shankari/mobilitynet_mongodump.zip)
• Install the e-mission server from the emevalzephyr branch https://github.com/e-mission/e-mission-server/tree/emevalzephyr
• mongorestore the dataset to the mongodb from your local installation (either docker or local install)
• Change the DATASTORE_URL in the notebook from http://cardshark.cs.berkeley.edu to http://localhost:8080
• Run the notebook

[darlev]

@shankari Thank you for your help! I was able to access the dataset via the API, but I've noticed that the "config/evaluation_spec" key does not exist anywhere in the dataset. Is there another way to access the ground truth trajectories and transportation modes of each trip?

[shankari]

@darlev first, a couple of high level updates:

• we are planning to run this as a challenge
• as part of this, we (aka @singhish) are actively working on hosting it in a better long-term location and modifying the scripts to match

Stay tuned for additional details.

Second, when you say "I was able to access the dataset via the API", you mean that you mongorestored the dataset and then loaded it from localhost?

but I've noticed that the "config/evaluation_spec" key does not exist anywhere in the dataset

can you indicate how you looked for this? @singhish has followed the same steps and is able to access the evaluation spec properly.

[darlev]

Second, when you say "I was able to access the dataset via the API", you mean that you mongorestored the dataset and then loaded it from localhost?

Yes, I downloaded the dataset and have worked on it locally. Regarding the "config/evaluation_spec" key, I found the issue. The BuiltinTimeSeries class here is missing some keys, such as "config/evaluation_spec". Adding it and other missing keys as self.timeseries_db types to the list seem to have done the trick.

Thanks again.

[shankari]

@darlev ah yes, you need to use the emevalzephyr branch on the e-mission server.
https://github.com/e-mission/e-mission-server/tree/emevalzephyr

I've edited #13 (comment)
to fix that

[shankari]

@darlev

Couple of notes:

• we finally merged a fix to support multiple temporal reroutes in the spec, and used it to fix the UCB timeline (Support reroutes in the ground truth #11)
• And I finally set up the alternate server (http://exv-91200.sb.dfki.de:6638) and updated the specs on it

So if you replace http://localhost:8080 with http://exv-91200.sb.dfki.de:6638:

• everything should still work
• you should get more data for the Berkeley timeline

Let me know if you have any problems.