Skip to content
This repository has been archived by the owner on Nov 11, 2020. It is now read-only.

S3 Seamless Census Sources fail #189

Closed
ansoncfit opened this issue Nov 16, 2018 · 1 comment
Closed

S3 Seamless Census Sources fail #189

ansoncfit opened this issue Nov 16, 2018 · 1 comment
Assignees
Labels

Comments

@ansoncfit
Copy link
Member

This may have to do with tiles not existing in the ocean. Fetching LODES data for Long Island, for example:

22:35:21.889 [pool-2-thread-3] INFO  c.c.data.census.SeamlessSource - Read 167000 features
22:35:21.907 [pool-2-thread-3] INFO  c.c.data.census.SeamlessSource - Read 168000 features
Exception in thread "pool-2-thread-3" com.amazonaws.services.s3.model.AmazonS3Exception: The specified key does not exist. (Service: Amazon S3; Status Code: 404; Error Code: NoSuchKey; Request ID: 072F0D38F9BCF756; S3 Extended Request ID: IX34emy8ScvS1eGqydWnm8Vby0JxAbLuEbo0o72q2wJnfLSHcl0720IcZcgDwiq9A39nQ6U7CjA=), S3 Extended Request ID: IX34emy8ScvS1eGqydWnm8Vby0JxAbLuEbo0o72q2wJnfLSHcl0720IcZcgDwiq9A39nQ6U7CjA=
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1632)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1304)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1058)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:743)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:717)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
        at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513)
        at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4365)
        at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4312)
        at com.amazonaws.services.s3.AmazonS3Client.getObject(AmazonS3Client.java:1415)
        at com.conveyal.taui.grids.SeamlessCensusGridExtractor$S3SeamlessSource.getInputStream(SeamlessCensusGridExtractor.java:40)
        at com.conveyal.data.census.SeamlessSource.extract(SeamlessSource.java:71)
        at com.conveyal.data.census.SeamlessSource.extract(SeamlessSource.java:41)
        at com.conveyal.taui.grids.SeamlessCensusGridExtractor.retrieveAndExtractCensusDataForBounds(SeamlessCensusGridExtractor.java:53)
        at com.conveyal.taui.controllers.OpportunityDatasetController.lambda$downloadLODES$3(OpportunityDatasetController.java:119)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
@ansoncfit ansoncfit added the bug label Nov 16, 2018
@ansoncfit ansoncfit self-assigned this Nov 16, 2018
This was referenced Nov 16, 2018
ansoncfit added a commit that referenced this issue Nov 16, 2018
@ansoncfit
Copy link
Member Author

It appears when we switched to manually triggered downloads, the try/catch block from seamless-census was not ported over to analysis-backend.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

1 participant