Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
VSO query result block level caching #1785
This checks each existing entry in the database and checks if the user already has the files for each Query Result Block.
Each VSO query result is a list of query response blocks. So there are cases in which 2 queries have a large number of same query response blocks. For example, if one query is a subset of another. So this feature will download files for a response block if :
Initially we download files for a Query A which has 5 response blocks, say 1, 2, 3, 4, 5.
Now we want to download files for a Query B which has 5 response blocks, say 4, 5, 6, 7, 8. So response blocks 4 and 5 are common with Query A, and their files have already been downloaded.
Old behaviour : Files for 4, 5, 6, 7 and 8 are downloaded.
New behaviour : Files for 6, 7 and 8 are downloaded.
Could you update this section of the docs as well: http://docs.sunpy.org/en/stable/guide/acquiring_data/database.html#adding-entries-using-the-vso-interface
The old cache is checked first. See here
The new caching only comes into play if there is no hit in the old caching system.
Consider a query A which is already made before.
Removing the old caching system will lead to considerable slowdown in Case 1.
referenced this pull request
Aug 11, 2016
The old caching system should be removed as it becomes a special case of the new one. Also it clashes with the new caching system.
So the behavior now should be :
Suggest edits :)
This pull request has been automatically marked as stale because it has not had any activity for the past five months. It will be closed if no further activity occurs. If the ideas in this pull request are still worth implementing, please make sure you open an issue to keep track of that idea!