Big refactoring and add features #147
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Rename update-tags command to tags Rename update-datatree command to datatree
Add requests-cache optional usage
Use @Property for self.provider in Fetchers Use local cache for pandas.Period for A, M, Q, W frequencies only Remove duplicate logger in update_mongo_collection() method Add fields and remove unused functions in Providers class Add new Categories class Refactoring datatree Update and fixes datasets_list method Add fields to Datasets class Add max errors process to Datasets.update_database() method Add SeriesIterator class for base of all Fetcher data class Big refactoring for Series class: - Use customs exceptions for reject series - Optimize bulk - Update only selected fields in bson - Refactoring revisions Update tests for _commons
Use cache for pandas.Period Refactoring data-tree Fixe frequency bug Clean code Refactoring all tests
Use cache for pandas.Period Remove PandaSDMX Refactoring all tests
Refactoring provider schema Change schemas for Datasets and Series
Fixes typos Update provider init Refactoring data tree Refactoring get_selected_datasets Add new test module Rename and skip tests for old test module
Refactoring data tree Refactoring get_selected_datasets Use SeriesIterator for data class Use generics tests
Add fixe for not compatible dimensions names Use SeriesIterator for data class Fixe dimensions and attributes storage Use generics tests
Refactoring all process Add compatibility for futures datasets Add limit update for WEO Use shared downloader Refactoring series structure Add generics tests
Use cache for pandas.Period Refactoring data tree Use SeriesIterator for data class Add process for split sdmx download (structure and data) Add RejectUpdatedSeries exception for updated series Use generics tests
Normalize datetime for download_xxx fields
Insee: optimize codelists loading
Move datasets_list from fetcher to _commons Move Series revision and update functions for testability Add fetchers.get_selected_datasets() method Add optional Fetcher.categories_filter and Fetcher.datasets_filter Add many controls for datas processing Add option for disable validation schema (codedict and series only) Add Fetcher.wrap_upsert_dataset() for call upsert_dataset in all fetchers Add Fetcher.get_calendar() method Remove automatic recording for Provider Remove old Fetcher.update_categories() method
Use cache for pandas.Period Remove tmp file after process Normalize base patch for all downloads (self.store_path) Use clean_datetime function Update frequencies using by dataset
MichelJuillard
added a commit
that referenced
this pull request
Feb 9, 2016
Big refactoring and add features
|
@srault95 I merged the PR, but OECD doesn't pass the tests |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.