Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Utilities refactoring #290

Merged
merged 35 commits into from Sep 26, 2023
Merged

Utilities refactoring #290

merged 35 commits into from Sep 26, 2023

Conversation

gmaze
Copy link
Member

@gmaze gmaze commented Sep 6, 2023

It's time to refactor the 4353 lines of argopy.utilities and its 58 functions or classes !

List of changes:

Argo related dataset and Meta-data fetchers

argopy.utilities.*argopy.related.<sub>.*

Initial location New location
argopy.utilities.TopoFetcher argopy.related.TopoFetcher, argopy.related.topography.TopoFetcher
argopy.utilities.OceanOPSDeployments argopy.related.OceanOPSDeployments, argopy.related.ocean_ops_deployments.OceanOPSDeployments
argopy.utilities.ArgoNVSReferenceTables argopy.related.ArgoNVSReferenceTables, argopy.related.reference_tables.ArgoNVSReferenceTables
argopy.utilities.ArgoDocs argopy.related.ArgoDocs, argopy.related.argo_documentation.ArgoDocs
argopy.utilities.get_coriolis_profile_id argopy.related.get_coriolis_profile_id, argopy.related.euroargo_api.get_coriolis_profile_id
argopy.utilities.get_ea_profile_page argopy.related.get_ea_profile_page, argopy.related.euroargo_api.get_ea_profile_page
argopy.related.gdac_snapshot.py argopy.related.doi_snapshot.py
------------------------------ ----------------------------
argopy.utilities.load_dict argopy.related.load_dict, argopy.related.utils.load_dict
argopy.utilities.mapp_dict argopy.related.mapp_dict, argopy.related.utils.mapp_dict

Checkers

argopy.utilities.*argopy.utils.checkers.*

Initial location New location
argopy.utilities.is_box argopy.utils.is_box, argopy.utils.checkers.is_box
argopy.utilities.is_indexbox argopy.utils.is_indexbox, argopy.utils.checkers.is_indexbox
argopy.utilities.is_list_of_strings argopy.utils.is_list_of_strings, argopy.utils.checkers.is_list_of_strings
argopy.utilities.is_list_of_dicts argopy.utils.is_list_of_dicts, argopy.utils.checkers.is_list_of_dicts
argopy.utilities.is_list_of_datasets argopy.utils.is_list_of_datasets, argopy.utils.checkers.is_list_of_datasets
argopy.utilities.is_list_equal argopy.utils.is_list_equal, argopy.utils.checkers.is_list_equal
argopy.utilities.is_wmo argopy.utils.is_wmo, argopy.utils.checkers.is_wmo
argopy.utilities.check_wmo argopy.utils.check_wmo, argopy.utils.checkers.check_wmo
argopy.utilities.is_cyc argopy.utils.is_cyc, argopy.utils.checkers.is_cyc
argopy.utilities.check_cyc argopy.utils.check_cyc, argopy.utils.checkers.check_cyc
argopy.utilities.check_index_cols argopy.utils.check_index_cols, argopy.utils.checkers.check_index_cols
argopy.utilities.check_gdac_path argopy.utils.check_gdac_path, argopy.utils.checkers.check_gdac_path

Data type casting

argopy.utilities.*argopy.utils.casting.*

Initial location New location
argopy.utilities.DATA_TYPES argopy.utils.DATA_TYPES, argopy.utils.casting.DATA_TYPES
argopy.utilities.cast_Argo_variable_type argopy.utils.cast_Argo_variable_type, argopy.utils.casting.cast_Argo_variable_type

Decorators

argopy.utilities.*argopy.utils.decorators.*

Initial location New location
argopy.utilities.deprecated argopy.utils.deprecated, argopy.utils.decorators.deprecated
argopy.utilities.doc_inherit argopy.utils.doc_inherit, argopy.utils.decorators.doc_inherit

Lists

argopy.utilities.*argopy.utils.lists.*

Initial location New location
argopy.utilities.list_available_data_src argopy.utils.list_available_data_src, argopy.utils.lists.list_available_data_src
argopy.utilities.list_available_index_src argopy.utils.list_available_index_src, argopy.utils.lists.list_available_index_src
argopy.utilities.list_standard_variables argopy.utils.list_standard_variables, argopy.utils.lists.list_standard_variables
argopy.utilities.list_multiprofile_file_variables argopy.utils.list_multiprofile_file_variables, argopy.utils.lists.list_multiprofile_file_variables

Cache management

argopy.utilities.*argopy.utils.caching.*

Initial location New location
argopy.utilities.clear_cache argopy.utils.clear_cache, argopy.utils.caching.clear_cache
argopy.utilities.lscache argopy.utils.lscache, argopy.utils.caching.lscache

Accessories

argopy.utilities.*argopy.utils.accessories.*

Initial location New location
argopy.utilities.Registry argopy.utils.Registry, argopy.utils.accessories.Registry
argopy.utilities.float_wmo argopy.utils.float_wmo, argopy.utils.accessories.float_wmo

Locals (environments, versions, systems)

argopy.utilities.*argopy.utils.locals.*

Initial location New location
argopy.utilities.show_versions argopy.utils.show_versions, argopy.utils.locals.show_versions
argopy.utilities.show_options argopy.utils.show_options, argopy.utils.locals.show_options
argopy.utilities.modified_environ argopy.utils.modified_environ, argopy.utils.locals.modified_environ
argopy.utilities.get_sys_info argopy.utils.locals.get_sys_info
argopy.utilities.netcdf_and_hdf5_versions argopy.utils.locals.netcdf_and_hdf5_versions

Geo (space/time data utilities)

argopy.utilities.*argopy.utils.geo.*

Initial location New location
argopy.utilities.wmo2box argopy.utils.wmo2box, argopy.utils.geo.wmo2box
argopy.utilities.wrap_longitude argopy.utils.wrap_longitude, argopy.utils.geo.wrap_longitude
argopy.utilities.toYearFraction argopy.utils.toYearFraction, argopy.utils.geo.toYearFraction
argopy.utilities.YearFraction_to_datetime argopy.utils.YearFraction_to_datetime, argopy.utils.geo.YearFraction_to_datetime

Mathematically or statistically compute something output of xarray objects

argopy.utilities.*argopy.utils.compute.*

Initial location New location
argopy.utilities.linear_interpolation_remap argopy.utils.linear_interpolation_remap, argopy.utils.compute.linear_interpolation_remap
argopy.utilities.groupby_remap argopy.utils.groupby_remap, argopy.utils.compute.groupby_remap

Manipulate/transform xarray objects or list of objects

argopy.utilities.*argopy.utils.transform.*

Initial location New location
argopy.utilities.fill_variables_not_in_all_datasets argopy.utils.fill_variables_not_in_all_datasets, argopy.utils.manip.fill_variables_not_in_all_datasets
argopy.utilities.drop_variables_not_in_all_datasets argopy.utils.drop_variables_not_in_all_datasets, argopy.utils.manip.drop_variables_not_in_all_datasets

Manipulate Argo formatted string and print/stdout formatters

argopy.utilities.*argopy.utils.format.*

Initial location New location
argopy.utilities.format_oneline argopy.utils.format_oneline, argopy.utils.format.format_oneline
argopy.utilities.argo_split_path argopy.utils.argo_split_path, argopy.utils.format.argo_split_path

Loggers

argopy.utilities.*argopy.utils.loggers.*

Initial location New location
argopy.utilities.warnUnless argopy.utils.warnUnless, argopy.utils.loggers.warnUnless
argopy.utilities.log_argopy_callerstack argopy.utils.log_argopy_callerstack, argopy.utils.loggers.log_argopy_callerstack

Misc

Initial location New location
argopy.monitored_threadpool.MyThreadPoolExecutor argopy.utils.MonitoredThreadPoolExecutor
argopy.utilities.Chunker argopy.utils.Chunker, argopy.utils.chunking.Chunker
argopy.utilities.monitor_status argopy.utils.monitor_status, argopy.utils.monitors.monitor_status

@gmaze gmaze added the internals Internal machinery label Sep 6, 2023
@gmaze
Copy link
Member Author

gmaze commented Sep 11, 2023

Everything refactored, now need to solve for all circular import !

re-inforce deprecation warning since 0.1.13
- Use module.submodule instead of module.__init__
- removed circular imports
warning message, ensure backward compatibility until 0.1.15
@gmaze
Copy link
Member Author

gmaze commented Sep 26, 2023

doc is fixed, ci tests ok, (failed upstream not related to this pr) so let's merge

@gmaze gmaze merged commit 5d1d0b7 into master Sep 26, 2023
21 of 24 checks passed
@gmaze gmaze deleted the refactor-utils branch September 26, 2023 10:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
internals Internal machinery
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant