[WIP] Data fetcher for a local GDAC ftp copy

[gmaze]

Many users or institutes work with a local copy of a GDAC ftp.

GDAC ftp (from Argo-France at Coriolis or Argo-US at AOML) are the official, and only reliable, Argo data access point.

Here, I want to merge work from https://github.com/euroargodev/argopy-osnet to the official argopy.

The resulting API may look like:

    from argopy import DataFetcher as ArgoDataFetcher
    argo_loader = ArgoDataFetcher(backend='localftp')

    argo_loader.region([-85,-45,10.,20.,0,1000.]).to_xarray()
    argo_loader.profile(6902746, 34).to_xarray()
    argo_loader.float([6902746, 6902747, 6902757, 6902766]).to_xarray()

[codecov]

Codecov Report

❗ No coverage uploaded for pull request base (master@8db54e3). Click here to learn what that means.
The diff coverage is 77.53%.

@@            Coverage Diff            @@
##             master       #5   +/-   ##
=========================================
  Coverage          ?   77.76%           
=========================================
  Files             ?       10           
  Lines             ?     1030           
  Branches          ?        0           
=========================================
  Hits              ?      801           
  Misses            ?      229           
  Partials          ?        0

Flag	Coverage Δ
#unittests	77.47% <77.07%> (?)

Impacted Files	Coverage Δ
setup.py	0% <ø> (ø)
argopy/errors.py	63.15% <63.15%> (ø)
argopy/xarray.py	66.09% <63.83%> (ø)
argopy/data_fetchers/proto.py	71.42% <71.42%> (ø)
argopy/data_fetchers/localftp.py	81.56% <81.56%> (ø)
argopy/utilities.py	65.95% <81.81%> (ø)
argopy/fetchers.py	90% <84.31%> (ø)
argopy/tutorial.py	91.37% <91.37%> (ø)
argopy/data_fetchers/erddap.py	82.82% <94.33%> (ø)
argopy/options.py	96.66% <96.66%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8db54e3...58f60fe. Read the comment docs.

[quai20]

I couldn't test much but no surprises so far. I let you merge the branch, I'll test more on master.

[gmaze]

What's new

Features and front-end API

• Added new data fetcher backend localftp in ArgoDataFetcher (c5f7cb6):

from argopy import DataFetcher as ArgoDataFetcher
argo_loader = ArgoDataFetcher(backend='localftp', path_ftp='/data/Argo/ftp_copy')
argo_loader.float(6902746).to_xarray()

• Introduced global OPTIONS to set values for: cache folder, dataset (eg:'phy' or 'bgc'), local ftp path, data fetcher ('erddap' or 'localftp') and user level ('standard' or 'expert'). Can be used in context with (83ccfb5):

with argopy.set_options(mode='expert', datasrc='erddap'):
    ds = argopy.DataFetcher().float(3901530).to_xarray()

• Added a argopy.tutorial module to be able to load sample data for documentation and unit testing (4af09b5):

ftproot, flist = argopy.tutorial.open_dataset('localftp')
txtfile = argopy.tutorial.open_dataset('weekly_index_prof')

• Improved xarray argo accessor. Added methods for casting data types, to filter variables according to data mode, to filter variables according to quality flags. Useful methods to transform collection of points into collection of profiles, and vice versa (14cda55):

ds = argopy.DataFetcher().float(3901530).to_xarray() # get a collection of points
dsprof = ds.argo.point2profile() # transform to profiles
ds = dsprof.argo.profile2point() # transform to points

• Changed License from MIT to Apache (25f90c9)

Inner machinery

• Add __all__ to control from argopy import * (83ccfb5)
• All data fetchers inherit from class ArgoDataFetcherProto in proto.py (44f45a5)
• Data fetchers use default options from global OPTIONS
• In Erddap fetcher: methods to cast data type, to filter by data mode and by QC flags are now delegated to the xarray argo accessor methods.
• Data fetchers methods to filter variables according to user mode are using variable lists defined in utilities.
• argopy.utilities augmented with listing functions of: backends, standard variables and multiprofile files variables.
• Introduce custom errors in errors.py (2563c9f)
• Front-end API ArgoDataFetcher uses a more general way of auto-discovering fetcher backend and their access points. Turned of the deployments access point, waiting for the index fetcher to do that.
• Improved xarray argo accessor. More reliable point2profile and data type casting with cast_type

Code management

• Add CI with github actions (ecbf9ba)
• Contribution guideline for data fetchers (b332495)
• Improve unit testing (all along commits)
• Introduce code coverage (b490ab5)
• Added explicit support for python 3.6 , 3.7 and 3.8 (58f60fe)