Using Data from CMEMS and fusing SST and SLA Data

[scallen01]

Gday,

I am trying to use pyeddytracker to detect eddies along the East Australian Coast is a very similar way to Baixin Li, et al (2022) ["A Dual-Attention Mechanism Deep Learning Network for Mesoscale Eddy Detection by Mining Spatiotemporal Characteristics"]. They used a dual-attention mechanism with CA, PA and ConvLSTM modules and referred to this repository as the open source for that network.

My question is, how do I get the algorithm to beginning the identification and tracking of eddies using either SST or SLA data retrieved from CMEMS (files in the format of .nc)?

Apologies if this a silly question but I did try to follow the README documents and other documents to get this working.

I hope this finds you well,

Scott

[AntSimi]

Sorry for the delay in my reply,
I am not sure to understand which help did you need.
Robust method available in py eddy tracker for eddy eddection are only for sla.
Did you achieved identification?
Could you be more specific ?

[scallen01]

Gday Antoine,

In addition to my below response, I was wondering if you used fsspec to create the Anticyclonic.zarr and Cyclonic.zarr files from the identification (Anticyclonic-*.nc) files?

Hope this finds you well,

Scott

[scallen01]

Good morning Antoine, I have now achieved identification and have a series of “Anticyconlic/Cyclonic”_*.nc files and am now trying to complete some statical plots and tracking plots. I can seem to either produce a file including these identification files like your “eddies_med_adt_allsat_dt2018” file or correctly write the code to just have the list of identification files. Is it better to have the .zarr files or to just list the .n identification files? If the .zarr file is better what is the process to produce these from the identification files? On a separate note, I have downloaded CMEMs data with daily images over a year within one .nc file, however, I can’t seem to get the identification steps to recognise the different days which are a third dimension of the files. Is this just the way py-eddy-tracker’ is designed? I have included my identification and statistical code files for reference. Thank you for the help, Scott

[AntSimi]

First zarr & nc are a format to store data, which could be used both for tracking or identification.

• If you want just do statistics on identification without tracking you could concatenate all your observations with concatenate method.
• If you want tracking before to do some statistics, you could follow this
• To do identification on specific index you could follow this

[scallen01]

Apologies @AntSimi if this is a very dumb question but once the tracking code has run and the four files are created. How would I go about completing say the Lifetime Histogram plots?

[AntSimi]

Just load your own file with load_file

[scallen01]

Good morning @AntSimi, I have tried a few plotting examples such as lifetime histogram and birth-and-death plot which use the zarr file which I have created through python from the cyclonic.nc or anticyclonic.nc from the tracking processes (keeping only the variable you have in your sample set). I am using the below code to plot, however, nothing shows on the plot. Is it something to do with the period?

`from matplotlib import pyplot as plt
from numpy import arange, ones
#from py_eddy_tracker.data import get_demo_path

from py_eddy_tracker.observations.tracking import TrackEddiesObservations`

a = TrackEddiesObservations.load_file("/Users/scottallen/Library/CloudStorage/OneDrive-UNSW/Honours/Thesis/Thesis_Data/Summer2020-2021_Tracks/Anticyclonic.zarr") c = TrackEddiesObservations.load_file("/Users/scottallen/Library/CloudStorage/OneDrive-UNSW/Honours/Thesis/Thesis_Data/Summer2020-2021_Tracks/Cyclonic.zarr") nb_year = (a.period[1] - a.period[0] + 1) / 365.25

`figure = plt.figure(figsize=(12, 8))
ax_ratio_cum = figure.add_axes([0.55, 0.06, 0.42, 0.34])
ax_ratio = figure.add_axes([0.07, 0.06, 0.46, 0.34])
ax_cum = figure.add_axes([0.55, 0.43, 0.42, 0.54])
ax = figure.add_axes([0.07, 0.43, 0.46, 0.54])
ax.set_ylabel("Eddies by year")
ax_ratio.set_ylabel("Ratio Cyclonic/Anticyclonic")
for ax_ in (ax, ax_cum, ax_ratio_cum, ax_ratio):
ax_.set_xlim(0, 400)
if ax_ in (ax, ax_cum):
ax_.set_ylim(1e-1, 1e4), ax_.set_yscale("log")
else:
ax_.set_xlabel("Lifetime in days (by week bins)")
ax_.set_ylim(0, 2)
ax_.axhline(1, color="g", lw=2)
ax_.grid()
ax_cum.xaxis.set_ticklabels([]), ax_cum.yaxis.set_ticklabels([])
ax.xaxis.set_ticklabels([]), ax_ratio_cum.yaxis.set_ticklabels([])

plot data

bin_hist = arange(7, 2000, 7)
x = (bin_hist[1:] + bin_hist[:-1]) / 2.0
a_nb, c_nb = a.nb_obs_by_track, c.nb_obs_by_track
a_nb, c_nb = a_nb[a_nb != 0], c_nb[c_nb != 0]
w_a, w_c = ones(a_nb.shape) / nb_year, ones(c_nb.shape) / nb_year
kwargs_a = dict(histtype="step", bins=bin_hist, x=a_nb, color="r", weights=w_a)
kwargs_c = dict(histtype="step", bins=bin_hist, x=c_nb, color="b", weights=w_c)
cum_a, _, _ = ax_cum.hist(cumulative=-1, **kwargs_a)
cum_c, _, _ = ax_cum.hist(cumulative=-1, **kwargs_c)
nb_a, _, _ = ax.hist(label="Anticyclonic", **kwargs_a)
nb_c, _, _ = ax.hist(label="Cyclonic", **kwargs_c)
ax_ratio_cum.plot(x, cum_c / cum_a)
ax_ratio.plot(x, nb_c / nb_a)
ax.legend()`

[AntSimi]

To help i will need more informations.
Share result of EddyInfos scripts

Run like this in your terminal
EddyInfos Anticyclonic.zarr

You must get a result like this:

-- Anticyclonic.zarr -- 
    | 32054678 observations from 15706.0 to 25633.0 (9928.0 days, ~3229 obs/day)
    |   Speed area      : 36.77 Mkm²/day
    |   Effective area  : 49.99 Mkm²/day
    ----Distribution in Amplitude:
    |   Amplitude bounds (cm)        0.00      1.00      2.00      3.00      4.00      5.00     10.00    500.00
    |   Percent of eddies         :      17.17     23.45     15.16     10.10      6.48     16.90     10.75
    ----Distribution in Radius:
    |   Speed radius (km)            0.00     15.00     30.00     45.00     60.00     75.00    100.00    200.00   2000.00
    |   Percent of eddies         :       0.03      8.51     32.86     23.91     14.37     12.15      8.05      0.12
    |   Effective radius (km)        0.00     15.00     30.00     45.00     60.00     75.00    100.00    200.00   2000.00
    |   Percent of eddies         :       0.02      6.60     24.67     21.16     15.74     16.49     14.97      0.37
    ----Distribution in Latitude
        Latitude bounds            -90.00    -60.00    -15.00     15.00     60.00     90.00
        Percent of eddies         :       5.69     48.38     13.32     28.99      3.63
        Percent of speed area     :       2.96     42.98     27.05     25.69      1.32
        Percent of effective area :       3.03     44.00     25.66     26.02      1.29
        Mean speed radius (km)    :      42.58     54.72     84.21     54.48     36.13
        Mean effective radius (km):      49.88     64.23     95.87     63.66     41.37
        Mean amplitude (cm)       :       3.71      5.45      2.08      4.69      2.79
    | 714863 tracks (44.84 obs/tracks, shorter 10 obs, longer 2629 obs)
    |   1668085 filled observations (2.33 obs/tracks, 5.20 % of total)
    |   Intepolated speed area      : 1.68 Mkm²/day
    |   Intepolated effective area  : 1.96 Mkm²/day
    |   Distance by day             : Mean 6.61 , Median 3.80 km/day
    |   Distance by track           : Mean 289.96 , Median 170.70 km/track
    ----Distribution in lifetime:
    |   Lifetime (days  )            1.00     30.00     90.00    180.00    270.00    365.00   1000.00  10000.00
    |   Percent of tracks         :      56.67     31.88      8.31      1.98      0.67      0.48      0.02
    |   Percent of eddies         :      21.75     35.62     22.70      9.50      4.64      5.30      0.48

[scallen01]

Hi there @AntSimi,

I have tried to produce the same result as you have above but the terminal returned this error:

(Thesis_Env) scottallen@scotts-air Summer2020-2021_Tracks % EddyInfos Anticyclonic.zarr
Traceback (most recent call last):
  File "/Users/scottallen/anaconda3/envs/Thesis_Env/bin/EddyInfos", line 33, in <module>
    sys.exit(load_entry_point('pyEddyTracker==0+unknown', 'console_scripts', 'EddyInfos')())
  File "/Users/scottallen/anaconda3/envs/Thesis_Env/lib/python3.7/site-packages/py_eddy_tracker/appli/eddies.py", line 131, in display_infos
    with Dataset(filename) as h:
  File "src/netCDF4/_netCDF4.pyx", line 2464, in netCDF4._netCDF4.Dataset.__init__
  File "src/netCDF4/_netCDF4.pyx", line 2027, in netCDF4._netCDF4._ensure_nc_success
OSError: [Errno -51] NetCDF: Unknown file format: 'Anticyclonic.zarr'

When I run it showing the Anticyclonic.nc file the tracking system produces it returns this:

-- Anticyclonic.nc -- 
/Users/scottallen/anaconda3/envs/Thesis_Env/lib/python3.7/site-packages/py_eddy_tracker/observations/observation.py:302: RuntimeWarning: invalid value encountered in true_divide
  v /= hist_numba(x, bins=bins)[0]
    | 13888 observations from 25902.0 to 25993.0 (92.0 days, ~151 obs/day)
    |   Speed area      : 1.64 Mkm²/day
    |   Effective area  : 2.27 Mkm²/day
    ----Distribution in Amplitude:
    |   Amplitude bounds (cm)        0.00      1.00      2.00      3.00      4.00      5.00     10.00    500.00
    |   Percent of eddies         :      10.30     18.15     14.44     10.67      7.88     19.77     18.79
    ----Distribution in Radius:
    |   Speed radius (km)            0.00     15.00     30.00     45.00     60.00     75.00    100.00    200.00   2000.00
    |   Percent of eddies         :       0.00      8.83     33.15     23.15     15.04     13.05      6.78      0.00
    |   Effective radius (km)        0.00     15.00     30.00     45.00     60.00     75.00    100.00    200.00   2000.00
    |   Percent of eddies         :       0.00      6.01     24.07     20.45     14.90     20.02     14.47      0.06
    ----Distribution in Latitude
        Latitude bounds            -90.00    -60.00    -15.00     15.00     60.00     90.00
        Percent of eddies         :       0.00     83.18     16.82      0.00      0.00
        Percent of speed area     :       0.00     74.42     25.58      0.00      0.00
        Percent of effective area :       0.00     76.18     23.82      0.00      0.00
        Mean speed radius (km)    :        nan     53.22     70.48       nan       nan
        Mean effective radius (km):        nan     63.50     80.77       nan       nan
        Mean amplitude (cm)       :        nan      7.48      2.13       nan       nan
    | 445 tracks (31.21 obs/tracks, shorter 10 obs, longer 92 obs)
    |   885 filled observations (1.99 obs/tracks, 6.37 % of total)
    |   Intepolated speed area      : 0.08 Mkm²/day
    |   Intepolated effective area  : 0.11 Mkm²/day
    |   Distance by day             : Mean 7.52 , Median 4.30 km/day
    |   Distance by track           : Mean 227.09 , Median 170.51 km/track
    ----Distribution in lifetime:
    |   Lifetime (days  )            1.00     30.00     90.00    180.00    270.00    365.00   1000.00  10000.00
    |   Percent of tracks         :      61.80     34.16      4.04      0.00      0.00      0.00      0.00
    |   Percent of eddies         :      34.28     53.84     11.88      0.00      0.00      0.00      0.00

Thank you again for your help,

Scott