NOAA NCEI Ocean Winds

[gabefair]

DO NOT START, SEE OTHER TICKETS BELOW

OceanWinds

• Agency: NOAA
• Agency Division: NCEI
• Data Type: Oceanographic
• Data Format:
• FTP/HTTP URL: ftp://podaac.jpl.nasa.gov/OceanWinds/

/OceanWinds/ has the following children:
aquarius/ 2/25/13, 12:00:00 AM : addressed in #236
ascat/ 5/7/13, 12:00:00 AM : duplicate of #237
ccmp/ 5/7/13, 12:00:00 AM : duplicate of #239
monthly_mean_atlas/ 5/7/13, 12:00:00 AM : duplicate of #327
nimbus7/ 10/25/11, 12:00:00 AM : duplicate of #326
nscat/ 5/10/13, 12:00:00 AM : duplicate of #263 & #196
oceansat2/ 5/10/13, 12:00:00 AM : duplicate of #264
poet_ifremer/ 4/25/12, 12:00:00 AM : addressed in #330
quikscat/ 6/22/16, 12:00:00 AM : duplicate of #265
rapidscat/ 5/14/16, 12:00:00 AM : duplicate of #266
seasat/ 5/10/13, 12:00:00 AM : duplicate of #209
seawinds/ 5/10/13, 12:00:00 AM : duplicate of #297
ssmi/ 5/10/13, 12:00:00 AM : duplicate of #295
windsat/ 7/29/13, 12:00:00 AM : duplicate of #296
woce/ 5/7/13, 12:00:00 AM : duplicate of #325

Once you are done, please share:

• The method/command you used to download
• The sha256 hash of the folder hashdeep -erl
• a url to the mirror or copy of the files (if you have it)

We are keeping this ticket open for logistical reasons until the child tickets are closed.

[baobrien]

Downloading now.

[gabefair]

@baobrien This one is massive. Hows it coming?
We can break it up if it would be better managed in smaller chuncks

[gabefair]

Did you use command: wget -N -m ftp://podaac.jpl.nasa.gov/OceanWinds/*
? The -N and -m will resume where you left off if you have to restart. We want to make sure we got the entire dataset.

[k15z]

I'm downloading it too, it's going slowly. Currently working on aquarius, ascat, and ccmp - downloaded ~200 GB so far.

[provos]

Getting a copy of this, too.

[JeremiahCurtis]

Are the above subfolders identical to ftp://podaac-ftp.jpl.nasa.gov/allData/seawinds/ and ftp://podaac-ftp.jpl.nasa.gov/allData/windsat/ ?

[bkirkbri]

Many of the directories in ftp://podaac.jpl.nasa.gov/OceanWinds seem to overlap with /allData/... which is broken into many issues.

[gabefair]

@provos , @k15z how is it coming along? Can you provide us links to your dataset once you are done? Thanks

[provos]

At 345G for allData/aquarius right now and 145G on allData/modis. I hope the 3TB I have for serving will be enough.

[provos]

At 490G for aquarius and 450G for modis.

[bkirkbri]

It does seem like /OceanWinds is symlinked into ../allData and duplicates the issues above.

@provos Did you fetch from ftp://podaac.jpl.nasa.gov/allData/aquarius rather than the issue URL of ftp://podaac.jpl.nasa.gov/OceanWinds/aquarius? If so, will you post completed mirror info in #236 as that issue is more specific?

[bkirkbri]

aquarius consists only of symlinks into ../allData/aquarius and is definitely a duplicate of #236

[bkirkbri]

@gabefair Thanks for tracking down the duplicates. I've verified that all content is a duplicate (creating a few new issues that didn't exist).

Anyone having a partial mirror of this issue should consult the duplicates list at the top and post their complete submirrors in the appropriate issue. Thank you!