[DataCap Application] Tech Greedy - NOAA Global Ensemble Forecast System (#3)

[xinaxu]

Data Owner Name

National Oceanic and Atmospheric Administration (NOAA)

Data Owner Country/Region

United States

Data Owner Industry

Environment

Website

https://registry.opendata.aws/noaa-gefs/

Social Media

https://registry.opendata.aws/noaa-gefs/

Total amount of DataCap being requested

5PiB

Weekly allocation of DataCap requested

1PiB

On-chain address for first allocation

f1l2cc5vuw5moppwsjd3b7cjtwa2exowqo36esklq

Data Type of Application

Public, Open Dataset (Research/Non-Profit)

Custom multisig

• Use Custom Multisig

Identifier

No response

Share a brief history of your project and organization

Project Detail: This dataset is an AWS open dataset that has yet been stored on Filecoin. We were planning to onboard this dataset with Slingshot V3, however, with the delay of the program, we want to start onboarding asap. Meanwhile, we want to reach out to storage providers outside of slingshot silo to expand the distribution variety further.

Organization Detail: Tech Greedy has been engaged in Filecoin ecosystem building including building data preparation and deal-making tool, participating in multiple rounds of slingshot and having mining operations.

Is this project associated with other projects/ecosystem stakeholders?

Yes

If answered yes, what are the other projects/ecosystem stakeholders

https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/1483
https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/1682

Describe the data being stored onto Filecoin

The Global Ensemble Forecast System (GEFS), previously known as the GFS Global ENSemble (GENS), is a weather forecast model made up of 21 separate forecasts, or ensemble members. The National Centers for Environmental Prediction (NCEP) started the GEFS to address the nature of uncertainty in weather observations, which is used to initialize weather forecast models. The GEFS attempts to quantify the amount of uncertainty in a forecast by generating an ensemble of multiple forecasts, each minutely different, or perturbed, from the original observations. With global coverage, GEFS is produced four times a day with weather forecasts going out to 16 days.

The total size of the dataset is ~2.0PiB so we will likely need more application in the future.

Where was the data currently stored in this dataset sourced from

AWS Cloud

If you answered "Other" in the previous question, enter the details here

No response

How do you plan to prepare the dataset

singularity

If you answered "other/custom tool" in the previous question, enter the details here

No response

Please share a sample of the data

https://noaa-nbm-pds.s3.amazonaws.com/index.html

Confirm that this is a public dataset that can be retrieved by anyone on the Network

• I confirm

If you chose not to confirm, what was the reason

No response

What is the expected retrieval frequency for this data

Monthly

For how long do you plan to keep this dataset stored on Filecoin

1.5 to 2 years

In which geographies do you plan on making storage deals

Greater China, Asia other than Greater China, North America, Europe

How will you be distributing your data to storage providers

HTTP or FTP server

How do you plan to choose storage providers

Slack, Big data exchange, Partners

If you answered "Others" in the previous question, what is the tool or platform you plan to use

No response

If you already have a list of storage providers to work with, fill out their names and provider IDs below

f01761579 - FeiGe IT LLC, Anhui, China
f0750779 - OuRuan IT LLC, Chengdu, China
f01969339 - Acrontech, Atlanta, Georgia, US
f01969323 - Acrontech, Atlanta, Georgia, US
f01969306 - Acrontech, Atlanta, Georgia, US
f01832393 - Tech Greedy, Seattle, Washington, US
f01985775 - Greater Heat, Dallas, Texas, US
f033462 - Greater Heat, Dallas, Texas, US
f01989866 - Weimin, Xi'an, CN
f01907545 - Weimin, Xi'an, CN

How do you plan to make deals to your storage providers

Singularity

If you answered "Others/custom tool" in the previous question, enter the details here

No response

Can you confirm that you will follow the Fil+ guideline

Yes

[large-datacap-requests]

Thanks for your request!
Everything looks good. 👌

A Governance Team member will review the information provided and contact you back pretty soon.

[Sunnyiscoming]

Datacap Request Trigger

Total DataCap requested

5PiB

Expected weekly DataCap usage rate

1PiB

Client address

f1l2cc5vuw5moppwsjd3b7cjtwa2exowqo36esklq

[large-datacap-requests]

DataCap Allocation requested

Multisig Notary address

f02049625

Client address

f1l2cc5vuw5moppwsjd3b7cjtwa2exowqo36esklq

DataCap allocation requested

256TiB

Id

47aa6982-f8f0-4a27-b113-4e45e4c4198d

[kernelogic]

In support for well known client and open dataset .

[kernelogic]

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacebpb5ekbpmibksvg7l6f6menttuwv3py5vdjw7sjy2fcbarnwvjrm

Address

f1l2cc5vuw5moppwsjd3b7cjtwa2exowqo36esklq

Datacap Allocated

256.00TiB

Signer Address

f1yjhnsoga2ccnepb7t3p3ov5fzom3syhsuinxexa

Id

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebpb5ekbpmibksvg7l6f6menttuwv3py5vdjw7sjy2fcbarnwvjrm

[NiwanDao]

LGTM

[NiwanDao]

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacebusbtlgcjc5xs6gy5d3ejkuvqx2wg5u34bptgl4ai2yr3xl7vxqw

Address

f1l2cc5vuw5moppwsjd3b7cjtwa2exowqo36esklq

Datacap Allocated

256.00TiB

Signer Address

f1a2lia2cwwekeubwo4nppt4v4vebxs2frozarz3q

Id

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebusbtlgcjc5xs6gy5d3ejkuvqx2wg5u34bptgl4ai2yr3xl7vxqw

[large-datacap-requests]

DataCap Allocation requested

Request number 2

Multisig Notary address

f02049625

Client address

f1l2cc5vuw5moppwsjd3b7cjtwa2exowqo36esklq

DataCap allocation requested

512TiB

Id

0e4506ed-1ad4-4499-99a6-301305214c4d

[large-datacap-requests]

Stats & Info for DataCap Allocation

Multisig Notary address

f02049625

Client address

f1l2cc5vuw5moppwsjd3b7cjtwa2exowqo36esklq

Rule to calculate the allocation request amount

10% of total dc amount requested

DataCap allocation requested

512TiB

Total DataCap granted for client so far

256TiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

4.75PiB

Stats

Number of deals	Number of storage providers	Previous DC Allocated	Top provider	Remaining DC
null	null	256TiB	null	69.18TiB

[flyworker]

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzaced2ie3bkbo5ylzgc35jbz6sc4itd63lm7grjdnpiggfynbxjcz7ce

Address

f1l2cc5vuw5moppwsjd3b7cjtwa2exowqo36esklq

Datacap Allocated

512.00TiB

Signer Address

f1hlubjsdkv4wmsdadihloxgwrz3j3ernf6i3cbpy

Id

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaced2ie3bkbo5ylzgc35jbz6sc4itd63lm7grjdnpiggfynbxjcz7ce

[cryptowhizzard]

-Respected community member
-Retrieval OK.

[xinaxu]

checker:manualTrigger

[filplus-checker-app]

DataCap and CID Checker Report Summary¹

Retrieval Statistics

• Overall Graphsync retrieval success rate: 6.16%
• Overall HTTP retrieval success rate: 42.97%
• Overall Bitswap retrieval success rate: 0.00%

Storage Provider Distribution

⚠️ 1 storage providers sealed more than 30% of total datacap - f01717477: 30.18%

Deal Data Replication

⚠️ 67.00% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients²

⚠️ CID sharing has been observed. (Top 3)

• 4.17 PiB - f1mshuxqsmbaxtov7akpua76orqthwseotvwfu5ky - NationalOceanic and Atmospheric Administration (NOAA)
• 3.98 PiB - f13vtwldyycj32sxhenrd7gmwj72hhatvuoydjxii - NationalOceanic and Atmospheric Administration (NOAA)
• 343.52 TiB - f1fjegj5ihpjn5ie75np6vagpwbvx3je4ni2ve4iq - NationalOceanic and Atmospheric Administration (NOAA)

Full report

Click here to view the CID Checker report.
Click here to view the Retrieval Dashboard.
Click here to view the Retrieval report.

Footnotes

1. To manually trigger this report, add a comment with text checker:manualTrigger ↩

2. To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ... ↩

[kernelogic]

checker:manualTrigger f1l2cc5vuw5moppwsjd3b7cjtwa2exowqo36esklq f1mshuxqsmbaxtov7akpua76orqthwseotvwfu5ky f13vtwldyycj32sxhenrd7gmwj72hhatvuoydjxii f1fjegj5ihpjn5ie75np6vagpwbvx3je4ni2ve4iq

[filplus-checker-app]

DataCap and CID Checker Report Summary¹

Other Addresses²

• f1mshuxqsmbaxtov7akpua76orqthwseotvwfu5ky - NationalOceanic and Atmospheric Administration (NOAA)

• f13vtwldyycj32sxhenrd7gmwj72hhatvuoydjxii - NationalOceanic and Atmospheric Administration (NOAA)

• f1fjegj5ihpjn5ie75np6vagpwbvx3je4ni2ve4iq - NationalOceanic and Atmospheric Administration (NOAA)

Retrieval Statistics

• Overall Graphsync retrieval success rate: 14.70%
• Overall HTTP retrieval success rate: 9.55%
• Overall Bitswap retrieval success rate: 0.02%

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

✔️ Data replication looks healthy.

Deal Data Shared with other Clients³

⚠️ CID sharing has been observed. (Top 3)

• 96.00 GiB - f1xw7jxdldohcdu4ec4a5mcnbtvutyjptawojmuzy - Solana Lab.

Full report

Click here to view the CID Checker report.
Click here to view the Retrieval Dashboard.
Click here to view the Retrieval report.

Footnotes

1. To manually trigger this report, add a comment with text checker:manualTrigger ↩

2. Deals from those addresses are combined into this report as they are specified with checker:manualTrigger ↩

3. To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ... ↩

[kernelogic]

After getting the report for the whole series I have a better understanding of the big picture. Distribution and retrieval success rate both satisfactory.

[kernelogic]

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacedcn24hzvrswbldzahz4jcnwwu5q7bbhh53idrt2lgh5u3tmrrmzs

Address

f1l2cc5vuw5moppwsjd3b7cjtwa2exowqo36esklq

Datacap Allocated

1.25PiB

Signer Address

f1yjhnsoga2ccnepb7t3p3ov5fzom3syhsuinxexa

Id

03935229-7d01-424d-ab3a-6df59f00befe

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacedcn24hzvrswbldzahz4jcnwwu5q7bbhh53idrt2lgh5u3tmrrmzs

[large-datacap-requests]

The issue reached the total datacap requested. This should be closed

[large-datacap-requests]

Stats & Info for DataCap Allocation

Multisig Notary address

f02049625

Client address

f1l2cc5vuw5moppwsjd3b7cjtwa2exowqo36esklq

Rule to calculate the allocation request amount

total dc reached

DataCap allocation requested

0

Total DataCap granted for client so far

1.1641532182693484e+53YiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

1.1641532182693484e+53YiB

Stats

Number of deals	Number of storage providers	Previous DC Allocated	Top provider	Remaining DC
164689	17	1.25PiB	21.17	101.90TiB

[xinaxu]

See aggregated stats here. #2087 (comment)

[NiwanDao]

Fair explanation. I will support this round.

[NiwanDao]

it seems like the bot is not ready for sign.

[xinaxu]

oh yeah, this has reached the max. I should close this one.