[DataCap Application] Tech Greedy - NOAA Global Ensemble Forecast System (#2)

[xinaxu]

Data Owner Name

National Oceanic and Atmospheric Administration (NOAA)

Data Owner Country/Region

United States

Data Owner Industry

Environment

Website

https://registry.opendata.aws/noaa-gefs/

Social Media

https://registry.opendata.aws/noaa-gefs/

Total amount of DataCap being requested

5PiB

Weekly allocation of DataCap requested

1PiB

On-chain address for first allocation

f1mshuxqsmbaxtov7akpua76orqthwseotvwfu5ky

Custom multisig

• Use Custom Multisig

Identifier

No response

Share a brief history of your project and organization

Project Detail: This dataset is an AWS open dataset that has yet been stored on Filecoin. We were planning to onboard this dataset with Slingshot V3, however, with the delay of the program, we want to start onboarding asap. Meanwhile, we want to reach out to storage providers outside of slingshot silo to expand the distribution variety further.

Organization Detail: Tech Greedy has been engaged in Filecoin ecosystem building including building data preparation and deal-making tool, participating in multiple rounds of slingshot and having mining operations.

Is this project associated with other projects/ecosystem stakeholders?

Yes

If answered yes, what are the other projects/ecosystem stakeholders

https://github.com/filecoin-project/filecoin-plus-large-datasets/issues/1483

Describe the data being stored onto Filecoin

The Global Ensemble Forecast System (GEFS), previously known as the GFS Global ENSemble (GENS), is a weather forecast model made up of 21 separate forecasts, or ensemble members. The National Centers for Environmental Prediction (NCEP) started the GEFS to address the nature of uncertainty in weather observations, which is used to initialize weather forecast models. The GEFS attempts to quantify the amount of uncertainty in a forecast by generating an ensemble of multiple forecasts, each minutely different, or perturbed, from the original observations. With global coverage, GEFS is produced four times a day with weather forecasts going out to 16 days.

The total size of the dataset is ~2.0PiB so we will likely need more application in the future.

Where was the data currently stored in this dataset sourced from

AWS Cloud

If you answered "Other" in the previous question, enter the details here

No response

How do you plan to prepare the dataset

singularity

If you answered "other/custom tool" in the previous question, enter the details here

No response

Please share a sample of the data

The data has already been prepared and hosted on
https://d.techgreedy.net/preparation/noaa-gefs-pds

To check what files each archive contains, use
https://d.techgreedy.net/generation-manifest/noaa-gefs-pds/<number>

To download and check the content
https://d.techgreedy.net/<cid>.car

Confirm that this is a public dataset that can be retrieved by anyone on the Network

• I confirm

If you chose not to confirm, what was the reason

No response

What is the expected retrieval frequency for this data

Monthly

For how long do you plan to keep this dataset stored on Filecoin

1.5 to 2 years

In which geographies do you plan on making storage deals

Greater China, Asia other than Greater China, North America, Europe

How will you be distributing your data to storage providers

HTTP or FTP server

How do you plan to choose storage providers

Slack, Big data exchange, Partners

If you answered "Others" in the previous question, what is the tool or platform you plan to use

No response

If you already have a list of storage providers to work with, fill out their names and provider IDs below

No response

How do you plan to make deals to your storage providers

Singularity

If you answered "Others/custom tool" in the previous question, enter the details here

No response

Can you confirm that you will follow the Fil+ guideline

Yes

[large-datacap-requests]

Thanks for your request!
Everything looks good. 👌

A Governance Team member will review the information provided and contact you back pretty soon.

[Sunnyiscoming]

Datacap Request Trigger

Total DataCap requested

5PiB

Expected weekly DataCap usage rate

1PiB

Client address

f1mshuxqsmbaxtov7akpua76orqthwseotvwfu5ky

[large-datacap-requests]

DataCap Allocation requested

Multisig Notary address

f02049625

Client address

f1mshuxqsmbaxtov7akpua76orqthwseotvwfu5ky

DataCap allocation requested

256TiB

Id

e7a60976-89fa-4126-baf2-3d71d985e134

[kernelogic]

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzacebq2nbzhtffubtzkljq5mlssh7h2ojq4z76ipbfjpmbabo6bt6v6i

Address

f1mshuxqsmbaxtov7akpua76orqthwseotvwfu5ky

Datacap Allocated

256.00TiB

Signer Address

f1yjhnsoga2ccnepb7t3p3ov5fzom3syhsuinxexa

Id

e7a60976-89fa-4126-baf2-3d71d985e134

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebq2nbzhtffubtzkljq5mlssh7h2ojq4z76ipbfjpmbabo6bt6v6i

[NiwanDao]

Based on the good reputation of Tech Greedy had on the community, I will support this tranche.

[NiwanDao]

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzaceaslvjpemllkuhctjx35jn2lswvrohi4uulfsvxqhkwbcj5f3emdy

Address

f1mshuxqsmbaxtov7akpua76orqthwseotvwfu5ky

Datacap Allocated

256.00TiB

Signer Address

f1a2lia2cwwekeubwo4nppt4v4vebxs2frozarz3q

Id

e7a60976-89fa-4126-baf2-3d71d985e134

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceaslvjpemllkuhctjx35jn2lswvrohi4uulfsvxqhkwbcj5f3emdy

[large-datacap-requests]

DataCap Allocation requested

Request number 2

Multisig Notary address

f02049625

Client address

f1mshuxqsmbaxtov7akpua76orqthwseotvwfu5ky

DataCap allocation requested

512TiB

Id

c01b1342-d09a-4486-9d95-300033d7445b

[large-datacap-requests]

Stats & Info for DataCap Allocation

Multisig Notary address

f02049625

Client address

f1mshuxqsmbaxtov7akpua76orqthwseotvwfu5ky

Rule to calculate the allocation request amount

80% of total dc amount requested

DataCap allocation requested

1.25PiB

Total DataCap granted for client so far

1.862645149230957e+37YiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

-2.25B

Stats

Number of deals	Number of storage providers	Previous DC Allocated	Top provider	Remaining DC
105647	12	2PiB	31.65	483.48TiB

[kernelogic]

Request Proposed

Your Datacap Allocation Request has been proposed by the Notary

Message sent to Filecoin Network

bafy2bzaceavdosit4i6pynw3ytj2efx5qjz776g2v5tavtkcvco7zccpargbc

Address

f1mshuxqsmbaxtov7akpua76orqthwseotvwfu5ky

Datacap Allocated

1.25PiB

Signer Address

f1yjhnsoga2ccnepb7t3p3ov5fzom3syhsuinxexa

Id

8007654e-6b2d-4712-ab21-7434288e33f2

You can check the status of the message here: https://filfox.info/en/message/bafy2bzaceavdosit4i6pynw3ytj2efx5qjz776g2v5tavtkcvco7zccpargbc

[NiwanDao]

checker:manualTrigger

[filplus-checker-app]

DataCap and CID Checker Report Summary¹

Storage Provider Distribution

⚠️ 1 storage providers sealed more than 30% of total datacap - f01969339: 31.93%

Deal Data Replication

⚠️ 88.92% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients²

⚠️ CID sharing has been observed. (Top 3)

• 4.35 PiB - f13vtwldyycj32sxhenrd7gmwj72hhatvuoydjxii - NationalOceanic and Atmospheric Administration (NOAA)
• 566.36 TiB - f1l2cc5vuw5moppwsjd3b7cjtwa2exowqo36esklq - NationalOceanic and Atmospheric Administration (NOAA)

Full report

Click here to view the full report.

Footnotes

1. To manually trigger this report, add a comment with text checker:manualTrigger ↩

2. To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ... ↩

[NiwanDao]

Please be aware of the number of replication in the next round.

[NiwanDao]

Request Approved

Your Datacap Allocation Request has been approved by the Notary

Message sent to Filecoin Network

bafy2bzacebsk7gjljoqsenl5pukxtt3ru7nhx5zyoke47v5iqdsmbnazgg2lw

Address

f1mshuxqsmbaxtov7akpua76orqthwseotvwfu5ky

Datacap Allocated

1.25PiB

Signer Address

f1a2lia2cwwekeubwo4nppt4v4vebxs2frozarz3q

Id

8007654e-6b2d-4712-ab21-7434288e33f2

You can check the status of the message here: https://filfox.info/en/message/bafy2bzacebsk7gjljoqsenl5pukxtt3ru7nhx5zyoke47v5iqdsmbnazgg2lw

[xinaxu]

checker:manualTrigger

[filplus-checker-app]

DataCap and CID Checker Report Summary¹

Retrieval Statistics

• Overall Graphsync retrieval success rate: 6.65%
• Overall HTTP retrieval success rate: 0.00%
• Overall Bitswap retrieval success rate: 0.06%

Storage Provider Distribution

✔️ Storage provider distribution looks healthy.

Deal Data Replication

⚠️ 73.78% of deals are for data replicated across less than 4 storage providers.

Deal Data Shared with other Clients²

⚠️ CID sharing has been observed. (Top 3)

• 4.54 PiB - f13vtwldyycj32sxhenrd7gmwj72hhatvuoydjxii - NationalOceanic and Atmospheric Administration (NOAA)
• 1.32 PiB - f1l2cc5vuw5moppwsjd3b7cjtwa2exowqo36esklq - NationalOceanic and Atmospheric Administration (NOAA)

Full report

Click here to view the CID Checker report.
Click here to view the Retrieval report.

Footnotes

1. To manually trigger this report, add a comment with text checker:manualTrigger ↩

2. To manually trigger this report with deals from other related addresses, add a comment with text checker:manualTrigger <other_address_1> <other_address_2> ... ↩

[large-datacap-requests]

The issue reached the total datacap requested. This should be closed

[large-datacap-requests]

Stats & Info for DataCap Allocation

Multisig Notary address

f02049625

Client address

f1mshuxqsmbaxtov7akpua76orqthwseotvwfu5ky

Rule to calculate the allocation request amount

total dc reached

DataCap allocation requested

0

Total DataCap granted for client so far

1.1641532182693484e+53YiB

Datacap to be granted to reach the total amount requested by the client (5PiB)

1.1641532182693484e+53YiB

Stats

Number of deals	Number of storage providers	Previous DC Allocated	Top provider	Remaining DC
156915	18	1.25PiB	21.73	287.76TiB