Skip to content

Releases: wookat/dataforge-pipelines

Open Data: China Macro & Trade Monthly (World Bank WDI table)

Choose a tag to compare

@devin-ai-integration devin-ai-integration released this 08 Aug 00:29
1c62bca

Open Data: China Macro & Trade Monthly — World Bank WDI annual indicators — part of the DataForge Open Data program.

Full production table, free for academic and personal use under CC BY-NC 4.0 (attribution + backlink required). Same file as served at https://dl.zalize.com/open-data/china-macro_normalized_indicators_annual_wb.parquet (machine-readable index: https://dl.zalize.com/open-data).

Packages

Package Rows Size SHA-256 Scope
china-macro_normalized_indicators_annual_wb.parquet 937 16 KB a8ab858457cc8dce690da8299f9df2a9ac68f625cae1b6ca382c0285bab1dfb7 World Bank WDI annual indicators for China (CC BY 4.0)

Note: only the World Bank WDI table is open data; IMF-sourced tables and the S/M/L bundles remain gated pending IMF written confirmation (fail-closed).

Data dictionary & datasheet

Field-level documentation lives under datasets/ and on the portal page: https://data.zalize.com/datasets/china-macro-trade-monthly-dataset

License

CC BY-NC 4.0 (DataForge curated layer) / commercial license — upstream: World Bank WDI CC BY 4.0 (retain World Bank attribution)

  • Academic / personal use: CC BY-NC 4.0 on the DataForge curated layer — attribution + backlink required.
  • Commercial use: requires a DataForge commercial license — contact us via the portal.
  • Upstream source: World Bank WDI (CC BY 4.0). Upstream license terms continue to apply.

Citation / attribution

DataForge (data.zalize.com), built from World Bank WDI (CC BY 4.0) — https://data.zalize.com

🔗 DataForge portal: https://data.zalize.com · All open data: https://data.zalize.com/open-data

Open Data: US Work-Visa Real Salaries (H-1B / PERM / LCA)

Choose a tag to compare

Open Data: US Work-Visa Real Salaries (H-1B / PERM / LCA) — part of the DataForge Open Data program.

Full production packages, free for academic and personal use under CC BY-NC 4.0 (attribution + backlink required). Same files as served at https://dl.zalize.com/open-data/<package_id> (machine-readable index: https://dl.zalize.com/open-data).

Packages

Package Tier Rows Size SHA-256 Scope
visa_salaries-tier1-S-2026-08-02 S 1,814,772 0.21 GB ef1e0012c66e3dd4ddfc0767bc41c5cab86487edc268444131d227ec40f8a197 Single-state snapshot: all California-worksite filings (latest full corpus) + CA wage benchmarks
visa_salaries-tier1-M-2026-08-02 M 915,005 0.11 GB 529f093717cded5635d4becb7fc34c346535dd9f1027b6c027bccb2e11622782 All programs, latest complete fiscal year (FY2025) incl. PERM + that year's benchmarks and employer profiles
visa_salaries-tier1-L-2026-08-02 L 14,535,607 1.68 GB 7536976acc639e0594100656b356eba01f7700fc840c28f02a292ca4a92d8d9f Full longitudinal library FY2008-FY2026: all filings, USCIS employer hub, employer visa-dependency profiles and SOCxstate wage benchmarks with normalized employer IDs

Data dictionary & datasheet

Field-level documentation lives in this repo under datasets/ and on the portal page: https://data.zalize.com/datasets/us-work-visa-salary-dataset

License

CC BY-NC 4.0 (DataForge curated layer) / commercial license — upstream: US federal public domain (17 U.S.C. §105)

  • Academic / personal use: CC BY-NC 4.0 on the DataForge curated layer — attribution + backlink required.
  • Commercial use: requires a DataForge commercial license — contact us via the portal.
  • Upstream source: US DOL OFLC disclosure files + USCIS H-1B data. Upstream license terms (public domain / CC BY / OGL where applicable) continue to apply to the underlying data.

Citation / attribution

Please credit:

DataForge (data.zalize.com), built from official US DOL/USCIS disclosure files — https://data.zalize.com

For attribution-licensed upstreams keep the upstream credit (PatentsView CC BY 4.0; "Contains public sector information licensed under the Open Government Licence v3.0").


🔗 DataForge portal: https://data.zalize.com · All open data: https://data.zalize.com/open-data

Open Data: US Federal Awards (USAspending, FY2008–present)

Choose a tag to compare

Open Data: US Federal Awards (USAspending, FY2008–present) — part of the DataForge Open Data program.

Full production packages, free for academic and personal use under CC BY-NC 4.0 (attribution + backlink required). Same files as served at https://dl.zalize.com/open-data/<package_id> (machine-readable index: https://dl.zalize.com/open-data).

Packages

Package Tier Rows Size SHA-256 Scope
usaspending-tier1-S-2026-08-05 S 1.45 GB a46321e8fae6e18f1645366d8c61aff6819727aadff4ed84dc166f4873e035a0 usaspending — usaspending-tier1-S-2026-08-05.zip
usaspending-tier1-M-2026-08-05 M 3.84 GB b7793cc3e33b5e5a2315b84b80899cc8b2bcc549a2999c1479e43481d08bd4cb usaspending — usaspending-tier1-M-2026-08-05.zip
usaspending-tier1-L-2026-08-05 L 1.04 GB 04e00c0cb20557e26b6c4d7a19a07c28ecdc4a0f4a4b848cd1b6f5facc7ae9ef usaspending — usaspending-tier1-L-2026-08-05.zip

Packages larger than GitHub's 2 GiB asset limit are split into .partNN volumes.
Reassemble with: cat <name>.zip.part* > <name>.zip then verify the SHA-256 above.

Data dictionary & datasheet

Field-level documentation lives in this repo under datasets/ and on the portal page: https://data.zalize.com/open-data

License

CC BY-NC 4.0 (DataForge curated layer) / commercial license — upstream: USAspending.gov public domain (17 U.S.C. §105)

  • Academic / personal use: CC BY-NC 4.0 on the DataForge curated layer — attribution + backlink required.
  • Commercial use: requires a DataForge commercial license — contact us via the portal.
  • Upstream source: USAspending.gov official award archives, FY2008–present. Upstream license terms (public domain / CC BY / OGL where applicable) continue to apply to the underlying data.

Citation / attribution

Please credit:

DataForge (data.zalize.com), built from official USAspending.gov archives — https://data.zalize.com

For attribution-licensed upstreams keep the upstream credit (PatentsView CC BY 4.0; "Contains public sector information licensed under the Open Government Licence v3.0").


🔗 DataForge portal: https://data.zalize.com · All open data: https://data.zalize.com/open-data

Open Data: US Residential Properties & Sale Events (government records)

Choose a tag to compare

Open Data: US Residential Properties & Sale Events (government records) — part of the DataForge Open Data program.

Full production packages, free for academic and personal use under CC BY-NC 4.0 (attribution + backlink required). Same files as served at https://dl.zalize.com/open-data/<package_id> (machine-readable index: https://dl.zalize.com/open-data).

Packages

Package Tier Rows Size SHA-256 Scope
realestate-tier1-S-2026-08-02 S 1,327,964 0.11 GB 5da022313d1afcbcab9b3d4e161c0412016283d1ffab127ddfc70f34b21aea17 Philadelphia PA: all OPA property records + recorded sale events
realestate-tier1-M-2026-08-02 M 1,423,038 0.13 GB 9f6280106f20b389850d16e63be8b6c70605de1b562d1cbedb637b710b30ce76 New York City (5 boroughs): PLUTO property records + DOF rolling sales
realestate-tier1-L-2026-08-02 L 7,009,514 0.50 GB 948c71304012cfae084118a269515027fda52b0577c94157f815568c751e6fa2 All covered counties: NYC + Cook County IL (Chicago) + Philadelphia — full properties + sale events

Data dictionary & datasheet

Field-level documentation lives in this repo under datasets/ and on the portal page: https://data.zalize.com/datasets/us-residential-properties-sale-events-dataset

License

CC BY-NC 4.0 (DataForge curated layer) / commercial license — upstream: US public property records

  • Academic / personal use: CC BY-NC 4.0 on the DataForge curated layer — attribution + backlink required.
  • Commercial use: requires a DataForge commercial license — contact us via the portal.
  • Upstream source: US county assessor / government property and sale-event records. Upstream license terms (public domain / CC BY / OGL where applicable) continue to apply to the underlying data.

Citation / attribution

Please credit:

DataForge (data.zalize.com) — https://data.zalize.com

For attribution-licensed upstreams keep the upstream credit (PatentsView CC BY 4.0; "Contains public sector information licensed under the Open Government Licence v3.0").


🔗 DataForge portal: https://data.zalize.com · All open data: https://data.zalize.com/open-data

Open Data: US Research Grants (NIH ExPORTER + NSF Awards)

Choose a tag to compare

Open Data: US Research Grants (NIH ExPORTER + NSF Awards) — part of the DataForge Open Data program.

Full production packages, free for academic and personal use under CC BY-NC 4.0 (attribution + backlink required). Same files as served at https://dl.zalize.com/open-data/<package_id> (machine-readable index: https://dl.zalize.com/open-data).

Packages

Package Tier Rows Size SHA-256 Scope
research_grants-tier1-S-2026-08-05 S 42,609 0.01 GB 707157ec89e2e5d679487c00205c950f161ce8b4a93da021de269de53521bb0c Single discipline, single year: every FY2025 National Cancer Institute (NCI) award with full NIH project schema, PI–award edges and per-institution funding aggregates
research_grants-tier1-M-2026-08-05 M 785,707 0.05 GB 3179d750ec6dc37fcaf56bd0be04df73c08d09119b32aa0c06900dbcf485ef3c NIH + NSF complete 2025: unified award table, full per-source schemas, PI normalization (profiles + edges), institution×year aggregates and project→publication counts
research_grants-tier1-L-2026-08-05 L 22,633,579 2.32 GB 747b3a036820a3bc6ebc25cf4d17bac0c052fad266d45822caa9a36a686f0cf1 The entire longitudinal library: NIH FY1985–FY2025 + NSF 1952–2026 unified awards, full per-source schemas, PI cumulative funding profiles, institution×year aggregates, publications/link-table/patents/clinical-studies reference tables (annual subscription)

Packages larger than GitHub's 2 GiB asset limit are split into .partNN volumes.
Reassemble with: cat <name>.zip.part* > <name>.zip then verify the SHA-256 above.

Data dictionary & datasheet

Field-level documentation lives in this repo under datasets/ and on the portal page: https://data.zalize.com/datasets/us-research-grants-nih-nsf-dataset

License

CC BY-NC 4.0 (DataForge curated layer) / commercial license — upstream: NIH/NSF public domain (17 U.S.C. §105)

  • Academic / personal use: CC BY-NC 4.0 on the DataForge curated layer — attribution + backlink required.
  • Commercial use: requires a DataForge commercial license — contact us via the portal.
  • Upstream source: NIH ExPORTER + NSF Awards API official bulk data. Upstream license terms (public domain / CC BY / OGL where applicable) continue to apply to the underlying data.

Citation / attribution

Please credit:

DataForge (data.zalize.com), built from official NIH/NSF public data — https://data.zalize.com

For attribution-licensed upstreams keep the upstream credit (PatentsView CC BY 4.0; "Contains public sector information licensed under the Open Government Licence v3.0").


🔗 DataForge portal: https://data.zalize.com · All open data: https://data.zalize.com/open-data

Open Data: US Granted Patents 1976-2024 + Citation Graph (PatentsView)

Choose a tag to compare

Open Data: US Granted Patents 1976-2024 + Citation Graph (PatentsView) — part of the DataForge Open Data program.

Full production packages, free for academic and personal use under CC BY-NC 4.0 (attribution + backlink required). Same files as served at https://dl.zalize.com/open-data/<package_id> (machine-readable index: https://dl.zalize.com/open-data).

Packages

Package Tier Rows Size SHA-256 Scope
patents-tier1-S-2026-08-02 S 9,075,421 7.92 GB 5117be5e96ff6ac46eedd2079cfbd0d987de6371c5009ba98caace48701a8b17 Core granted-patents table: 9.08M US patents 1976-2024 with title, abstract, dates, CPC primary, first assignee (Parquet + CSV + JSONL)
patents-tier1-M-2026-08-02 M 255,347,795 17.24 GB 9309dad24a37cbc1bda8ba9be45dbc372bf1de11b26606d642f673f30218192c Everything in S plus full relationship tables (assignees, inventors, 56.8M CPC assignments, 145.9M citations), 5 company/technology intelligence tables, and ready-made slices by company / CPC section / grant year / country

Packages larger than GitHub's 2 GiB asset limit are split into .partNN volumes.
Reassemble with: cat <name>.zip.part* > <name>.zip then verify the SHA-256 above.

Data dictionary & datasheet

Field-level documentation lives in this repo under datasets/ and on the portal page: https://data.zalize.com/datasets/us-patents-citation-company-dataset

License

CC BY-NC 4.0 (DataForge curated layer) / commercial license — upstream: USPTO PatentsView CC BY 4.0

  • Academic / personal use: CC BY-NC 4.0 on the DataForge curated layer — attribution + backlink required.
  • Commercial use: requires a DataForge commercial license — contact us via the portal.
  • Upstream source: USPTO / PatentsView bulk downloads. Upstream license terms (public domain / CC BY / OGL where applicable) continue to apply to the underlying data.

Citation / attribution

Please credit:

PatentsView (CC BY 4.0); packaged by DataForge (data.zalize.com) — https://data.zalize.com

For attribution-licensed upstreams keep the upstream credit (PatentsView CC BY 4.0; "Contains public sector information licensed under the Open Government Licence v3.0").


🔗 DataForge portal: https://data.zalize.com · All open data: https://data.zalize.com/open-data

Open Data: US Grid Hourly & Power Plants (EIA)

Choose a tag to compare

Open Data: US Grid Hourly & Power Plants (EIA) — part of the DataForge Open Data program.

Full production packages, free for academic and personal use under CC BY-NC 4.0 (attribution + backlink required). Same files as served at https://dl.zalize.com/open-data/<package_id> (machine-readable index: https://dl.zalize.com/open-data).

Packages

Package Tier Rows Size SHA-256 Scope
energy-tier1-S-2026-08-04 S 209,518 0.00 GB 0e74e06144cc36d92d5439a9b0547b00a93f38b6a5fd4cfdf2e9ca1641d9b02c One ISO, one year: CAISO (CISO) full 2025 — hourly demand/forecast/generation/interchange, load-zone subregion demand, hourly fuel mix, plus the CAISO BA dimension row and gap report
energy-tier1-M-2026-08-04 M 7,929,603 0.07 GB a6a66595c50e4f995f01baca19951ac362b8d23ae424c514b468218ef62493a8 Full US, one year (2025): every balancing authority's hourly demand/forecast/generation/interchange + fuel mix + subregions, all interconnection flows, EIA-860 plant/generator/utility master data, EIA-923 2025 plant-month generation, 2025 state retail prices and generation
energy-tier2-L-2026-08-04 L 60,657,958 0.49 GB 88a58ac655108e201b6603ea4aab74e59f6bf313ddd3fa134c9147260c19d48e Full history, monthly refresh subscription: hourly grid operations for every BA since 2015 (~150M observations incl. fuel mix, interchange, subregions), EIA-923 plant-month generation 2017-2025, full state price/generation history since 2001, EIA-860 master data

Data dictionary & datasheet

Field-level documentation lives in this repo under datasets/ and on the portal page: https://data.zalize.com/datasets/us-grid-hourly-power-plants-dataset

License

CC BY-NC 4.0 (DataForge curated layer) / commercial license — upstream: US EIA public domain (17 U.S.C. §105)

  • Academic / personal use: CC BY-NC 4.0 on the DataForge curated layer — attribution + backlink required.
  • Commercial use: requires a DataForge commercial license — contact us via the portal.
  • Upstream source: US EIA hourly grid (EBA) + EIA-860/923 plant master data. Upstream license terms (public domain / CC BY / OGL where applicable) continue to apply to the underlying data.

Citation / attribution

Please credit:

DataForge (data.zalize.com), built from official US EIA bulk data — https://data.zalize.com

For attribution-licensed upstreams keep the upstream credit (PatentsView CC BY 4.0; "Contains public sector information licensed under the Open Government Licence v3.0").


🔗 DataForge portal: https://data.zalize.com · All open data: https://data.zalize.com/open-data

Open Data: Government Tenders & Contract Awards (SAM.gov + official portals)

Choose a tag to compare

Open Data: Government Tenders & Contract Awards (SAM.gov + official portals) — part of the DataForge Open Data program.

Full production packages, free for academic and personal use under CC BY-NC 4.0 (attribution + backlink required). Same files as served at https://dl.zalize.com/open-data/<package_id> (machine-readable index: https://dl.zalize.com/open-data).

Packages

Package Tier Rows Size SHA-256 Scope
tenders-tier1-S-2026-08-02 S 1,046,568 0.63 GB 9bf8c3f3213ff08b13cebb720ace1239a8daa8f53144e696d9b27af6025df7f4 US federal snapshot: all SAM.gov contract opportunities, awards and amendments (FY2024-FY2026 archives + current cycle)
tenders-tier2-M-2026-08-02 M 5,956,897 0.95 GB ded9bc7e48d79ceced71fa24286a5515f1b0aa430c1e28ba5c066a4456e6d860 Global awards intelligence: all award notices across 10 official portals + full opportunity-to-award linkage table + resolved supplier entities
tenders-tier3-L-2026-08-02 L 11,453,832 2.16 GB eae4b340010acb95d5dfa624c010b389e0f466792b2882618a71bc0b0037ffee Full corpus: all notices (opportunities + awards + amendments), event stream, opportunity-to-award links and resolved suppliers across all 10 official portals

Packages larger than GitHub's 2 GiB asset limit are split into .partNN volumes.
Reassemble with: cat <name>.zip.part* > <name>.zip then verify the SHA-256 above.

Data dictionary & datasheet

Field-level documentation lives in this repo under datasets/ and on the portal page: https://data.zalize.com/datasets/government-tenders-contract-awards-dataset

License

CC BY-NC 4.0 (DataForge curated layer) / commercial license — upstream: official government procurement portals (SAM.gov public domain)

  • Academic / personal use: CC BY-NC 4.0 on the DataForge curated layer — attribution + backlink required.
  • Commercial use: requires a DataForge commercial license — contact us via the portal.
  • Upstream source: SAM.gov Contract Opportunities + official multi-country procurement portals. Upstream license terms (public domain / CC BY / OGL where applicable) continue to apply to the underlying data.

Citation / attribution

Please credit:

DataForge (data.zalize.com), built from official government procurement data — https://data.zalize.com

For attribution-licensed upstreams keep the upstream credit (PatentsView CC BY 4.0; "Contains public sector information licensed under the Open Government Licence v3.0").


🔗 DataForge portal: https://data.zalize.com · All open data: https://data.zalize.com/open-data

Open Data: US Court Opinions Metadata, Dockets & Judges (CourtListener)

Choose a tag to compare

Open Data: US Court Opinions Metadata, Dockets & Judges (CourtListener) — part of the DataForge Open Data program.

Full production packages, free for academic and personal use under CC BY-NC 4.0 (attribution + backlink required). Same files as served at https://dl.zalize.com/open-data/<package_id> (machine-readable index: https://dl.zalize.com/open-data).

Packages

Package Tier Rows Size SHA-256 Scope
courtlistener-tier1-S-2026-08-04 S 2,286,889 0.07 GB 3cec1e8d268d5d05e94a5bdd55df9c5fd06411a1f578ddbaae4efe4af29061c6 US Supreme Court complete decision metadata (1754+): every SCOTUS opinion cluster with SCDB crosswalk, citations, opinion-level metadata, judges and court reference tables
courtlistener-tier1-M-2026-08-04 M 7,395,271 0.37 GB 02dc9e0e11de170162a197a99196331fb537bfb1bff6918ce5ffc3a1d4e944bf All 13 federal circuit courts + SCOTUS: decision metadata, opinion-level records with author IDs, reporter citations, judge profiles + positions, court-year volume stats
courtlistener-tier1-L-2026-08-04 L 46,485,000 2.30 GB 3228bc0623bde6a62be8c8a499a858cca0364c785e3435f15059cc74dd1f42eb Entire CourtListener decision library (all courts, all years) + company litigation profiles + judges + oral-argument metadata; citation-graph edge list (77M rows) and dockets full table (72M rows) delivered alongside as B2B Parquet

Packages larger than GitHub's 2 GiB asset limit are split into .partNN volumes.
Reassemble with: cat <name>.zip.part* > <name>.zip then verify the SHA-256 above.

Data dictionary & datasheet

Field-level documentation lives in this repo under datasets/ and on the portal page: https://data.zalize.com/datasets/us-court-opinions-dockets-judges-dataset

License

CC BY-NC 4.0 (DataForge curated layer) / commercial license — upstream: CourtListener Public Domain Mark 1.0 (no opinion text)

  • Academic / personal use: CC BY-NC 4.0 on the DataForge curated layer — attribution + backlink required.
  • Commercial use: requires a DataForge commercial license — contact us via the portal.
  • Upstream source: CourtListener / Free Law Project official bulk data (opinion clusters, dockets, judges; metadata + derived signals only). Upstream license terms (public domain / CC BY / OGL where applicable) continue to apply to the underlying data.

Citation / attribution

Please credit:

DataForge (data.zalize.com), built from CourtListener / Free Law Project public-domain bulk data — https://data.zalize.com

For attribution-licensed upstreams keep the upstream credit (PatentsView CC BY 4.0; "Contains public sector information licensed under the Open Government Licence v3.0").


🔗 DataForge portal: https://data.zalize.com · All open data: https://data.zalize.com/open-data

Open Data: Tech Job Postings & Salaries (ATS structured fields)

Choose a tag to compare

Open Data: Tech Job Postings & Salaries (ATS structured fields) — part of the DataForge Open Data program.

Full production packages, free for academic and personal use under CC BY-NC 4.0 (attribution + backlink required). Same files as served at https://dl.zalize.com/open-data/<package_id> (machine-readable index: https://dl.zalize.com/open-data).

Packages

Package Tier Rows Size SHA-256 Scope
jobs-tier1-S-2026-08-01 S 35,417 0.00 GB e4f4bc2c36ae00edbabc054482640927bc1dc5205a90c367a92fc56365b0ebe9 Single tech vertical: postings whose extracted skills include python
jobs-tier1-M-2026-08-01 M 19,311 0.00 GB a7ad425f5215dd22c96a699ed56d83411942c6ca2a7cbdb240d3d5dae2ef54a1 Engineering departments across US + major EU countries
jobs-tier1-L-2026-08-01 L 394,300 0.04 GB b9099ae46aeb89485f174e8b3d15dfa043781607f38571edbca8cf9c247f457f All postings across all covered ATS platforms and countries

Data dictionary & datasheet

Field-level documentation lives in this repo under datasets/ and on the portal page: https://data.zalize.com/datasets/tech-job-postings-salary-dataset

License

CC BY-NC 4.0 (DataForge curated layer) / commercial license

  • Academic / personal use: CC BY-NC 4.0 on the DataForge curated layer — attribution + backlink required.
  • Commercial use: requires a DataForge commercial license — contact us via the portal.
  • Upstream source: Public ATS job boards (Greenhouse / Lever / Ashby), parsed structured fields. Upstream license terms (public domain / CC BY / OGL where applicable) continue to apply to the underlying data.

Citation / attribution

Please credit:

DataForge (data.zalize.com) — https://data.zalize.com

For attribution-licensed upstreams keep the upstream credit (PatentsView CC BY 4.0; "Contains public sector information licensed under the Open Government Licence v3.0").


🔗 DataForge portal: https://data.zalize.com · All open data: https://data.zalize.com/open-data