From f9a6451515ab579ff837ade5a511da65bffff699 Mon Sep 17 00:00:00 2001 From: Max Jones <14077947+maxrjones@users.noreply.github.com> Date: Mon, 6 Oct 2025 15:11:36 -0400 Subject: [PATCH 1/2] Fix cut-off sentence in GPU blog post --- src/posts/gpu-pipeline/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/posts/gpu-pipeline/index.md b/src/posts/gpu-pipeline/index.md index d160cac67..2df9b0ee7 100644 --- a/src/posts/gpu-pipeline/index.md +++ b/src/posts/gpu-pipeline/index.md @@ -101,7 +101,7 @@ PyTorch’s `DataLoader` includes options like `num_workers`, `pin_memory`, and ## Hackathon: Strategies Explored! -During the hackathon, we tested the following strategies to improve the data loading performance. In the end, we were able to achieve +During the hackathon, we tested the following strategies to improve the data loading performance. In the end, we were able to achieve at least ~17x improvement on 1 GPU in training throughput by optimizing data loading and preprocessing steps. ### Step 1: Optimized Chunking & Compression From c3934782aa819a8771bf6e81b48f14bf02fd8507 Mon Sep 17 00:00:00 2001 From: Deepak Cherian Date: Fri, 17 Oct 2025 19:31:08 -0600 Subject: [PATCH 2/2] Update index.md --- src/posts/gpu-pipeline/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/posts/gpu-pipeline/index.md b/src/posts/gpu-pipeline/index.md index 2df9b0ee7..1d268715a 100644 --- a/src/posts/gpu-pipeline/index.md +++ b/src/posts/gpu-pipeline/index.md @@ -198,7 +198,7 @@ With nvCOMP, all steps of data loading including reading from disk, decompressio To unlock this, we would need zarr-python to support GPU-based decompression codecs, with one for Zstandard (Zstd) currently being implemented in [this PR](https://github.com/zarr-developers/zarr-python/pull/2863). -We tested the performance of GPU-based decompression using nvCOMP with Zarr-Python 3 and KvikIO, and compared it to CPU-based decompression using [this data reading benchmark here](https://github.com/pangeo-data/ncar-hackathon-xarray-on-gpus/tree/v1.0/benchmarks/era5_zarr_benchmark.py). +We tested the performance of GPU-based decompression using nvCOMP with Zarr-Python 3 and KvikIO, and compared it to CPU-based decompression using [this data reading benchmark here](https://github.com/pangeo-data/ncar-hackathon-xarray-on-gpus/blob/v1.0/benchmarks/era5_zarr_benchmark.py). Here are the results: