MGARD (MultiGrid Adaptive Reduction of Data) is a technique for multilevel lossy compression and refactoring of scientific data based on the theory of multigrid methods. We encourage you to make a GitHub issue if you run into any problems using MGARD, have any questions or suggestions, etc.
MGARD framework consists of the following modules. Please see the detailed instructions for each module to build and install MGARD.
MGARD-CPU is design for running compression on CPUs. See detailed user guide in here
MGARD-CUDA is designed for accelerating compression specifically using NVIDIA GPUs. See detailed user guide in here.
MGARD-X is designed for portable compression on NVIDIA GPUs, AMD GPUs, and CPUs. See detailed user guide in here.
MGARD-DR and MGARD-XDR are designed for enabling fine-grain data refactoring and progressive data reconstruction. See detailed user guide in here.
MGARD-ROI is designed for preserving region-of-interest during data compression. See detailed user guide in here.
MGARD-QOI is designed for preserving linear quantity-of-interest during data compression. See detailed user guide in here.
MGARD-Lambda is designed for preserving non-linear quantity-of-interest during data compression. This is an experimental part of MGARD. Currently only support certain QoIs derived from XGC 5D data. See theory in here and example in here.
Data produced by MGARD, MGARD-X, and MDR-X are designed to follow a unified self-describing format. See format details in here.
- Xin Liang et al. MGARD+: Optimizing Multilevel Methods for Error-bounded Scientific Data Reduction. IEEE Transactions on Computers, 2021
- Mark Ainsworth et al. Multilevel Techniques for Compression and Reduction of Scientific Data—The Unstructured Case. SIAM Journal on Scientific Computing, 42 (2), A1402–A1427, 2020.
- Mark Ainsworth et al. Multilevel Techniques for Compression and Reduction of Scientific Data—Quantitative Control of Accuracy in Derived Quantities. SIAM Journal on Scientific Computing 41 (4), A2146–A2171, 2019.
- Mark Ainsworth et al. Multilevel Techniques for Compression and Reduction of Scientific Data—The Multivariate Case. SIAM Journal on Scientific Computing 41 (2), A1278–A1303, 2019.
- Mark Ainsworth et al. Multilevel Techniques for Compression and Reduction of Scientific Data—The Univariate Case. Computing and Visualization in Science 19, 65–76, 2018.
- Ben Whitney. Multilevel Techniques for Compression and Reduction of Scientific Data. PhD thesis, Brown University, 2018.
- Tania Banerjee et al. Scalable Hybrid Learning Techniques for Scientific Data Compression., Arxiv, 2022
- Qian Gong et al. Region-adaptive, Error-controlled Scientific Data Compression using Multilevel Decomposition. the 34th International Conference on Scientific and Statistical Database Management, Jul. 2022
- Tania Benerjee et al. An algorithmic and software pipeline for very large scale scientific data compression with error guarantees. International Conference on High Performance Computing, Data, and Analytics, 2022
- Jaemoon Lee et al. Error-bounded learned scientific data compression with preservation of derived quantities. Applied Sciences, 2022
- Qian Gong et al. Maintaining trust in reduction: Preserving the accuracy of quantities of interest for lossy compression. 21st Smoky Mountains Computational Sciences and Engineering Conference, Oct. 2021
- Jinzheng Wang et al. Improving Progressive Retrieval for HPC Scientific Data using Deep Neural Network. IEEE International Conference on Data Engineering (ICDE), 2023
- Xin Liang et al. Error-controlled, progressive, and adaptable retrieval of scientific data with multilevel decomposition. the International Conference for High Performance Computing, Networking, Storage and Analysis 2021, Nov, 2021
- Jieyang Chen et al. Scalable Multigrid-based Hierarchical Scientific Data Refactoring on GPUs. Arxiv
- Jieyang Chen et al. Accelerating Multigrid-based Hierarchical Scientific Data Refactoring on GPUs. 35th IEEE International Parallel & Distributed Processing Symposium, May 17–21, 2021.
- Lipeng Wan et al. RAPIDS: Reconciling Availability, Accuracy, and Performance in Managing Geo-Distributed Scientific Data. the International ACM Symposium on High-Performance Parallel and Distributed Computing, Jun. 2023
- Xinying Wang et al. Unbalanced Parallel I/O: An Often-Neglected Side Effect of Lossy Scientific Data Compression. 7th International Workshop on Data Analysis and Reduction for Big Scientific Data, Nov. 2021