Repository navigation
Export to Compressed Parquet File #1376
Description
Is your feature request related to a problem? Please describe.
I want to save data from PBID to a compressed file, which can store a lot of data, which supports data types, which is popular.
Describe the solution you'd like
PBID can now import Parquet files with the "Parquet.Document" function (there is also a "Parquet.Metadata" function). It would be great if we could also export data to Parquet files.
Describe alternatives you've considered
I now that I can use python for this task, but I can not install anything on my work computer.
I've read about the Delta format. I think Parquet is ubiquitous and that would be a better solution.
Maybe Microsoft will introduce this option to PBID.
Additional context
Activity
dgosbell commentedon Jun 13, 2025
It should be possible to do this as there are a couple of libraries that provide parquet support for c#. However, I'm curious as to how you would use exports to parquet if you cannot install tools like Python. Do you have some way of consuming parquet files?
Bizkapish commentedon Jun 13, 2025
The parquet files would be used as a repository.
PBID does not have a partial refresh capability, so every month I consolidate about 1200 (this number is growing) Excel files into one PBID database. Together, these files are about 10 GB, and importing from Excel files is slow.
I would like to consolidate the old data into a file format that is compressed and fast to load. For me, the only solution is to use a PBID file as a repository for the old data, and then load the data partly from that file and partly from the latest Excel files, into final PBID file. This is a good solution, but not the best.
After reading about the new Parquet.Document function, I realized that this would be the best solution for me. To store old data in parquet files. The only problem is how to create Parquet files on my work computer where I only have a limited set of applications.
Microsoft has empowered us to work with large datasets in PBID, but for some of us there remains no efficient solution for storing and sharing that data.
dgosbell commentedon Oct 24, 2025
This feature has been added in the recent 3.4.0 release