A PySpark notebook to convert XML files to CSV format. I needed the Stack Exchange Data Dump in CSV format for my project. All the converters available online are good for small files but when you have xmls ranging from hundreds of MB to GB, they were taking too much time.
This one works in parallel, utilizing Spark's RDDs and complete the conversion in few minutes with a minimal 2-core and 2 GB RAM Spark setup.
adich23/XmltoCsv_StackExchange
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
About
Xml to Csv converter for Large files using Apache Spark
Topics
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published