Skip to content

Commit fcb8da0

Browse files
committed
feat: csv to ndjson converter script ✨
1 parent c72c50e commit fcb8da0

File tree

3 files changed

+135
-57
lines changed

3 files changed

+135
-57
lines changed

CSV_TO_NDJSON/README.md

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
# README.md
2+
3+
# CSV to NDJSON Converter
4+
5+
This Python script converts a CSV file located in this folder into an NDJSON file saved in the same folder.
6+
7+
## What is NDJSON?
8+
9+
NDJSON (Newline Delimited JSON) is a convenient format for streaming JSON objects, where each line is a valid JSON object.
10+
It’s widely used in data pipelines and tools such as **Google Cloud BigQuery**, **ElasticSearch**, and many other data processing platforms.
11+
12+
## How to use
13+
14+
1. Place your CSV file in this folder.
15+
2. Make sure you have Python 3 installed.
16+
3. Run the script from this folder with:
17+
18+
```bash
19+
python csv_to_ndjson.py input.csv output.ndjson
20+
21+
22+
#### Example
23+
If you have a CSV file like this:
24+
25+
```csv
26+
name,age,city
27+
Alice,30,New York
28+
Bob,25,Los Angeles
29+
```
30+
31+
The output NDJSON will be:
32+
33+
```json
34+
{"name":"Alice","age":"30","city":"New York"}
35+
{"name":"Bob","age":"25","city":"Los Angeles"}
36+
```
37+
38+
Feel free to modify or extend it as needed.

CSV_TO_NDJSON/csv_to_ndjson.py

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
# csv_to_ndjson.py
2+
import csv
3+
import json
4+
import sys
5+
import os
6+
7+
8+
def csv_to_ndjson(csv_filename, ndjson_filename):
9+
base_dir = os.path.dirname(os.path.abspath(__file__))
10+
11+
csv_path = os.path.join(base_dir, csv_filename)
12+
ndjson_path = os.path.join(base_dir, ndjson_filename)
13+
14+
try:
15+
with (
16+
open(csv_path, mode="r", encoding="utf-8") as f_csv,
17+
open(ndjson_path, mode="w", encoding="utf-8") as f_ndjson,
18+
):
19+
reader = csv.DictReader(f_csv)
20+
for row in reader:
21+
f_ndjson.write(
22+
json.dumps(row, ensure_ascii=False, separators=(",", ":")) + "\n"
23+
)
24+
25+
print(f"Successfully converted '{csv_path}' to '{ndjson_path}'")
26+
27+
except FileNotFoundError:
28+
print(f"Error: CSV file '{csv_path}' not found.")
29+
except Exception as e:
30+
print(f"An error occurred: {e}")
31+
32+
33+
if __name__ == "__main__":
34+
if len(sys.argv) != 3:
35+
print("Usage: python csv_to_ndjson.py input.csv output.ndjson")
36+
else:
37+
csv_file = sys.argv[1]
38+
ndjson_file = sys.argv[2]
39+
csv_to_ndjson(csv_file, ndjson_file)

0 commit comments

Comments
 (0)