When having a csv source, sling treats the first row as the header. Is there a way to treat the first row as first row in database target?

[alberttwong]

Can it make up column names?

Here is how the csv looks like.

1|AAAAAAAABAAAAAAA|980124|7135|32946|2452238|2452208|Mr.|Javier|Lewis|Y|9|12|1936|CHILE|""|Javier.Lewis@VFAxlnZEvOx.org|2452508
2|AAAAAAAACAAAAAAA|819667|1461|31655|2452318|2452288|Dr.|Amy|Moses|Y|9|4|1966|TOGO|""|Amy.Moses@Ovk9KjHH.com|2452318

[flarco]

Yes, set `source_options: { header: false }`

…

On Wed, Apr 17, 2024, 8:22 PM Albert T. Wong ***@***.***> wrote: Here is how the csv looks like. ***@***.***|2452508 ***@***.***|2452318 — Reply to this email directly, view it on GitHub <#265>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB2QZYWIEM6G6VAQJ4OVFO3Y53725AVCNFSM6AAAAABGMGNVU6VHI2DSMVQWIX3LMV43ERDJONRXK43TNFXW4OZWGUZDCNJXG4> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[alberttwong]

doesn't seem to work

atwong@Albert-CelerData tpcds-csv % sling run --src-stream 'file://customer.csv' --source-options '{ header: false }' --tgt-conn POSTGRESLOCAL source_options: --tgt-object 'public.customer' --mode full-refresh
5:04PM INF connecting to target database (postgres)
5:04PM INF reading from source file system (file)
5:04PM INF writing to target database [mode: full-refresh]
5:04PM INF streaming data
5:04PM INF created table "public"."customer"
5:04PM INF inserted 99999 rows into "public"."customer" in 0 secs [143,154 r/s]
5:04PM INF execution succeeded

[flarco]

Ah, try --src-options '{ header: false }' not --source-options '{ header: false }'

sling run --src-stream 'file://customer.csv' --src-options '{ header: false }' --tgt-conn POSTGRESLOCAL --tgt-object 'public.customer' --mode full-refresh

[alberttwong]

Works. I would add this to the docs. I couldn't find that header option.

atwong@Albert-CelerData tpcds-csv % sling run --src-stream 'file://customer.csv' --src-options '{ header: false }' --tgt-conn POSTGRESLOCAL source_options: --tgt-object 'public.customer' --mode full-refresh
6:26PM INF connecting to target database (postgres)
6:26PM INF reading from source file system (file)
6:26PM INF writing to target database [mode: full-refresh]
6:26PM INF streaming data
6:26PM INF created table "public"."customer"
6:26PM INF inserted 100000 rows into "public"."customer" in 0 secs [138,955 r/s]
6:26PM INF execution succeeded