csvstack: Allow file pattern instead of filenames

[darwinbandoy]

I try to implement csvstack for 160K files csvstack --filenames *.csv > ./merged.csv and I get argument list too long error.

Anyone managed to solve this?

[jpmckinney]

“Argument list too long” is a shell error, not a csvkit error. The shell expands the * into a list of arguments.

The alternative would be to provide the file pattern to csvkit, and for it to expand it.

I’ll set this as a feature request.

[darwinbandoy]

I did managed to get around the wildcard (*) argument list too long issue by using ls | grep csv for my customized code. But when I do this for csvstack I only get filenames in a row. This is the code I am using:

ls | grep csv | csvstack --filenames > ./merged.csv

[jpmckinney]

Yeah, --filenames is to "Use the filename of each input file as its grouping value. When specified, -g will be ignored.". There isn't yet any option for providing filenames via stdin.