Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A strange cron problem with csvsql #342

Closed
aborruso opened this issue Oct 5, 2014 · 13 comments
Closed

A strange cron problem with csvsql #342

aborruso opened this issue Oct 5, 2014 · 13 comments

Comments

@aborruso
Copy link

aborruso commented Oct 5, 2014

Hi all,
if I run this script, it runs properly
/usr/local/bin/csvsql --query "select * from archiviotemp_05 group by numero_protocollo" /var/www/test/archiviotemp_05.csv > /var/www/test/06_archivio.csv

If I run it via cron I have a 0 kb file. This my cron job:
*/5 * * * * root /bin/bash /usr/local/bin/csvsql --query "select * from archiviotemp_05 group by numero_protocollo" /var/www/test/archiviotemp_05.csv > /var/www/test/06_archivio.csv > /dev/null 2>&1

It seems to me that there is some problem related to csvsql and cron, because I use cron without any problems with the other csvkit commands. But it could be stupid error of mine.

Thank you very much

@aborruso
Copy link
Author

aborruso commented Oct 5, 2014

The same without /bin/bash (I think it's not necessary):
*/5 * * * * root /usr/local/bin/csvsql --query "select * from archiviotemp_05 group by numero_protocollo" /var/www/test/archiviotemp_05.csv > /var/www/test/06_archivio.csv > /dev/null 2>&1

Thank you

@aborruso
Copy link
Author

Hi I know that this is probably not an issue, but I need to know if anyone is able to run some csvsql command via cron job. If yes, I will know it's an error of mine and I will work on it.

Thank you very much

@onyxfish
Copy link
Collaborator

Hi Andrea, it's def possible to use the csvkit commands with cron. Looking at your command my best guess would be it's something to do with the second redirect on your command: > /dev/null 2>&1. Or it could be that universal devil: permissions. Either way, I'm 90% sure this isn't a csvkit issue.

@aborruso
Copy link
Author

Hi @onyxfish ,
I'm a beginner, so I may be wrong.
In my test csvsql does not work using cron.

It works only if write the command in this way:
csvsql --query "select * from archiviotemp group by numero_protocollo" --tables archiviotemp < $folder/archiviotemp.csv > $folder/archivio.csv

This solution has been provided to me by my friend. Moreover he looked at your code and he told me to tell you:
"I see you add stdin as input file if it's not a tty, but there is no tty when the script runs as a cron job, and it tries to read from stdin. On line 58-63 you should consider this variant."

I hope to be helpful.

Thank you

@dazl
Copy link

dazl commented Dec 20, 2014

thank you @aborruso, in my bash script I had to change

csvsql --db $DB --insert --no-create $f

to

csvsql --db $DB --insert --no-create --table $table < $f

for it to work within a rundeck task

@jommgoncalves
Copy link

Hi,

I have the same problem. The proposed workaround it works for one table but for multiple tables it doesn't work :\

Anyone knows how can I solve this problem?

Thank you very much,

@jpmckinney
Copy link
Member

@jommgoncalves You can pass multiple table names with --tables.

@jommgoncalves
Copy link

@jpmckinney Thank you for your quickly reply. It works with your suggestion!

@imwikaash
Copy link

Hi all,
I am using csvsql in a script . it is executing fine. But the same is not working using a crob job.

Please help.

@aborruso
Copy link
Author

@imwikaash please paste here your crone command

@imwikaash
Copy link

@aborruso Here is my crone command
29 * * * * python /mnt/scripts/marketplace007/files/checkImsVimSync.py >> /mnt/scripts/marketplace007/files/error.log 2>&1

here is command in checkImsVimSync python file
path="/mnt/scripts/marketplace007/files"
os.system('csvsql --query "select * from test" --tables test<'+path+'/test.csv > '+path+'/nonsynced.csv')

@aborruso
Copy link
Author

@imwikaash try to put the absolute path of csvsql inside your py file

@imwikaash
Copy link

Thanks @aborruso
working fine after adding the absolute path of csvsql ..

Gui13 added a commit to Gui13/csvkit that referenced this issue Sep 1, 2021
Add crontab use case, where you need to pass the file through piping rather than as an argument to the input.

See wireservice#342
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants