-
Notifications
You must be signed in to change notification settings - Fork 95
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ERROR: http response code 404 from gpfdist #30
Comments
ssh to vitrina03 and run |
Hello. Sincerely, Sergey Berezin. |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| 2022-11-08 16:57:03 81125 INFO Before opening listening sockets - following listening sockets are available: 2022-11-08 16:57:13 81125 INFO [0:1:6:8] request end |
This means gpfdist is working properly and simply couldn't find any files. Did the generate data step finish? Look for the log file on that host named generate_data.x.log. Refer to this: |
Here is the data in the Log. ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| GEN_DATA_SCALE: 10000 |
This means the dsdgen binary didn't get copied to the host. https://github.com/RunningJon/TPC-DS/blob/master/00_compile_tpcds/rollout.sh#L33 Do you see this file? 00_compile_tpcds/tools/dsqgen If so, does segment_hosts.txt have all of the correct segment hosts in it including this host? |
Yes, the dsqgen file is in place. Sincerely, Sergey Berezin. |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| -rwxrwxr-x 1 gpadmin gpadmin 455416 Oct 31 15:53 dsdgen ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| [root@vitrina01 ~]# find / -name segment_hosts.txt |
10TB with only 15 segments on 3 nodes will take a very long time to complete. The error was on vitrina03: But you said it is in place. Correct? |
The file is in a different location. Sincerely, Sergey Berezin. ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| [root@vitrina01 tools]# ls -la /home/gpadmin/dsdgen |
dsdgen is the program that is compiled on vitrina01 (master) in /pivotalguru/TPC-DS/00_compile_tpcds/tools and copied to every segment host (vitrina02, vitrina03, vitrina04). The error message said that the file didn't exist on vitrina03. |
Sincerely, Sergey Berezin. ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| |
https://github.com/RunningJon/TPC-DS/blob/master/00_compile_tpcds/rollout.sh#L33 Are you able to ssh between the nodes as gpadmin? Can you scp files from the master node to every segment host? |
Yes, there is a connection. ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| [root@vitrina01 TPC-DS-master]# scp tpcds.log vitrina02:/home/gpadmin/ |
I would set RUN_COMPILE_TPCDS to "true" and RUN_GEN_DATA to "true" in tpcds_variables.sh and then run the benchmark again. When it ran the compile the first time, it did not copy the binary to the segment hosts. |
Now the log is showing: |
You need to kill all of the dsdgen processes on the segment hosts and make sure the benchmark isn't running. |
Hello. Where can i change variable for this one, if this is possible? Sincerely, Sergey Berezin. |
It is running |
Please tell me how to run more or less than five worker processes? Sincerely, Sergey Berezin. |
for analyze? You could hard code it in the script but it really won't help much. That isn't what is taking a long time. |
Hello
I am getting an error while running the script.
Help me figure it out
Sincerely, Sergey Berezin.
stop gpfdist on all ports
92962
killing 93075
killing 93128
93166
killing 93204
280453
killing 280505
killing280543
killing280581
killing280670
killing290935
killing291026
killing291069
killing291107
killing291145
executing on vitrina02 ./start_gpfdist.sh 4001 /data1/primary/gpseg0/pivotalguru
Started gpfdist on port 4001
executing on vitrina02 ./start_gpfdist.sh 4002 /data1/primary/gpseg1/pivotalguru
Started gpfdist on port 4002
executing on vitrina02 ./start_gpfdist.sh 4003 /data1/primary/gpseg2/pivotalguru
Started gpfdist on port 4003
executing on vitrina02 ./start_gpfdist.sh 4004 /data1/primary/gpseg3/pivotalguru
Started gpfdist on port 4004
executing on vitrina02 ./start_gpfdist.sh 4005 /data1/primary/gpseg4/pivotalguru
Started gpfdist on port 4005
executing on vitrina03 ./start_gpfdist.sh 4001 /data1/primary/gpseg5/pivotalguru
Started gpfdist on port 4001
executing on vitrina03 ./start_gpfdist.sh 4002 /data1/primary/gpseg6/pivotalguru
Started gpfdist on port 4002
executing on vitrina03 ./start_gpfdist.sh 4003 /data1/primary/gpseg7/pivotalguru
Started gpfdist on port 4003
executing on vitrina03 ./start_gpfdist.sh 4004 /data1/primary/gpseg8/pivotalguru
Started gpfdist on port 4004
executing on vitrina03 ./start_gpfdist.sh 4005 /data1/primary/gpseg9/pivotalguru
Started gpfdist on port 4005
executing on vitrina04 ./start_gpfdist.sh 4001 /data1/primary/gpseg10/pivotalguru
Started gpfdist on port 4001
executing on vitrina04 ./start_gpfdist.sh 4002 /data1/primary/gpseg11/pivotalguru
Started gpfdist on port 4002
executing on vitrina04 ./start_gpfdist.sh 4003 /data1/primary/gpseg12/pivotalguru
Started gpfdist on port 4003
executing on vitrina04 ./start_gpfdist.sh 4004 /data1/primary/gpseg13/pivotalguru
Started gpfdist on port 4004
executing on vitrina04 ./start_gpfdist.sh 4005 /data1/primary/gpseg14/pivotalguru
Started gpfdist on port 4005
psql -v ON_ERROR_STOP=1 -f /pivotalguru/TPC-DS/04_load/001.gpdb.time_dim.sql | grep INSERT | awk -F ' ' '{print $3}'
psql:/pivotalguru/TPC-DS/04_load/001.gpdb.time_dim.sql:1: ERROR: http response code 404 from gpfdist (gpfdist://vitrina03:4002/time_dim_[0-9]_[0-9 ].dat): HTTP/1.0 404 file not found (seg6 slice1 192.168.11.24:10001 pid=97255)
The text was updated successfully, but these errors were encountered: