Squall Cluster Configs

We will explain the content of a config file on INSTALL_DIR/dip/SQLtoQueryPlanPlugin/confs/1G_hyracks_parallel:

DIP_DISTRIBUTED true
DIP_QUERY_NAME hyracks

DIP_TOPOLOGY_NAME_PREFIX teamX
DIP_DATA_ROOT /export/home/avitorovic/queries/tpch/
DIP_SQL_ROOT ../dip/SQLtoQueryPlanPlugin/SQLqueries/

# DIP_DB_SIZE is in GBs
DIP_DB_SIZE 1
DIP_MAX_SRC_PAR 4

#below are unlikely to change
DIP_EXTENSION .tbl
DIP_READ_SPLIT_DELIMITER \|
DIP_GLOBAL_ADD_DELIMITER |
DIP_GLOBAL_SPLIT_DELIMITER \|

DIP_ACK_EVERY_TUPLE true
DIP_KILL_AT_THE_END true
#used only in distributed mode
DIP_NIMBUS_HOST icdatasrv2
DIP_STORM_ZOOKEEPER_SERVERS icdatasrv2

Config file 1G_hyracks_parallel is the same as in 0.1G_hyracks_serial, except:

1. DIP_DISTRIBUTED is set to true.

2. DIP_TOPOLOGY_NAME_PREFIX is an optional parameter. It is used for distinguishing different users possibly running the same query at the same time on the cluster.

3. DIP_DATA_ROOT refers to a location on the cluster.

4. DIP_MAX_SRC_PAR is set to 4 to exploit parallelism in the cluster.

You can find more examples of config files in INSTALL_DIR/dip/SQLtoQueryPlanPlugin/confs.

Then, run the following commands:

cd $INSTALL_DIR/bin
./pluginSQLClusterRun.sh $CONFIG_FILE

where CONFIG_FILE is a file from INSTALL_DIR/dip/SQLtoQueryPlanPlugin/confs which ends up with _parallel.

DDDD As before, you cannot run arbitrary large database with small component parallelism. The way you control it here is through MAX_SRC_PAR parameter - the larger the parameter is, bigger database can be processed. TODO