Skip to content

Latest commit

 

History

History
61 lines (39 loc) · 2.72 KB

SCRIPT_OUTPUT.md

File metadata and controls

61 lines (39 loc) · 2.72 KB
Raw

UDF script output

Exasol allows capturing combined output (STDOUT + STDERR) of UDF scripts. It is very helpful for debugging and to extract additional statistics from scripts running in production.

In order to use this feature, user has to run TCP server and provide address via session parameters. For example:

ALTER SESSION SET SCRIPT_OUTPUT_ADDRESS = 'myserver:16442';

Exasol may run UDFs in parallel using large amount of VM's. Each VM opens individual connection to TCP Server and keeps it opened until the end of execution. TCP server must be prepared for large number of simultaneous connections.

PyEXASOL provides such TCP server for your convenience. It can work in two different modes:

DEBUG MODE

Debug mode is useful for manual debugging during UDF script development.

Connections are accepted from all VM's. Output of first connection is displayed. Outputs of other connections are discarded.

Server runs forever until stopped by user.

How to use it:

  1. Run server in debug mode;
python -m pyexasol_utils.script_output
  1. Copy-paste provided SQL query to your SQL client and execute it;
  2. Run queries with UDF scripts, see output in terminal;
  3. Stop server with (Ctrl + C) when you finish debugging;

Please note: if you have problems getting script output immediately out of VM's, please make sure you flush STDOUT / STDERR in your UDF script. Some programming languages (like Python) may buffer output by default.

SCRIPT MODE

Script mode is useful for production usage and during last stages of development.

Connections are accepted from all VM's. Output of each VM is stored into separate log file.

Server runs for single SQL statement and stops automatically.

How to use it:

  1. (optional) Create base directory for UDF script logs and set it using udf_output_dir connection option.
  2. Execute query with UDF script using function execute_udf_output().
  3. Read and process files returned by function.

Example:

stmt, log_files = C.execute_udf_output("SELECT my_script(user_id) FROM table")

printer.pprint(stmt.fetchall())
printer.pprint(log_files[0].read_text())

You are responsible for deletion of log files.

Connectivity problem

Unlike HTTP Transport, script output TCP server is real server. It receives incoming connections from Exasol nodes. Those connections might be blocked by firewalls and various network policies. You are responsible for making host with TCP server available for incoming connections.

If you want to bind TCP server to specific address and port, you may use --host, --port arguments for debug mode and udf_output_bind_address, udf_output_connect_address connection options for script mode.