Implement SQL driver

Actually three features are introduced with this commit.

## SQL driver

Now test-run recognizes *.test.sql files.

The content of those files are SQL statements written line-by-line. A
long statement may be splitted, see the next section.

The implementation leans on '\set language [lua|sql]' command of
tarantool console: appropriate language is set depending of a file name:
*.test.lua or *.test.sql before executing commands for a test file.

A value of 'engine' option when it is provided by a configuration has a
special meaning for *.test.sql tests (a configuration is a file set by
'config' option in suite.ini) . If it is 'memtx' or 'vinyl', then
corresponding default engine will be set for SQL subsystem before
executing commands for a test file. The implementation uses "pragma
sql_default_engine='memtx|vinyl'" command.

It is possible to mix Lua and SQL code in *.test.lua and *.test.sql
tests. Consider an example:

 | -- Verify output of large integers.
 | CREATE TABLE test (id INTEGER PRIMARY KEY)
 | INSERT INTO test VALUES (9223372036854775807)
 | SELECT * FROM test
 | \set language lua
 | box.space.TEST:select()
 | \set language sql
 | DROP TABLE test

## Line carrying with backslash

Consider the following code:

 | test_run:cmd("setopt delimiter ';'")
 | function echo(...)
 |     return ...
 | end
 | test_run:cmd("setopt delimiter ''");

Now it may be rewritten in the following way:

 | function echo(...) \
 |     return ...     \
 | end

This ability also works in SQL test files: a long statement may be
written in this way:

 | CREATE TABLE t1 (s1 VARCHAR(10) PRIMARY KEY)
 | CREATE TABLE t2 (s1 VARCHAR(10) PRIMARY KEY, s2 VARCHAR(10) \
 |     REFERENCES t1 ON DELETE CASCADE)

## New result file format

A new result file format is introduced. The differences are following.

1. A result of a Lua command or SQL statement is indented to easier
   distinguish from a command / statement itself.

Before:

 | CREATE TABLE t1 (s1 VARCHAR(10) PRIMARY KEY)
 | ---
 | - row_count: 1
 | ...

After:

 | CREATE TABLE t1 (s1 VARCHAR(10) PRIMARY KEY)
 |  | ---
 |  | - row_count: 1
 |  | ...

2. Empty lines from a test file are preserved in a result file.

3. A small bug when a comment in a setopt-delimiter block is written
   before a previous command was fixed.

4. The first line of a result file in the new format is a version line.

 | -- test-run result file version 2

A result file for a new test will be written in the new format, but when
a result file already exists (and has the old format), test-run performs
formatting of a test output in the old way. So the new test-run version
is backward compatible with previous ones and no changes are needed for
existing test and result files.

If you want to update a result file to the new format, remove it and run
a test.

If you want to produce a result file in the old format for a new test,
create an empty result file, run a test and move *.reject to *.result.

Fixes tarantool/tarantool#4123

README.md

@@ -30,9 +30,9 @@ Field `core` must be one of:
 
 ### Test
 
-Each test consists of files `*.test(.lua|.py)?`, `*.result`, and may have skip
-condition file `*.skipcond`.  On first run (without `.result`) `.result` is
-generated from output.  Each run, in the beggining, `.skipcond` file is
+Each test consists of files `*.test(.lua|.sql|.py)?`, `*.result`, and may have
+skip condition file `*.skipcond`.  On first run (without `.result`) `.result`
+is generated from output.  Each run, in the beggining, `.skipcond` file is
 executed. In the local env there's object `self`, that's `Test` object. If test
 must be skipped - you must put `self.skip = 1` in this file. Next,
 `.test(.lua|.py)?` is executed and file `.reject` is created, then `.reject` is
@@ -68,6 +68,11 @@ engine = test_run:get_cfg('engine')
 -- second run engine is 'sophia'
 ```
 
+"engine" value has a special meaning for *.test.sql files: if it is "memtx" or
+"vinyl", then the corresponding engine will be set using "pragma
+sql_default_engine='memtx|vinyl'" command before executing commands from a test
+file.
+
 #### Python
 
 Files: `<name>.test.py`, `<name>.result` and `<name>.skipcond`(optionaly).
@@ -234,6 +239,21 @@ test
 ...
 ```
 
+It is possible to use backslash at and of a line to carry it.
+
+```lua
+function echo(...) \
+    return ...     \
+end
+```
+
+#### SQL
+
+*.test.sql files are just SQL statements written line-by-line.
+
+It is possible to mix SQL and Lua commands using `\set language lua` and `\set
+language sql` commands.
+
 ##### Interaction with the test environment
 
 In lua test you can use `test_run` module to interact with the test

lib/tarantool_server.py

@@ -31,6 +31,7 @@
 from lib.utils import format_process
 from lib.utils import signame
 from lib.utils import warn_unix_socket
+from lib.utils import prefix_each_line
 from test import TestRunGreenlet, TestExecutionError
 
 
@@ -51,51 +52,246 @@ def save_join(green_obj, timeout=None):
 
 
 class LuaTest(Test):
+    """ Handle *.test.lua and *.test.sql test files. """
+
     TIMEOUT = 60 * 10
+    RESULT_FILE_VERSION_INITIAL = 1
+    RESULT_FILE_VERSION_DEFAULT = 2
+    RESULT_FILE_VERSION_LINE_RE = re.compile(
+        r'^-- test-run result file version (?P<version>\d+)$')
+    RESULT_FILE_VERSION_TEMPLATE = '-- test-run result file version {}'
+
+    def __init__(self, *args, **kwargs):
+        super(LuaTest, self).__init__(*args, **kwargs)
+        if self.name.endswith('.test.lua'):
+            self.default_language = 'lua'
+        else:
+            assert self.name.endswith('.test.sql')
+            self.default_language = 'sql'
+        self.result_file_version = self.result_file_version()
+
+    def result_file_version(self):
+        """ If a result file is not exists, return a default
+            version (last known by test-run).
+            If it exists, but does not contain a valid result file
+            header, return 1.
+            If it contains a version, return the version.
+        """
+        if not os.path.isfile(self.result):
+            return self.RESULT_FILE_VERSION_DEFAULT
+
+        with open(self.result, 'r') as f:
+            line = f.readline().rstrip('\n')
+
+            # An empty line or EOF.
+            if not line:
+                return self.RESULT_FILE_VERSION_INITIAL
+
+            # No result file header.
+            m = self.RESULT_FILE_VERSION_LINE_RE.match(line)
+            if not m:
+                return self.RESULT_FILE_VERSION_INITIAL
+
+            # A version should be integer.
+            try:
+                return int(m.group('version'))
+            except ValueError:
+                return self.RESULT_FILE_VERSION_INITIAL
+
+    def write_result_file_version_line(self):
+        # The initial version of a result file does not have a
+        # version line.
+        if self.result_file_version < 2:
+            return
+        sys.stdout.write(self.RESULT_FILE_VERSION_TEMPLATE.format(
+                         self.result_file_version) + '\n')
+
+    def execute_pretest_clean(self, ts):
+        """ Clean globals, loaded packages, spaces, users, roles
+            and so on before each test if the option is set.
+
+            Return True as success (or if this feature is disabled
+            in suite.ini) and False in case of an error.
+        """
+        if not self.suite_ini['pretest_clean']:
+            return True
+
+        command = "require('pretest_clean').clean()"
+        result = self.send_command(command, ts, 'lua')
+        result = result.replace('\r\n', '\n')
+        if result != '---\n...\n':
+            sys.stdout.write(result)
+            return False
+
+        return True
+
+    def execute_pragma_sql_default_engine(self, ts):
+        """ Set default engine for an SQL test if it is provided
+            in a configuration.
+
+            Return True if the command is successful or when it is
+            not performed, otherwise (when got an unexpected
+            result for the command) return False.
+        """
+        # Pass the command only for *.test.sql test files, because
+        # hence we sure tarantool supports SQL.
+        if self.default_language != 'sql':
+            return True
+
+        # Skip if no 'memtx' or 'vinyl' engine is provided.
+        ok = self.run_params and 'engine' in self.run_params and \
+            self.run_params['engine'] in ('memtx', 'vinyl')
+        if not ok:
+            return True
+
+        engine = self.run_params['engine']
+        command = "pragma sql_default_engine='{}'".format(engine)
+        result = self.send_command(command, ts, 'sql')
+        result = result.replace('\r\n', '\n')
+        if result != '---\n- row_count: 0\n...\n':
+            sys.stdout.write(result)
+            return False
+
+        return True
+
+    def set_language(self, ts, language):
+        for conn in ts.curcon:
+            conn(r'\set language ' + language, silent=True)
+
+    def send_command(self, command, ts, language=None):
+        if language:
+            self.set_language(ts, language)
+        result = ts.curcon[0](command, silent=True)
+        for conn in ts.curcon[1:]:
+            conn(command, silent=True)
+        # gh-24 fix
+        if result is None:
+            result = '[Lost current connection]\n'
+        return result
+
+    def flush(self, ts, command_log, command_exe):
+        # Write a command to a result file.
+        command = command_log.getvalue()
+        sys.stdout.write(command)
+
+        # Drop a previous command.
+        command_log.seek(0)
+        command_log.truncate()
+
+        if not command_exe:
+            return
+
+        # Send a command to tarantool console.
+        result = self.send_command(command_exe.getvalue(), ts)
+
+        # Convert and prettify a command result.
+        result = result.replace('\r\n', '\n')
+        if self.result_file_version >= 2:
+            result = prefix_each_line(' | ', result)
+
+        # Write a result of the command to a result file.
+        sys.stdout.write(result)
+
+        # Drop a previous command.
+        command_exe.seek(0)
+        command_exe.truncate()
 
     def exec_loop(self, ts):
-        cmd = None
-
-        def send_command(command):
-            result = ts.curcon[0](command, silent=True)
-            for conn in ts.curcon[1:]:
-                conn(command, silent=True)
-            # gh-24 fix
-            if result is None:
-                result = '[Lost current connection]\n'
-            return result
-
-        if self.suite_ini['pretest_clean']:
-            result = send_command("require('pretest_clean').clean()") \
-                .replace("\r\n", "\n")
-            if result != '---\n...\n':
-                sys.stdout.write(result)
-                return
+        self.write_result_file_version_line()
+        if not self.execute_pretest_clean(ts):
+            return
+        if not self.execute_pragma_sql_default_engine(ts):
+            return
+
+        # Set default language for the test.
+        self.set_language(ts, self.default_language)
+
+        # Use two buffers: one to commands that are logged in a
+        # result file and another that contains commands that
+        # actually executed on a tarantool console.
+        command_log = StringIO()
+        command_exe = StringIO()
+
+        # A newline from a source that is not end of a command is
+        # replaced with the following symbols.
+        newline_log = '\n'
+        newline_exe = ' '
+
+        # A backslash from a source is replaced with the following
+        # symbols.
+        backslash_log = '\\'
+        backslash_exe = ''
+
+        # A newline that marks end of a command is replaced with
+        # the following symbols.
+        eoc_log = '\n'
+        eoc_exe = '\n'
 
         for line in open(self.name, 'r'):
-            if not line.endswith('\n'):
-                line += '\n'
-            # context switch for inspector after each line
-            if not cmd:
-                cmd = StringIO()
-            if line.find('--') == 0:
-                sys.stdout.write(line)
-            else:
-                if line.strip() or cmd.getvalue():
-                    cmd.write(line)
-                delim_len = -len(ts.delimiter) if len(ts.delimiter) else None
-                end_line = line.endswith(ts.delimiter + '\n')
-                cmd_value = cmd.getvalue().strip()[:delim_len].strip()
-                if end_line and cmd_value:
-                    sys.stdout.write(cmd.getvalue())
-                    rescom = cmd.getvalue()[:delim_len].replace('\n\n', '\n')
-                    result = send_command(rescom)
-                    sys.stdout.write(result.replace("\r\n", "\n"))
-                    cmd.close()
-                    cmd = None
-            # join inspector handler
+            # Normalize a line.
+            line = line.rstrip('\n')
+
+            # Show empty lines / comments in a result file, but
+            # don't send them to tarantool.
+            line_is_empty = line.strip() == ''
+            if line_is_empty or line.find('--') == 0:
+                if self.result_file_version >= 2:
+                    command_log.write(line + eoc_log)
+                    self.flush(ts, command_log, None)
+                elif line_is_empty:
+                    # Compatibility mode: don't add empty lines to
+                    # a result file in except when a delimiter is
+                    # set.
+                    if command_log.getvalue():
+                        command_log.write(eoc_log)
+                else:
+                    # Compatibility mode: write a comment and only
+                    # then a command before it when a delimiter is
+                    # set.
+                    sys.stdout.write(line + eoc_log)
+                self.inspector.sem.wait()
+                continue
+
+            # A delimiter is set and found at end of the line:
+            # send the command.
+            if ts.delimiter and line.endswith(ts.delimiter):
+                delimiter_len = len(ts.delimiter)
+                command_log.write(line + eoc_log)
+                command_exe.write(line[:-delimiter_len] + eoc_exe)
+                self.flush(ts, command_log, command_exe)
+                self.inspector.sem.wait()
+                continue
+
+            # A backslash found at end of the line: continue
+            # collecting input. Send / log a backslash as is when
+            # it is inside a block with set delimiter.
+            if line.endswith('\\') and not ts.delimiter:
+                command_log.write(line[:-1] + backslash_log + newline_log)
+                command_exe.write(line[:-1] + backslash_exe + newline_exe)
+                self.inspector.sem.wait()
+                continue
+
+            # A delimiter is set, but not found at the end of the
+            # line: continue collecting input.
+            if ts.delimiter:
+                command_log.write(line + newline_log)
+                command_exe.write(line + newline_exe)
+                self.inspector.sem.wait()
+                continue
+
+            # A delimiter is not set, backslash is not found at
+            # end of the line: send the command.
+            command_log.write(line + eoc_log)
+            command_exe.write(line + eoc_exe)
+            self.flush(ts, command_log, command_exe)
             self.inspector.sem.wait()
-        # stop any servers created by the test, except the default one
+
+        # Free StringIO() buffers.
+        command_log.close()
+        command_exe.close()
+
+        # Stop any servers created by the test, except the default
+        # one.
         ts.stop_nondefault()
 
     def killall_servers(self, server, ts, crash_occured):
@@ -167,6 +363,8 @@ def execute(self, server):
 
 
 class PythonTest(Test):
+    """ Handle *.test.py test files. """
+
     def execute(self, server):
         super(PythonTest, self).execute(server)
         execfile(self.name, dict(locals(), test_run_current_test=self,
@@ -872,13 +1070,21 @@ def test_debug(self):
     def find_tests(test_suite, suite_path):
         test_suite.ini['suite'] = suite_path
 
-        def get_tests(pattern):
-            return sorted(glob.glob(os.path.join(suite_path, pattern)))
+        def get_tests(*patterns):
+            res = []
+            for pattern in patterns:
+                path_pattern = os.path.join(suite_path, pattern)
+                res.extend(sorted(glob.glob(path_pattern)))
+            return res
 
+        # Add Python tests.
         tests = [PythonTest(k, test_suite.args, test_suite.ini)
                  for k in get_tests("*.test.py")]
 
-        for k in get_tests("*.test.lua"):
+        # Add Lua and SQL tests. One test can appear several times
+        # with different configuration names (as configured in a
+        # file set by 'config' suite.ini option, usually *.cfg).
+        for k in get_tests("*.test.lua", "*.test.sql"):
             runs = test_suite.get_multirun_params(k)
 
             def is_correct(run_name):

lib/test_suite.py

@@ -174,6 +174,7 @@ def start_server(self, server):
         inspector.start()
         # fixme: remove this string if we fix all legacy tests
         suite_name = os.path.basename(self.suite_path)
+        # Set 'lua' type for *.test.lua and *.test.sql test files.
         server.tests_type = 'python' if suite_name.endswith('-py') else 'lua'
         server.deploy(silent=False)
         return inspector

lib/utils.py

@@ -252,3 +252,9 @@ def process_file(filepath):
                                 time_a,
                                 time_b)
     color_stdout.writeout_unidiff(diff)
+
+
+def prefix_each_line(prefix, data):
+    data = data.rstrip('\n')
+    lines = [(line + '\n') for line in data.split('\n')]
+    return prefix + prefix.join(lines)