From 6696a5fb43d9b6644ad9ed54a29841be04dafbcf Mon Sep 17 00:00:00 2001 From: Pavel Semyonov Date: Tue, 5 Nov 2024 17:41:32 +0700 Subject: [PATCH 1/5] Add tt-ee 2.5.1 authentication features --- doc/tooling/tt_cli/connect.rst | 6 +- doc/tooling/tt_cli/export.rst | 110 ++++++++++++++++++++++++++++++++- doc/tooling/tt_cli/import.rst | 108 ++++++++++++++++++++++++++++++++ 3 files changed, 219 insertions(+), 5 deletions(-) diff --git a/doc/tooling/tt_cli/connect.rst b/doc/tooling/tt_cli/connect.rst index 58783b79da..443a362ad1 100644 --- a/doc/tooling/tt_cli/connect.rst +++ b/doc/tooling/tt_cli/connect.rst @@ -104,9 +104,9 @@ If no credentials are provided for a remote connection, the user is automaticall Encrypted connection ~~~~~~~~~~~~~~~~~~~~ -To connect to instances that use SSL encryption, provide the SSL certificate and -SSL key files in the ``--sslcertfile`` and ``--sslkeyfile`` options. If necessary, -add other SSL parameters -- ``--sslcafile`` and ``--sslciphers``. +To connect to instances that use :ref:`SSL encryption `, +provide the SSL certificate and SSL key files in the ``--sslcertfile`` and ``--sslkeyfile`` options. +If necessary, add other SSL parameters -- ``--sslcafile`` and ``--sslciphers``. Script evaluation ~~~~~~~~~~~~~~~~~ diff --git a/doc/tooling/tt_cli/export.rst b/doc/tooling/tt_cli/export.rst index 3b708f1a6d..c8ca8a8c7e 100644 --- a/doc/tooling/tt_cli/export.rst +++ b/doc/tooling/tt_cli/export.rst @@ -130,7 +130,15 @@ the ``customers.jsonl`` file: $ tt tdg2 export localhost:3301 customers:customers.jsonl -If the ``customers`` type has four fields (``id``, ``firstname``, ``lastname``, and ``age``), the file with exported data might look like this: +If token authentication is enabled in TDG2, pass the application token in the ``-token`` option: + +.. code-block:: console + + $ tt tdg2 export localhost:3301 customers:customers.jsonl \ + --token=2fc136cf-8cae-4655-a431-7c318967263d + +If the ``customers`` type has four fields (``id``, ``firstname``, ``lastname``, and ``age``), +the file with exported data might look like this: .. code-block:: json @@ -138,7 +146,7 @@ If the ``customers`` type has four fields (``id``, ``firstname``, ``lastname``, {"age":41,"first_name":"Fay","id":2,"second_name":"Rivers"} {"age":74,"first_name":"Milo","id":4,"second_name":"Walters"} -If an object contains a ``null`` value in a field, this field skipped: +``null`` field values are skipped: .. code-block:: json @@ -168,11 +176,55 @@ Set the ``tt tdg2 export`` batch size less or equal to 1000: $ tt tdg2 export localhost:3301 customers:customers.jsonl --batch-size=1000 +.. _tt-export-auth: + +Authentication +-------------- + +When connecting to the cluster with enabled authentication, specify access credentials +in the ``--username`` and ``--password`` command options: + +.. code-block:: console + + $ tt crud export localhost:3301 customers:customers.csv \ + --username myuser -password p4$$w0rD + +.. _tt-export-ssl: + +Encrypted connection +-------------------- + +To connect to instances that use :ref:`SSL encryption `, +provide the SSL certificate and SSL key files in the ``--sslcertfile`` and ``--sslkeyfile`` options. +If necessary, add other SSL parameters in the ``--ssl*`` options. + +.. code-block:: console + + $ tt crud export localhost:3301 customers:customers.csv \ + --username myuser -password p4$$w0rD \ + --auth pap-sha256 --sslcertfile certs/server.crt \ + --sslkeyfile certs/server.key + +For connections that use SSL but don't require additional parameters, add the ``--use-ssl`` +option: + +.. code-block:: console + + $ tt crud export localhost:3301 customers:customers.csv \ + --username myuser -password p4$$w0rD \ + --use--ssl + .. _tt-export-options: Options ------- +.. option:: --auth STRING + + **Applicable to:** ``tt crud export``, ``tt tdg2 export`` + + Authentication type: ``chap-sha1`` or ``pap-sha256``. + .. option:: --batch-queue-size INT The maximum number of tuple batches in a queue between a fetch and write threads (the default is ``32``). @@ -226,6 +278,60 @@ Options Export data using a :ref:`read view `. +.. option:: --sslcafile STRING + + **Applicable to:** ``tt crud export``, ``tt tdg2 export`` + + The path to a trusted certificate authorities (CA) file for encrypted connections. + + See also :ref:`tt-export-ssl`. + +.. option:: --sslcertfile STRING + + **Applicable to:** ``tt crud export``, ``tt tdg2 export`` + + The path to an SSL certificate file for encrypted connections. + + See also :ref:`tt-export-ssl`. + +.. option:: --sslciphersfile STRING + + **Applicable to:** ``tt crud export``, ``tt tdg2 export`` + + The list of SSL cipher suites used for encrypted connections, separated by colons (``:``). + + See also :ref:`tt-export-ssl`. + +.. option:: --sslkeyfile STRING + + **Applicable to:** ``tt crud export``, ``tt tdg2 export`` + + The path to a private SSL key file for encrypted connections. + + See also :ref:`tt-export-ssl`. + +.. option:: --sslpassword STRING + + **Applicable to:** ``tt crud export``, ``tt tdg2 export`` + + The password for the SSL key file for encrypted connections. + + See also :ref:`tt-export-ssl`. + +.. option:: --sslpasswordfile STRING + + **Applicable to:** ``tt crud export``, ``tt tdg2 export`` + + A file with list of passwords to the SSL key file for encrypted connections. + + See also :ref:`tt-export-auth`. + +.. option:: --token STRING + + **Applicable to:** ``tt tdg2 export`` + + An application token for connecting to TDG2. + .. option:: --username STRING A username for connecting to the instance. diff --git a/doc/tooling/tt_cli/import.rst b/doc/tooling/tt_cli/import.rst index 277dc201b2..92ca4981fd 100644 --- a/doc/tooling/tt_cli/import.rst +++ b/doc/tooling/tt_cli/import.rst @@ -184,6 +184,13 @@ The objects are described in the ``customers.jsonl`` file. $ tt tdg2 import localhost:3301 customers.jsonl:customers +If token authentication is enabled in TDG2, pass the application token in the ``-token`` option: + +.. code-block:: console + + $ tt tdg2 import localhost:3301 customers.jsonl:customers \ + --token=2fc136cf-8cae-4655-a431-7c318967263d + The input file can look like this: .. code-block:: json @@ -220,11 +227,58 @@ To automatically confirm a batch import operation, add the ``--force`` option: --force +.. _tt-import-auth: + +Authentication +-------------- + +When connecting to the cluster with enabled authentication, specify access credentials +in the ``--username`` and ``--password`` command options: + +.. code-block:: console + + $ tt crud import localhost:3301 customers.csv:customers \ + --header --match=header \ + --username myuser -password p4$$w0rD + +.. _tt-import-ssl: + +Encrypted connection +-------------------- + +To connect to instances that use :ref:`SSL encryption `, +provide the SSL certificate and SSL key files in the ``--sslcertfile`` and ``--sslkeyfile`` options. +If necessary, add other SSL parameters in the ``--ssl*`` options. + +.. code-block:: console + + $ tt crud import localhost:3301 customers.csv:customers \ + --header --match=header \ + --username myuser -password p4$$w0rD \ + --auth pap-sha256 --sslcertfile certs/server.crt \ + --sslkeyfile certs/server.key + +For connections that use SSL but don't require additional parameters, add the ``--use-ssl`` +option: + +.. code-block:: console + + $ tt crud import localhost:3301 customers.csv:customers \ + --header --match=header \ + --username myuser -password p4$$w0rD \ + --use--ssl + .. _tt-import-options: Options ------- +.. option:: --auth STRING + + **Applicable to:** ``tt crud import``, ``tt tdg2 import`` + + Authentication type: ``chap-sha1`` or ``pap-sha256``. + .. option:: --batch-size INT **Applicable to:** ``tt crud import``, ``tt tdg2 import`` @@ -377,6 +431,54 @@ Options For CSV, double quotes are used by default (``"``). The double symbol of this option acts as the escaping symbol within input data. +.. option:: --sslcafile STRING + + **Applicable to:** ``tt crud import``, ``tt tdg2 import`` + + The path to a trusted certificate authorities (CA) file for encrypted connections. + + See also :ref:`tt-import-ssl`. + +.. option:: --sslcertfile STRING + + **Applicable to:** ``tt crud import``, ``tt tdg2 import`` + + The path to an SSL certificate file for encrypted connections. + + See also :ref:`tt-import-ssl`. + +.. option:: --sslciphersfile STRING + + **Applicable to:** ``tt crud import``, ``tt tdg2 import`` + + The list of SSL cipher suites used for encrypted connections, separated by colons (``:``). + + See also :ref:`tt-import-ssl`. + +.. option:: --sslkeyfile STRING + + **Applicable to:** ``tt crud import``, ``tt tdg2 import`` + + The path to a private SSL key file for encrypted connections. + + See also :ref:`tt-import-ssl`. + +.. option:: --sslpassword STRING + + **Applicable to:** ``tt crud import``, ``tt tdg2 import`` + + The password for the SSL key file for encrypted connections. + + See also :ref:`tt-import-ssl`. + +.. option:: --sslpasswordfile STRING + + **Applicable to:** ``tt crud import``, ``tt tdg2 import`` + + A file with list of passwords to the SSL key file for encrypted connections. + + See also :ref:`tt-import-auth`. + .. option:: -success STRING The name of a file with rows that were imported (the default is ``success``). @@ -394,6 +496,12 @@ Options Symbols specified in this option cannot intersect with ``--dec-sep``. +.. option:: --token STRING + + **Applicable to:** ``tt tdg2 import`` + + An application token for connecting to TDG2. + .. option:: --username STRING A username for connecting to the instance. From c4c473a1100b2f5de06adf6bd4e10fc4f4d0ad9e Mon Sep 17 00:00:00 2001 From: Pavel Semyonov Date: Tue, 5 Nov 2024 17:53:31 +0700 Subject: [PATCH 2/5] Fix dashes --- doc/tooling/tt_cli/export.rst | 4 ++-- doc/tooling/tt_cli/import.rst | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/doc/tooling/tt_cli/export.rst b/doc/tooling/tt_cli/export.rst index c8ca8a8c7e..11548de97d 100644 --- a/doc/tooling/tt_cli/export.rst +++ b/doc/tooling/tt_cli/export.rst @@ -130,7 +130,7 @@ the ``customers.jsonl`` file: $ tt tdg2 export localhost:3301 customers:customers.jsonl -If token authentication is enabled in TDG2, pass the application token in the ``-token`` option: +If token authentication is enabled in TDG2, pass the application token in the ``--token`` option: .. code-block:: console @@ -212,7 +212,7 @@ option: $ tt crud export localhost:3301 customers:customers.csv \ --username myuser -password p4$$w0rD \ - --use--ssl + --use-ssl .. _tt-export-options: diff --git a/doc/tooling/tt_cli/import.rst b/doc/tooling/tt_cli/import.rst index 92ca4981fd..ace7ec7236 100644 --- a/doc/tooling/tt_cli/import.rst +++ b/doc/tooling/tt_cli/import.rst @@ -184,7 +184,7 @@ The objects are described in the ``customers.jsonl`` file. $ tt tdg2 import localhost:3301 customers.jsonl:customers -If token authentication is enabled in TDG2, pass the application token in the ``-token`` option: +If token authentication is enabled in TDG2, pass the application token in the ``--token`` option: .. code-block:: console @@ -266,7 +266,7 @@ option: $ tt crud import localhost:3301 customers.csv:customers \ --header --match=header \ --username myuser -password p4$$w0rD \ - --use--ssl + --use-ssl .. _tt-import-options: From 435952c67a4c6ce71c972f6b53ba4f1e6a6037f1 Mon Sep 17 00:00:00 2001 From: Pavel Semyonov Date: Tue, 5 Nov 2024 18:38:52 +0700 Subject: [PATCH 3/5] Apply suggestions from code review Co-authored-by: Alexander Tulchinskiy <38702085+DerekBum@users.noreply.github.com> --- doc/tooling/tt_cli/export.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/doc/tooling/tt_cli/export.rst b/doc/tooling/tt_cli/export.rst index 11548de97d..c0e082575f 100644 --- a/doc/tooling/tt_cli/export.rst +++ b/doc/tooling/tt_cli/export.rst @@ -187,7 +187,7 @@ in the ``--username`` and ``--password`` command options: .. code-block:: console $ tt crud export localhost:3301 customers:customers.csv \ - --username myuser -password p4$$w0rD + --username myuser --password p4$$w0rD .. _tt-export-ssl: @@ -201,7 +201,7 @@ If necessary, add other SSL parameters in the ``--ssl*`` options. .. code-block:: console $ tt crud export localhost:3301 customers:customers.csv \ - --username myuser -password p4$$w0rD \ + --username myuser --password p4$$w0rD \ --auth pap-sha256 --sslcertfile certs/server.crt \ --sslkeyfile certs/server.key @@ -211,7 +211,7 @@ option: .. code-block:: console $ tt crud export localhost:3301 customers:customers.csv \ - --username myuser -password p4$$w0rD \ + --username myuser --password p4$$w0rD \ --use-ssl .. _tt-export-options: From 6a922481fd27a73a24975d930ece6d5f512621a5 Mon Sep 17 00:00:00 2001 From: Pavel Semyonov Date: Tue, 5 Nov 2024 18:43:59 +0700 Subject: [PATCH 4/5] Review fixes --- doc/tooling/tt_cli/export.rst | 8 +++++++- doc/tooling/tt_cli/import.rst | 33 ++++++++++++++++++++------------- 2 files changed, 27 insertions(+), 14 deletions(-) diff --git a/doc/tooling/tt_cli/export.rst b/doc/tooling/tt_cli/export.rst index c0e082575f..69197f3402 100644 --- a/doc/tooling/tt_cli/export.rst +++ b/doc/tooling/tt_cli/export.rst @@ -223,7 +223,7 @@ Options **Applicable to:** ``tt crud export``, ``tt tdg2 export`` - Authentication type: ``chap-sha1`` or ``pap-sha256``. + Authentication type: ``chap-sha1``, ``pap-sha256``, or ``auto``. .. option:: --batch-queue-size INT @@ -332,6 +332,12 @@ Options An application token for connecting to TDG2. +.. option:: --use-ssl STRING + + Use SSL without providing any additional SSL parameters. + + See also :ref:`tt-export-ssl`. + .. option:: --username STRING A username for connecting to the instance. diff --git a/doc/tooling/tt_cli/import.rst b/doc/tooling/tt_cli/import.rst index ace7ec7236..26101b7559 100644 --- a/doc/tooling/tt_cli/import.rst +++ b/doc/tooling/tt_cli/import.rst @@ -239,7 +239,7 @@ in the ``--username`` and ``--password`` command options: $ tt crud import localhost:3301 customers.csv:customers \ --header --match=header \ - --username myuser -password p4$$w0rD + --username myuser --password p4$$w0rD .. _tt-import-ssl: @@ -254,7 +254,7 @@ If necessary, add other SSL parameters in the ``--ssl*`` options. $ tt crud import localhost:3301 customers.csv:customers \ --header --match=header \ - --username myuser -password p4$$w0rD \ + --username myuser --password p4$$w0rD \ --auth pap-sha256 --sslcertfile certs/server.crt \ --sslkeyfile certs/server.key @@ -265,7 +265,7 @@ option: $ tt crud import localhost:3301 customers.csv:customers \ --header --match=header \ - --username myuser -password p4$$w0rD \ + --username myuser --password p4$$w0rD \ --use-ssl .. _tt-import-options: @@ -277,7 +277,7 @@ Options **Applicable to:** ``tt crud import``, ``tt tdg2 import`` - Authentication type: ``chap-sha1`` or ``pap-sha256``. + Authentication type: ``chap-sha1``, ``pap-sha256``, or ``auto``. .. option:: --batch-size INT @@ -431,6 +431,17 @@ Options For CSV, double quotes are used by default (``"``). The double symbol of this option acts as the escaping symbol within input data. +.. option:: --rollback-on-error + + **Applicable to:** ``tt crud import`` + + Specify whether any operation failed on a storage leads to rollback of a batch + import on this storage. + + .. note:: + + ``tt tdg2 import`` always works as if ``--rollback-on-error`` is ``true``. + .. option:: --sslcafile STRING **Applicable to:** ``tt crud import``, ``tt tdg2 import`` @@ -502,17 +513,13 @@ Options An application token for connecting to TDG2. -.. option:: --username STRING +.. option:: --use-ssl STRING - A username for connecting to the instance. - -.. option:: --rollback-on-error + Use SSL without providing any additional SSL parameters. - **Applicable to:** ``tt crud import`` + See also :ref:`tt-import-ssl`. - Specify whether any operation failed on a storage leads to rollback of a batch - import on this storage. +.. option:: --username STRING - .. note:: + A username for connecting to the instance. - ``tt tdg2 import`` always works as if ``--rollback-on-error`` is ``true``. From ff26e962a44ebe13170303971c01de2327c8d5c3 Mon Sep 17 00:00:00 2001 From: Pavel Semyonov Date: Wed, 6 Nov 2024 10:14:58 +0700 Subject: [PATCH 5/5] Apply suggestions from code review Co-authored-by: Elena Shebunyaeva --- doc/tooling/tt_cli/import.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/tooling/tt_cli/import.rst b/doc/tooling/tt_cli/import.rst index 26101b7559..dad406789a 100644 --- a/doc/tooling/tt_cli/import.rst +++ b/doc/tooling/tt_cli/import.rst @@ -435,7 +435,7 @@ Options **Applicable to:** ``tt crud import`` - Specify whether any operation failed on a storage leads to rollback of a batch + Specify whether any operation failed on a storage leads to rolling back batch import on this storage. .. note:: @@ -486,7 +486,7 @@ Options **Applicable to:** ``tt crud import``, ``tt tdg2 import`` - A file with list of passwords to the SSL key file for encrypted connections. + A file with a list of passwords to the SSL key file for encrypted connections. See also :ref:`tt-import-auth`.