MDB-29481 Add setting to restore without data copy.

[MikhailBurdukov]

No description provided.

[MikhailBurdukov]

config or cli knob?

[MikhailBurdukov]

Without this version restriction:

    Given we use the same object storage bucket for clickhouse01 as on clickhouse02 # tests/integration/steps/common.py:53 0.006s
      Traceback (most recent call last):
        File "/data/disk1/home/mburdukov/workspace/ch-backup/venv/lib/python3.10/site-packages/behave/model.py", line 1329, in run
          match.run(runner.context)
        File "/data/disk1/home/mburdukov/workspace/ch-backup/venv/lib/python3.10/site-packages/behave/matchers.py", line 98, in run
          self.func(context, *args, **kwargs)
        File "tests/integration/steps/common.py", line 58, in step_sync_storage_config
          copy_between_containers(
        File "/data/disk1/home/mburdukov/workspace/ch-backup/tests/integration/modules/docker.py", line 61, in copy_between_containers
          assert container_to.put_archive(path=path_to, data=data)
        File "/data/disk1/home/mburdukov/workspace/ch-backup/venv/lib/python3.10/site-packages/docker/models/containers.py", line 334, in put_archive
          return self.client.api.put_archive(self.id, path, data)
        File "/data/disk1/home/mburdukov/workspace/ch-backup/venv/lib/python3.10/site-packages/docker/utils/decorators.py", line 19, in wrapped
          return f(self, resource_id, *args, **kwargs)
        File "/data/disk1/home/mburdukov/workspace/ch-backup/venv/lib/python3.10/site-packages/docker/api/container.py", line 965, in put_archive
          res = self._put(url, params=params, data=data)
        File "/data/disk1/home/mburdukov/workspace/ch-backup/venv/lib/python3.10/site-packages/docker/utils/decorators.py", line 46, in inner
          return f(self, *args, **kwargs)
        File "/data/disk1/home/mburdukov/workspace/ch-backup/venv/lib/python3.10/site-packages/docker/api/client.py", line 241, in _put
          return self.put(url, **self._set_request_timeout(kwargs))
        File "/data/disk1/home/mburdukov/workspace/ch-backup/venv/lib/python3.10/site-packages/requests/sessions.py", line 649, in put
          return self.request("PUT", url, data=data, **kwargs)
        File "/data/disk1/home/mburdukov/workspace/ch-backup/venv/lib/python3.10/site-packages/requests/sessions.py", line 589, in request
          resp = self.send(prep, **send_kwargs)
        File "/data/disk1/home/mburdukov/workspace/ch-backup/venv/lib/python3.10/site-packages/requests/sessions.py", line 703, in send
          r = adapter.send(request, **kwargs)
        File "/data/disk1/home/mburdukov/workspace/ch-backup/venv/lib/python3.10/site-packages/requests/adapters.py", line 486, in send
          resp = conn.urlopen(
        File "/data/disk1/home/mburdukov/workspace/ch-backup/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 715, in urlopen
          httplib_response = self._make_request(
        File "/data/disk1/home/mburdukov/workspace/ch-backup/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 414, in _make_request
          conn.request_chunked(method, url, **httplib_request_kw)
      AttributeError: 'UnixHTTPConnection' object has no attribute 'request_chunked'

[MikhailBurdukov]

For 8.4.0 tenacity build failed. BTW python3.6 for the newest lib versions out of support: https://tenacity.readthedocs.io/en/latest/changelog.html#upgrade-notes

env PATH=venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin PYTHON_VERSION=3.10 PYTHONIOENCODING=utf8 CLICKHOUSE_VERSION=latest COMPOSE_HTTP_TIMEOUT=300 python -m tests.integration.env_control create
Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/data/disk1/home/mburdukov/workspace/ch-backup/tests/integration/env_control.py", line 12, in <module>
    from tests.integration.modules import compose, docker, minio, templates
  File "/data/disk1/home/mburdukov/workspace/ch-backup/tests/integration/modules/minio.py", line 9, in <module>
    from tenacity import retry, retry_if_exception_type, stop_after_attempt, wait_fixed
  File "/data/disk1/home/mburdukov/workspace/ch-backup/venv/lib/python3.10/site-packages/tenacity/__init__.py", line 653, in <module>
    from tenacity.asyncio import AsyncRetrying  # noqa:E402,I100
ModuleNotFoundError: No module named 'tenacity.asyncio'
make: *** [Makefile:167: .session_conf.sav] Error 1

[Alex-Burmak]

Suggest to use the same host for testing in-place restore. Otherwise it's not in-place :)

And it would be great to check that restore is actually in-place. For example, check object count in S3 or check that references to S3 objects in local metadata of S3 disks after restore are the same as at backup creation time.