Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Сan't initialize storage/router if no UUIDs #407

Closed
askalt opened this issue Dec 28, 2023 · 0 comments · Fixed by #408
Closed

Сan't initialize storage/router if no UUIDs #407

askalt opened this issue Dec 28, 2023 · 0 comments · Fixed by #408
Assignees
Labels
2sp bug Something isn't working teamE

Comments

@askalt
Copy link

askalt commented Dec 28, 2023

Assume, we have cluster config:

credentials:
  users:
    replicator:
      password: a
      roles: [replication]
    storage:
      password: a
      roles: [sharding]

iproto:
  advertise:
    peer:
      login: replicator
    sharding:
      login: storage

sharding:
  bucket_count: 1000

groups:
  storages:
    app:
      module: storage
    sharding:
      roles: [storage]
    replication:
      failover: manual
    replicasets:
      storage-a:
        leader: storage-a-001
        instances:
          storage-a-001:
            iproto:
              listen:
              - uri: 'localhost:3301'
          storage-a-002:
            iproto:
              listen:
              - uri: 'localhost:3302'
      storage-b:
        leader: storage-b-001
        instances:
          storage-b-001:
            iproto:
              listen:
              - uri: 'localhost:3303'
          storage-b-002:
            iproto:
              listen:
              - uri: 'localhost:3304'
  routers:
    app:
      module: router
    sharding:
      roles: [router]
    replicasets:
      router-a:
        instances:
          router-a-001:
            iproto:
              listen:
              - uri: 'localhost:3305'

router.lua:

local vshard = require('vshard')
local crud   = require('crud')

vshard.router.bootstrap()
crud.init_router()

storage.lua:

local crud = require('crud')
local vshard = require('vshard')

crud.init_storage()

instances.yml (is required to start with tt):

storage-a-001:
storage-a-002:
storage-b-001:
storage-b-002:
router-a-001:
$ tt start myapp
   • Starting an instance [myapp:storage-a-001]...
   • Starting an instance [myapp:storage-a-002]...
   • Starting an instance [myapp:storage-b-001]...
   • Starting an instance [myapp:storage-b-002]...
   • Starting an instance [myapp:router-a-001]...
$ tt status myapp
INSTANCE                STATUS          PID
myapp:storage-b-002     RUNNING         222904
myapp:router-a-001      RUNNING         222905
myapp:storage-a-001     NOT RUNNING     
myapp:storage-a-002     RUNNING         222902
myapp:storage-b-001     NOT RUNNING   

Check in logs, what happened with stopped instances:

2023-12-28 12:14:25.959 [222951] main utils.c:679 E> LuajitError: .../myapp/.rocks/share/tarantool/crud.lua:179: Failed to find a vshard configuration for  replicaset with replicaset_uuid f2469bba-0e1f-46b0-af0b-a3491f1e8519.

vshard config (which was applied by tarantool itself):

---
...
sharding:
  storage-a:
    master: auto
    replicas:
      storage-a-002:
        uuid: null
        uri:
          password: a
          uri: localhost:3302
          login: storage
      storage-a-001:
        uuid: null
        uri:
          password: a
          uri: localhost:3301
          login: storage
    uuid: null
  storage-b:
    master: auto
    replicas:
      storage-b-002:
        uuid: null
        uri:
          password: a
          uri: localhost:3304
          login: storage
      storage-b-001:
        uuid: null
        uri:
          password: a
          uri: localhost:3303
...

Here is how tarantool builds vshard config: https://github.com/tarantool/tarantool/blob/57c1f625ffb280aea950030962cff104b3f3a47d/src/box/lua/config/configdata.lua#L144-L231

@DifferentialOrange DifferentialOrange self-assigned this Dec 28, 2023
@DifferentialOrange DifferentialOrange added the bug Something isn't working label Dec 28, 2023
DifferentialOrange added a commit that referenced this issue Dec 28, 2023
PR #404 has introduced vshard 0.1.25 + Tarantool 3.0 "name as key"
identification mode based on UUIDs extraction. If works fine if
vshard configuration (or Tarantool 3.0 configuration which builds
vshard one) provides UUIDs, but fails if it isn't. Since UUIDs are
optional and won't be provided in most cases, it makes crud fails
to work on most Tarantool 3.0 vshard clusters. This patch fixes
the issue.

Now the code uses name as key, if corresponding mode is enabled, and
uuid otherwise. Patch doesn't cover `select_old` since it runs only
on pre-3.0 Tarantool. Unfortunately, code relies on vshard internals
since now there is no other way [1].

This patch covers new mode support for readview code as well. It likely
was broken before this patch even if UUIDs were provided.

1. tarantool/vshard#460

Follows #404
Closes #407
DifferentialOrange added a commit that referenced this issue Dec 28, 2023
PR #404 has introduced vshard 0.1.25 + Tarantool 3.0 "name as key"
identification mode based on UUIDs extraction. If works fine if
vshard configuration (or Tarantool 3.0 configuration which builds
vshard one) provides UUIDs, but fails if it isn't. Since UUIDs are
optional and won't be provided in most cases, it makes crud fails
to work on most Tarantool 3.0 vshard clusters. This patch fixes
the issue.

Now the code uses name as key, if corresponding mode is enabled, and
uuid otherwise. Patch doesn't cover `select_old` since it runs only
on pre-3.0 Tarantool. Unfortunately, code relies on vshard internals
since now there is no other way [1].

This patch covers new mode support for readview code as well. It likely
was broken before this patch even if UUIDs were provided.

1. tarantool/vshard#460

Follows #404
Closes #407
DifferentialOrange added a commit that referenced this issue Dec 29, 2023
PR #404 has introduced vshard 0.1.25 + Tarantool 3.0 "name as key"
identification mode based on UUIDs extraction. If works fine if
vshard configuration (or Tarantool 3.0 configuration which builds
vshard one) provides UUIDs, but fails if it isn't. Since UUIDs are
optional and won't be provided in most cases, it makes crud fails
to work on most Tarantool 3.0 vshard clusters. This patch fixes
the issue.

Now the code uses name as key, if corresponding mode is enabled, and
uuid otherwise. Patch doesn't cover `select_old` since it runs only
on pre-3.0 Tarantool. Unfortunately, code relies on vshard internals
since now there is no other way [1].

This patch covers new mode support for readview code as well. It likely
was broken before this patch even if UUIDs were provided.

1. tarantool/vshard#460

Follows #404
Closes #407
DifferentialOrange added a commit that referenced this issue Dec 29, 2023
PR #404 has introduced vshard 0.1.25 + Tarantool 3.0 "name as key"
identification mode based on UUIDs extraction. If works fine if
vshard configuration (or Tarantool 3.0 configuration which builds
vshard one) provides UUIDs, but fails if it isn't. Since UUIDs are
optional and won't be provided in most cases, it makes crud fails
to work on most Tarantool 3.0 vshard clusters. This patch fixes
the issue.

Now the code uses name as key, if corresponding mode is enabled, and
uuid otherwise. Patch doesn't cover `select_old` since it runs only
on pre-3.0 Tarantool. Unfortunately, code relies on vshard internals
since now there is no other way [1].

This patch covers new mode support for readview code as well. It likely
was broken before this patch even if UUIDs were provided.

1. tarantool/vshard#460

Follows #404
Part of #407
@DifferentialOrange DifferentialOrange changed the title can't initialize storage/router with cluster config Сan't initialize storage/router if no UUIDs Dec 29, 2023
DifferentialOrange added a commit that referenced this issue Dec 29, 2023
PR #404 has introduced vshard 0.1.25 + Tarantool 3.0 "name as key"
identification mode based on UUIDs extraction. If works fine if
vshard configuration (or Tarantool 3.0 configuration which builds
vshard one) provides UUIDs, but fails if it isn't. Since UUIDs are
optional and won't be provided in most cases, it makes crud fails
to work on most Tarantool 3.0 vshard clusters. This patch fixes
the issue.

Now the code uses name as key, if corresponding mode is enabled, and
uuid otherwise. Patch doesn't cover `select_old` since it runs only
on pre-3.0 Tarantool. Unfortunately, code relies on vshard internals
since now there is no other way [1].

This patch covers new mode support for readview code as well. It likely
was broken before this patch even if UUIDs were provided.

1. tarantool/vshard#460

Follows #404
Closes #407
DifferentialOrange added a commit that referenced this issue Jan 9, 2024
PR #404 has introduced vshard 0.1.25 + Tarantool 3.0 "name as key"
identification mode based on UUIDs extraction. If works fine if
vshard configuration (or Tarantool 3.0 configuration which builds
vshard one) provides UUIDs, but fails if it isn't. Since UUIDs are
optional and won't be provided in most cases, it makes crud fails
to work on most Tarantool 3.0 vshard clusters. This patch fixes
the issue.

Now the code uses name as key, if corresponding mode is enabled, and
uuid otherwise. Patch doesn't cover `select_old` since it runs only
on pre-3.0 Tarantool. Unfortunately, code relies on vshard internals
since now there is no other way [1].

This patch covers new mode support for readview code as well. It likely
was broken before this patch even if UUIDs were provided.

1. tarantool/vshard#460

Follows #404
Closes #407
DifferentialOrange added a commit that referenced this issue Jan 9, 2024
PR #404 has introduced vshard 0.1.25 + Tarantool 3.0 "name as key"
identification mode based on UUIDs extraction. If works fine if
vshard configuration (or Tarantool 3.0 configuration which builds
vshard one) provides UUIDs, but fails if it isn't. Since UUIDs are
optional and won't be provided in most cases, it makes crud fails
to work on most Tarantool 3.0 vshard clusters. This patch fixes
the issue.

Now the code uses name as key, if corresponding mode is enabled, and
uuid otherwise. Patch doesn't cover `select_old` since it runs only
on pre-3.0 Tarantool. Unfortunately, code relies on vshard internals
since now there is no other way [1].

This patch covers new mode support for readview code as well. It likely
was broken before this patch even if UUIDs were provided.

1. tarantool/vshard#460

Follows #404
Closes #407
DifferentialOrange added a commit that referenced this issue Jan 10, 2024
PR #404 has introduced vshard 0.1.25 + Tarantool 3.0 "name as key"
identification mode based on UUIDs extraction. If works fine if
vshard configuration (or Tarantool 3.0 configuration which builds
vshard one) provides UUIDs, but fails if it isn't. Since UUIDs are
optional and won't be provided in most cases, it makes crud fails
to work on most Tarantool 3.0 vshard clusters. This patch fixes
the issue.

Now the code uses name as key, if corresponding mode is enabled, and
uuid otherwise. Patch doesn't cover `select_old` since it runs only
on pre-3.0 Tarantool. Unfortunately, code relies on vshard internals
since now there is no other way [1].

This patch covers new mode support for readview code as well. It likely
was broken before this patch even if UUIDs were provided.

1. tarantool/vshard#460

Follows #404
Closes #407
DifferentialOrange added a commit that referenced this issue Feb 5, 2024
Overview

  This release introduces compatibility with several
  Tarantool 3 + vshard 0.1.25 features, as well as critical scan fix.

Fixed
  * Compatibility with vshard configuration if UUIDs are omitted (#407).
  * Compatibility with automatic master discovery in vshard (#409).
  * Secondary conditions for index operands with operations `>=`, `<=`,
    `>`, `<` no longer cause missing part of the actual result for scan
    operations (`crud.select`, `crud.pairs`, `crud.count`,
    `readview:select`, `readview:pairs`) (#418).
DifferentialOrange added a commit that referenced this issue Feb 5, 2024
Overview

  This release introduces compatibility with several
  Tarantool 3 + vshard 0.1.25 features, as well as critical scan fix.

Fixed
  * Compatibility with vshard configuration if UUIDs are omitted (#407).
  * Compatibility with automatic master discovery in vshard (#409).
  * Secondary conditions for index operands with operations `>=`, `<=`,
    `>`, `<` no longer cause missing part of the actual result for scan
    operations (`crud.select`, `crud.pairs`, `crud.count`,
    `readview:select`, `readview:pairs`) (#418).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2sp bug Something isn't working teamE
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants