ref(migrations): add revert command by MeredithAnya · Pull Request #6966 · getsentry/snuba

MeredithAnya · 2025-03-12T20:43:57Z

Addresses #6964

New Usage

Revert all migrations

snuba migrations revert --force

Revert all migrations, including the MigrationGroup.SYSTEM (aka migrations_local/dist table(s))

snuba migrations revert --include-system --force

Revert all migrations for a group

snuba migrations revert --group profile_chunks --force

Fake reverting all migrations for a group

snuba migrations revert --group profile_chunks --fake --force

Notes

The --include-system is only relevant for when one wants to revert all migrations, it doesn't do anything when you revert with a specific group
These new commands shouldn't be added to snuba admin as they are not something we really want to do in production, but more to add flexibility when setting up clusters/regions for the first time

sentry · 2025-03-12T20:44:08Z

🔍 Existing Issues For Review

Your pull request is modifying functions with the following pre-existing issues:

📄 File: snuba/cli/migrations.py

Function	Unhandled Issue
`migrate`	InvalidMigrationState: Missing migrations: 0033_items_attribute_table_v1 ... `Event Count:` 2

_{Did you find this useful? React with a 👍 or 👎}

codecov · 2025-03-12T21:07:40Z

❌ 1 Tests Failed:

Tests completed	Failed	Passed	Skipped
1283	1	1282	10

View the top 1 failed test(s) by shortest run time

tests.migrations.test_runner::test_reverse_idempotency_all

Stack Traces | 18.8s run time

Traceback (most recent call last):
  File ".../snuba/clickhouse/native.py", line 206, in execute
    result_data = query_execute()
                  ^^^^^^^^^^^^^^^
  File ".../snuba/clickhouse/native.py", line 189, in query_execute
    return conn.execute(  # type: ignore
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../local/lib/python3.11.............../site-packages/clickhouse_driver/client.py", line 382, in execute
    rv = self.process_ordinary_query(
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../local/lib/python3.11.............../site-packages/clickhouse_driver/client.py", line 580, in process_ordinary_query
    return self.receive_result(with_column_types=with_column_types,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../local/lib/python3.11.../sentry_sdk/integrations/clickhouse_driver.py", line 112, in _inner_end
    res = f(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^
  File ".../local/lib/python3.11.............../site-packages/clickhouse_driver/client.py", line 212, in receive_result
    return result.get_result()
           ^^^^^^^^^^^^^^^^^^^
  File ".../local/lib/python3.11.../site-packages/clickhouse_driver/result.py", line 50, in get_result
    for packet in self.packet_generator:
  File ".../local/lib/python3.11.............../site-packages/clickhouse_driver/client.py", line 228, in packet_generator
    packet = self.receive_packet()
             ^^^^^^^^^^^^^^^^^^^^^
  File ".../local/lib/python3.11.............../site-packages/clickhouse_driver/client.py", line 245, in receive_packet
    raise packet.exception
clickhouse_driver.errors.ServerException: Code: 122.
DB::Exception: Table columns structure in ZooKeeper is different from local table structure. Local columns:
columns format version: 1
37 columns:
`organization_id` UInt64
`project_id` UInt64
`group_id` UInt64
`search_title` String
`primary_hash` UUID
`fingerprint` Array(String)
`occurrence_id` UUID
`occurrence_type_id` UInt16
`detection_timestamp` DateTime
`event_id` Nullable(UUID)
`trace_id` Nullable(UUID)
`platform` LowCardinality(String)
`environment` LowCardinality(Nullable(String))
`release` LowCardinality(Nullable(String))
`dist` LowCardinality(Nullable(String))
`receive_timestamp` DateTime
`client_timestamp` DateTime
`tags.key` Array(String)
`tags.value` Array(String)
`user` Nullable(String)
`user_hash` Nullable(UInt64)	MATERIALIZED	cityHash64(user)
`user_id` Nullable(String)
`user_name` Nullable(String)
`user_email` Nullable(String)
`ip_address_v4` Nullable(IPv4)
`ip_address_v6` Nullable(IPv6)
`sdk_name` LowCardinality(Nullable(String))
`sdk_version` LowCardinality(Nullable(String))
`contexts.key` Array(String)
`contexts.value` Array(String)
`http_method` LowCardinality(Nullable(String))
`http_referer` Nullable(String)
`deleted` UInt8
`message_timestamp` DateTime
`partition` UInt16
`offset` UInt64
`retention_days` UInt16

Zookeeper columns:
columns format version: 1
38 columns:
`organization_id` UInt64
`project_id` UInt64
`group_id` UInt64
`search_title` String
`primary_hash` UUID
`fingerprint` Array(String)
`occurrence_id` UUID
`occurrence_type_id` UInt16
`detection_timestamp` DateTime
`event_id` Nullable(UUID)
`trace_id` Nullable(UUID)
`platform` LowCardinality(String)
`environment` LowCardinality(Nullable(String))
`release` LowCardinality(Nullable(String))
`dist` LowCardinality(Nullable(String))
`receive_timestamp` DateTime
`client_timestamp` DateTime
`tags.key` Array(String)
`tags.value` Array(String)
`_tags_hash_map` Array(UInt64)	MATERIALIZED	arrayMap((k, v) -> cityHash64(concat(replaceRegexpAll(k, \'(\\\\=|\\\\\\\\)\', \'\\\\\\\\\\\\1\'), \'=\', v)), tags.key, tags.value)
`user` Nullable(String)
`user_hash` Nullable(UInt64)	MATERIALIZED	cityHash64(user)
`user_id` Nullable(String)
`user_name` Nullable(String)
`user_email` Nullable(String)
`ip_address_v4` Nullable(IPv4)
`ip_address_v6` Nullable(IPv6)
`sdk_name` LowCardinality(Nullable(String))
`sdk_version` LowCardinality(Nullable(String))
`contexts.key` Array(String)
`contexts.value` Array(String)
`http_method` LowCardinality(Nullable(String))
`http_referer` Nullable(String)
`deleted` UInt8
`message_timestamp` DateTime
`partition` UInt16
`offset` UInt64
`retention_days` UInt16
. Stack trace:

0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c61ff37 in ....................................................................................................../usr/bin/clickhouse
1. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007132547 in ....................................................................................................../usr/bin/clickhouse
2. DB::StorageReplicatedMergeTree::checkTableStructure(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, bool) @ 0x0000000012748488 in ....................................................................................................../usr/bin/clickhouse
3. DB::StorageReplicatedMergeTree::StorageReplicatedMergeTree(String const&, String const&, bool, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, std::shared_ptr<DB::Context>, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>, bool, DB::RenamingRestrictions, bool) @ 0x000000001273b3b9 in ....................................................................................................../usr/bin/clickhouse
4. DB::create(DB::StorageFactory::Arguments const&) @ 0x0000000012f37318 in ....................................................................................................../usr/bin/clickhouse
5. DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr<DB::Context>, std::shared_ptr<DB::Context>, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, bool) const @ 0x0000000012663e1b in ....................................................................................................../usr/bin/clickhouse
6. DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr<DB::DDLGuard, std::default_delete<DB::DDLGuard>>&) @ 0x0000000011cf707e in ....................................................................................................../usr/bin/clickhouse
7. DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x0000000011cf04bd in ....................................................................................................../usr/bin/clickhouse
8. DB::InterpreterCreateQuery::execute() @ 0x0000000011cfd920 in ....................................................................................................../usr/bin/clickhouse
9. DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000122bfe15 in ....................................................................................................../usr/bin/clickhouse
10. DB::executeQuery(String const&, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) @ 0x00000000122bb5b5 in ....................................................................................................../usr/bin/clickhouse
11. DB::TCPHandler::runImpl() @ 0x0000000013137519 in ....................................................................................................../usr/bin/clickhouse
12. DB::TCPHandler::run() @ 0x00000000131498f9 in ....................................................................................................../usr/bin/clickhouse
13. Poco::Net::TCPServerConnection::start() @ 0x0000000015b42834 in ....................................................................................................../usr/bin/clickhouse
14. Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b43a31 in ....................................................................................................../usr/bin/clickhouse
15. Poco::PooledThread::run() @ 0x0000000015c7a667 in ....................................................................................................../usr/bin/clickhouse
16. Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c7893c in ....................................................................................................../usr/bin/clickhouse
17. ? @ 0x00007ff38d264609 in ?
18. ? @ 0x00007ff38d189353 in ?


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File ".../tests/migrations/test_runner.py", line 364, in test_reverse_idempotency_all
    runner.run_all(force=True)
  File ".../snuba/migrations/runner.py", line 255, in run_all
    self._run_migration_impl(
  File ".../snuba/migrations/runner.py", line 338, in _run_migration_impl
    migration.forwards(context, dry_run, columns_states)
  File ".../snuba/migrations/migration.py", line 170, in forwards
    op.execute()
  File ".../snuba/migrations/operations.py", line 81, in execute
    connection.execute(self.format_sql(), settings=self._settings)
  File ".../snuba/clickhouse/native.py", line 291, in execute
    raise ClickhouseError(e.message, code=e.code) from e
snuba.clickhouse.errors.ClickhouseError: DB::Exception: Table columns structure in ZooKeeper is different from local table structure. Local columns:
columns format version: 1
37 columns:
`organization_id` UInt64
`project_id` UInt64
`group_id` UInt64
`search_title` String
`primary_hash` UUID
`fingerprint` Array(String)
`occurrence_id` UUID
`occurrence_type_id` UInt16
`detection_timestamp` DateTime
`event_id` Nullable(UUID)
`trace_id` Nullable(UUID)
`platform` LowCardinality(String)
`environment` LowCardinality(Nullable(String))
`release` LowCardinality(Nullable(String))
`dist` LowCardinality(Nullable(String))
`receive_timestamp` DateTime
`client_timestamp` DateTime
`tags.key` Array(String)
`tags.value` Array(String)
`user` Nullable(String)
`user_hash` Nullable(UInt64)	MATERIALIZED	cityHash64(user)
`user_id` Nullable(String)
`user_name` Nullable(String)
`user_email` Nullable(String)
`ip_address_v4` Nullable(IPv4)
`ip_address_v6` Nullable(IPv6)
`sdk_name` LowCardinality(Nullable(String))
`sdk_version` LowCardinality(Nullable(String))
`contexts.key` Array(String)
`contexts.value` Array(String)
`http_method` LowCardinality(Nullable(String))
`http_referer` Nullable(String)
`deleted` UInt8
`message_timestamp` DateTime
`partition` UInt16
`offset` UInt64
`retention_days` UInt16

Zookeeper columns:
columns format version: 1
38 columns:
`organization_id` UInt64
`project_id` UInt64
`group_id` UInt64
`search_title` String
`primary_hash` UUID
`fingerprint` Array(String)
`occurrence_id` UUID
`occurrence_type_id` UInt16
`detection_timestamp` DateTime
`event_id` Nullable(UUID)
`trace_id` Nullable(UUID)
`platform` LowCardinality(String)
`environment` LowCardinality(Nullable(String))
`release` LowCardinality(Nullable(String))
`dist` LowCardinality(Nullable(String))
`receive_timestamp` DateTime
`client_timestamp` DateTime
`tags.key` Array(String)
`tags.value` Array(String)
`_tags_hash_map` Array(UInt64)	MATERIALIZED	arrayMap((k, v) -> cityHash64(concat(replaceRegexpAll(k, \'(\\\\=|\\\\\\\\)\', \'\\\\\\\\\\\\1\'), \'=\', v)), tags.key, tags.value)
`user` Nullable(String)
`user_hash` Nullable(UInt64)	MATERIALIZED	cityHash64(user)
`user_id` Nullable(String)
`user_name` Nullable(String)
`user_email` Nullable(String)
`ip_address_v4` Nullable(IPv4)
`ip_address_v6` Nullable(IPv6)
`sdk_name` LowCardinality(Nullable(String))
`sdk_version` LowCardinality(Nullable(String))
`contexts.key` Array(String)
`contexts.value` Array(String)
`http_method` LowCardinality(Nullable(String))
`http_referer` Nullable(String)
`deleted` UInt8
`message_timestamp` DateTime
`partition` UInt16
`offset` UInt64
`retention_days` UInt16
. Stack trace:

0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c61ff37 in ....................................................................................................../usr/bin/clickhouse
1. DB::Exception::Exception<String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&) @ 0x0000000007132547 in ....................................................................................................../usr/bin/clickhouse
2. DB::StorageReplicatedMergeTree::checkTableStructure(String const&, std::shared_ptr<DB::StorageInMemoryMetadata const> const&, bool) @ 0x0000000012748488 in ....................................................................................................../usr/bin/clickhouse
3. DB::StorageReplicatedMergeTree::StorageReplicatedMergeTree(String const&, String const&, bool, DB::StorageID const&, String const&, DB::StorageInMemoryMetadata const&, std::shared_ptr<DB::Context>, String const&, DB::MergeTreeData::MergingParams const&, std::unique_ptr<DB::MergeTreeSettings, std::default_delete<DB::MergeTreeSettings>>, bool, DB::RenamingRestrictions, bool) @ 0x000000001273b3b9 in ....................................................................................................../usr/bin/clickhouse
4. DB::create(DB::StorageFactory::Arguments const&) @ 0x0000000012f37318 in ....................................................................................................../usr/bin/clickhouse
5. DB::StorageFactory::get(DB::ASTCreateQuery const&, String const&, std::shared_ptr<DB::Context>, std::shared_ptr<DB::Context>, DB::ColumnsDescription const&, DB::ConstraintsDescription const&, bool) const @ 0x0000000012663e1b in ....................................................................................................../usr/bin/clickhouse
6. DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::unique_ptr<DB::DDLGuard, std::default_delete<DB::DDLGuard>>&) @ 0x0000000011cf707e in ....................................................................................................../usr/bin/clickhouse
7. DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&) @ 0x0000000011cf04bd in ....................................................................................................../usr/bin/clickhouse
8. DB::InterpreterCreateQuery::execute() @ 0x0000000011cfd920 in ....................................................................................................../usr/bin/clickhouse
9. DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000122bfe15 in ....................................................................................................../usr/bin/clickhouse
10. DB::executeQuery(String const&, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) @ 0x00000000122bb5b5 in ....................................................................................................../usr/bin/clickhouse
11. DB::TCPHandler::runImpl() @ 0x0000000013137519 in ....................................................................................................../usr/bin/clickhouse
12. DB::TCPHandler::run() @ 0x00000000131498f9 in ....................................................................................................../usr/bin/clickhouse
13. Poco::Net::TCPServerConnection::start() @ 0x0000000015b42834 in ....................................................................................................../usr/bin/clickhouse
14. Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b43a31 in ....................................................................................................../usr/bin/clickhouse
15. Poco::PooledThread::run() @ 0x0000000015c7a667 in ....................................................................................................../usr/bin/clickhouse
16. Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c7893c in ....................................................................................................../usr/bin/clickhouse
17. ? @ 0x00007ff38d264609 in ?
18. ? @ 0x00007ff38d189353 in ?

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

onkar

LGTM

MeredithAnya force-pushed the meredith/SNUBA-6964 branch from 531431a to 7736cdc Compare March 17, 2025 19:08

MeredithAnya marked this pull request as ready for review March 17, 2025 19:16

MeredithAnya requested a review from a team as a code owner March 17, 2025 19:16

MeredithAnya requested a review from mwarkentin March 17, 2025 19:16

onkar reviewed Mar 18, 2025

View reviewed changes

MeredithAnya force-pushed the meredith/SNUBA-6964 branch from 7736cdc to 8000606 Compare March 24, 2025 20:38

onkar approved these changes Mar 25, 2025

View reviewed changes

MeredithAnya added 6 commits March 25, 2025 10:25

ref(migrations): add revert command

f448484

some tests

0f05099

fix typing and test

1109e1d

tuple default, simplfy through check

d91dbc0

smol fixes

01eb443

fix decorator

e4c85dd

MeredithAnya force-pushed the meredith/SNUBA-6964 branch from f17da5a to e4c85dd Compare March 25, 2025 17:25

MeredithAnya enabled auto-merge (squash) March 25, 2025 17:25

MeredithAnya merged commit e7bc512 into master Mar 25, 2025

MeredithAnya deleted the meredith/SNUBA-6964 branch March 25, 2025 17:54

MeredithAnya mentioned this pull request Mar 25, 2025

Support multiple reverse migrations #6964

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

ref(migrations): add revert command#6966

ref(migrations): add revert command#6966
MeredithAnya merged 6 commits into
masterfrom
meredith/SNUBA-6964

MeredithAnya commented Mar 12, 2025 •

edited

Loading

Uh oh!

sentry Bot commented Mar 12, 2025

Uh oh!

codecov Bot commented Mar 12, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

onkar left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

MeredithAnya commented Mar 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sentry Bot commented Mar 12, 2025

🔍 Existing Issues For Review

Uh oh!

codecov Bot commented Mar 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

❌ 1 Tests Failed:

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

onkar left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

MeredithAnya commented Mar 12, 2025 •

edited

Loading

codecov Bot commented Mar 12, 2025 •

edited

Loading