Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CI Failure (ErrorCommiting transaction) in TxUpgradeTest.upgrade_does_not_change_tx_coordinator_assignment_test #18184

Open
vbotbuildovich opened this issue Apr 30, 2024 · 10 comments · Fixed by #20743
Open

CI Failure (ErrorCommiting transaction) in TxUpgradeTest.upgrade_does_not_change_tx_coordinator_assignment_test #18184

vbotbuildovich opened this issue Apr 30, 2024 · 10 comments · Fixed by #20743
Assignees
@mmaslankaprv
Labels
area/transactions auto-triaged used to know which issues have been opened from a CI job ci-failure ci-rca/test CI Root Cause Analysis - Test Issue

Comments

@vbotbuildovich
Copy link
Collaborator

vbotbuildovich commented Apr 30, 2024 •
edited by jira bot
Loading

https://buildkite.com/redpanda/redpanda/builds/48458
https://buildkite.com/redpanda/redpanda/builds/48467
https://buildkite.com/redpanda/redpanda/builds/48475
https://buildkite.com/redpanda/vtools/builds/13348
https://buildkite.com/redpanda/redpanda/builds/48482
https://buildkite.com/redpanda/vtools/builds/13346

Module: rptest.tests.transactions_test
Class: TxUpgradeTest
Method: upgrade_does_not_change_tx_coordinator_assignment_test

test_id:    TxUpgradeTest.upgrade_does_not_change_tx_coordinator_assignment_test
status:     FAIL
run time:   152.840 seconds

KafkaException(KafkaError{FATAL,code=INVALID_TXN_STATE,val=48,str="Failed to add partitions to transaction: Broker: Producer attempted a transactional operation in an invalid state"})
Traceback (most recent call last):
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/ducktape/tests/runner_client.py", line 184, in _do_run
    data = self.run_test()
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/ducktape/tests/runner_client.py", line 276, in run_test
    return self.test_context.function(self.test)
  File "/root/tests/rptest/services/cluster.py", line 103, in wrapped
    r = f(self, *args, **kwargs)
  File "/root/tests/rptest/tests/transactions_test.py", line 1346, in upgrade_does_not_change_tx_coordinator_assignment_test
    self._populate_tx_coordinator(topic.name)
  File "/root/tests/rptest/tests/transactions_test.py", line 1310, in _populate_tx_coordinator
    producer.commit_transaction()
cimpl.KafkaException: KafkaError{FATAL,code=INVALID_TXN_STATE,val=48,str="Failed to add partitions to transaction: Broker: Producer attempted a transactional operation in an invalid state"}

JIRA Link: CORE-2733
@vbotbuildovich vbotbuildovich added auto-triaged used to know which issues have been opened from a CI job ci-failure labels Apr 30, 2024
@vbotbuildovich
Copy link
Collaborator Author

*https://buildkite.com/redpanda/vtools/builds/13383
*https://buildkite.com/redpanda/vtools/builds/13384

@vbotbuildovich
Copy link
Collaborator Author

*https://buildkite.com/redpanda/redpanda/builds/48602
*https://buildkite.com/redpanda/redpanda/builds/48605
*https://buildkite.com/redpanda/vtools/builds/13406

@piyushredpanda
Copy link
Contributor

Presumed to be fixed by #18187

@piyushredpanda piyushredpanda added the ci-rca/redpanda CI Root Cause Analysis - Redpanda Issue label May 8, 2024
@vbotbuildovich
Copy link
Collaborator Author

*https://buildkite.com/redpanda/vtools/builds/14734
*https://buildkite.com/redpanda/redpanda/builds/50321
*https://buildkite.com/redpanda/redpanda/builds/50323
*https://buildkite.com/redpanda/vtools/builds/14763
*https://buildkite.com/redpanda/vtools/builds/14768
*https://buildkite.com/redpanda/vtools/builds/14777
*https://buildkite.com/redpanda/redpanda/builds/50380
*https://buildkite.com/redpanda/redpanda/builds/50396
*https://buildkite.com/redpanda/vtools/builds/14864
*https://buildkite.com/redpanda/vtools/builds/14865

@vbotbuildovich
Copy link
Collaborator Author

*https://buildkite.com/redpanda/redpanda/builds/50477
*https://buildkite.com/redpanda/redpanda/builds/50497
*https://buildkite.com/redpanda/redpanda/builds/50504
*https://buildkite.com/redpanda/vtools/builds/14929
*https://buildkite.com/redpanda/redpanda/builds/50547
*https://buildkite.com/redpanda/vtools/builds/14952
*https://buildkite.com/redpanda/vtools/builds/14966
*https://buildkite.com/redpanda/vtools/builds/14975
*https://buildkite.com/redpanda/vtools/builds/14967
*https://buildkite.com/redpanda/redpanda/builds/50565
*https://buildkite.com/redpanda/vtools/builds/14982
*https://buildkite.com/redpanda/vtools/builds/14990
*https://buildkite.com/redpanda/vtools/builds/14998
*https://buildkite.com/redpanda/redpanda/builds/50633
*https://buildkite.com/redpanda/redpanda/builds/50638
*https://buildkite.com/redpanda/redpanda/builds/50634
*https://buildkite.com/redpanda/redpanda/builds/50676
*https://buildkite.com/redpanda/redpanda/builds/50693
*https://buildkite.com/redpanda/redpanda/builds/50719
*https://buildkite.com/redpanda/redpanda/builds/50721
*https://buildkite.com/redpanda/vtools/builds/15058
*https://buildkite.com/redpanda/vtools/builds/15057
*https://buildkite.com/redpanda/redpanda/builds/50783
*https://buildkite.com/redpanda/redpanda/builds/50786

@vbotbuildovich
Copy link
Collaborator Author

*https://buildkite.com/redpanda/vtools/builds/15093
*https://buildkite.com/redpanda/vtools/builds/15094
*https://buildkite.com/redpanda/redpanda/builds/50818
*https://buildkite.com/redpanda/redpanda/builds/50828

@mmaslankaprv mmaslankaprv self-assigned this Jun 28, 2024
@mmaslankaprv mmaslankaprv changed the title CI Failure (key symptom) in TxUpgradeTest.upgrade_does_not_change_tx_coordinator_assignment_test CI Failure (ErrorCommiting transaction) in TxUpgradeTest.upgrade_does_not_change_tx_coordinator_assignment_test Jun 28, 2024
@mmaslankaprv mmaslankaprv added ci-rca/test CI Root Cause Analysis - Test Issue and removed ci-rca/redpanda CI Root Cause Analysis - Redpanda Issue labels Jun 28, 2024
mmaslankaprv added a commit to mmaslankaprv/redpanda that referenced this issue Jun 28, 2024
@mmaslankaprv
tests: do not set transaction timeout in tests that do not need it
797ac33 
Most of our transactions tests were build based with the same code which
often were copy pasted. The test were setting the default transaction
timeout to 10 seconds as they might have been based on the test that
validated the transactions expiration. Updated the tests to make sure
that the timeout is set in the test that need it.

The timeout made some tests to fail intermittently.

Fixes: redpanda-data#18184

Signed-off-by: Michał Maślanka <michal@redpanda.com>
@mmaslankaprv mmaslankaprv mentioned this issue Jun 28, 2024
Merged
7 tasks
vbotbuildovich pushed a commit to vbotbuildovich/redpanda that referenced this issue Jun 28, 2024
@mmaslankaprv @vbotbuildovich
tests: do not set transaction timeout in tests that do not need it
df7b41c 
Most of our transactions tests were build based with the same code which
often were copy pasted. The test were setting the default transaction
timeout to 10 seconds as they might have been based on the test that
validated the transactions expiration. Updated the tests to make sure
that the timeout is set in the test that need it.

The timeout made some tests to fail intermittently.

Fixes: redpanda-data#18184

Signed-off-by: Michał Maślanka <michal@redpanda.com>
(cherry picked from commit 797ac33)
vbotbuildovich pushed a commit to vbotbuildovich/redpanda that referenced this issue Jun 28, 2024
@mmaslankaprv @vbotbuildovich
tests: do not set transaction timeout in tests that do not need it
aa2f0eb 
Most of our transactions tests were build based with the same code which
often were copy pasted. The test were setting the default transaction
timeout to 10 seconds as they might have been based on the test that
validated the transactions expiration. Updated the tests to make sure
that the timeout is set in the test that need it.

The timeout made some tests to fail intermittently.

Fixes: redpanda-data#18184

Signed-off-by: Michał Maślanka <michal@redpanda.com>
(cherry picked from commit 797ac33)
This was referenced Jun 28, 2024
Open
Open
@vbotbuildovich
Copy link
Collaborator Author

*https://buildkite.com/redpanda/redpanda/builds/50925
*https://buildkite.com/redpanda/redpanda/builds/50932
*https://buildkite.com/redpanda/vtools/builds/15136
*https://buildkite.com/redpanda/vtools/builds/15151
*https://buildkite.com/redpanda/redpanda/builds/50934
*https://buildkite.com/redpanda/vtools/builds/15163
*https://buildkite.com/redpanda/vtools/builds/15177
*https://buildkite.com/redpanda/redpanda/builds/50988
*https://buildkite.com/redpanda/vtools/builds/15196
*https://buildkite.com/redpanda/vtools/builds/15195
*https://buildkite.com/redpanda/redpanda/builds/50989
*https://buildkite.com/redpanda/vtools/builds/15217
*https://buildkite.com/redpanda/vtools/builds/15239
*https://buildkite.com/redpanda/redpanda/builds/51095

@vbotbuildovich vbotbuildovich reopened this Jul 4, 2024
@vbotbuildovich
Copy link
Collaborator Author

*https://buildkite.com/redpanda/redpanda/builds/51131
*https://buildkite.com/redpanda/redpanda/builds/51133
*https://buildkite.com/redpanda/redpanda/builds/51133

@vbotbuildovich
Copy link
Collaborator Author

*https://buildkite.com/redpanda/redpanda/builds/51170

@vbotbuildovich
Copy link
Collaborator Author

*https://buildkite.com/redpanda/redpanda/builds/51180
*https://buildkite.com/redpanda/redpanda/builds/51182
*https://buildkite.com/redpanda/vtools/builds/15368

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mmaslankaprv mmaslankaprv

Labels
area/transactions auto-triaged used to know which issues have been opened from a CI job ci-failure ci-rca/test CI Root Cause Analysis - Test Issue
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Do not set transaction timeout in tests that do not need it mmaslankaprv/redpanda
4 participants
@dotnwat @mmaslankaprv @vbotbuildovich @piyushredpanda