rptest: Basic transaction produce test

[savex]

Basic produce tests and some fixes for consume basic tests.

Backports Required

• none - not a bug fix
• none - this is a backport
• none - issue does not exist in previous branches
• none - papercut/not impactful enough to backport
• v23.3.x
• v23.2.x
• v23.1.x

Release Notes

• none

[savex]

This will be updated once venv will be prepared to use inside task executor. Main reason is that once graph generates jobs that run python code from java - it needs pyflink packages accessible outside flink service. And we do not want to install them at system level on ducktape worker node.

[vbotbuildovich]

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/43456#018cd742-40a5-4c32-af83-b9c1331b2f91

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/43511#018cdbfa-461c-40fb-b2f1-13c0b154771a

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/43577#018cec4c-4b28-42a5-8f81-c6ce1d680d0a

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/43624#018cf145-4776-44ed-8be2-d32f6978bca6

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/43624#018cf156-ed0f-4f46-89f7-7a2dae00e931

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/43624#018cf156-ed09-4d7f-8676-47c3bb21a2fd

[vbotbuildovich]

new failures in https://buildkite.com/redpanda/redpanda/builds/43456#018cd742-40ab-403e-ad86-53b6ac575fea:

"rptest.tests.cloud_storage_timing_stress_test.CloudStorageTimingStressTest.test_cloud_storage.cleanup_policy=delete"

new failures in https://buildkite.com/redpanda/redpanda/builds/43577#018cec24-3ec7-4523-b0e2-0f598c011092:

"rptest.tests.flink_basic_test.FlinkBasicTests.test_transaction_workload"

new failures in https://buildkite.com/redpanda/redpanda/builds/43624#018cf145-476c-4f56-97a6-61b2745a441f:

"rptest.tests.cluster_features_test.FeaturesMultiNodeUpgradeTest.test_upgrade"

new failures in https://buildkite.com/redpanda/redpanda/builds/43624#018cf145-4776-44ed-8be2-d32f6978bca6:

"rptest.tests.cluster_features_test.FeaturesMultiNodeUpgradeTest.test_rollback"

new failures in https://buildkite.com/redpanda/redpanda/builds/43624#018cf145-476f-4dcc-ba4d-8be5475b070b:

"rptest.tests.cluster_features_test.FeaturesSingleNodeUpgradeTest.test_upgrade"
"rptest.tests.cluster_features_test.FeaturesNodeJoinTest.test_old_node_join"

new failures in https://buildkite.com/redpanda/redpanda/builds/43624#018cf156-ed06-459f-bc34-de63574722eb:

"rptest.tests.cluster_features_test.FeaturesMultiNodeUpgradeTest.test_rollback"

new failures in https://buildkite.com/redpanda/redpanda/builds/43624#018cf156-ed09-4d7f-8676-47c3bb21a2fd:

"rptest.tests.cluster_features_test.FeaturesMultiNodeUpgradeTest.test_upgrade"

new failures in https://buildkite.com/redpanda/redpanda/builds/43624#018cf156-ed0c-4b66-9808-3e12469a9a20:

"rptest.tests.cluster_features_test.FeaturesSingleNodeUpgradeTest.test_upgrade"
"rptest.tests.cluster_features_test.FeaturesNodeJoinTest.test_old_node_join"

[savex]

As of now, consumer sets latest offset to +2 comparing to last offset produced. Investigation pending on why this happens.

[savex]

Updated test logic so it would not wait for Consumer to finish but check for data produced instead. Also, instead of offset, index is used in validation of data.