Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[8.8](backport #35780) Upgrade to Go 1.19.10 and handle changes in golang Debian base image #35841

Merged
merged 3 commits into from
Jun 20, 2023

Conversation

mergify[bot]
Copy link
Contributor

@mergify mergify bot commented Jun 20, 2023

This is an automatic backport of pull request #35780 done by Mergify.
Cherry-pick of 62374dd has failed:

On branch mergify/bp/8.8/pr-35780
Your branch is up to date with 'origin/8.8'.

You are currently cherry-picking commit 62374dd251.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   .go-version
	modified:   .golangci.yml
	modified:   CHANGELOG.next.asciidoc
	modified:   auditbeat/Dockerfile
	modified:   dev-tools/mage/pytest.go
	modified:   dev-tools/mage/target/integtest/integtest.go
	modified:   heartbeat/Dockerfile
	modified:   libbeat/tests/system/requirements.txt
	modified:   libbeat/tests/system/requirements_aix.txt
	modified:   metricbeat/Dockerfile
	modified:   metricbeat/magefile.go
	modified:   metricbeat/module/aerospike/_meta/Dockerfile
	modified:   metricbeat/module/haproxy/_meta/Dockerfile
	modified:   metricbeat/module/kafka/_meta/Dockerfile
	modified:   metricbeat/module/mongodb/_meta/Dockerfile
	modified:   metricbeat/module/munin/_meta/Dockerfile
	modified:   metricbeat/module/rabbitmq/_meta/Dockerfile
	modified:   packetbeat/Dockerfile
	modified:   testing/environments/docker/kafka/Dockerfile
	modified:   x-pack/functionbeat/Dockerfile
	modified:   x-pack/metricbeat/magefile.go

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   libbeat/docs/version.asciidoc

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/github/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally


Mergify commands and options

More conditions and actions can be found in the documentation.

You can also trigger Mergify actions by commenting on this pull request:

  • @Mergifyio refresh will re-evaluate the rules
  • @Mergifyio rebase will rebase this PR on its base branch
  • @Mergifyio update will merge the base branch into this PR
  • @Mergifyio backport <destination> will backport this PR on <destination> branch

Additionally, on Mergify dashboard you can:

  • look at your merge queues
  • generate the Mergify configuration with the config editor.

Finally, you can contact us on https://mergify.com

…35780)

* Explicitly install netcat-openbsd everywhere.

The plain netcat package appears to have been removed in the latest
golang:1.19.10 base image.

* Setup a virtualenv in Docker containers.

This fixes the PEP668 error about "externally managed environments"
which prevents modifying the system python installation.

* Revert "Revert "[Automation] Bump Golang version to 1.19.10 (#35751)" (#35784)"

This reverts commit 27b1799.

* Reword CHANGELOG.next.asciidoc

Co-authored-by: subham sarkar <sarkar.subhams2@gmail.com>

* Improve PEP668 comments.

* Install latest PyYAML in test containers.

* Add check to see if already in a venv.

* Upgrade pytest and py.

Remove dependency on pytest-otel as it prevented upgrading pytest.

* Check for pre-set virtualenv earlier.

* Always create a fresh virtualenv in Docker.

Remove duplicate dependencies from the docker files.

* Fix metricbeat integration tests.

* Force install Python dependencies for Metricbeat.

---------

Co-authored-by: subham sarkar <sarkar.subhams2@gmail.com>
(cherry picked from commit 62374dd)

# Conflicts:
#	libbeat/docs/version.asciidoc
@mergify mergify bot requested review from a team as code owners June 20, 2023 19:30
@mergify mergify bot added the backport label Jun 20, 2023
@mergify mergify bot requested a review from a team as a code owner June 20, 2023 19:30
@mergify mergify bot added the conflicts There is a conflict in the backported pull request label Jun 20, 2023
@mergify mergify bot requested a review from a team as a code owner June 20, 2023 19:30
@mergify mergify bot requested review from ycombinator and rdner and removed request for a team June 20, 2023 19:30
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jun 20, 2023
@botelastic
Copy link

botelastic bot commented Jun 20, 2023

This pull request doesn't have a Team:<team> label.

@elasticmachine
Copy link
Collaborator

elasticmachine commented Jun 20, 2023

💔 Tests Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2023-06-20T19:33:09.847+0000

  • Duration: 74 min 21 sec

Test stats 🧪

Test Results
Failed 4
Passed 26408
Skipped 1978
Total 28390

Test errors 4

Expand to view the tests failures

Build&Test / x-pack/metricbeat-pythonIntegTest / test_queue_metrics_collected – x-pack.metricbeat.module.activemq.test_activemq.ActiveMqTest_0
    Expand to view the error details

     Failed: Timeout >90.0s 
    

    Expand to view the stacktrace

     self = <test_activemq.ActiveMqTest_0 testMethod=test_queue_metrics_collected>
    
        @unittest.skipUnless(metricbeat.INTEGRATION_TESTS, 'integration test')
        def test_queue_metrics_collected(self):
    >       self.verify_destination_metrics_collection('queue')
    
    module/activemq/test_activemq.py:96: 
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
    module/activemq/test_activemq.py:70: in verify_destination_metrics_collection
        conn.disconnect()
    /opt/venv/lib/python3.11/site-packages/stomp/connect.py:185: in disconnect
        self.transport.stop()
    /opt/venv/lib/python3.11/site-packages/stomp/transport.py:122: in stop
        self.__receiver_thread_exit_condition.wait()
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
    
    self = <Condition(<unlocked _thread.RLock object owner=0 count=0 at 0x7f3752a36c40>, 0)>
    timeout = None
    
        def wait(self, timeout=None):
            """Wait until notified or until a timeout occurs.
        
            If the calling thread has not acquired the lock when this method is
            called, a RuntimeError is raised.
        
            This method releases the underlying lock, and then blocks until it is
            awakened by a notify() or notify_all() call for the same condition
            variable in another thread, or until the optional timeout occurs. Once
            awakened or timed out, it re-acquires the lock and returns.
        
            When the timeout argument is present and not None, it should be a
            floating point number specifying a timeout for the operation in seconds
            (or fractions thereof).
        
            When the underlying lock is an RLock, it is not released using its
            release() method, since this may not actually unlock the lock when it
            was acquired multiple times recursively. Instead, an internal interface
            of the RLock class is used, which really unlocks it even when it has
            been recursively acquired several times. Another internal interface is
            then used to restore the recursion level when the lock is reacquired.
        
            """
            if not self._is_owned():
                raise RuntimeError("cannot wait on un-acquired lock")
            waiter = _allocate_lock()
            waiter.acquire()
            self._waiters.append(waiter)
            saved_state = self._release_save()
            gotit = False
            try:    # restore state no matter what (e.g., KeyboardInterrupt)
                if timeout is None:
    >               waiter.acquire()
    E               Failed: Timeout >90.0s
    
    /usr/lib/python3.11/threading.py:320: Failed 
    

Build&Test / x-pack/metricbeat-pythonIntegTest / test_topic_metrics_collected – x-pack.metricbeat.module.activemq.test_activemq.ActiveMqTest_0
    Expand to view the error details

     Failed: Timeout >90.0s 
    

    Expand to view the stacktrace

     self = <test_activemq.ActiveMqTest_0 testMethod=test_topic_metrics_collected>
    
        @unittest.skipUnless(metricbeat.INTEGRATION_TESTS, 'integration test')
        def test_topic_metrics_collected(self):
    >       self.verify_destination_metrics_collection('topic')
    
    module/activemq/test_activemq.py:100: 
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
    module/activemq/test_activemq.py:70: in verify_destination_metrics_collection
        conn.disconnect()
    /opt/venv/lib/python3.11/site-packages/stomp/connect.py:185: in disconnect
        self.transport.stop()
    /opt/venv/lib/python3.11/site-packages/stomp/transport.py:122: in stop
        self.__receiver_thread_exit_condition.wait()
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
    
    self = <Condition(<unlocked _thread.RLock object owner=0 count=0 at 0x7f3752bdd780>, 0)>
    timeout = None
    
        def wait(self, timeout=None):
            """Wait until notified or until a timeout occurs.
        
            If the calling thread has not acquired the lock when this method is
            called, a RuntimeError is raised.
        
            This method releases the underlying lock, and then blocks until it is
            awakened by a notify() or notify_all() call for the same condition
            variable in another thread, or until the optional timeout occurs. Once
            awakened or timed out, it re-acquires the lock and returns.
        
            When the timeout argument is present and not None, it should be a
            floating point number specifying a timeout for the operation in seconds
            (or fractions thereof).
        
            When the underlying lock is an RLock, it is not released using its
            release() method, since this may not actually unlock the lock when it
            was acquired multiple times recursively. Instead, an internal interface
            of the RLock class is used, which really unlocks it even when it has
            been recursively acquired several times. Another internal interface is
            then used to restore the recursion level when the lock is reacquired.
        
            """
            if not self._is_owned():
                raise RuntimeError("cannot wait on un-acquired lock")
            waiter = _allocate_lock()
            waiter.acquire()
            self._waiters.append(waiter)
            saved_state = self._release_save()
            gotit = False
            try:    # restore state no matter what (e.g., KeyboardInterrupt)
                if timeout is None:
    >               waiter.acquire()
    E               Failed: Timeout >90.0s
    
    /usr/lib/python3.11/threading.py:320: Failed 
    

Build&Test / x-pack/metricbeat-pythonIntegTest / test_queue_metrics_collected – x-pack.metricbeat.module.activemq.test_activemq.ActiveMqTest_1
    Expand to view the error details

     Failed: Timeout >90.0s 
    

    Expand to view the stacktrace

     self = <test_activemq.ActiveMqTest_1 testMethod=test_queue_metrics_collected>
    
        @unittest.skipUnless(metricbeat.INTEGRATION_TESTS, 'integration test')
        def test_queue_metrics_collected(self):
    >       self.verify_destination_metrics_collection('queue')
    
    module/activemq/test_activemq.py:96: 
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
    module/activemq/test_activemq.py:70: in verify_destination_metrics_collection
        conn.disconnect()
    /opt/venv/lib/python3.11/site-packages/stomp/connect.py:185: in disconnect
        self.transport.stop()
    /opt/venv/lib/python3.11/site-packages/stomp/transport.py:122: in stop
        self.__receiver_thread_exit_condition.wait()
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
    
    self = <Condition(<unlocked _thread.RLock object owner=0 count=0 at 0x7f3753f657c0>, 0)>
    timeout = None
    
        def wait(self, timeout=None):
            """Wait until notified or until a timeout occurs.
        
            If the calling thread has not acquired the lock when this method is
            called, a RuntimeError is raised.
        
            This method releases the underlying lock, and then blocks until it is
            awakened by a notify() or notify_all() call for the same condition
            variable in another thread, or until the optional timeout occurs. Once
            awakened or timed out, it re-acquires the lock and returns.
        
            When the timeout argument is present and not None, it should be a
            floating point number specifying a timeout for the operation in seconds
            (or fractions thereof).
        
            When the underlying lock is an RLock, it is not released using its
            release() method, since this may not actually unlock the lock when it
            was acquired multiple times recursively. Instead, an internal interface
            of the RLock class is used, which really unlocks it even when it has
            been recursively acquired several times. Another internal interface is
            then used to restore the recursion level when the lock is reacquired.
        
            """
            if not self._is_owned():
                raise RuntimeError("cannot wait on un-acquired lock")
            waiter = _allocate_lock()
            waiter.acquire()
            self._waiters.append(waiter)
            saved_state = self._release_save()
            gotit = False
            try:    # restore state no matter what (e.g., KeyboardInterrupt)
                if timeout is None:
    >               waiter.acquire()
    E               Failed: Timeout >90.0s
    
    /usr/lib/python3.11/threading.py:320: Failed 
    

Build&Test / x-pack/metricbeat-pythonIntegTest / test_topic_metrics_collected – x-pack.metricbeat.module.activemq.test_activemq.ActiveMqTest_1
    Expand to view the error details

     Failed: Timeout >90.0s 
    

    Expand to view the stacktrace

     self = <test_activemq.ActiveMqTest_1 testMethod=test_topic_metrics_collected>
    
        @unittest.skipUnless(metricbeat.INTEGRATION_TESTS, 'integration test')
        def test_topic_metrics_collected(self):
    >       self.verify_destination_metrics_collection('topic')
    
    module/activemq/test_activemq.py:100: 
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
    module/activemq/test_activemq.py:70: in verify_destination_metrics_collection
        conn.disconnect()
    /opt/venv/lib/python3.11/site-packages/stomp/connect.py:185: in disconnect
        self.transport.stop()
    /opt/venv/lib/python3.11/site-packages/stomp/transport.py:122: in stop
        self.__receiver_thread_exit_condition.wait()
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
    
    self = <Condition(<unlocked _thread.RLock object owner=0 count=0 at 0x7f37526ac140>, 0)>
    timeout = None
    
        def wait(self, timeout=None):
            """Wait until notified or until a timeout occurs.
        
            If the calling thread has not acquired the lock when this method is
            called, a RuntimeError is raised.
        
            This method releases the underlying lock, and then blocks until it is
            awakened by a notify() or notify_all() call for the same condition
            variable in another thread, or until the optional timeout occurs. Once
            awakened or timed out, it re-acquires the lock and returns.
        
            When the timeout argument is present and not None, it should be a
            floating point number specifying a timeout for the operation in seconds
            (or fractions thereof).
        
            When the underlying lock is an RLock, it is not released using its
            release() method, since this may not actually unlock the lock when it
            was acquired multiple times recursively. Instead, an internal interface
            of the RLock class is used, which really unlocks it even when it has
            been recursively acquired several times. Another internal interface is
            then used to restore the recursion level when the lock is reacquired.
        
            """
            if not self._is_owned():
                raise RuntimeError("cannot wait on un-acquired lock")
            waiter = _allocate_lock()
            waiter.acquire()
            self._waiters.append(waiter)
            saved_state = self._release_save()
            gotit = False
            try:    # restore state no matter what (e.g., KeyboardInterrupt)
                if timeout is None:
    >               waiter.acquire()
    E               Failed: Timeout >90.0s
    
    /usr/lib/python3.11/threading.py:320: Failed 
    

Steps errors 4

Expand to view the steps failures

x-pack/metricbeat-pythonIntegTest - mage pythonIntegTest
  • Took 19 min 50 sec . View more details here
  • Description: mage pythonIntegTest
x-pack/metricbeat-pythonIntegTest - mage pythonIntegTest
  • Took 12 min 25 sec . View more details here
  • Description: mage pythonIntegTest
x-pack/metricbeat-pythonIntegTest - mage pythonIntegTest
  • Took 12 min 20 sec . View more details here
  • Description: mage pythonIntegTest
Error signal
  • Took 0 min 0 sec . View more details here
  • Description: Error 'hudson.AbortException: script returned exit code 1'

🐛 Flaky test report

❕ There are test failures but not known flaky tests.

Expand to view the summary

Genuine test errors 4

💔 There are test failures but not known flaky tests, most likely a genuine test failure.

  • Name: Build&Test / x-pack/metricbeat-pythonIntegTest / test_queue_metrics_collected – x-pack.metricbeat.module.activemq.test_activemq.ActiveMqTest_0
  • Name: Build&Test / x-pack/metricbeat-pythonIntegTest / test_topic_metrics_collected – x-pack.metricbeat.module.activemq.test_activemq.ActiveMqTest_0
  • Name: Build&Test / x-pack/metricbeat-pythonIntegTest / test_queue_metrics_collected – x-pack.metricbeat.module.activemq.test_activemq.ActiveMqTest_1
  • Name: Build&Test / x-pack/metricbeat-pythonIntegTest / test_topic_metrics_collected – x-pack.metricbeat.module.activemq.test_activemq.ActiveMqTest_1

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@cmacknz cmacknz merged commit 789ff76 into 8.8 Jun 20, 2023
@cmacknz cmacknz deleted the mergify/bp/8.8/pr-35780 branch June 20, 2023 21:18
@cmacknz
Copy link
Member

cmacknz commented Jun 20, 2023

Saw the same activemq failure in the original PR, was fixed by re-running the test. Ignoring.

@reakaleek reakaleek mentioned this pull request Jul 19, 2023
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport conflicts There is a conflict in the backported pull request needs_team Indicates that the issue/PR needs a Team:* label
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants