Don't double install mac deps, cleanups #2513

ankith26 · 2023-10-09T12:49:25Z

This PR is one part of the series of changes I'm making. The goal of this PR is to update the scripts to only install mac deps in the custom prefix (and not in /usr/local) and do related cleanups that should eventually help us get on using self-compiled deps on windows, and also being able to use systems like meson for the dependencies that either only support it, or recommend using it

MyreMylar

LGTM and, more importantly here, the mac CI is happy.

buildconfig/manylinux-build/docker_base/opus/build-opus.sh

Starbuck5 · 2023-10-15T02:57:12Z

Question: does this lead to a noticeable speedup of the mac deps generation? That's not necessary, but it would be a quantifiable win for this PR.

also being able to use systems like meson

I used Meson for a C/C++ project last year and I had a good impression of it. Other libraries seem to be getting good use of it, I think numpy and scipy use meson now.

Starbuck5 · 2023-10-15T03:09:21Z

@oddbookworm or @novialriptide Would it be convenient for either of you to give these mac wheels a test?

ankith26 · 2023-10-15T06:55:18Z

Question: does this lead to a noticeable speedup of the mac deps generation?

In theory it should, but the speedup would be small. There is usually a lot of randomness in the time the build step takes, for some reason. But it runs rarely so I guess it won't be a big deal.

novialriptide · 2023-11-02T13:26:08Z

I got multiple errors running macOS 14.0 (M2 Max 32GB)

`ankith26-mac-build` branch

======================================================================
ERROR: test_get__owned_empty_type (pygame.tests.scrap_test.ScrapModuleTest.test_get__owned_empty_type)
Ensures get works when there is no data of the requested type
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/pygame/tests/scrap_test.py", line 64, in test_get__owned_empty_type
    scrap.put(pygame.SCRAP_TEXT, b"text to clipboard")
pygame.error: content could not be placed in clipboard.

======================================================================
ERROR: test_init__reinit (pygame.tests.scrap_test.ScrapModuleTest.test_init__reinit)
Ensures reinitializing the scrap module doesn't clear its data.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/pygame/tests/scrap_test.py", line 36, in test_init__reinit
    scrap.put(data_type, expected_data)
pygame.error: content could not be placed in clipboard.

======================================================================
ERROR: test_put (pygame.tests.scrap_test.ScrapModuleTest.test_put)
Ensures put can place data into the clipboard
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/pygame/tests/scrap_test.py", line 113, in test_put
    scrap.put(DATA_TYPE, b"buf")
pygame.error: content could not be placed in clipboard.

======================================================================
ERROR: test_put__bmp_image (pygame.tests.scrap_test.ScrapModuleTest.test_put__bmp_image)
Ensures put can place a BMP image into the clipboard.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/pygame/tests/scrap_test.py", line 103, in test_put__bmp_image
    scrap.put(pygame.SCRAP_BMP, expected_bytes)
pygame.error: content could not be placed in clipboard.

======================================================================
ERROR: test_put__text (pygame.tests.scrap_test.ScrapModuleTest.test_put__text)
Ensures put can place text into the clipboard.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/pygame/tests/scrap_test.py", line 90, in test_put__text
    scrap.put(pygame.SCRAP_TEXT, b"Hello world")
pygame.error: content could not be placed in clipboard.

======================================================================
FAIL: test_multiple_timers (pygame.tests.time_test.TimeModuleTest.test_multiple_timers)
Test for multiple timers running together, test repeat and dict
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/pygame/tests/time_test.py", line 287, in test_multiple_timers
    self.assertAlmostEqual(
AssertionError: 14.290542007074691 != 10 within 4 delta (4.290542007074691 difference)

======================================================================
FAIL: test_box_blur (pygame.tests.transform_test.TransformDisplayModuleTest.test_box_blur)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/pygame/tests/transform_test.py", line 1501, in test_box_blur
    self.assertTrue(sf_b2.get_at(pos) == data2[pos])
AssertionError: False is not true

======================================================================
FAIL: test_gaussian_blur (pygame.tests.transform_test.TransformDisplayModuleTest.test_gaussian_blur)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/pygame/tests/transform_test.py", line 1543, in test_gaussian_blur
    self.assertTrue(sf_b1.get_at(pos) == data1[pos])
AssertionError: False is not true

======================================================================
FAIL: test_position (pygame.tests.window_test.WindowTypeTest.test_position)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/pygame/tests/window_test.py", line 130, in test_position
    self.assertTupleEqual(self.win.position, (12, 34))
AssertionError: Tuples differ: (12, 72) != (12, 34)

First differing element 1:
72
34

- (12, 72)
?      ^^

+ (12, 34)
?      ^^


----------------------------------------------------------------------
Ran 2095 tests in 47.046s

FAILED (failures=4, errors=5)

However, I'm getting errors before the changes anyways.

version `2.3.2`

======================================================================
ERROR: test_get__owned_empty_type (pygame.tests.scrap_test.ScrapModuleTest.test_get__owned_empty_type)
Ensures get works when there is no data of the requested type
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/pygame/tests/scrap_test.py", line 64, in test_get__owned_empty_type
    scrap.put(pygame.SCRAP_TEXT, b"text to clipboard")
pygame.error: content could not be placed in clipboard.

======================================================================
ERROR: test_init__reinit (pygame.tests.scrap_test.ScrapModuleTest.test_init__reinit)
Ensures reinitializing the scrap module doesn't clear its data.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/pygame/tests/scrap_test.py", line 36, in test_init__reinit
    scrap.put(data_type, expected_data)
pygame.error: content could not be placed in clipboard.

======================================================================
ERROR: test_put (pygame.tests.scrap_test.ScrapModuleTest.test_put)
Ensures put can place data into the clipboard
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/pygame/tests/scrap_test.py", line 113, in test_put
    scrap.put(DATA_TYPE, b"buf")
pygame.error: content could not be placed in clipboard.

======================================================================
ERROR: test_put__bmp_image (pygame.tests.scrap_test.ScrapModuleTest.test_put__bmp_image)
Ensures put can place a BMP image into the clipboard.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/pygame/tests/scrap_test.py", line 103, in test_put__bmp_image
    scrap.put(pygame.SCRAP_BMP, expected_bytes)
pygame.error: content could not be placed in clipboard.

======================================================================
ERROR: test_put__text (pygame.tests.scrap_test.ScrapModuleTest.test_put__text)
Ensures put can place text into the clipboard.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/pygame/tests/scrap_test.py", line 90, in test_put__text
    scrap.put(pygame.SCRAP_TEXT, b"Hello world")
pygame.error: content could not be placed in clipboard.

======================================================================
FAIL: test_multiple_timers (pygame.tests.time_test.TimeModuleTest.test_multiple_timers)
Test for multiple timers running together, test repeat and dict
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/pygame/tests/time_test.py", line 287, in test_multiple_timers
    self.assertAlmostEqual(
AssertionError: 43.70808300154749 != 10 within 4 delta (33.70808300154749 difference)

======================================================================
FAIL: test_box_blur (pygame.tests.transform_test.TransformDisplayModuleTest.test_box_blur)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/pygame/tests/transform_test.py", line 1501, in test_box_blur
    self.assertTrue(sf_b2.get_at(pos) == data2[pos])
AssertionError: False is not true

======================================================================
FAIL: test_gaussian_blur (pygame.tests.transform_test.TransformDisplayModuleTest.test_gaussian_blur)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/pygame/tests/transform_test.py", line 1543, in test_gaussian_blur
    self.assertTrue(sf_b1.get_at(pos) == data1[pos])
AssertionError: False is not true

======================================================================
FAIL: test_pg_version_consistency (pygame.tests.version_test.VersionTest.test_pg_version_consistency)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/pygame/tests/version_test.py", line 38, in test_pg_version_consistency
    self.assertEqual(pgh_minor, version.vernum[1])
AssertionError: 4 != 3

----------------------------------------------------------------------
Ran 2046 tests in 40.783s

FAILED (failures=4, errors=5)

novialriptide · 2023-11-02T13:32:35Z

It could be because I'm using ARM (Apple Silicon), not x86 (Intel)?

ankith26 · 2023-11-02T13:37:13Z

I get those scrap fails locally too so that's fine IG. What's weird is the timer fail and the blur fail (but again, both happen on the older version too). I don't think either is caused by this PR though. Could you also go though a few examples and stuff to check those are not broken

novialriptide · 2023-11-02T13:41:26Z

I did a quick run of the following

aliens.py
go_over_there.py
liquid.py (this was pretty cool)
stars.py

Nothing seems broken so far.

novialriptide

Since the errors seem like they're irrelevant to this PR, I'll give it the thumbsup

ankith26 force-pushed the ankith26-mac-build branch 5 times, most recently from 829d36b to 596354b Compare October 9, 2023 14:47

ankith26 marked this pull request as ready for review October 9, 2023 15:52

ankith26 requested a review from a team as a code owner October 9, 2023 15:52

Don't double install mac deps, cleanups

f880811

ankith26 force-pushed the ankith26-mac-build branch from 596354b to f880811 Compare October 9, 2023 15:58

ankith26 added Code quality/robustness Code quality and resilience to changes dependencies labels Oct 9, 2023

ankith26 added this to the 2.4.0 milestone Oct 9, 2023

MyreMylar approved these changes Oct 9, 2023

View reviewed changes

Starbuck5 reviewed Oct 15, 2023

View reviewed changes

buildconfig/manylinux-build/docker_base/opus/build-opus.sh Show resolved Hide resolved

MyreMylar requested a review from oddbookworm October 15, 2023 10:15

Merge branch 'main' into ankith26-mac-build

3ad1a97

novialriptide approved these changes Nov 2, 2023

View reviewed changes

ankith26 merged commit 89c2465 into main Nov 5, 2023
29 checks passed

ankith26 deleted the ankith26-mac-build branch November 5, 2023 18:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Don't double install mac deps, cleanups #2513

Don't double install mac deps, cleanups #2513

ankith26 commented Oct 9, 2023 •

edited

MyreMylar left a comment

Starbuck5 commented Oct 15, 2023

Starbuck5 commented Oct 15, 2023

ankith26 commented Oct 15, 2023

novialriptide commented Nov 2, 2023 •

edited

novialriptide commented Nov 2, 2023

ankith26 commented Nov 2, 2023

novialriptide commented Nov 2, 2023

novialriptide left a comment

Don't double install mac deps, cleanups #2513

Don't double install mac deps, cleanups #2513

Conversation

ankith26 commented Oct 9, 2023 • edited

MyreMylar left a comment

Choose a reason for hiding this comment

Starbuck5 commented Oct 15, 2023

Starbuck5 commented Oct 15, 2023

ankith26 commented Oct 15, 2023

novialriptide commented Nov 2, 2023 • edited

ankith26-mac-build branch

version 2.3.2

novialriptide commented Nov 2, 2023

ankith26 commented Nov 2, 2023

novialriptide commented Nov 2, 2023

novialriptide left a comment

Choose a reason for hiding this comment

ankith26 commented Oct 9, 2023 •

edited

novialriptide commented Nov 2, 2023 •

edited

`ankith26-mac-build` branch

version `2.3.2`