Stateful test failure using randomly seed 1697538841 looks like a faker issue

[cofiem]

Describe the bug

When running warehouse tests, this randomly seed fails (seed is from #15398 ):

 make tests TESTARGS="--randomly-seed=1697538841 -vvv" T=tests/unit/admin/views/test_emails.py::TestEmailList

with the two errors

FAILED tests/unit/admin/views/test_emails.py::TestEmailList::test_wildcard_query - AssertionError: assert equals failed
  -{'emails': <paginate.Page: Page 1/1>, 'query': 'ubrown@example.or%'}                              +{
                                                                                                    +  'emails': [
                                                                                                    +    EmailMessage(created=datetime.datetime(2025, 3, 7, 14, 35, 9, 91506), status=<EmailStatuses.
                                                                                                    +Accepted: 'Accepted'>, message_id='IqdyfEuXOtQR', from_='andrew89@example.com', to='ubrown@examp
                                                                                                    +le.org', subject='Before send window.', missing=False, id=UUID('836af322-8b4a-49d1-af21-978e19f5
                                                                                                    +26a0')),
                                                                                                    +  ],
                                                                                                    +  'query': 'ubrown@example.or%',
                                                                                                    +}
FAILED tests/unit/admin/views/test_emails.py::TestEmailList::test_basic_query - AssertionError: assert equals failed
  -{'emails': <paginate.Page: Page 1/1>, 'query': 'ubrown@example.org'}                              +{
                                                                                                    +  'emails': [
                                                                                                    +    EmailMessage(created=datetime.datetime(2025, 3, 7, 14, 35, 9, 91506), status=<EmailStatuses.
                                                                                                    +Accepted: 'Accepted'>, message_id='IqdyfEuXOtQR', from_='andrew89@example.com', to='ubrown@examp
                                                                                                    +le.org', subject='Before send window.', missing=False, id=UUID('fc1ba556-b2ed-4321-b993-1db542de
                                                                                                    +2486')),
                                                                                                    +  ],
                                                                                                    +  'query': 'ubrown@example.org',
                                                                                                    +}

Results (21.19s):
       7 passed
       2 failed
         - tests/unit/admin/views/test_emails.py:104 TestEmailList.test_wildcard_query
         - tests/unit/admin/views/test_emails.py:57 TestEmailList.test_basic_query

where <paginate.Page: Page 1/1> is

[EmailMessage(
created=datetime.datetime(2025, 3, 7, 14, 11, 25, 124019), 
status=<EmailStatuses.Accepted: 'Accepted'>,
message_id='IqdyfEuXOtQR', 
from_='andrew89@example.com',
to='ubrown@example.org', 
subject='Before send window.',
missing=False,
id=UUID('971b4791-4ca6-4e4c-b550-a8cc2a830422')), 
EmailMessage(created=datetime.datetime(2025, 3, 6, 8, 10, 10, 717537), status=<EmailStatuses.Accepted: 'Accepted'>,
message_id='NNidOFjCuWWu', 
from_='smithmorgan@example.org', 
to='ubrown@example.org', 
subject='Trip enter office near include language provide.', 
missing=False, 
id=UUID('ae299de1-9a51-4516-aa20-304748f22e47'))
]

I looks like this is an instance of FactoryBoy/factory_boy#820 where the same value is produced more than once.

Others have fixed this by not using faker and generating strings with incrementing numbers instead, e.g.

     message_id = factory.Faker("pystr", max_chars=12)
-    from_ = factory.Faker("safe_email")
-    to = factory.Faker("safe_email")
+    from_ = factory.Sequence(lambda n: f'person_from{n}@example.com')
+    to = factory.Sequence(lambda n: f'person_to{n}@example.com')
     subject = factory.Faker("sentence")

The tests seem to pass with the above change.

Expected behavior
The tests should pass.

To Reproduce
See above.

My Platform
main branch at 6fefaa1

[cofiem]

In fact, the failures from some other seeds seem to be due to the same issue.

Seeds:

• 2228510538: tests/unit/admin/views/test_ipaddresses.py::TestIpAddressList
• 1862154588: tests/unit/admin/views/test_prohibited_user_names.py::TestProhibitedUserNameList

[miketheman]

Thanks! We've been collecting these on #15398 - had you come across this one already?

Would you like to propose an update to the test factory, as you've shown?