Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Eliminate or fix racy unit tests #3786

Closed
17 of 20 tasks
Aaronontheweb opened this issue May 16, 2019 · 6 comments
Closed
17 of 20 tasks

Eliminate or fix racy unit tests #3786

Aaronontheweb opened this issue May 16, 2019 · 6 comments
Labels
confirmed bug tests
Milestone
1.4.0

Comments

@Aaronontheweb
Copy link
Member

Aaronontheweb commented May 16, 2019 •
edited
Loading

Version: Akka.NET v1.3.13 or later.

The time has come, once again, to clean up the mess that impedes our work on Akka.NET as a result of some racy tests.

Try to fix them, but failing that - mark them as Skipped inside XUnit.
@Aaronontheweb Aaronontheweb added this to the 1.4.0 milestone May 16, 2019
@Aaronontheweb Aaronontheweb mentioned this issue May 16, 2019
Merged
Aaronontheweb added a commit to Aaronontheweb/akka.net that referenced this issue May 21, 2019
@Aaronontheweb
working on akkadotnet#3786 - made RemoteConnection_should_send_and_de…
72fce5c 
…code_messages async
@Aaronontheweb Aaronontheweb mentioned this issue May 21, 2019
Merged
Aaronontheweb added a commit to Aaronontheweb/akka.net that referenced this issue May 21, 2019
@Aaronontheweb
akkadotnet#3786 - fixed Passivation_of_inactive_entities spec
d381628
@Aaronontheweb Aaronontheweb mentioned this issue May 21, 2019
Closed
Aaronontheweb added a commit to Aaronontheweb/akka.net that referenced this issue May 21, 2019
@Aaronontheweb
Revert "working on akkadotnet#3786 - made RemoteConnection_should_sen…
ec73639 
…d_and_decode_messages async"

This reverts commit 72fce5c.
Aaronontheweb added a commit to Aaronontheweb/akka.net that referenced this issue May 21, 2019
@Aaronontheweb
fix akkadotnet#3786 - switched from parsing IPAddress to using IPAddr…
baf6fcb 
…ess.Loopback
Aaronontheweb added a commit to Aaronontheweb/akka.net that referenced this issue May 21, 2019
@Aaronontheweb
Revert "fix akkadotnet#3786 - switched from parsing IPAddress to usin…
f844856 
…g IPAddress.Loopback"

This reverts commit baf6fcb.
Aaronontheweb added a commit that referenced this issue May 21, 2019
@Aaronontheweb
[SpecFix] made RemoteConnection_should_send_and_decode_messages async (…
2bfc579 
…#3800)

* working on #3786 - skipping RemoteConnection_should_send_and_decode_messages async
Aaronontheweb added a commit to Aaronontheweb/akka.net that referenced this issue May 21, 2019
@Aaronontheweb
akkadotnet#3786 - removed two Cluster must cancel join prematurely specs
6035064
@Aaronontheweb Aaronontheweb mentioned this issue May 21, 2019
Merged
Aaronontheweb added a commit to Aaronontheweb/akka.net that referenced this issue May 21, 2019
@Aaronontheweb
fix PersistentActor_should_be_able_to_persist_events_that_happen_duri…
8df73dc 
…ng_recovery

akkadotnet#3786 - fix PersistentActor_should_be_able_to_persist_events_that_happen_during_recovery by porting akka/akka#22232
@Aaronontheweb Aaronontheweb mentioned this issue May 21, 2019
Merged
Aaronontheweb added a commit that referenced this issue May 22, 2019
@Aaronontheweb
fix PersistentActor_should_be_able_to_persist_events_that_happen_duri…
c11f5b7 
…ng_recovery (#3805)

* fix PersistentActor_should_be_able_to_persist_events_that_happen_during_recovery

#3786 - fix PersistentActor_should_be_able_to_persist_events_that_happen_during_recovery by porting akka/akka#22232

* force PersistentActorRecoverySpec to use new SteppingMemoryJournal instance each time
Aaronontheweb added a commit that referenced this issue May 22, 2019
@Aaronontheweb
#3786 - removed two Cluster must cancel join prematurely specs (#3803)
0ddfa3c
Aaronontheweb added a commit to Aaronontheweb/akka.net that referenced this issue May 22, 2019
@Aaronontheweb
akkadotnet#3786 - working on fixing AtLeastOnceDeliveryCrashSpec
cd298fb
@Aaronontheweb Aaronontheweb mentioned this issue May 22, 2019
Merged
Aaronontheweb added a commit that referenced this issue May 26, 2019
@Aaronontheweb
[SpecFix] working on fixing AtLeastOnceDeliveryCrashSpec (#3806)
a3963cd 
* #3786 - working on fixing AtLeastOnceDeliveryCrashSpec

* fixed issue with SnapshotDirectoryFailureSpec.LocalSnapshotStore_configured_with_a_failing_directory

* fixed Akka.Persistence.Tests.Actor_PipeTo_should_not_be_delayed_by_async_receive

* looks like issue might be related to parallel execution and thread blocking

* added logging for AtLeastOnceDelivery_must_tolerate_and_recover_from_random_failures
Aaronontheweb added a commit to Aaronontheweb/akka.net that referenced this issue Jun 4, 2019
@Aaronontheweb
part of akkadotnet#3786 - fixed issue with multiple specs all trying …
27e1156 
…to use same Akka.Remote port at the same time
@Aaronontheweb Aaronontheweb mentioned this issue Jun 4, 2019
Merged
@Aaronontheweb
Copy link
Member Author

Important detail about why Akka.Remote.Tests.RemotingSpec.Remoting_must_lookup_actors_across_node_boundaries fails occasionally

[DEBUG][6/4/2019 3:04:54 PM][Thread 0137][LocalActorRefProvider(akka://RemotingSpec-101)] Resolve of path sequence [/temp/k] failed

Looks like a racy issue with looking up temporary actors when the actor on the other side of the network sets up a reply - might need to take a look at the VirtualPathContainer if a round-trip I/O operation is able to complete before the recipient FutureActorRef is able to register with it.

@Aaronontheweb
Copy link
Member Author

Relevant logs: https://dev.azure.com/dotnet/Akka.NET/_build/results?buildId=17879&view=ms.vss-test-web.build-test-results-tab&runId=1043496&paneView=attachments&resultId=100886

Aaronontheweb added a commit to Aaronontheweb/akka.net that referenced this issue Jun 26, 2019
@Aaronontheweb
akkadotnet#3786 - fix bind on port 0 error with DistributedPubSub specs
b4e39a0
@Aaronontheweb Aaronontheweb mentioned this issue Jun 26, 2019
Merged
Aaronontheweb added a commit that referenced this issue Jun 26, 2019
@Aaronontheweb
#3786 - fix bind on port 0 error with DistributedPubSub specs (#3831)
ea0d054
Aaronontheweb added a commit that referenced this issue Jun 26, 2019
@Aaronontheweb
fixed issue with multiple specs all trying to use same Akka.Remote po…
dbc25db 
…rt at the same time (#3817)

* part of #3786 - fixed issue with multiple specs all trying to use same Akka.Remote port at the same time

* updated specs to use Collections XUnit attribute instead

* use port 0 in all specs by default
@Aaronontheweb Aaronontheweb mentioned this issue Jul 7, 2019
Merged
Aaronontheweb added a commit to Aaronontheweb/akka.net that referenced this issue Jul 30, 2019
@Aaronontheweb
[SpecFix] working on fixing AtLeastOnceDeliveryCrashSpec (akkadotnet#…
53ca61f 
…3806)

* akkadotnet#3786 - working on fixing AtLeastOnceDeliveryCrashSpec

* fixed issue with SnapshotDirectoryFailureSpec.LocalSnapshotStore_configured_with_a_failing_directory

* fixed Akka.Persistence.Tests.Actor_PipeTo_should_not_be_delayed_by_async_receive

* looks like issue might be related to parallel execution and thread blocking

* added logging for AtLeastOnceDelivery_must_tolerate_and_recover_from_random_failures
Aaronontheweb added a commit to Aaronontheweb/akka.net that referenced this issue Jul 30, 2019
@Aaronontheweb
akkadotnet#3786 - fix bind on port 0 error with DistributedPubSub spe…
ca5c397 
…cs (akkadotnet#3831)
Aaronontheweb added a commit to Aaronontheweb/akka.net that referenced this issue Jul 30, 2019
@Aaronontheweb
fixed issue with multiple specs all trying to use same Akka.Remote po…
a57228f 
…rt at the same time (akkadotnet#3817)

* part of akkadotnet#3786 - fixed issue with multiple specs all trying to use same Akka.Remote port at the same time

* updated specs to use Collections XUnit attribute instead

* use port 0 in all specs by default
@Aaronontheweb Aaronontheweb mentioned this issue Dec 2, 2019
Closed
@Aaronontheweb
Copy link
Member Author

Akka NET+Racy+Test+Debug+Decision+Tree

cc @IgorFedchenko @nagytech - so here's the list of the tests with the highest flip rates: https://dev.azure.com/dotnet/Akka.NET/_test/analytics?definitionId=84&contextType=build

I'd recommend adding a comment to here and say which test you're working on. Some tests might just be crappy / bad tests that need to be deleted or disabled. Others might reveal a real underlying problem with the infrastructure and take some time to debug. Others might be simple fixes (i.e. using AwaitAssert instead of a simple assertion, for testing a volatile condition)

@ghost ghost mentioned this issue Dec 12, 2019
Merged
@IgorFedchenko
Copy link
Contributor

IgorFedchenko commented Jan 23, 2020 •
edited
Loading

Working on
Akka.Cluster.Sharding.Tests.CoordinatedShutdownShardingSpec.Sharding_and_CoordinatedShutdown_must_run_successfully
Akka.Cluster.Sharding.Tests.DisabledInactiveEntityPassivationSpec.Passivation_of_inactive_entities_must_not_passivate_when_passivation_is_disabled
Akka.Tests.IO.TcpIntegrationSpec.Should_fail_writing_when_buffer_is_filled

This was referenced Jan 23, 2020
Merged
Merged
Merged
Merged
Open
Aaronontheweb added a commit to Aaronontheweb/akka.net that referenced this issue Feb 4, 2020
@Aaronontheweb
akkadotnet#3786 - make CoordinatedShutdownShardingSpec use murmur has…
6b591da 
…h for shard distributions
@Aaronontheweb Aaronontheweb mentioned this issue Feb 4, 2020
Merged
Aaronontheweb added a commit that referenced this issue Feb 8, 2020
@Aaronontheweb
#3786 - make CoordinatedShutdownShardingSpec use murmur hash for shar…
4e2aaf5 
…d distributions (#4186)
@Aaronontheweb Aaronontheweb mentioned this issue Feb 9, 2020
Merged
Aaronontheweb added a commit to Aaronontheweb/akka.net that referenced this issue Feb 12, 2020
@Aaronontheweb
per akkadotnet#3786 - skipping racy FlowDelaySpecs
f925922
@Aaronontheweb Aaronontheweb mentioned this issue Feb 12, 2020
Merged
Aaronontheweb added a commit to Aaronontheweb/akka.net that referenced this issue Feb 25, 2020
@Aaronontheweb
Revert "per akkadotnet#3786 - skipping racy FlowDelaySpecs"
05d20a4 
This reverts commit f925922.
Aaronontheweb added a commit to Aaronontheweb/akka.net that referenced this issue Feb 25, 2020
@Aaronontheweb
Revert "Revert "per akkadotnet#3786 - skipping racy FlowDelaySpecs""
269c886 
This reverts commit 05d20a4.
Aaronontheweb added a commit that referenced this issue Feb 26, 2020
@Aaronontheweb
Racy specs: Akka.Streams (#4209)
c102dd9 
* per #3786 - skipping racy FlowDelaySpecs

* disabled racy FlowIdleInjectSpecs

* Revert "per #3786 - skipping racy FlowDelaySpecs"

This reverts commit f925922.

* Revert "disabled racy FlowIdleInjectSpecs"

This reverts commit 23a673c.

* Revert "Revert "per #3786 - skipping racy FlowDelaySpecs""

This reverts commit 05d20a4.

* Revert "Revert "disabled racy FlowIdleInjectSpecs""

This reverts commit 8f92a83.

* disable IdleTimeout_must_fail_if_time_between_elements_is_too_large

* marking other worst offenders from Akka.Streams as skipped

* marking FlowDelaySpec.A_Delay_must_add_delay_to_initialDelay_if_exists_upstream as skipped

* disabled ActorPublisher_should_be_able_to_define_a_subscription_timeout_after_which_it_should_shut_down

* skipped Akka.Streams.Tests.Dsl.FlowDelaySpec.A_Delay_must_properly_delay_according_to_buffer_size

* disabled more racy specs

* disabling more racy specs
This was referenced Feb 26, 2020
Merged
Merged
Merged
Merged
Aaronontheweb added a commit to Aaronontheweb/akka.net that referenced this issue Feb 27, 2020
@Aaronontheweb
per akkadotnet#3786 - harden DData.Tests.WriteAggregatorSpecs
c7ad9db
@Aaronontheweb Aaronontheweb mentioned this issue Feb 27, 2020
Merged
Aaronontheweb added a commit that referenced this issue Feb 27, 2020
@Aaronontheweb
per #3786 - harden DData.Tests.WriteAggregatorSpecs (#4269)
549ff8a
@Aaronontheweb
Copy link
Member Author

Looking good now - had several big PRs run the full test suite without any racy issues. Going to mark this as closed for the time being.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
confirmed bug tests
Projects
None yet
Milestone
1.4.0
Development

No branches or pull requests
2 participants
@Aaronontheweb @IgorFedchenko