Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

YARN-10951. Move async scheduling fields to a new AsyncSchedulingConfiguration #3438

Closed
wants to merge 1 commit into from
Closed

YARN-10951. Move async scheduling fields to a new AsyncSchedulingConfiguration #3438

wants to merge 1 commit into from

Conversation

JackWangCS
Copy link
Contributor

Description of PR

Move async scheduling logic to a new class

How was this patch tested?

For code changes:

  • [Yes] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • [No] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • [No] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • [No] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 43s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 31m 50s trunk passed
+1 💚 compile 1m 5s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 compile 0m 53s trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 checkstyle 0m 49s trunk passed
+1 💚 mvnsite 1m 1s trunk passed
+1 💚 javadoc 0m 47s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javadoc 0m 44s trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 spotbugs 1m 54s trunk passed
+1 💚 shadedclient 30m 46s branch has no errors when building and testing our client artifacts.
-0 ⚠️ patch 31m 4s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 54s the patch passed
+1 💚 compile 0m 57s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javac 0m 57s the patch passed
+1 💚 compile 0m 47s the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 javac 0m 47s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 0m 38s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 0 new + 122 unchanged - 5 fixed = 122 total (was 127)
+1 💚 mvnsite 0m 50s the patch passed
+1 💚 javadoc 0m 37s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javadoc 0m 34s the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 spotbugs 1m 55s the patch passed
-1 ❌ shadedclient 28m 54s patch has errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 73m 52s /patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt hadoop-yarn-server-resourcemanager in the patch passed.
-1 ❌ asflicense 0m 34s /results-asflicense.txt The patch generated 1 ASF License warnings.
180m 34s
Reason Tests
Failed junit tests hadoop.yarn.server.resourcemanager.scheduler.fair.converter.TestFSConfigToCSConfigConverterMain
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerWeightMode
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestQueueState
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestWorkPreservingRMRestartForNodeLabel
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestAbsoluteResourceConfiguration
hadoop.yarn.server.resourcemanager.TestWorkPreservingUnmanagedAM
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueACLs
hadoop.yarn.server.resourcemanager.resourcetracker.TestNMReconnect
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestAutoCreatedQueueDeletionPolicy
hadoop.yarn.server.resourcemanager.webapp.TestRMWebApp
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerSchedulingRequestUpdate
hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA
hadoop.yarn.server.resourcemanager.scheduler.constraint.algorithm.TestLocalAllocationTagsManager
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesSchedulerActivities
hadoop.yarn.server.resourcemanager.reservation.TestCapacitySchedulerPlanFollower
hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA
hadoop.yarn.server.resourcemanager.TestResourceTrackerService
hadoop.yarn.server.resourcemanager.scheduler.TestSchedulingWithAllocationRequestId
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesReservation
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerWithMultiResourceTypes
hadoop.yarn.server.resourcemanager.TestApplicationMasterServiceFair
hadoop.yarn.server.resourcemanager.security.TestClientToAMTokens
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesConfigurationMutation
hadoop.yarn.server.resourcemanager.TestAMAuthorization
hadoop.yarn.server.resourcemanager.scheduler.fair.converter.TestConvertedConfigValidator
hadoop.yarn.server.resourcemanager.TestResourceManagerMXBean
hadoop.yarn.server.resourcemanager.security.TestRMDelegationTokens
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestQueueStateManager
hadoop.yarn.server.resourcemanager.TestRMHAForNodeLabels
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerNewQueueAutoCreation
hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServiceAppsNodelabel
hadoop.yarn.server.resourcemanager.TestApplicationCleanup
hadoop.yarn.server.resourcemanager.rmcontainer.TestRMContainerImpl
hadoop.yarn.server.resourcemanager.scheduler.TestAbstractYarnScheduler
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestQueueConfigurationAutoRefreshPolicy
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerAutoQueueCreation
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerConfigValidator
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesSchedulerActivitiesWithMultiNodesEnabled
hadoop.yarn.server.resourcemanager.federation.TestFederationRMStateStoreService
hadoop.yarn.server.resourcemanager.TestDecommissioningNodesWatcher
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerAmbiguousLeafs
hadoop.yarn.server.resourcemanager.TestAppManager
hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRMRPCResponseId
hadoop.yarn.server.resourcemanager.webapp.TestAppPage
hadoop.yarn.server.resourcemanager.TestApplicationMasterLauncher
hadoop.yarn.server.resourcemanager.TestSignalContainer
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueMappingFactory
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerMultiNodesWithPreemption
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerSurgicalPreemption
hadoop.yarn.server.resourcemanager.TestResourceManager
hadoop.yarn.server.resourcemanager.TestRMHAMetrics
hadoop.yarn.server.resourcemanager.TestContainerResourceUsage
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestAbsoluteResourceWithAutoQueue
hadoop.yarn.server.resourcemanager.TestApplicationMasterServiceCapacity
hadoop.yarn.server.resourcemanager.scheduler.policy.TestFairOrderingPolicy
hadoop.yarn.server.resourcemanager.TestRMEmbeddedElector
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestLeafQueue
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestQueueManagementDynamicEditPolicy
hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRMRPCNodeUpdates
hadoop.yarn.server.resourcemanager.TestRMHA
hadoop.yarn.server.resourcemanager.TestOpportunisticContainerAllocatorAMService
hadoop.yarn.server.resourcemanager.TestRMStoreCommands
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestReservations
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySched
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerAutoCreatedQueuePreemption
hadoop.yarn.server.resourcemanager.rmapp.attempt.TestAMLivelinessMonitor
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodeLabels
hadoop.yarn.server.resourcemanager.TestRM
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerDynamicBehavior
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler
hadoop.yarn.server.resourcemanager.TestNodeBlacklistingOnAMFailures
hadoop.yarn.server.resourcemanager.security.TestAMRMTokens
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationLimits
hadoop.yarn.server.resourcemanager.TestRMAdminService
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerNodeLabelUpdate
hadoop.yarn.server.resourcemanager.monitor.capacity.TestProportionalCapacityPreemptionPolicy
hadoop.yarn.server.resourcemanager.TestReservationSystemWithRMHA
hadoop.yarn.server.resourcemanager.TestRMTimelineService
hadoop.yarn.server.resourcemanager.scheduler.constraint.TestAllocationTagsManager
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestIncreaseAllocationExpirer
hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer
hadoop.yarn.server.resourcemanager.scheduler.TestSchedulerUtils
hadoop.yarn.server.resourcemanager.rmapp.TestApplicationLifetimeMonitor
hadoop.yarn.server.resourcemanager.TestRMHATimelineCollectors
hadoop.yarn.server.resourcemanager.webapp.TestRedirectionErrorPage
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerOvercommit
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestQueueMappings
hadoop.yarn.server.resourcemanager.scheduler.TestSchedulerHealth
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesForCSWithPartitions
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationLimitsByPartition
hadoop.yarn.server.resourcemanager.TestClientRMService
hadoop.yarn.server.resourcemanager.resource.TestResourceProfiles
hadoop.yarn.server.resourcemanager.monitor.TestSchedulingMonitor
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation
hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.TestLeveldbConfigurationStore
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerMultiNodes
hadoop.yarn.server.resourcemanager.TestRMProxyUsersConf
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerAsyncScheduling
hadoop.yarn.server.resourcemanager.TestApplicationACLs
hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart
hadoop.yarn.server.resourcemanager.nodelabels.TestRMDelegatedNodeLabelsUpdater
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.TestZKConfigurationStore
hadoop.yarn.server.resourcemanager.volume.csi.TestVolumeProcessor
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerResizing
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerMaxParallelApps
hadoop.yarn.server.resourcemanager.scheduler.constraint.TestPlacementProcessor
hadoop.yarn.server.resourcemanager.placement.TestPlacementManager
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriority
hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification
hadoop.yarn.server.resourcemanager.TestLeaderElectorService
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerWorkflowPriorityMapping
hadoop.yarn.server.resourcemanager.rmapp.TestNodesListManager
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySchedDynamicConfig
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestSchedulingRequestContainerAllocation
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriorityACLs
hadoop.yarn.server.resourcemanager.TestRMRestart
hadoop.yarn.server.resourcemanager.scheduler.constraint.TestPlacementConstraintsUtil
hadoop.yarn.server.resourcemanager.reservation.TestReservationSystem
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestQueueParsing
hadoop.yarn.server.resourcemanager.webapp.TestNodesPage
hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerLazyPreemption
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCSAllocateCustomResource
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestSchedulingRequestContainerAllocationAsync
hadoop.yarn.server.resourcemanager.nodelabels.TestRMNodeLabelsManager
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3438/1/artifact/out/Dockerfile
GITHUB PR #3438
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname Linux 1a0105391d88 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 419b8e0
Default Java Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3438/1/testReport/
Max. process+thread count 5408 (vs. ulimit of 5500)
modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3438/1/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@JackWangCS JackWangCS marked this pull request as draft September 16, 2021 02:43
@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 43s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 33m 20s trunk passed
+1 💚 compile 1m 3s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 compile 0m 57s trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 checkstyle 0m 49s trunk passed
+1 💚 mvnsite 0m 59s trunk passed
+1 💚 javadoc 0m 48s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javadoc 0m 43s trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 spotbugs 1m 54s trunk passed
+1 💚 shadedclient 29m 21s branch has no errors when building and testing our client artifacts.
-0 ⚠️ patch 29m 41s Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 50s the patch passed
+1 💚 compile 0m 56s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javac 0m 56s the patch passed
+1 💚 compile 0m 48s the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 javac 0m 48s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 0m 40s /results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 1 new + 122 unchanged - 5 fixed = 123 total (was 127)
+1 💚 mvnsite 0m 50s the patch passed
+1 💚 javadoc 0m 37s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javadoc 0m 35s the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 spotbugs 1m 52s the patch passed
-1 ❌ shadedclient 28m 44s patch has errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 73m 58s /patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt hadoop-yarn-server-resourcemanager in the patch passed.
-1 ❌ asflicense 0m 33s /results-asflicense.txt The patch generated 1 ASF License warnings.
180m 48s
Reason Tests
Failed junit tests hadoop.yarn.server.resourcemanager.scheduler.fair.converter.TestFSConfigToCSConfigConverterMain
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerWeightMode
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestQueueState
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestWorkPreservingRMRestartForNodeLabel
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestAbsoluteResourceConfiguration
hadoop.yarn.server.resourcemanager.TestWorkPreservingUnmanagedAM
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueACLs
hadoop.yarn.server.resourcemanager.resourcetracker.TestNMReconnect
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestAutoCreatedQueueDeletionPolicy
hadoop.yarn.server.resourcemanager.webapp.TestRMWebApp
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerSchedulingRequestUpdate
hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA
hadoop.yarn.server.resourcemanager.scheduler.constraint.algorithm.TestLocalAllocationTagsManager
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesSchedulerActivities
hadoop.yarn.server.resourcemanager.reservation.TestCapacitySchedulerPlanFollower
hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA
hadoop.yarn.server.resourcemanager.TestResourceTrackerService
hadoop.yarn.server.resourcemanager.scheduler.TestSchedulingWithAllocationRequestId
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesReservation
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerWithMultiResourceTypes
hadoop.yarn.server.resourcemanager.TestApplicationMasterServiceFair
hadoop.yarn.server.resourcemanager.security.TestClientToAMTokens
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesConfigurationMutation
hadoop.yarn.server.resourcemanager.TestAMAuthorization
hadoop.yarn.server.resourcemanager.scheduler.fair.converter.TestConvertedConfigValidator
hadoop.yarn.server.resourcemanager.TestResourceManagerMXBean
hadoop.yarn.server.resourcemanager.security.TestRMDelegationTokens
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestQueueStateManager
hadoop.yarn.server.resourcemanager.TestRMHAForNodeLabels
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerNewQueueAutoCreation
hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServiceAppsNodelabel
hadoop.yarn.server.resourcemanager.TestApplicationCleanup
hadoop.yarn.server.resourcemanager.rmcontainer.TestRMContainerImpl
hadoop.yarn.server.resourcemanager.scheduler.TestAbstractYarnScheduler
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestQueueConfigurationAutoRefreshPolicy
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerAutoQueueCreation
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerConfigValidator
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesSchedulerActivitiesWithMultiNodesEnabled
hadoop.yarn.server.resourcemanager.federation.TestFederationRMStateStoreService
hadoop.yarn.server.resourcemanager.TestDecommissioningNodesWatcher
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerAmbiguousLeafs
hadoop.yarn.server.resourcemanager.TestAppManager
hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRMRPCResponseId
hadoop.yarn.server.resourcemanager.webapp.TestAppPage
hadoop.yarn.server.resourcemanager.TestApplicationMasterLauncher
hadoop.yarn.server.resourcemanager.TestSignalContainer
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueMappingFactory
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerMultiNodesWithPreemption
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerSurgicalPreemption
hadoop.yarn.server.resourcemanager.TestResourceManager
hadoop.yarn.server.resourcemanager.TestRMHAMetrics
hadoop.yarn.server.resourcemanager.TestContainerResourceUsage
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestAbsoluteResourceWithAutoQueue
hadoop.yarn.server.resourcemanager.TestApplicationMasterServiceCapacity
hadoop.yarn.server.resourcemanager.scheduler.policy.TestFairOrderingPolicy
hadoop.yarn.server.resourcemanager.TestRMEmbeddedElector
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestLeafQueue
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestQueueManagementDynamicEditPolicy
hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRMRPCNodeUpdates
hadoop.yarn.server.resourcemanager.TestRMHA
hadoop.yarn.server.resourcemanager.TestOpportunisticContainerAllocatorAMService
hadoop.yarn.server.resourcemanager.TestRMStoreCommands
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestReservations
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySched
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerAutoCreatedQueuePreemption
hadoop.yarn.server.resourcemanager.rmapp.attempt.TestAMLivelinessMonitor
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodeLabels
hadoop.yarn.server.resourcemanager.TestRM
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerDynamicBehavior
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler
hadoop.yarn.server.resourcemanager.TestNodeBlacklistingOnAMFailures
hadoop.yarn.server.resourcemanager.security.TestAMRMTokens
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationLimits
hadoop.yarn.server.resourcemanager.TestRMAdminService
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerNodeLabelUpdate
hadoop.yarn.server.resourcemanager.monitor.capacity.TestProportionalCapacityPreemptionPolicy
hadoop.yarn.server.resourcemanager.TestReservationSystemWithRMHA
hadoop.yarn.server.resourcemanager.TestRMTimelineService
hadoop.yarn.server.resourcemanager.scheduler.constraint.TestAllocationTagsManager
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestIncreaseAllocationExpirer
hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer
hadoop.yarn.server.resourcemanager.scheduler.TestSchedulerUtils
hadoop.yarn.server.resourcemanager.rmapp.TestApplicationLifetimeMonitor
hadoop.yarn.server.resourcemanager.TestRMHATimelineCollectors
hadoop.yarn.server.resourcemanager.webapp.TestRedirectionErrorPage
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerOvercommit
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestQueueMappings
hadoop.yarn.server.resourcemanager.scheduler.TestSchedulerHealth
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesForCSWithPartitions
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationLimitsByPartition
hadoop.yarn.server.resourcemanager.TestClientRMService
hadoop.yarn.server.resourcemanager.resource.TestResourceProfiles
hadoop.yarn.server.resourcemanager.monitor.TestSchedulingMonitor
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation
hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.TestLeveldbConfigurationStore
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerMultiNodes
hadoop.yarn.server.resourcemanager.TestRMProxyUsersConf
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerAsyncScheduling
hadoop.yarn.server.resourcemanager.TestApplicationACLs
hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart
hadoop.yarn.server.resourcemanager.nodelabels.TestRMDelegatedNodeLabelsUpdater
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.TestZKConfigurationStore
hadoop.yarn.server.resourcemanager.volume.csi.TestVolumeProcessor
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerResizing
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerMaxParallelApps
hadoop.yarn.server.resourcemanager.scheduler.constraint.TestPlacementProcessor
hadoop.yarn.server.resourcemanager.placement.TestPlacementManager
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriority
hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification
hadoop.yarn.server.resourcemanager.TestLeaderElectorService
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerWorkflowPriorityMapping
hadoop.yarn.server.resourcemanager.rmapp.TestNodesListManager
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySchedDynamicConfig
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestSchedulingRequestContainerAllocation
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriorityACLs
hadoop.yarn.server.resourcemanager.TestRMRestart
hadoop.yarn.server.resourcemanager.scheduler.constraint.TestPlacementConstraintsUtil
hadoop.yarn.server.resourcemanager.reservation.TestReservationSystem
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestQueueParsing
hadoop.yarn.server.resourcemanager.webapp.TestNodesPage
hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerLazyPreemption
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCSAllocateCustomResource
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestSchedulingRequestContainerAllocationAsync
hadoop.yarn.server.resourcemanager.nodelabels.TestRMNodeLabelsManager
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3438/2/artifact/out/Dockerfile
GITHUB PR #3438
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname Linux 576325d05996 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / c622865
Default Java Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3438/2/testReport/
Max. process+thread count 5407 (vs. ulimit of 5500)
modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3438/2/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@JackWangCS JackWangCS changed the title YARN-10951. Move async scheduling logic to a new AsyncSchedulingManager YARN-10951. Move async scheduling fields to a new AsyncSchedulingConfiguration Sep 21, 2021
@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 44s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 💚 mvninstall 32m 15s trunk passed
+1 💚 compile 1m 4s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 compile 0m 57s trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 checkstyle 0m 48s trunk passed
+1 💚 mvnsite 1m 1s trunk passed
+1 💚 javadoc 0m 46s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javadoc 0m 44s trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 spotbugs 1m 58s trunk passed
+1 💚 shadedclient 21m 3s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 54s the patch passed
+1 💚 compile 0m 55s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javac 0m 55s the patch passed
+1 💚 compile 0m 46s the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 javac 0m 46s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 0m 39s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 0 new + 112 unchanged - 1 fixed = 112 total (was 113)
+1 💚 mvnsite 0m 50s the patch passed
+1 💚 javadoc 0m 36s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javadoc 0m 35s the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 spotbugs 2m 0s the patch passed
+1 💚 shadedclient 20m 50s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 96m 13s hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚 asflicense 0m 32s The patch does not generate ASF License warnings.
185m 37s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3438/3/artifact/out/Dockerfile
GITHUB PR #3438
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname Linux 98d28b8e3d07 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 6465d7d
Default Java Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3438/3/testReport/
Max. process+thread count 918 (vs. ulimit of 5500)
modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3438/3/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@JackWangCS JackWangCS marked this pull request as ready for review September 22, 2021 02:40
@szilard-nemeth
Copy link
Contributor

Hi @JackWangCS ,
Closing this in favor of #3800
The upstream codebase had changes a lot so I had to redo all the changes from scratch.
Thanks for your initial contribution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
3 participants
@JackWangCS @hadoop-yetus @szilard-nemeth