Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AMBARI-22781. Fix OneFS blueprint installation (amagyar) #485

Merged
merged 3 commits into from Mar 2, 2018

Conversation

zeroflag
Copy link
Contributor

What changes were proposed in this pull request?

OneFS blueprint installation failed because SingleHostTopologyUpdater tried to determine the namenode host which doesn't exist in a onefs setup.

How was this patch tested?

I used the following minimal blueprint to create a cluster with OneFS.

{
  "configurations": [
      {
          "onefs": {
              "properties": {
                  "onefs_host": "scisilon.fqdn"
              }
          }
      }
  ],
  "host_groups": [{
      "name": "host_group_1",
      "components": [
          { "name": "ZOOKEEPER_SERVER" },
          { "name": "ZOOKEEPER_CLIENT" },
          { "name": "ONEFS_CLIENT" }
      ],
      "cardinality": "1"
  }],
  "Blueprints": {
      "stack_name": "HDP",
      "stack_version": "2.6"
  }
 }

}

@Test
public void testHasHadoopCompatibleFsWhenThereIsNoHCFS() throws Exception {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this should be testDoesNotHave...

@@ -1542,6 +1542,10 @@ public String updateForClusterCreate(String propertyName,
return origValue;
}

if (topology.hasHadoopCompatibleFileSystem()) {
return origValue;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this condition should also include a check that the component being processed is one from OneFS. Otherwise it will fail to throw IllegalArgumentException for other components when it should.

To verify this, make the following change in BlueprintConfigurationProcessorTest and run it:

@@ -8244,7 +8244,11 @@ public class BlueprintConfigurationProcessorTest extends EasyMockSupport {

     replay(bp, topologyRequestMock);

-    ClusterTopology topology = new ClusterTopologyImpl(ambariContext, topologyRequestMock);
+    ClusterTopology topology = new ClusterTopologyImpl(ambariContext, topologyRequestMock) {
+      public boolean hasHadoopCompatibleFileSystem() {
+        return true;
+      }
+    };
     topology.setConfigRecommendationStrategy(ConfigRecommendationStrategy.NEVER_APPLY);

     return topology;

Result:

[ERROR]   BlueprintConfigurationProcessorTest.testDoUpdateForClusterCreate_SingleHostProperty__MissingComponent:2344 IllegalArgumentException should have been thrown
[ERROR]   BlueprintConfigurationProcessorTest.testDoUpdateForClusterCreate_SingleHostProperty__MultipleMatchingHostGroupsError:2386 IllegalArgumentException should have been thrown

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch.

@asfgit
Copy link

asfgit commented Feb 27, 2018

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Github-PullRequest-Builder/851/
Test PASSed.

@asfgit
Copy link

asfgit commented Feb 28, 2018

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Github-PullRequest-Builder/861/
Test FAILed.
Test FAILured.

@zeroflag
Copy link
Contributor Author

retest this please

@asfgit
Copy link

asfgit commented Feb 28, 2018

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Github-PullRequest-Builder/864/
Test PASSed.

@@ -259,6 +261,14 @@ public ComponentInfo getComponentInfo(String component) {
return componentInfo;
}

public Optional<ServiceInfo> getServiceInfo(String serviceName) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Javadoc.

@asfgit
Copy link

asfgit commented Feb 28, 2018

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Github-PullRequest-Builder/867/
Test FAILed.
Test FAILured.

@zeroflag
Copy link
Contributor Author

zeroflag commented Mar 1, 2018

retest this please

@asfgit
Copy link

asfgit commented Mar 1, 2018

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Github-PullRequest-Builder/907/
Test PASSed.

@zeroflag zeroflag merged commit 991a219 into apache:trunk Mar 2, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants