[AMBARI-24926] Apply user-defined configuration for Add Service request.

[adoroszlai]

What changes were proposed in this pull request?

Apply configuration specified in Add Service request (cluster- and service-level), instead of using stack defaults.

Config groups and stack advisor config recommendations are not yet implemented.

https://issues.apache.org/jira/browse/AMBARI-24926

How was this patch tested?

Tested API request on existing cluster:

$ curl -X POST -d @- "http://${AMBARI_SERVER}:8080/api/v1/clusters/TEST/services" <<-EOF
{
  "operation_type": "ADD_SERVICE",
  "stack_name": "HDP",
  "stack_version": "3.0",
  "components": [
    { "component_name": "KAFKA_BROKER", "fqdn": "c7402.ambari.apache.org" },
    { "component_name": "METRICS_COLLECTOR", "fqdn": "c7402.ambari.apache.org" },
    { "component_name": "METRICS_MONITOR", "fqdn": "c7401.ambari.apache.org" },
    { "component_name": "METRICS_MONITOR", "fqdn": "c7402.ambari.apache.org" }
  ],
  "configurations": [
    {
      "cluster-env": {
        "properties": {
          "custom-property": "whatever"
        }
      }
    },
    {
      "zoo.cfg": {
        "properties": {
          "syncLimit": "7"
        }
      }
    },
    {
      "core-site": {
        "properties": {
          "fs.defaultFS": "ZZZ"
        }
      }
    },
    {
      "kafka-broker": {
        "properties": {
          "zookeeper.connect": "c7401.ambari.apache.org:2181,c7402.ambari.apache.org:2181"
        }
      }
    },
    {
      "ams-site": {
        "properties": {
          "timeline.metrics.hbase.init.check.enabled": "false"
        }
      }
    }
  ]
}
EOF

HTTP/1.1 202 Accepted
...
{
  "href" : "http://c7401.ambari.apache.org:8080/api/v1/clusters/TEST/requests/12",
  "Requests" : {
    "id" : 12,
    "status" : "Accepted"
  }
}

After the request completed, I have verified that the configs from the request have been applied for both new and existing services by exporting the cluster as blueprint. Note that the config core-site was ignored, since HDFS was not present in the cluster, neither was it added as a new service in the request.

$ curl "http://c7401.ambari.apache.org:8080/api/v1/clusters/TEST?format=blueprint"
{
  "configurations" : [
    {
      "zoo.cfg" : {
        "properties" : {
          "syncLimit" : "7"
        }
      }
    },
    {
      "kafka-broker" : {
        "properties" : {
          "zookeeper.connect" : "%HOSTGROUP::host_group_1%:2181,%HOSTGROUP::host_group_2%:2181"
        }
      }
    },
    {
      "cluster-env" : {
        "properties" : {
          "custom-property" : "whatever"
        }
      }
    },
    {
      "ams-site" : {
        "properties" : {
          "timeline.metrics.hbase.init.check.enabled" : "false"
        }
      }
    }
  ],
  "host_groups" : [
    {
      "components" : [
        { "name" : "KAFKA_BROKER" },
        { "name" : "ZOOKEEPER_SERVER" },
        { "name" : "METRICS_MONITOR" },
        { "name" : "ZOOKEEPER_CLIENT" },
        { "name" : "METRICS_COLLECTOR" }
      ],
      "name" : "host_group_2",
      "cardinality" : "1"
    },
    {
      "components" : [
        { "name" : "ZOOKEEPER_SERVER" },
        { "name" : "AMBARI_SERVER" },
        { "name" : "METRICS_MONITOR" },
        { "name" : "ZOOKEEPER_CLIENT" }
      ],
      "name" : "host_group_1",
      "cardinality" : "1"
    }
  ],
  "Blueprints" : {
    "stack_name" : "HDP",
    "stack_version" : "3.0"
  }
}

Also checked service config versions on the web UI.

[asfgit]

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Github-PullRequest-Builder/4602/
Test FAILed.
Test FAILured.

[adoroszlai]

retest this please

[asfgit]

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Github-PullRequest-Builder/4603/
Test PASSed.