Skip to content
This repository has been archived by the owner on Jun 7, 2021. It is now read-only.

APEXCORE-488: Issues in SSL communication with StrAM #357

Merged
merged 1 commit into from
Jul 19, 2016
Merged

APEXCORE-488: Issues in SSL communication with StrAM #357

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
7 changes: 6 additions & 1 deletion engine/src/main/java/com/datatorrent/stram/StreamingAppMasterService.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,7 @@
import com.datatorrent.stram.security.StramDelegationTokenManager;
import com.datatorrent.stram.security.StramUserLogin;
import com.datatorrent.stram.security.StramWSFilterInitializer;
import com.datatorrent.stram.util.ConfigUtils;
import com.datatorrent.stram.util.SecurityUtils;
import com.datatorrent.stram.webapp.AppInfo;
import com.datatorrent.stram.webapp.StramWebApp;
Expand Down Expand Up @@ -614,7 +615,11 @@ protected void serviceStart() throws Exception
}
WebApp webApp = WebApps.$for("stram", StramAppContext.class, appContext, "ws").with(config).start(new StramWebApp(this.dnmgr));
LOG.info("Started web service at port: " + webApp.port());
this.appMasterTrackingUrl = NetUtils.getConnectAddress(webApp.getListenerAddress()).getHostName() + ":" + webApp.port();
appMasterTrackingUrl = NetUtils.getConnectAddress(webApp.getListenerAddress()).getHostName() + ":" + webApp.port();

if (ConfigUtils.isSSLEnabled(config)) {
appMasterTrackingUrl = "https://" + appMasterTrackingUrl;
}
LOG.info("Setting tracking URL to: " + appMasterTrackingUrl);
} catch (Exception e) {
LOG.error("Webapps failed to start. Ignoring for now:", e);
Expand Down
9 changes: 8 additions & 1 deletion engine/src/main/java/com/datatorrent/stram/client/StramAgent.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -204,7 +204,14 @@ private UriBuilder getStramWebURIBuilder(WebServicesClient webServicesClient, St
if (info != null) {
//ws = wsClient.resource("http://" + info.appMasterTrackingUrl).path(WebServices.PATH).path(info.version).path("stram");
// the filter should convert to the right version
ub = UriBuilder.fromUri("http://" + info.appMasterTrackingUrl).path(WebServices.PATH).path(WebServices.VERSION).path("stram");
String url;
if (!info.appMasterTrackingUrl.startsWith("http://")
&& !info.appMasterTrackingUrl.startsWith("https://")) {
url = "http://" + info.appMasterTrackingUrl;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should be logged, saying falling back to http

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it is a secure cluster, then fallback should be https?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its not falling back. When protocol scheme is not specified in Tracking URL, it shall use the default protocol scheme i.e. 'http://'

} else {
url = info.appMasterTrackingUrl;
}
ub = UriBuilder.fromUri(url).path(WebServices.PATH).path(WebServices.VERSION).path("stram");
WebServicesVersionConversion.Converter versionConverter = WebServicesVersionConversion.getConverter(info.version);
if (versionConverter != null) {
VersionConversionFilter versionConversionFilter = new VersionConversionFilter(versionConverter);
Expand Down
19 changes: 17 additions & 2 deletions engine/src/main/java/com/datatorrent/stram/util/ConfigUtils.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,15 +68,30 @@ public static String getRMUsername(Configuration conf)
return principal;
}

public static String getSchemePrefix(YarnConfiguration conf)
public static boolean isSSLEnabled(Configuration conf)
{
if (HttpConfig.Policy.HTTPS_ONLY == HttpConfig.Policy.fromString(
conf.get(YarnConfiguration.YARN_HTTP_POLICY_KEY, YarnConfiguration.YARN_HTTP_POLICY_DEFAULT))) {
return true;
}
return false;
}

public static String getSchemePrefix(Configuration conf)
{
if (HttpConfig.Policy.HTTPS_ONLY == HttpConfig.Policy.fromString(conf.get(YarnConfiguration.YARN_HTTP_POLICY_KEY, YarnConfiguration.YARN_HTTP_POLICY_DEFAULT))) {
if (isSSLEnabled(conf)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think @vrozov also pointed out this earlier, but this function is not required.. I see all the usages passing Yarnconfiguration

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean the getSchemePrefix(YarnConfiguration...) method? Needed for binary compatibility.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you good with that explanation @gauravgopi123

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@PramodSSImmaneni : I am talking about getSchemePrefix(Configuration) method...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@PramodSSImmaneni : In-fact I won't make any change in this file. I would just add following code in StreamingAppMasterService

if ("https//:".equals.(ConfigUtils. getSchemePrefix(config))) { appMasterTrackingUrl = "https://" + appMasterTrackingUrl; }

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@PramodSSImmaneni : I missed one thing. New method is needed as AppMaster has Configuration and YarnConfiguration object..

Changes look good

return "https://";
} else {
return "http://";
}
}

@Deprecated
public static String getSchemePrefix(YarnConfiguration conf)
{
return getSchemePrefix((Configuration)conf);
}

public static String getYarnLogDir()
{
if (yarnLogDir != null) {
Expand Down