HIVE-24426. Spark job fails with fixed LlapTaskUmbilicalServer port. #1705
Conversation
kgyrtkirk
added
tests pending
tests passed
and removed
tests failed
tests pending
labels
| portFound = true; | ||
| } else { | ||
| int maxPort = Integer.parseInt(portRange[1]); | ||
| for (int i = minPort; i < maxPort; i++) { |
There was a problem hiding this comment.
can you use RangeValidator from HiveConf and make sure the min is 1024 and max is 65535 which is the port range for user space ports.
| portFound = true; | ||
| break; | ||
| } catch (BindException be) { | ||
| // Ignore and move ahead, in search of a free port. |
There was a problem hiding this comment.
Log at warn level to say which port is being tried and what error message received for debugging.
| .setNumHandlers(numHandlers).setSecretManager(jobTokenSecretManager) | ||
| .build(); | ||
| if (conf | ||
| .getBoolean(CommonConfigurationKeysPublic.HADOOP_SECURITY_AUTHORIZATION, |
There was a problem hiding this comment.
nit: you could move this to a private variable instead of searching conf object every time (now that startServer is in a loop)
| "LLAP task umbilical server RPC port"), | ||
| LLAP_TASK_UMBILICAL_SERVER_PORT("hive.llap.daemon.umbilical.port", "0", | ||
| "LLAP task umbilical server RPC port or range of ports to try in case " | ||
| + "the first port is occupied"), |
There was a problem hiding this comment.
add what the min and max values for the range can be (1024 - 65535)
| portFound = true; | ||
| break; | ||
| } catch (BindException be) { | ||
| // Ignore and move ahead, in search of a free port. |
| startServerInternal(conf, minPort, numHandlers, jobTokenSecretManager); | ||
| portFound = true; | ||
| } else { | ||
| int maxPort = Integer.parseInt(portRange[1]); |
There was a problem hiding this comment.
same here (use RangeValidator)
| .setNumHandlers(numHandlers).setSecretManager(jobTokenSecretManager) | ||
| .build(); | ||
|
|
||
| if (conf |
There was a problem hiding this comment.
nit: same here to move this to private variable.
kgyrtkirk
added
tests pending
tests passed
and removed
tests passed
tests pending
labels
|
Thanx @prasanthj for the review, I have handled the review comments. Please have a look |
kgyrtkirk
added
tests pending
tests unstable
and removed
tests passed
tests pending
labels
kgyrtkirk
added
tests pending
tests passed
and removed
tests unstable
tests pending
labels
|
Thanx @prasanthj for the review, I have added the success log as well. Please have a check. |
https://issues.apache.org/jira/browse/HIVE-24426