Skip to content
Permalink
Browse files

[SPARK-23089][STS] Recreate session log directory if it doesn't exist

## What changes were proposed in this pull request?

When creating a session directory, Thrift should create the parent directory (i.e. /tmp/base_session_log_dir) if it is not present. It is common that many tools delete empty directories, so the directory may be deleted. This can cause the session log to be disabled.

This was fixed in HIVE-12262: this PR brings it in Spark too.

## How was this patch tested?

manual tests

Author: Marco Gaido <marcogaido91@gmail.com>

Closes #20281 from mgaido91/SPARK-23089.

(cherry picked from commit e41400c)
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
  • Loading branch information...
mgaido91 authored and cloud-fan committed Jan 19, 2018
1 parent ef7989d commit b7a81999df8f43223403c77db9c1aedddb58370d
@@ -223,6 +223,18 @@ private void configureSession(Map<String, String> sessionConfMap) throws HiveSQL

@Override
public void setOperationLogSessionDir(File operationLogRootDir) {
if (!operationLogRootDir.exists()) {
LOG.warn("The operation log root directory is removed, recreating: " +
operationLogRootDir.getAbsolutePath());
if (!operationLogRootDir.mkdirs()) {
LOG.warn("Unable to create operation log root directory: " +
operationLogRootDir.getAbsolutePath());
}
}
if (!operationLogRootDir.canWrite()) {
LOG.warn("The operation log root directory is not writable: " +
operationLogRootDir.getAbsolutePath());
}
sessionLogDir = new File(operationLogRootDir, sessionHandle.getHandleIdentifier().toString());
isOperationLogEnabled = true;
if (!sessionLogDir.exists()) {

0 comments on commit b7a8199

Please sign in to comment.
You can’t perform that action at this time.