Adding a fix/workaround when fs.append() unable to return a valid outputstream #388
Conversation
| @@ -69,6 +69,18 @@ | |||
| try { | |||
| this.output = fs.append(path, bufferSize); | |||
| } catch (RemoteException e) { | |||
| if (e.getMessage().contains("not sufficiently replicated yet")) { | |||
There was a problem hiding this comment.
Question : I see that RemoteException supports optional error-codes. Not sure if you see an error-code for this case. If you do see an error-code, can we use that to match instead of string matching ?
There was a problem hiding this comment.
If you take a look at the error-codes, they are generic. To give you an example I ran the case to show the error code generated :
It's best to not rely on the generic error code.
There was a problem hiding this comment.
Sounds good. Thanks for the snapshot.
There was a problem hiding this comment.
can we pull the err msg string into a final variable..
| @@ -69,6 +69,18 @@ | |||
| try { | |||
| this.output = fs.append(path, bufferSize); | |||
| } catch (RemoteException e) { | |||
| if (e.getMessage().contains("not sufficiently replicated yet")) { | |||
There was a problem hiding this comment.
Sounds good. Thanks for the snapshot.
| @@ -69,6 +69,18 @@ | |||
| try { | |||
| this.output = fs.append(path, bufferSize); | |||
| } catch (RemoteException e) { | |||
| if (e.getMessage().contains("not sufficiently replicated yet")) { | |||
There was a problem hiding this comment.
can we pull the err msg string into a final variable..
n3nash
force-pushed
the
hdfs_append_issue_fix
branch
from
b7c1822
to
219ed0f
Compare
|
@vinothchandar addressed your comment. |
n3nash
force-pushed
the
hdfs_append_issue_fix
branch
from
219ed0f
to
88e3718
Compare
Depends on #387 to be merged first.
Added a fix to handle unable to append due to under-replicated issue. If such an issue is encountered, a new log file is created (rolled over)
@vinothchandar @bvaradar