Log decryption Error and saved to fail folder

[zhinobu38]

Hi,

I'm running on Windows Server 2012R2 with Python 3.8.6 version.
Some log file cause below error at decryption state.
Could you please suggest on how to fix this issue?

Thank you.

2020-11-04 15:00:07,760 INFO Successfully downloaded file from URL https://logs1.incapsula.com/6827_1XXX7/6827_21977.log
2020-11-04 15:00:07,760 WARNING Keys Dir: C:\incapsula\logs-downloader-master2-Test\config\keys
2020-11-04 15:00:07,776 INFO Syslog enabled, using UDP
2020-11-04 15:00:14,870 INFO Saving file 6827_21977.log locally to the 'fail' folder 'charmap' codec can't encode character '\u014c' in position 0: character maps to Traceback (most recent call last):
File "C:\incapsula\logs-downloader-master2-Test\script\LogsDownloader.py", line 222, in handle_file
self.handle_log_decrypted_content(logfile, decrypted_file)
File "C:\incapsula\logs-downloader-master2-Test\script\LogsDownloader.py", line 276, in handle_log_decrypted_content
local_file.writelines(decrypted_file)
File "C:\Python38\lib\encodings\cp874.py", line 19, in encode
return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u014c' in position 0: character maps to

[zhinobu38]

As I searchIt seems to be window's terminal issue that not support some encoded character

So I try to change the script, line 255 as below, to avoid this issue.
from : decrypted_file = decrypted_file.decode('utf-8')
to : decrypted_file = decrypted_file.decode('utf-8', 'ignore')

But it still running the same error.
Please help suggest.

Thank you so much in advanced.

the log which i test today is below :
2020-11-06 11:01:49,027 INFO Sleeping for 2 seconds before fetching the next logs file
2020-11-06 11:01:57,216 INFO Successfully downloaded file from URL https://logs1.incapsula.com/6827_XXXXX/6827_26545.log
2020-11-06 11:01:57,216 WARNING Keys Dir: C:\incapsula\logs-downloader-master2-Test\config\keys
2020-11-06 11:01:57,232 INFO Syslog enabled, using UDP
2020-11-06 11:01:58,966 INFO Saving file 6827_26545.log locally to the 'fail' folder 'charmap' codec can't encode character '\xe3' in position 0: character maps to Traceback (most recent call last):
File "C:\incapsula\logs-downloader-master2-Test\script\LogsDownloader.py", line 222, in handle_file
self.handle_log_decrypted_content(logfile, decrypted_file)
File "C:\incapsula\logs-downloader-master2-Test\script\LogsDownloader.py", line 277, in handle_log_decrypted_content
local_file.writelines(decrypted_file)
File "C:\Python38\lib\encodings\cp874.py", line 19, in encode
return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\xe3' in position 0: character maps to

2020-11-06 11:01:58,966 ERROR Failed to download file 6827_26545.log. Error is - write() argument must be str, not bytes , Traceback (most recent call last):
File "C:\incapsula\logs-downloader-master2-Test\script\LogsDownloader.py", line 222, in handle_file
self.handle_log_decrypted_content(logfile, decrypted_file)
File "C:\incapsula\logs-downloader-master2-Test\script\LogsDownloader.py", line 277, in handle_log_decrypted_content
local_file.writelines(decrypted_file)
File "C:\Python38\lib\encodings\cp874.py", line 19, in encode
return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\xe3' in position 0: character maps to

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\incapsula\logs-downloader-master2-Test\script\LogsDownloader.py", line 143, in get_log_files
success = self.handle_file(next_file)
File "C:\incapsula\logs-downloader-master2-Test\script\LogsDownloader.py", line 233, in handle_file
file.write(result[1])
TypeError: write() argument must be str, not bytes

2020-11-06 11:02:46,998 INFO Successfully downloaded file from URL https://logs1.incapsula.com/6827_XXXXX/6827_26545.log

[zhinobu38]

I found solution from Imperva Communities. The issue is about encoding issue with Windows system running in Asia locale that lead to unsupported encoding character.

To fix this replace inline 275 on Logdownloader.py with :
local_file = open(self.config.PROCESS_DIR + filename, "a+", encoding="UTF-8")

This address my problem.