Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Logging of insertion errors failing on Windows #59

Closed
SecSuperN0va opened this issue Jun 11, 2022 · 0 comments
Closed

Logging of insertion errors failing on Windows #59

SecSuperN0va opened this issue Jun 11, 2022 · 0 comments
Assignees
Labels
bug Something isn't working

Comments

@SecSuperN0va
Copy link

When a record fails insertion, I believe a log message should be written to *_invalid.log.
On a windows machine, the formatting of the timestamp leads to an invalid file path being generated (: characters are not valid in Windows file-paths), and the logger being unable to create the file to record the insertion failure.

An example of the exception is as follows:

00:31:46.394 [main] INFO  com.bayer.dt.tdl.loader - buffered-read: total: 2,929, rate: 29,585.86/s
java.io.FileNotFoundException: 2022-06-12-00:31:46\files_invalid.log (The filename, directory name, or volume label syntax is incorrect)
        at java.base/java.io.FileOutputStream.open0(Native Method)
        at java.base/java.io.FileOutputStream.open(FileOutputStream.java:298)
        at java.base/java.io.FileOutputStream.<init>(FileOutputStream.java:237)
        at java.base/java.io.FileOutputStream.<init>(FileOutputStream.java:158)
        at java.base/java.io.FileWriter.<init>(FileWriter.java:82)
        at io.FileLogger.logInvalid(FileLogger.java:64)
        at generator.EntityGenerator.write(EntityGenerator.java:70)
        at loader.AsyncLoaderWorker.lambda$asyncWrite$0(AsyncLoaderWorker.java:415)
        at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
        at loader.AsyncLoaderWorker.lambda$asyncWrite$1(AsyncLoaderWorker.java:413)
        at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
java.io.FileNotFoundException: 2022-06-12-00:31:46\files_invalid.log (The filename, directory name, or volume label syntax is incorrect)
        at java.base/java.io.FileOutputStream.open0(Native Method)
        at java.base/java.io.FileOutputStream.open(FileOutputStream.java:298)
        at java.base/java.io.FileOutputStream.<init>(FileOutputStream.java:237)
        at java.base/java.io.FileOutputStream.<init>(FileOutputStream.java:158)
        at java.base/java.io.FileWriter.<init>(FileWriter.java:82)
        at io.FileLogger.logInvalid(FileLogger.java:64)
        at generator.EntityGenerator.write(EntityGenerator.java:70)
        at loader.AsyncLoaderWorker.lambda$asyncWrite$0(AsyncLoaderWorker.java:415)
        at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
        at loader.AsyncLoaderWorker.lambda$asyncWrite$1(AsyncLoaderWorker.java:413)
        at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
java.io.FileNotFoundException: 2022-06-12-00:31:46\files_invalid.log (The filename, directory name, or volume label syntax is incorrect)
        at java.base/java.io.FileOutputStream.open0(Native Method)
        at java.base/java.io.FileOutputStream.open(FileOutputStream.java:298)
        at java.base/java.io.FileOutputStream.<init>(FileOutputStream.java:237)
        at java.base/java.io.FileOutputStream.<init>(FileOutputStream.java:158)
        at java.base/java.io.FileWriter.<init>(FileWriter.java:82)
        at io.FileLogger.logInvalid(FileLogger.java:64)
        at generator.EntityGenerator.write(EntityGenerator.java:70)
        at loader.AsyncLoaderWorker.lambda$asyncWrite$0(AsyncLoaderWorker.java:415)
        at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
        at loader.AsyncLoaderWorker.lambda$asyncWrite$1(AsyncLoaderWorker.java:413)
        at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
java.io.FileNotFoundException: 2022-06-12-00:31:46\files_invalid.log (The filename, directory name, or volume label syntax is incorrect)
        at java.base/java.io.FileOutputStream.open0(Native Method)
        at java.base/java.io.FileOutputStream.open(FileOutputStream.java:298)
        at java.base/java.io.FileOutputStream.<init>(FileOutputStream.java:237)
        at java.base/java.io.FileOutputStream.<init>(FileOutputStream.java:158)
        at java.base/java.io.FileWriter.<init>(FileWriter.java:82)
        at io.FileLogger.logInvalid(FileLogger.java:64)
        at generator.EntityGenerator.write(EntityGenerator.java:70)
        at loader.AsyncLoaderWorker.lambda$asyncWrite$0(AsyncLoaderWorker.java:415)
        at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
        at loader.AsyncLoaderWorker.lambda$asyncWrite$1(AsyncLoaderWorker.java:413)
        at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
@flyingsilverfin flyingsilverfin self-assigned this Jul 6, 2022
@flyingsilverfin flyingsilverfin added the bug Something isn't working label Jul 6, 2022
flyingsilverfin added a commit that referenced this issue Jul 7, 2022
## What is the goal of this PR?

Issue #59 reports that `:` cannot be used in file names for error logging on Windows. To fix this, we replace `:` with `_` in the log file name.

## What are the changes implemented in this PR?
* Replace `:` with `_` with in the FileLogger for invalid csv lines
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants