JSON Writer leave json file in correct state #205
Comments
|
Sounds plausible. Would you provide a pull request? |
|
I'm having some trouble building tinylog.. Do you have an idea why those errors are occuring? I assume this leads to the following error when trying to only build env |
|
It looks like that the Maven profile "dev" was not active. However, I just updated the parent pom.xml, so that activation of this Maven profile is no longer required. I hope it will compile now. |
- Everytime an entry is written, the needed closing array bracket is written too. - Every entry checks for itself, if it's the first one and appends a comma accordingly - No post processing is needed anymore - To append, the file is checked for a valid array
|
Yes, your changes fixed it, thanks.:) Please have a look at the PR. |
|
Currently, the JSON writer will place the correct closing |
|
Is there a reason why we try to enclose the file's content with I'd also recommend to not tie JSON with a file, it should be possible to write it anywhere (e.g. standard out). |
|
I think I have the same problem: Any suggestions? |
|
@davidmoshal usually the check for a valid json executes only once during initialization, e.g. on startup. Was this state of the file created by the writer? If yes, is there any chance for reproduction? |
The initial idea was to generate a JSON conform output so that it can be parsed by JSON parsers. Is the JSON line format valid JSON, i.e. parseable by common parsers?? |
https://en.wikipedia.org/wiki/JSON_streaming#Line-delimited_JSON
Generally the parser (like say Jackson core JsonParser) doesn't need to support it directly. That is, the log reader/publisher reads lines and then for each line uses the parser (say Jackson JsonParser) to parse the line. In this way the check for valid JSON is per line. Things like ElasticSearch take line delimited JSON. So for example, if the logging output is being send to ElasticSearch you can send it to the bulk API: |
|
I agree that any large scale, high performance or streaming based use case may require this lighter format. However, I wanted to avoid solutions shown in this SO question only for simply parsing json. I do think that a configurable format for JSON or e.g. LDJSON would make sense. What are your thoughts? |
I think that is somewhat unrelated in that I don't think we ever in practice turn X Mb of logs into a So as I see it, I don't think that SO answer is practical, useful, helpful wrt logs, in that it will work with really small amounts of data only. With logs we are more conceptually dealing with a stream than a list.
My expectation is that LDJSON is the expected practical output for logging as JSON. That is what I've seen and used in the past via ElasticSearch ECS - Elastic Common Schema.
https://www.elastic.co/guide/en/ecs-logging/overview/master/intro.html |
|
Thank for all the great and useful input. I see four necessary steps:
|
[
{
"level" : "INFO" ,
"source" : "au21.engine.framework.commands.EngineCommandHandler.handle()" ,
"message" : "{\"session_id\":\"\",\"command_json\":[],\"simplename\":\"DbDeleteAuctionsCommand\",\"classname\":\"au21.engine.domain.de.commands.DbDeleteAuctionsCommand\"}"
},
// ... removed hundreds of similar objects ...
{
"level" : "INFO" ,
"source" : "au21.engine.framework.commands.EngineCommandHandler.handle()" ,
"message" : "{\"session_id\":\"\",\"command_json\":[Initially I thought that it was due to the test runner killing the process before it could flush the last message to the log file. |
|
@davidmoshal Is According to the algorithm, something should be written also afterwards. It would be also interesting to know what JSON object should be written there as message. |
|
This closed issue has been locked automatically. |
Describe the issue
Currently, the writer leaves the
]out and keeps the last,which is invalid json. I think we should let the json log file always stay in a syntactic correct state. One would not be able to parse the file while it does not match JSON spec.Currently the close method performs the last steps needed to bring it in that state by calling
postProcess. Maybe we could handle the logic ofpostProcesseverytime we write an entry. The new close method should then only handlewriter.flush()andwriter.close()The file currently looks like this:
[ { ... },To Reproduce
Environment
tinylog version: 2.4
Java version: 15
The text was updated successfully, but these errors were encountered: