Just wanted to say thanks

[cooleydw494]

I've been building a GPT-powered automation framework that does a lot of file-writing using function calling. It has been a huge hassle trying to figure out a good way to handle broken JSON, and it happens too often to ignore. Using json-repair worked better than my own solution, and allowed me to remove some bulky, annoying-to-maintain code.

So Thanks!

P.S. I got even better results from simply adding "using JSON compatible unicode characters only" to the description of the relevant parameter of my function (again, for GPT function calling). When there's an issue its usually handled pretty well by json-repair, but I went from having issues 30-50% of the time, to 5% after doing that.

[brettrp]

For simple output I actually gave up on JSON and got it to generate simple CSV instead, and then turned that back into structured data afterwards.

I suspect one issue is that if you want a higher temperature to get varied content, then the LLM is not going to just vary the content of the JSON more, but also its structure.

[cooleydw494]

makes sense. Sounds like you're using LLMs a little closer to the metal potentially than I am. If you use GPT function calling, the parameters always come back as JSON, so not much else to be done except try to minimize issues. Then, I've found that getting results that don't include extraneous text when not using function calling (just generating text response) is quite possible but also inconsistent and bulky.

the function calling is allowing me to coerce the response in more significant (usually) ways, so I'm kind of locked in! For instance, I often use a function that doesn't even do anything. I have a finalize_output function that takes a string and returns that string unchanged. Funny how LLMs work, GPT's pre-training for function calling allows the simple fact that it is a function to align it better to the fact that I'm asking for no extraneous data!

It's a bit late here so maybe I didn't articulate that well, but its super interesting.

[mangiucugna]

Hi David, thank you.
If you would be so kind to share some examples of invalid json that is not repaired, I will be very happy to add them to the test cases and make sure those are fixed. This is a passion project and my aim was always to fix 100% of the cases (at least with LLMs)

[cooleydw494]

I need to get to bed but I ran an automation with some basic prints added in. I can figure out a better logging method tomorrow so you can actually see the characters. In this example (the common case it seems) the JSON is being fixed, maybe the best that it can be, but it seems that some kind of newline character is being removed when I'd hope it would be possible to replace it with something compatible. Apologies if this doesn't make any sense, I really need to get to sleep haha.

Context: I'm testing an automation that scaffolds a project based on a txt file with architecture details, using a twitter posting tool as an example (I know, this wouldn't really work because they've changed their API since 2021).

[cooleydw494]

mostly unhelpful in the way that it is printed, but points to a potential explanation. Will try to get some better logging tomorrow.

[mangiucugna]

Thanks for the output, indeed json_repair was getting rid of newlines and breaking everything in your python function. I have added your function to the tests and published 0.1.9 that fixes that test case. Let me know if it works for you!

[mangiucugna]

Also maybe to clarify, I kept the part in which the library removes trailing spaces at the start and end of the string because those are not valid

[cooleydw494]

To @brettrp's point I can imagine a case where the original GPT response actually included invalid characters, and they WERE correctly removed. I think it would log to the console the same either way the way that I wrote it so I can't be sure. I'll investigate tonight.

I see that you fixed the problem with removing escaped newline characters (nice!). That likely fixes my issue (the test proves it fixes removal of valid newlines, if that is what I had), but if it's still happening my guess would be that there are invalid characters that would be good to replace with escaped newlines potentially. I'll let you know.

[mangiucugna]

I have tested a bunch of unicode characters and the output is all escaped, I would be curious to see which characters (if any) break the lib.
If you can find some examples please let me know, and of course let me know if the latest version fixes your issue!

[cooleydw494]

Will do!

[cooleydw494]

Btw @mangiucugna, I forgot to update here, but I don't think there was any edge case. Looks like everything is working great now! Thanks so much

[ArslanKAS]

Thank you so much for this library @mangiucugna . I was struggling a lot with LLM responses when asking them to give me JSON outputs. Wrote multiple functions to handle all sorts of broken Json but still that wasn't enough. Your library seems to handle all sorts of broken jsons so far. Great work. Loved it

[mangiucugna]

Very happy to hear! You are welcome!

[ArslanKAS]

Could you please add a Logo or something for your package? I think more people should know your package and it needs to be integrated into other LLM-Tools such as LangChain, LLaMaIndex and Ollama. I've been testing LLaMa3 using Ollama and their JSON-formatter is super slow. The LLM time suddenly jumps from 8 seconds to 90 seconds when I turn that option on in Ollama. Maybe if they integrate your package then it'd come down to only 10 seconds or so.

[mangiucugna]

sincere question, why you think a logo would help? I am not keen on spending too much time on design since all the free time I can allow for this package I spend in fixing issue and adding features.

I would be happy to help integrating into other packages if someone is spearheading the conversation there, I don't want that to come across as me trying to push my own view.

[ArslanKAS]

Apologies, just wanted the logo so it'd appeal more people. I can understand you would've a busy life and can't spend much time but I was having a thought that maybe if this package isn't updated in the future then it'd be real loss, so in order to keep it alive more people should know about it and maybe that'd motivate you and others to contribute more into it.