Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JSON returned by json_repair is not in expected format #20

Closed
pd2871 opened this issue Apr 8, 2024 · 6 comments
Closed

JSON returned by json_repair is not in expected format #20

pd2871 opened this issue Apr 8, 2024 · 6 comments
Assignees

Comments

@pd2871
Copy link

pd2871 commented Apr 8, 2024

Describe the bug
Returned json by json_repair is not in correct format which was expected.

To Reproduce
Steps to reproduce the behavior:
Run the file in python environment with langchain and json_repair installed

Expected behavior
I expected the repair_json to repair the output json.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • OS: Windows
  • Browser Chrome
  • Version 123.0.6312.106

Additional context
OpenAI has returned incomplete json due to limit in completion prompt. The json is missing ":" after 'answer40' at the end of the response. I expected json_repair to fix the issue but it has created unexpected json.

Code to get the json:

import json_repair
from langchain_core.messages.ai import AIMessage

response = AIMessage(content='', additional_kwargs={'function_call': {'arguments': '{""answer"":[{""traits"":""Female aged 60+"",""answer1"":""5"",""answer2"":""Labrador Retriever"",""answer3"":""75"",""answer4"":""Buddy"",""answer5"":""It was my late husband\'s nickname"",""answer6"":""Bud"",""answer7"":""Before 2020"",""answer8"":""yes"",""answer9"":""6"",""answer10"":""Instant connection"",""answer11"":""It\'s taught me more about unconditional love"",""answer12"":""5"",""answer13"":""Family Member"",""answer14"":""Because he\'s part of the family"",""answer15"":""Paw shake, about a week"",""answer16"":""14, in a dog bed by my bedside"",""answer17"":""yes"",""answer18"":""$1200"",""answer19"":""20%"",""answer20"":""60%"",""answer21"":""$100"",""answer22"":""Spent less in March"",""answer23"":""Monthly"",""answer24"":""Dry food"",""answer25"":""I know a lot about the brand"",""answer26"":""5"",""answer27"":""3"",""answer28"":""5"",""answer29"":""4"",""answer30"":""5"",""answer31"":""4"",""answer32"":""2"",""answer33"":""Blue Buffalo"",""answer34"":""High quality, my dog loves it, good value"",""answer35"":""5"",""answer36"":""4"",""answer37"":""5"",""answer38"":""4"",""answer39"":""5"",""answer40"":""4"",""answer41"":""3"",""answer42"":""3"",""answer43"":""I interact more"",""answer44"":""The ongoing pandemic"",""answer45"":""No"",""answer46"":""Yes"",""answer47"":""Car"",""answer48"":""Female"",""answer49"":""63"",""answer50"":""1"",""answer51"":""Graduate degree"",""answer52"":""Retired"",""answer53"":""Less than $25,000"",""answer54"":""Sarasota"",""answer55"":""Florida""},{""traits"":""Female aged 60+"",""answer1"":""3"",""answer2"":""Beagle"",""answer3"":""20"",""answer4"":""Scout"",""answer5"":""To Kill a Mockingbird is my favorite book"",""answer6"":""Scooty"",""answer7"":""After 2020"",""answer8"":""no"",""answer9"":""8"",""answer10"":""Overwhelmed with joy"",""answer11"":""I\'ve become more active"",""answer12"":""5"",""answer13"":""Best Friend"",""answer14"":""We do everything together"",""answer15"":""Stay, a couple of days"",""answer16"":""16, on the living room couch"",""answer17"":""yes"",""answer18"":""$800"",""answer19"":""30%"",""answer20"":""50%"",""answer21"":""$70"",""answer22"":""Spent more in March"",""answer23"":""Bi-weekly"",""answer24"":""Mix"",""answer25"":""I know just the basics about the brand"",""answer26"":""5"",""answer27"":""4"",""answer28"":""5"",""answer29"":""3"",""answer30"":""5"",""answer31"":""2"",""answer32"":""4"",""answer33"":""Purina"",""answer34"":""Affordable, accessible, and Scout enjoys it"",""answer35"":""4"",""answer36"":""5"",""answer37"":""4"",""answer38"":""3"",""answer39"":""5"",""answer40"":""2"",""answer41"":""4"",""answer42"":""2"",""answer43"":""No change"",""answer44"":""The pandemic, but we\'ve adapted"",""answer45"":""Yes"",""answer46"":""No"",""answer47"":""N/A-I\'m not planning to travel with my dog this Spring"",""answer48"":""Female"",""answer49"":""68"",""answer50"":""2"",""answer51"":""Bachelor\'s degree"",""answer52"":""Retired"",""answer53"":""$50,000 to $74,999"",""answer54"":""Boise"",""answer55"":""Idaho""},{""traits"":""Female aged 60+"",""answer1"":""7"",""answer2"":""Golden Retriever"",""answer3"":""65"",""answer4"":""Sunny"",""answer5"":""Her cheerful personality"",""answer6"":""Sun"",""answer7"":""Before 2020"",""answer8"":""yes"",""answer9"":""10"",""answer10"":""Pure happiness"",""answer11"":""I appreciate the simple joys more"",""answer12"":""5"",""answer13"":""Loyal companion"",""answer14"":""Sunny is always by my side"",""answer15"":""Open doors, took about a month"",""answer16"":""12, on her plush bed in the sunroom"",""answer17"":""yes"",""answer18"":""$1500"",""answer19"":""25%"",""answer20"":""40%"",""answer21"":""$130"",""answer22"":""Spent about the same in March"",""answer23"":""Weekly"",""answer24"":""Dry food"",""answer25"":""I know a lot about the brand"",""answer26"":""5"",""answer27"":""2"",""answer28"":""5"",""answer29"":""4"",""answer30"":""5"",""answer31"":""5"",""answer32"":""1"",""answer33"":""Royal Canin"",""answer34"":""Superb quality, vet recommended"",""answer35"":""5"",""answer36"":""3"",""answer37"":""5"",""answer38"":""4"",""answer39"":""5"",""answer40"":""5"",""answer41"":""2"",""answer42"":""2"",""answer43"":""I interact more"",""answer44"":""Pandemic, we\'re staying home more"",""answer45"":""Not Sure"",""answer46"":""Yes"",""answer47"":""Car"",""answer48"":""Female"",""answer49"":""65"",""answer50"":""1"",""answer51"":""Some college, no degree"",""answer52"":""Employed part-time"",""answer53"":""$25,000 to $49,999"",""answer54"":""Tucson"",""answer55"":""Arizona""},{""traits"":""Female aged 60+"",""answer1"":""9"",""answer2"":""Dachshund"",""answer3"":""14"",""answer4"":""Frankie"",""answer5"":""His long body reminds me of a hotdog"",""answer6"":""Frank"",""answer7"":""In 2020"",""answer8"":""no"",""answer9"":""48"",""answer10"":""Amused by his quirky attitude"",""answer11"":""Laughs are a daily occurrence now"",""answer12"":""5"",""answer13"":""Protector"",""answer14"":""Fearless despite his size"",""answer15"":""Dance, surprisingly one weekend"",""answer16"":""10, in my bed"",""answer17"":""yes"",""answer18"":""$500"",""answer19"":""20%"",""answer20"":""40%"",""answer21"":""$40"",""answer22"":""Spent less in March"",""answer23"":""Monthly"",""answer24"":""Wet food"",""answer25"":""I\'m not very familiar with the brand"",""answer26"":""5"",""answer27"":""3"",""answer28"":""4"",""answer29"":""3"",""answer30"":""4"",""answer31"":""3"",""answer32"":""3"",""answer33"":""Hill\'s Science Diet"",""answer34"":""Recommended by my vet, Frankie\'s health"",""answer35"":""4"",""answer36"":""4"",""answer37"":""4"",""answer38"":""3"",""answer39"":""4"",""answer40"":""4"",""answer41"":""3"",""answer42"":""4"",""answer43"":""I interact more"",""answer44"":""Retirement, I have more time"",""answer45"":""Yes"",""answer46"":""No"",""answer47"":""N/A-I\'m not planning to travel with my dog this Spring"",""answer48"":""Female"",""answer49"":""70"",""answer50"":""1"",""answer51"":""Associate degree"",""answer52"":""Retired"",""answer53"":""Less than $25,000"",""answer54"":""Springfield"",""answer55"":""Missouri""},{""traits"":""Female aged 60+"",""answer1"":""2"",""answer2"":""Poodle"",""answer3"":""10"",""answer4"":""Coco"",""answer5"":""Her fur\'s chocolate color"",""answer6"":""Cokes"",""answer7"":""After 2020"",""answer8"":""yes"",""answer9"":""3"",""answer10"":""Adoration"",""answer11"":""I feel less lonely"",""answer12"":""5"",""answer13"":""Best Friend"",""answer14"":""Coco follows me everywhere"",""answer15"":""Fetch, just a few sessions"",""answer16"":""18, on a designer doggy bed"",""answer17"":""yes"",""answer18"":""$2000"",""answer19"":""10%"",""answer20"":""70%"",""answer21"":""$160"",""answer22"":""Spent more in March"",""answer23"":""Weekly"",""answer24"":""Mix"",""answer25"":""I know a lot about the brand"",""answer26"":""5"",""answer27"":""2"",""answer28"":""5"",""answer29"":""5"",""answer30"":""5"",""answer31"":""3"",""answer32"":""5"",""answer33"":""Orijen"",""answer34"":""Coco\'s improvement, natural ingredients"",""answer35"":""5"",""answer36"":""3"",""answer37"":""5"",""answer38"":""5"",""answer39"":""5"",""answer40"":""3"",""answer41"":""5"",""answer42"":""3"",""answer43"":""I interact more"",""answer44"":""Having moved to a pet-friendly community"",""answer45"":""Yes"",""answer46"":""Yes"",""answer47"":""Car"",""answer48"":""Female"",""answer49"":""61"",""answer50"":""1"",""answer51"":""Graduate degree"",""answer52"":""Self-employed"",""answer53"":""$100,000 to $149,000"",""answer54"":""San Diego"",""answer55"":""California""},{""traits"":""Female aged 60+"",""answer1"":""11"",""answer2"":""Yorkshire Terrier"",""answer3"":""7"",""answer4"":""Pixie"",""answer5"":""Her tiny, fairy-like appearance"",""answer6"":""Pix"",""answer7"":""Before 2020"",""answer8"":""no"",""answer9"":""12"",""answer10"":""A bit anxious, she was so small"",""answer11"":""I enjoy everyday moments more"",""answer12"":""5"",""answer13"":""Family Member"",""answer14"":""She\'s been with me through thick and thin"",""answer15"":""Spin, just took two weeks"",""answer16"":""13, her cushioned crate"",""answer17"":""yes"",""answer18"":""$700"",""answer19"":""15%"",""answer20"":""45%"",""answer21"":""$60"",""answer22"":""Spent about the same in March"",""answer23"":""Bi-weekly"",""answer24"":""Dry food"",""answer25"":""I know just the basics about the brand"",""answer26"":""4"",""answer27"":""3"",""answer28"":""4"",""answer29"":""5"",""answer30"":""5"",""answer31"":""1"",""answer32"":""4"",""answer33"":""Iams"",""answer34"":""My Pixie\'s health, cost-effective"",""answer35"":""4"",""answer36"":""5"",""answer37"":""4"",""answer38"":""4"",""answer39"":""5"",""answer40"":""1"",""answer41"":""4"",""answer42"":""5"",""answer43"":""I interact more"",""answer44"":""The pandemic has us spending more time indoors"",""answer45"":""Not Sure"",""answer46"":""No"",""answer47"":""N/A-I\'m not planning to travel with my dog this Spring"",""answer48"":""Female"",""answer49"":""72"",""answer50"":""2"",""answer51"":""High school diploma or equivalent"",""answer52"":""Retired"",""answer53"":""Less than $25,000"",""answer54"":""Macon"",""answer55"":""Georgia""},{""traits"":""Female aged 60+"",""answer1"":""4"",""answer2"":""Border Collie"",""answer3"":""45"",""answer4"":""Shep"",""answer5"":""Traditional name for a sheepdog"",""answer6"":""Sheppy"",""answer7"":""Before 2020"",""answer8"":""yes"",""answer9"":""5"",""answer10"":""I felt protective"",""answer11"":""Daily physical activity is a must"",""answer12"":""5"",""answer13"":""Loyal companion"",""answer14"":""Always ready to work and play"",""answer15"":""We\'ve mastered herding techniques, took months"",""answer16"":""10, in the utility room on his mat"",""answer17"":""yes"",""answer18"":""$1300"",""answer19"":""10%"",""answer20"":""55%"",""answer21"":""$110"",""answer22"":""Spent about the same in March"",""answer23"":""Weekly"",""answer24"":""Dry food"",""answer25"":""I know a lot about the brand"",""answer26"":""5"",""answer27"":""4"",""answer28"":""5"",""answer29"":""3"",""answer30"":""5"",""answer31"":""3"",""answer32"":""2"",""answer33"":""Acana"",""answer34"":""High-quality, breed-specific formula"",""answer35"":""5"",""answer36"":""4"",""answer37"":""5"",""answer38"":""4"",""answer39"":""5"",""answer40"":""3"",""answer41"":""2"",""answer42"":""4"",""answer43"":""No change"",""answer44"":""Seasonal allergies, we adjust our time outside"",""answer45"":""No"",""answer46"":""No"",""answer47"":""N/A-I\'m not planning to travel with my dog this Spring"",""answer48"":""Female"",""answer49"":""67"",""answer50"":""1"",""answer51"":""Bachelor\'s degree"",""answer52"":""Employed part-time"",""answer53"":""$25,000 to $49,999"",""answer54"":""Topeka"",""answer55"":""Kansas""},{""traits"":""Female aged 60+"",""answer1"":""10"",""answer2"":""Chihuahua"",""answer3"":""5"",""answer4"":""Tiny"",""answer5"":""Her petite size"",""answer6"":""T"",""answer7"":""After 2020"",""answer8"":""yes"",""answer9"":""24"",""answer10"":""Amused by her sassiness"",""answer11"":""It\'s important to love fiercely"",""answer12"":""5"",""answer13"":""Best Friend"",""answer14"":""Tiny is always by my side, my constant tiny shadow"",""answer15"":""Sit pretty, she picked it up in days"",""answer16"":""14, she curls up in the bed under the window"",""answer17"":""yes"",""answer18"":""$550"",""answer19"":""40%"",""answer20"":""30%"",""answer21"":""$50"",""answer22"":""Spent less in March"",""answer23"":""Bi-weekly"",""answer24"":""Mix"",""answer25"":""I know just the basics about the brand"",""answer26"":""5"",""answer27"":""5"",""answer28"":""5"",""answer29"":""2"",""answer30"":""5"",""answer31"":""2"",""answer32"":""1"",""answer33"":""Cesar"",""answer34"":""Tiny loves it, it\'s affordable, easy to store"",""answer35"":""5"",""answer36"":""5"",""answer37"":""4"",""answer38"":""2"",""answer39"":""5"",""answer40"":""2"",""answer41"":""1"",""answer42"":""5"",""answer43"":""No change"",""answer44"":""The seasons changing is the biggest influence"",""answer45"":""Yes"",""answer46"":""Yes"",""answer47"":""Car"",""answer48"":""Female"",""answer49"":""69"",""answer50"":""1"",""answer51"":""Some college, no degree"",""answer52"":""Retired"",""answer53"":""$50,000 to $74,999"",""answer54"":""El Paso"",""answer55"":""Texas""},{""traits"":""Female aged 60+"",""answer1"":""6"",""answer2"":""Bulldog"",""answer3"":""50"",""answer4"":""Winston"",""answer5"":""His stately, British-like demeanor"",""answer6"":""Win"",""answer7"":""Before 2020"",""answer8"":""no"",""answer9"":""3"",""answer10"":""Awe, he\'s got so much character"",""answer11"":""I\'ve become more patient"",""answer12"":""5"",""answer13"":""Family Member"",""answer14"":""He\'s more like a child to me than a pet"",""answer15"":""Speak on command, a stubborn month or so"",""answer16"":""12, sprawled across the hallway rug"",""answer17"":""yes"",""answer18"":""$1100"",""answer19"":""15%"",""answer20"":""45%"",""answer21"":""$90"",""answer22"":""Spent about the same in March"",""answer23"":""Monthly"",""answer24"":""Dry food"",""answer25"":""I know a lot about the brand"",""answer26"":""4"",""answer27"":""2"",""answer28"":""4"",""answer29"":""5"",""answer30"":""5"",""answer31"":""3"",""answer32"":""3"",""answer33"":""Nutro"",""answer34"":""Non-GMO ingredients, Winston\'s preference"",""answer35"":""4"",""answer36"":""3"",""answer37"":""4"",""answer38"":""5"",""answer39"":""5"",""answer40"":""3"",""answer41"":""3"",""answer42"":""3"",""answer43"":""I interact less"",""answer44"":""Winston\'s aging, he is more independent"",""answer45"":""Not Sure"",""answer46"":""No"",""answer47"":""N/A-I\'m not planning to travel with my dog this Spring"",""answer48"":""Female"",""answer49"":""65"",""answer50"":""2"",""answer51"":""Graduate degree"",""answer52"":""Retired"",""answer53"":""Prefer not to say"",""answer54"":""Charleston"",""answer55"":""South Carolina""},{""traits"":""Female aged 60+"",""answer1"":""8"",""answer2"":""Shih Tzu"",""answer3"":""12"",""answer4"":""Gizmo"",""answer5"":""His playful and curious nature"",""answer6"":""Gizzy"",""answer7"":""In 2020"",""answer8"":""yes"",""answer9"":""60"",""answer10"":""Instant love, he\'s such a fluffball"",""answer11"":""I\'m more outgoing, he needs lots of socializing"",""answer12"":""5"",""answer13"":""Best Friend"",""answer14"":""We have an unbreakable bond"",""answer15"":""Roll over, a week\'s full of treats and praise"",""answer16"":""15, on his favorite armchair"",""answer17"":""yes"",""answer18"":""$950"",""answer19"":""10%"",""answer20"":""55%"",""answer21"":""$80"",""answer22"":""Spent about the same in March"",""answer23"":""Monthly"",""answer24"":""Dry food"",""answer25"":""I know just the basics about the brand"",""answer26"":""5"",""answer27"":""4"",""answer28"":""5"",""answer29"":""3"",""answer30"":""5"",""answer31"":""1"",""answer32"":""4"",""answer33"":""Science Diet"",""answer34"":""Gizzy likes it, vet approved, easy to find"",""answer35"":""4"",""answer36"":""4"",""answer37"":""5"",""answer38"":""3"",""answer39"":""5"",""answer40"":""1"",""answer41"":""4"",""answer42"":""4"",""answer43"":""No change"",""answer44"":""Seasonal weather, we keep our routine"",""answer45"":""Yes"",""answer46"":""Not Sure"",""answer47"":""N/A-I\'m not planning to travel with my dog this Spring"",""answer48"":""Female"",""answer49"":""64"",""answer50"":""1"",""answer51"":""Bachelor\'s degree"",""answer52"":""Employed full-time"",""answer53"":""$50,000 to $74,999"",""answer54"":""Reno"",""answer55"":""Nevada""},{""traits"":""Female aged 60+"",""answer1"":""1"",""answer2"":""Australian Shepherd"",""answer3"":""30"",""answer4"":""Blue"",""answer5"":""His striking blue eyes"",""answer6"":""Boo"",""answer7"":""After 2020"",""answer8"":""no"",""answer9"":""2"",""answer10"":""Thrilled, he was full of energy"",""answer11"":""I\'ve got a new sense of purpose"",""answer12"":""5"",""answer13"":""Loyal companion"",""answer14"":""He\'s loyal and always there for support"",""answer15"":""Fetch with a frisbee, about two weeks"",""answer16"":""14, on a runner in the hall"",""answer17"":""yes"",""answer18"":""$1000"",""answer19"":""5%"",""answer20"":""65%"",""answer21"":""$85"",""answer22"":""Spent more in March"",""answer23"":""Weekly"",""answer24"":""Dry food"",""answer25"":""I know a lot about the brand"",""answer26"":""5"",""answer27"":""3"",""answer28"":""5"",""answer29"":""4"",""answer30"":""5"",""answer31"":""4"",""answer32"":""1"",""answer33"":""Taste of the Wild"",""answer34"":""Grain-free, Blue\'s coat looks amazing"",""answer35"":""5"",""answer36"":""4"",""answer37"":""5"",""answer38"":""4"",""answer39"":""5"",""answer40', 'name': 'Answers'}})

final_json = json_repair.loads(response.additional_kwargs['function_call']['arguments'])['answer']

JSON return by json_repair:

[{'traits': '', 60: '', 'answer1': '', 5: ',', 'answer2': '', 'Retriever': '', 'answer3': '', 75: ',', 'answer4': '', 'Buddy': '', 'answer5': '', 'nickname': '', 'answer6': '', 'Bud': '', 'answer7': '', 2020: ',', 'answer8': '', 'yes': '', 'answer9': '', 6: ',', 'answer10': '', 'connection': '', 'answer11': '', 'love': '', 'answer12': '', 'answer13': '', 'Member': '', 'answer14': '', 'family': '', 'answer15': '', 'week': '', 'answer16': '', 14: 'in a dog bed by my bedside', ',': 'answer18', ':': 1200, 'answer19': '', 20: '', 'answer20': '', 'answer21': '', 100: ',', 'answer22': '', 'March': '', 'answer23': '', 'Monthly': '', 'answer24': '', 'food': '', 'answer25': '', 'brand': '', 'answer26': '', 'answer27': '', 3: ',', 'answer28': '', 'answer29': '', 4: ',', 'answer30': '', 'answer31': '', 'answer32': '', 2: ',', 'answer33': '', 'Buffalo': '', 'answer34': '', 'value': '', 'answer35': '', 'answer36': '', 'answer37': '', 'answer38': '', 'answer39': '', 'answer40': '', 'answer41': '', 'answer42': '', 'answer43': '', 'more': '', 'answer44': '', 'pandemic': '', 'answer45': '', 'No': '', 'answer46': '', 'Yes': '', 'answer47': '', 'Car': '', 'answer48': '', 'Female': '', 'answer49': '', 63: ',', 'answer50': '', 1: ',', 'answer51': '', 'degree': '', 'answer52': '', 'Retired': '', 'answer53': '', 25: 0, 'answer54': '', 'Sarasota': '', 'answer55': '', 'Florida': ''}, {'traits': '', 60: '', 'answer1': '', 3: ',', 'answer2': '', 'Beagle': '', 'answer3': '', 20: ',', 'answer4': '', 'Scout': '', 'answer5': '', 'book': '', 'answer6': '', 'Scooty': '', 'answer7': '', 2020: ',', 'answer8': '', 'no': '', 'answer9': '', 8: ',', 'answer10': '', 'joy': '', 'answer11': '', 'active': '', 'answer12': '', 5: ',', 'answer13': '', 'Friend': '', 'answer14': '', 'together': '', 'answer15': '', 'days': '', 'answer16': '', 16: 'on the living room couch', ',': 'answer18', ':': 800, 'answer19': '', 30: '', 'answer20': '', 50: 0, 'answer21': '', 70: ',', 'answer22': '', 'March': '', 'answer23': '', 'Bi-weekly': '', 'answer24': '', 'Mix': '', 'answer25': '', 'brand': '', 'answer26': '', 'answer27': '', 4: ',', 'answer28': '', 'answer29': '', 'answer30': '', 'answer31': '', 2: ',', 'answer32': '', 'answer33': '', 'Purina': '', 'answer34': '', 'it': '', 'answer35': '', 'answer36': '', 'answer37': '', 'answer38': '', 'answer39': '', 'answer40': '', 'answer41': '', 'answer42': '', 'answer43': '', 'change': '', 'answer44': '', 'adapted': '', 'answer45': '', 'Yes': '', 'answer46': '', 'No': '', 'answer47': '', 'Spring': '', 'answer48': '', 'Female': '', 'answer49': '', 68: ',', 'answer50': '', 'answer51': '', 'degree': '', 'answer52': '', 'Retired': '', 'answer53': '', 74: 999, 'answer54': '', 'Boise': '', 'answer55': '', 'Idaho': ''}, {'traits': '', 60: '', 'answer1': '', 7: ',', 'answer2': '', 'Retriever': '', 'answer3': '', 65: ',', 'answer4': '', 'Sunny': '', 'answer5': '', 'personality': '', 'answer6': '', 'Sun': '', 'answer7': '', 2020: ',', 'answer8': '', 'yes': '', 'answer9': '', 10: ',', 'answer10': '', 'happiness': '', 'answer11': '', 'more': '', 'answer12': '', 5: ',', 'answer13': '', 'companion': '', 'answer14': '', 'side': '', 'answer15': '', 'month': '', 'answer16': '', 12: 'on her plush bed in the sunroom', ',': 'answer18', ':': 1500, 'answer19': '', 25: 0, 'answer20': '', 40: '', 'answer21': '', 130: ',', 'answer22': '', 'March': '', 'answer23': '', 'Weekly': '', 'answer24': '', 'food': '', 'answer25': '', 'brand': '', 'answer26': '', 'answer27': '', 2: ',', 'answer28': '', 'answer29': '', 4: ',', 'answer30': '', 'answer31': '', 'answer32': '', 1: ',', 'answer33': '', 'Canin': '', 'answer34': '', 'recommended': '', 'answer35': '', 'answer36': '', 3: ',', 'answer37': '', 'answer38': '', 'answer39': '', 'answer40': '', 'answer41': '', 'answer42': '', 'answer43': '', 'answer44': '', 'answer45': '', 'Sure': '', 'answer46': '', 'Yes': '', 'answer47': '', 'Car': '', 'answer48': '', 'Female': '', 'answer49': '', 'answer50': '', 'answer51': '', 'degree': '', 'answer52': '', 'part-time': '', 'answer53': '', 49: 999, 'answer54': '', 'Tucson': '', 'answer55': '', 'Arizona': ''}, {'traits': '', 60: '', 'answer1': '', 9: ',', 'answer2': '', 'Dachshund': '', 'answer3': '', 14: ',', 'answer4': '', 'Frankie': '', 'answer5': '', 'hotdog': '', 'answer6': '', 'Frank': '', 'answer7': '', 2020: ',', 'answer8': '', 'no': '', 'answer9': '', 48: ',', 'answer10': '', 'attitude': '', 'answer11': '', 'now': '', 'answer12': '', 5: ',', 'answer13': '', 'Protector': '', 'answer14': '', 'size': '', 'answer15': '', 'weekend': '', 'answer16': '', 10: 'in my bed', ',': 'answer18', ':': 500, 'answer19': '', 20: '', 'answer20': '', 40: ',', 'answer21': '', 'answer22': '', 'March': '', 'answer23': '', 'Monthly': '', 'answer24': '', 'food': '', 'answer25': '', 'brand': '', 'answer26': '', 'answer27': '', 3: ',', 'answer28': '', 4: ',', 'answer29': '', 'answer30': '', 'answer31': '', 'answer32': '', 'answer33': '', 'Diet': '', 'answer34': '', 'health': '', 'answer35': '', 'answer36': '', 'answer37': '', 'answer38': '', 'answer39': '', 'answer40': '', 'answer41': '', 'answer42': '', 'answer43': '', 'more': '', 'answer44': '', 'time': '', 'answer45': '', 'Yes': '', 'answer46': '', 'No': '', 'answer47': '', 'Spring': '', 'answer48': '', 'Female': '', 'answer49': '', 70: ',', 'answer50': '', 1: ',', 'answer51': '', 'degree': '', 'answer52': '', 'Retired': '', 'answer53': '', 25: 0, 'answer54': '', 'Springfield': '', 'answer55': '', 'Missouri': ''}, {'traits': '', 60: '', 'answer1': '', 2: ',', 'answer2': '', 'Poodle': '', 'answer3': '', 10: '', 'answer4': '', 'Coco': '', 'answer5': '', 'color': '', 'answer6': '', 'Cokes': '', 'answer7': '', 2020: ',', 'answer8': '', 'yes': '', 'answer9': '', 3: ',', 'answer10': '', 'Adoration': '', 'answer11': '', 'lonely': '', 'answer12': '', 5: ',', 'answer13': '', 'Friend': '', 'answer14': '', 'everywhere': '', 'answer15': '', 'sessions': '', 'answer16': '', 18: 'on a designer doggy bed', ',': 'answer18', ':': 2000, 'answer19': '', 'answer20': '', 70: '', 'answer21': '', 160: ',', 'answer22': '', 'March': '', 'answer23': '', 'Weekly': '', 'answer24': '', 'Mix': '', 'answer25': '', 'brand': '', 'answer26': '', 'answer27': '', 'answer28': '', 'answer29': '', 'answer30': '', 'answer31': '', 'answer32': '', 'answer33': '', 'Orijen': '', 'answer34': '', 'ingredients': '', 'answer35': '', 'answer36': '', 'answer37': '', 'answer38': '', 'answer39': '', 'answer40': '', 'answer41': '', 'answer42': '', 'answer43': '', 'more': '', 'answer44': '', 'community': '', 'answer45': '', 'Yes': '', 'answer46': '', 'answer47': '', 'Car': '', 'answer48': '', 'Female': '', 'answer49': '', 61: ',', 'answer50': '', 1: ',', 'answer51': '', 'degree': '', 'answer52': '', 'Self-employed': '', 'answer53': '', 100: 0, 149: 0, 'answer54': '', 'Diego': '', 'answer55': '', 'California': ''}, {'traits': '', 60: ',', 'answer1': '', 11: ',', 'answer2': '', 'Terrier': '', 'answer3': '', 7: ',', 'answer4': '', 'Pixie': '', 'answer5': '', 'appearance': '', 'answer6': '', 'Pix': '', 'answer7': '', 2020: ',', 'answer8': '', 'no': '', 'answer9': '', 12: ',', 'answer10': '', 'small': '', 'answer11': '', 'more': '', 'answer12': '', 5: ',', 'answer13': '', 'Member': '', 'answer14': '', 'thin': '', 'answer15': '', 'weeks': '', 'answer16': '', 13: 'her cushioned crate', ',': 'answer18', ':': 700, 'answer19': '', 15: '', 'answer20': '', 45: '', 'answer21': '', 'answer22': '', 'March': '', 'answer23': '', 'Bi-weekly': '', 'answer24': '', 'food': '', 'answer25': '', 'brand': '', 'answer26': '', 4: ',', 'answer27': '', 3: ',', 'answer28': '', 'answer29': '', 'answer30': '', 'answer31': '', 1: ',', 'answer32': '', 'answer33': '', 'Iams': '', 'answer34': '', 'cost-effective': '', 'answer35': '', 'answer36': '', 'answer37': '', 'answer38': '', 'answer39': '', 'answer40': '', 'answer41': '', 'answer42': '', 'answer43': '', 'answer44': '', 'indoors': '', 'answer45': '', 'Sure': '', 'answer46': '', 'No': '', 'answer47': '', 'Spring': '', 'answer48': '', 'Female': '', 'answer49': '', 72: ',', 'answer50': '', 2: ',', 'answer51': '', 'equivalent': '', 'answer52': '', 'Retired': '', 'answer53': '', 25: 0, 'answer54': '', 'Macon': '', 'answer55': '', 'Georgia': ''}, {'traits': '', 60: '', 'answer1': '', 4: ',', 'answer2': '', 'Collie': '', 'answer3': '', 45: ',', 'answer4': '', 'Shep': '', 'answer5': '', 'sheepdog': '', 'answer6': '', 'Sheppy': '', 'answer7': '', 2020: ',', 'answer8': '', 'yes': '', 'answer9': '', 5: ',', 'answer10': '', 'protective': '', 'answer11': '', 'must': '', 'answer12': '', 'answer13': '', 'companion': '', 'answer14': '', 'play': '', 'answer15': '', 'months': '', 'answer16': '', 10: '', ',': 'answer18', ':': 1300, 'answer19': '', 'answer20': '', 55: '', 'answer21': '', 110: ',', 'answer22': '', 'March': '', 'answer23': '', 'Weekly': '', 'answer24': '', 'food': '', 'answer25': '', 'brand': '', 'answer26': '', 'answer27': '', 'answer28': '', 'answer29': '', 3: ',', 'answer30': '', 'answer31': '', 'answer32': '', 2: ',', 'answer33': '', 'Acana': '', 'answer34': '', 'formula': '', 'answer35': '', 'answer36': '', 'answer37': '', 'answer38': '', 'answer39': '', 'answer40': '', 'answer41': '', 'answer42': '', 'answer43': '', 'change': '', 'answer44': '', 'outside': '', 'answer45': '', 'No': '', 'answer46': '', 'answer47': '', 'Spring': '', 'answer48': '', 'Female': '', 'answer49': '', 67: ',', 'answer50': '', 1: ',', 'answer51': '', 'degree': '', 'answer52': '', 'part-time': '', 'answer53': '', 25: 0, 49: 999, 'answer54': '', 'Topeka': '', 'answer55': '', 'Kansas': ''}, {'traits': '', 60: '', 'answer1': '', 10: ',', 'answer2': '', 'Chihuahua': '', 'answer3': '', 5: ',', 'answer4': '', 'Tiny': '', 'answer5': '', 'size': '', 'answer6': '', 'T': '', 'answer7': '', 2020: ',', 'answer8': '', 'yes': '', 'answer9': '', 24: ',', 'answer10': '', 'sassiness': '', 'answer11': '', 'fiercely': '', 'answer12': '', 'answer13': '', 'Friend': '', 'answer14': '', 'shadow': '', 'answer15': '', 'days': '', 'answer16': '', 14: 'she curls up in the bed under the window', ',': 'answer18', ':': 550, 'answer19': '', 40: '', 'answer20': '', 30: '', 'answer21': '', 50: 0, 'answer22': '', 'March': '', 'answer23': '', 'Bi-weekly': '', 'answer24': '', 'Mix': '', 'answer25': '', 'brand': '', 'answer26': '', 'answer27': '', 'answer28': '', 'answer29': '', 2: ',', 'answer30': '', 'answer31': '', 'answer32': '', 1: ',', 'answer33': '', 'Cesar': '', 'answer34': '', 'store': '', 'answer35': '', 'answer36': '', 'answer37': '', 4: ',', 'answer38': '', 'answer39': '', 'answer40': '', 'answer41': '', 'answer42': '', 'answer43': '', 'change': '', 'answer44': '', 'influence': '', 'answer45': '', 'Yes': '', 'answer46': '', 'answer47': '', 'Car': '', 'answer48': '', 'Female': '', 'answer49': '', 69: ',', 'answer50': '', 'answer51': '', 'degree': '', 'answer52': '', 'Retired': '', 'answer53': '', 74: 999, 'answer54': '', 'Paso': '', 'answer55': '', 'Texas': ''}, {'traits': '', 60: '', 'answer1': '', 6: ',', 'answer2': '', 'Bulldog': '', 'answer3': '', 50: ',', 'answer4': '', 'Winston': '', 'answer5': '', 'demeanor': '', 'answer6': '', 'Win': '', 'answer7': '', 2020: ',', 'answer8': '', 'no': '', 'answer9': '', 3: ',', 'answer10': '', 'character': '', 'answer11': '', 'patient': '', 'answer12': '', 5: ',', 'answer13': '', 'Member': '', 'answer14': '', 'pet': '', 'answer15': '', 'so': '', 'answer16': '', 12: 'sprawled across the hallway rug', ',': 'answer18', ':': 1100, 'answer19': '', 15: '', 'answer20': '', 45: '', 'answer21': '', 90: ',', 'answer22': '', 'March': '', 'answer23': '', 'Monthly': '', 'answer24': '', 'food': '', 'answer25': '', 'brand': '', 'answer26': '', 4: ',', 'answer27': '', 2: ',', 'answer28': '', 'answer29': '', 'answer30': '', 'answer31': '', 'answer32': '', 'answer33': '', 'Nutro': '', 'answer34': '', 'preference': '', 'answer35': '', 'answer36': '', 'answer37': '', 'answer38': '', 'answer39': '', 'answer40': '', 'answer41': '', 'answer42': '', 'answer43': '', 'less': '', 'answer44': '', 'independent': '', 'answer45': '', 'Sure': '', 'answer46': '', 'No': '', 'answer47': '', 'Spring': '', 'answer48': '', 'Female': '', 'answer49': '', 65: ',', 'answer50': '', 'answer51': '', 'degree': '', 'answer52': '', 'Retired': '', 'answer53': '', 'say': '', 'answer54': '', 'Charleston': '', 'answer55': '', 'Carolina': ''}, {'traits': '', 60: ',', 'answer1': '', 8: ',', 'answer2': '', 'Tzu': '', 'answer3': '', 12: ',', 'answer4': '', 'Gizmo': '', 'answer5': '', 'nature': '', 'answer6': '', 'Gizzy': '', 'answer7': '', 2020: ',', 'answer8': '', 'yes': '', 'answer9': '', 'answer10': '', 'fluffball': '', 'answer11': '', 'socializing': '', 'answer12': '', 5: ',', 'answer13': '', 'Friend': '', 'answer14': '', 'bond': '', 'answer15': '', 'praise': '', 'answer16': '', 15: 'on his favorite armchair', ',': 'answer18', ':': 950, 'answer19': '', 10: '', 'answer20': '', 55: '', 'answer21': '', 80: ',', 'answer22': '', 'March': '', 'answer23': '', 'Monthly': '', 'answer24': '', 'food': '', 'answer25': '', 'brand': '', 'answer26': '', 'answer27': '', 4: ',', 'answer28': '', 'answer29': '', 3: ',', 'answer30': '', 'answer31': '', 1: ',', 'answer32': '', 'answer33': '', 'Diet': '', 'answer34': '', 'find': '', 'answer35': '', 'answer36': '', 'answer37': '', 'answer38': '', 'answer39': '', 'answer40': '', 'answer41': '', 'answer42': '', 'answer43': '', 'change': '', 'answer44': '', 'routine': '', 'answer45': '', 'Yes': '', 'answer46': '', 'Sure': '', 'answer47': '', 'Spring': '', 'answer48': '', 'Female': '', 'answer49': '', 64: ',', 'answer50': '', 'answer51': '', 'degree': '', 'answer52': '', 'full-time': '', 'answer53': '', 50: 0, 74: 999, 'answer54': '', 'Reno': '', 'answer55': '', 'Nevada': ''}, {'traits': '', 60: '', 'answer1': '', 1: ',', 'answer2': '', 'Shepherd': '', 'answer3': '', 30: ',', 'answer4': '', 'Blue': '', 'answer5': '', 'eyes': '', 'answer6': '', 'Boo': '', 'answer7': '', 2020: ',', 'answer8': '', 'no': '', 'answer9': '', 2: ',', 'answer10': '', 'energy': '', 'answer11': '', 'purpose': '', 'answer12': '', 5: ',', 'answer13': '', 'companion': '', 'answer14': '', 'support': '', 'answer15': '', 'weeks': '', 'answer16': '', 14: 'on a runner in the hall', ',': 'answer18', ':': 1000, 'answer19': '', 'answer20': '', 65: '', 'answer21': '', 85: ',', 'answer22': '', 'March': '', 'answer23': '', 'Weekly': '', 'answer24': '', 'food': '', 'answer25': '', 'brand': '', 'answer26': '', 'answer27': '', 3: ',', 'answer28': '', 'answer29': '', 4: ',', 'answer30': '', 'answer31': '', 'answer32': '', 'answer33': '', 'Wild': '', 'answer34': '', 'amazing': '', 'answer35': '', 'answer36': '', 'answer37': '', 'answer38': '', 'answer39': ''}]

@mangiucugna
Copy link
Owner

that's a super weird return value with those double double quotes, I will look into it. Do you know by any chance why the output is formatted like that?

@pd2871
Copy link
Author

pd2871 commented Apr 8, 2024

that's a super weird return value with those double double quotes, I will look into it. Do you know by any chance why the output is formatted like that?

This is extracted from AIMessage returned from Langchain. I tried to use my custom parser instead of parsers provided by Langchain because they were not able to handle broken JSONs.

@mangiucugna
Copy link
Owner

I have released 0.12.0 to support this format, let me know if it works for you as it passed all tests for me

@pd2871
Copy link
Author

pd2871 commented Apr 8, 2024

I have released 0.12.0 to support this format, let me know if it works for you as it passed all tests for me

Thank you for this quick fix. I will check it and let you know in any case of failure with new similar data.

@nikolaysm
Copy link

nikolaysm commented Apr 19, 2024

Hi @mangiucugna,

Thank you for your efforts on this. I've encountered a similar issue with the output from the LLM. It seems that the repair_json function isn't handling certain cases correctly.

For instance, when trying to repair the following JSON string:

json_str = '{\n"html": "<h3 id="title">Waarom meer dan 200 Technical Experts - "Passie voor techniek"?</h3>"}'
data = repair_json(json_str, return_objects=True)

The current output is:

{
    'html': '<h3 id=', 
    'techniek': 'h3>',
    'title': u'Waarom meer dan 200 Technical Experts - '
}

However, the expected output should be:

{
    'html': '<h3 id="title">Waarom meer dan 200 Technical Experts - "Passie voor techniek"?</h3>'
}

It seems like the function is having trouble handling certain characters or nested structures properly. Would you mind looking into this further?

Thank you again for your attention to this matter.

@mangiucugna
Copy link
Owner

mangiucugna commented Apr 19, 2024

Hi @nikolaysm can you open a new issue for that? The issue is with the fact that the right format is <h3 id='title'> so I am not 100% sure I can support this use case but is definitely a distinct use case from this issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants