Response in 0.1.6 as a string

[OlegRuban-ai]

Hello! In version 0.1.6, the response comes in the form of a string, but not in json format for further selection of the correct response. In addition, there are a lot of unnecessary special characters from the site layout.

[dsdanielpark]

I'm developing an algorithm to parse this payload. Since Google responds including 3rd party services, there are significantly more types to parse than before. However, some nonce values are not being returned in my account currently, preventing me from debugging and updating. Please try the following function temporarily. It's still incomplete, but it might be of some help.

You can start from here.

response_items = response_text.lstrip("')]}\'\n\n").split("\n")[1].split("\\")
response_items = [x for x in response_items if x]

Or, try this temporary parsing methods.

def parse_method1(response_text): 
    # Initial processing of the response string
    response_items = response_text.lstrip("')]}\'\n\n").split("\n")[1].split("\\")
    response_items = [item for item in response_items if item]
    processed_items = [x for x in response_items if x[0] == "n"]

    # Extracting information into JSON format
    temp_dict = {}
    json_data = []
    for item in processed_items:
        key, value = item.split(": ", 1)
        if key == 'nsnippet' and temp_dict:
            json_data.append(temp_dict)
            temp_dict = {}
        temp_dict[key] = value
    if temp_dict:
        json_data.append(temp_dict)

    # Restructuring the JSON data
    restructured_data = {}
    for index, item in enumerate(json_data, start=1):
        choice_key = f'choice{index:02}'
        choice_value = {}
        for key, value in item.items():
            new_key = key[1:]  # Remove the 'n' from the key
            choice_value[new_key] = value
        restructured_data[choice_key] = choice_value
    try:
        restructured_data['text'] = restructured_data['choice01']['snippet']
    except:
        pass                               
    return restructured_data

def parse_method2(response_text):
    response_items = response_text.lstrip("')]}\'\n\n").split("\n")[1].split("\\")
    response_items = [item for item in response_items if item]
    processed_items = [x for x in response_items if x[0] == "n" or "https://" in x or "http://" in x or "rc_" in x]
    processed_items = [x for x in processed_items if "encrypted" not in x and "[Image" not in x]
    cleand_items = [item.lstrip('n').lstrip('"').replace("  ", " ") for item in processed_items]
    cleand_items = [item for item in cleand_items if item]

    result = {"text": ""}
    choice_count = 0
    current_key = None

    for item in cleand_items:
        if item.startswith('rc_'):
            choice_count += 1
            current_key = f'choice{choice_count:02}'
            result[current_key] = {"response_choice": item, "links": [], "text": ""}
        elif 'http' in item and current_key:
            result[current_key]["links"].append(item)
        elif current_key:
            if result[current_key]["text"]:
                result[current_key]["text"] += "\n" + item
            else:
                result[current_key]["text"] = item
        
        else:  
            if result["text"]:
                result["text"] += "\n" + item
            else:
                result["text"] = item

    if 'choice01' in result:
        result['text'] = result['choice01']['text']
    return result

[dsdanielpark]

We are currently retesting with the method of parsing based on correct Json by default. The code above is temporary code for a specific purpose.

[dsdanielpark]

OlegRuban-ai

Please try to use version 1.0.4!

Thank you!!