Missing [[ radio ]] placeholder in exercise questions with radio buttons

[benjaoming]

Summary

@aronasorman wrote in #5131

I've tested a locally built content pack and most of the problem questions seem to be gone now! However, there still are a few questions that don't show answer areas. This time, they seem to be focused on True/False questions. Here's an example I found:

http://127.0.0.1:8008/learn/khan/humanities/us-history/road-to-revolution/declaration-of-independence/aspen-us-declaration-of-independence-quiz/

Branch or installer method

• Branch: 0.16.x

Screenshots

Real-life consequences (anything community should be aware, for instance how it affects your deployment)

We still don't know the count of affected questions.

[benjaoming]

I'm afraid this issue is not seeing enough progress to be fixed in a 0.16 patch release, let's postpone it for 0.17.

The vast amount of exercises have already been fixed, so it's okay to relax and enjoy that, too :)

[benjaoming]

Same exercise, still reproducing

[benjaoming]

This question works:

And this, mysteriously doesn't:

[mrpau-eugene]

Hi @benjaoming it seems like this issue is not just about True/False questions. I looked at the JSON data for each question that isn't generating properly. This is just a hunch, but maybe there was a problem while generating these JSON data. In addition, as far as I have observed, most question that don't generate/display correctly has an author_name ['James Lewis']

This question is a a multiple choice kind

pk: 7477
id: x5cea7a853e9aa640

{"hints":[{"content":"In the 18th century, many Americans were influenced by the ideas of the Enlightenment. One movement that grew out of the Enlightenment was Deism. Deists did not ascribe to the specific beliefs of religious groups of their time and believed that while God had created the universe, He also did not interfere in human affairs. \n\n[See this video to review these ideas] /learn/khan/humanities/us-history/road-to-revolution/declaration-of-independence/life-liberty-and-the-pursuit-of-happiness/"}],"question":{"content":"Deists… ","widgets":{"radio 1":{"type":"radio","options":{"choices":[{"content":"pushed for the establishment of a state-supported church. ","correct":false},{"content":"believed in a Creator, but did not hold to a specific religious dogma. ","correct":true},{"content":"believed that the Declaration should not refer to a God of any sort.","correct":false},{"content":"were banned in the 13 colonies. ","correct":false}],"randomize":true}}}},"answerArea":{"type":"radio","options":{"choices":[{"content":"pushed for the establishment of a state-supported church. ","correct":false},{"content":"believed in a Creator, but did not hold to a specific religious dogma. ","correct":true},{"content":"believed that the Declaration should not refer to a God of any sort.","correct":false},{"content":"were banned in the 13 colonies. ","correct":false}],"randomize":true}}}

and it displays this

[mrpau-eugene]

Here is another one that isn't a True/False question.

pk: 20744
id: x699f189dd5bcc110

{"hints":[{"content":"The authors of the Declaration struck a balance in their justification of natural rights. Their explanations looked to both self-evident truths discerned by human reason and the fact that the Creator was the source of these rights. The authors refer to these rights as unalienable which means that no king or government could ever legitimately violate them. \n\n[Watch this video to review these ideas] /learn/khan/humanities/us-history/road-to-revolution/declaration-of-independence/life-liberty-and-the-pursuit-of-happiness/\n\n "}],"question":{"content":""endowed by their Creator with certain unalienable rights" \n\nThis above phrase reflects the authors of the Declaration's belief that\n\n","widgets":{"radio 1":{"type":"radio","options":{"choices":[{"content":"kings ought to have absolute power since they ruled by divine right.","correct":false},{"content":"all people enjoyed certain rights which could never legitimately be taken away from them. ","correct":true},{"content":"there should be no separation between church and state in their new country. ","correct":false},{"correct":false,"content":"that people should enjoy absolute freedom to do as they pleased in their new country. "}],"randomize":true}}}},"answerArea":{"type":"radio","options":{"choices":[{"correct":false,"content":"kings ought to have absolute power since they ruled by divine right."},{"content":"all people enjoyed certain rights which could never legitimately be taken away from them. ","correct":true},{"content":"there should be no separation between church and state in their new country. ","correct":false},{"content":"that people should enjoy absolute freedom to do as they pleased in their new country. ","correct":false}],"randomize":true}}}

Dropdown type of questions seem to generate fine.

[benjaoming]

@mrpau-eugene you can manipulate the exercise data directly in ~/.kalite/database/content_khan_en.sqlite to see if you can pinpoint the problem through the JSON data?

For the record (because I haven't made note of it anywhere else): This error doesn't seem to give a javascript exception or traceback.

[mrpau-eugene]

@benjaoming okay. will try to see if I the culprit is really in the JSON data

[mrpau-eugene]

@benjaoming I think this piece of text that is highlighted is required for the questions to display correctly. After adding this line, it works fine 🤣

Working:

[benjaoming]

Great find @mrpau-eugene !

Let's ask @rtibbles what he thinks about the solution. Is this because of an error in upstream data, an old version of Perseus or should we try sanitizing these data in the ka-lite-content-packs?

[rtibbles]

It is the [[ radio 1]] that is vital to proper rendering, this is what indicates that the radio boxes should be present.

This looks like an upstream data issue, but you can double check by looking at the item data on Khan Academy at http://www.khanacademy.org/api/internal/assessment_items/

[benjaoming]

http://www.khanacademy.org/api/internal/assessment_items/

@rtibbles ⬆️ is a 404

@mrpau-eugene can you see if the broken questions work here? -> https://www.khanacademy.org/humanities/ap-us-history/period-3/apush-declaration-of-independence/e/aspen-us-declaration-of-independence-quiz

[rtibbles]

Oh, my <item_id> addendum disappeared.

http://www.khanacademy.org/api/internal/assessment_items/x699f189dd5bcc110

[rtibbles]

Seems like the [[ radio 1]] is missing in the source data above too.

[benjaoming]

@rtibbles is there advice for @mrpau-eugene about how to report the issue upstream? I guess we don't want to fix such issues in ka-lite-content-packs...

[rtibbles]

My guess would be that this might be flagged to not be shown on the main KA site, so if you can't see it there, it may be that we shouldn't be downloading it.

[benjaoming]

@mrpau-eugene worth investigating if we are including questions that should be skipped because they are flagged as @rtibbles suggests they might be.

In case they are flagged, we would have to check up on whether the ka-lite-content-packs is properly skipping these exercise questions and whether we have to build a new English content pack for that reason.

[mrpau-eugene]

@benjaoming @rtibbles
It seems like they aren't flagged as a question to be skipped in the main site as shown here:

I kinda don't understand but the placeholder has a value of 2 while other questions has a value of 1. Can someone explain to me what this value is for or what does it do? For a clarification.

[[☃ radio 2]]

Btw, here is the unicode of the snowman emoji. I found it somewhere in some questions \u2603.

[benjaoming]

@mrpau-eugene what's the diff between the KA.org JSON data for the above question, and the JSON data found in the KA Lite content database?

[mrpau-eugene]

@benjaoming aside from the missing [[\u2603 radio 2]], JSON data coming from KhanAcademy:

The widgets contains this two objects
"wdigets": {"radio 1": {....}, "radio 2":{...}} while the contents from the KA Lite content database only has "widgets":{"radio 1": {...}}

also, the "answerArea" object coming from KhanAcademy website contains only contains these:

while the contents in the KA Lite Database contains:

[benjaoming]

@mrpau-eduard I'm getting a feeling that it's worth seeing if this issue and the data in our content database can be made to match what's on KA.org by building a new English content pack en-zip, at least for further investigation. Perhaps ask @mrpau-eduard to build it so you can see if this will correct the issue somehow?

As you can see, it was reported on Jun 17, 2016 - so it has persisted several of our content packs.

[mrpau-eugene]

@benjaoming okay thanks for the suggestion :) I'll do some more investigations, too. I'll also talk to @mrpau-eduard regarding about this.

[mrpau-eugene]

@benjaoming @mrpau-eduard I would just like to take note here that this question has been removed from Khan Academy and confirmed that it no longer exists in the newly built content packs.

[benjaoming]

Nice find @mrpau-eugene ! I'm closing this now, just added a release note about.