Parsing error with updated response options

[EMWPhD]

Awesome package! I think the issue that I am experiencing is related to issue #278, but I am experiencing some different functionality that I was hoping to understand more about.

When I use fetch_survey() I seem to receive a parsing error for any field where the response choice options for any questions have been updated in Qualtrics. Looking at the metadata for the survey, the original response choice seems to be stored under "choiceText" whereas the updated values are stored under "description" (example below).

Oddly enough, if I run fetch_survey() with the option import_id = T I do not get a parsing error and the updated text appears as the value labels and the columns are labelled using the QIDs (as expected).

I am just trying to understand this functionality better and see if there is a way to avoid these parsing errors using qnames instead of qids.

$choices$2
$choices$2$recode
[1] "2"

$choices$2$description
[1] "Updated Text"

$choices$2$choiceText
[1] "Original Text"

[juliasilge]

I tried to play around with this and I think this is once more coming back to the need for #267. Do you think that's right @jmobrien?

[jmobrien]

That would be my strong guess, yes--though as for what we should do about it directly I'm less clear.

I'm suspicious that much of qualtRics's conversion code (what's being referenced here) may predate many of Qualtrics's current features. So, while--for us--this is pragmatically about transitioning from the v2 to v3 API, I'm betting we'd be encountering issues like this even if they were still on the v2 system. And it won't be nearly as simple as just redirecting existing approaches to look at the V3 endpoints.

That said, this does make me wonder about some stuff. To help with that, @EMWPhD, is when you were "looking at the metadata", was that from metadata(), or the metadata from fetch_description()? (I know the names are confusing, sorry. We're working on it.).

Also, what exactly did you mean had been done when you said "where the response choice options for any questions have been updated in Qualtrics"? (Again, sorry--your wording was fine, I just have to ask for some subtle technical reasons).

[EMWPhD]

No need to apologize @jmobrien!  I am happy to clarify as best I can.

When I said the response choice options were updated in Qualtrics, I was referring to the "variable naming" feature that shows up in the "recode values" section of the platform for each question.  For a multiple choice question, this feature allows you to display a set of response options to the respondent, but change the wording on those response options for the exported results.  I am including the documentation from Qualtrics and an example below. 

In the example, the text displayed to the respondent is on the right and the updated text that is to be exported is in the middle column in pink (e.g. "Full-time", "Part-time" without the parentheticals).  When this feature is used, fetch_survey() results in a parsing error unless you include the option import_id = T. When that option is added, fetch_survey() correctly applies the updated labels.

The example that I included above was using metadata().  It seems like the original values are being stored as "choiceText" and the updated values are stored as "description."  The location in the list returned from metadata() is:
questions$<QID>$choices$1$choicetext/description 

I tried using fetch_desc() as well and the information is there too, but in a different location in the list.  The original text is under:
questions$<QID>$Choices$1$Display

And the renames values are under:
questions$<QID>$VariableNaming$1

"Variable Naming" Documentation:
https://www.qualtrics.com/support/survey-platform/survey-module/question-options/recode-values/

Example: 

[jmobrien]

Thanks for the extra info. Yes, that seems about right. Looking at this more closely, I think it's a v2/v3 issue in the sense that we're looking at the kinds of extra features things that they tried to build around in v3, but as you have probably noticed it's complex either way.

Fortunately, I'm now thinking it might be possible to fix this for now in a relatively simple way, without requiring it to be a part of a more wholesale redesign. But I need to check a few things first, and could use your help.

With a configuration exactly as you screenshotted just above, would you mind copying here the full content (from metadata()) of the relevant question? So, whatever QID that is, just the list that prints from <metadata_df>$questions$<QID>? It will be kind of long, I know, but it would be useful to see a few things all together. (Depending on what I see I might end up asking you for a bit more, but let's hold off for now.)

[EMWPhD]

Absolutely, happy to help in any way that I can. Below is the full list that prints from that QID. I hope this helps, but if there is any additional information that I can provide please let me know.

$questionType
$questionType$type
[1] "MC"

$questionType$selector
[1] "SAVR"

$questionType$subSelector
[1] "TX"


$questionText
[1] "Which of the following best describes your employment status:"

$questionLabel
NULL

$validation
$validation$doesForceResponse
[1] FALSE


$questionName
[1] "Q1"

$choices
$choices$`1`
$choices$`1`$recode
[1] "1"

$choices$`1`$description
[1] "Full-time"

$choices$`1`$choiceText
[1] "Full-time (greater than 30 hours per week)"

$choices$`1`$imageDescription
NULL

$choices$`1`$variableName
[1] "Full-time"

$choices$`1`$analyze
[1] TRUE


$choices$`2`
$choices$`2`$recode
[1] "2"

$choices$`2`$description
[1] "Part-time"

$choices$`2`$choiceText
[1] "Part-time (less than 30 hours per week)"

$choices$`2`$imageDescription
NULL

$choices$`2`$variableName
[1] "Part-time"

$choices$`2`$analyze
[1] TRUE

[jmobrien]

Great, this is useful. Could you perhaps do the same using fetch_description()? Then, can you deselect the "variable naming" box (leaving everything else the same), and do it again?

Don't need to post both outputs, necessarily, but I'm guessing there might be one or two elements that vary and what those are would be good to note what those are if you see them.

(And thanks. I would usually check these sorts of thing myself but I'm limited right now).

[EMWPhD]

Sure thing, happy to help. Below are the three examples. I ran fetch_description() first and then I ran both metadata() and fetch_description() with the variable naming option deselected on the survey.

using fetch_description() with variable naming on:

$QuestionText
[1] "Which of the following best describes your employment status:"

$DataExportTag
[1] "Q1"

$QuestionType
[1] "MC"

$Selector
[1] "SAVR"

$SubSelector
[1] "TX"

$Configuration
$Configuration$QuestionDescriptionOption
[1] "UseText"


$QuestionDescription
[1] "Which of the following best describes your employment status:"

$Choices
$Choices$`1`
$Choices$`1`$Display
[1] "Full-time (greater than 30 hours per week)"


$Choices$`2`
$Choices$`2`$Display
[1] "Part-time (less than 30 hours per week)"



$ChoiceOrder
$ChoiceOrder[[1]]
[1] 1

$ChoiceOrder[[2]]
[1] 2


$Validation
$Validation$Settings
$Validation$Settings$ForceResponse
[1] "OFF"

$Validation$Settings$Type
[1] "None"



$Language
list()

$NextChoiceId
[1] 3

$NextAnswerId
[1] 1

$QuestionID
[1] "QID1"

$DataVisibility
$DataVisibility$Private
[1] FALSE

$DataVisibility$Hidden
[1] FALSE


$RecodeValues
$RecodeValues$`1`
[1] "1"

$RecodeValues$`2`
[1] "2"


$VariableNaming
$VariableNaming$`1`
[1] "Full-time"

$VariableNaming$`2`
[1] "Part-time"


$QuestionText_Unsafe
[1] "Which of the following best describes your employment status:"

using metadata() with variable naming off:

$questionType
$questionType$type
[1] "MC"

$questionType$selector
[1] "SAVR"

$questionType$subSelector
[1] "TX"


$questionText
[1] "Which of the following best describes your employment status:"

$questionLabel
NULL

$validation
$validation$doesForceResponse
[1] FALSE


$questionName
[1] "Q1"

$choices
$choices$`1`
$choices$`1`$recode
[1] "1"

$choices$`1`$description
[1] "Full-time (greater than 30 hours per week)"

$choices$`1`$choiceText
[1] "Full-time (greater than 30 hours per week)"

$choices$`1`$imageDescription
NULL

$choices$`1`$variableName
NULL

$choices$`1`$analyze
[1] TRUE


$choices$`2`
$choices$`2`$recode
[1] "2"

$choices$`2`$description
[1] "Part-time (less than 30 hours per week)"

$choices$`2`$choiceText
[1] "Part-time (less than 30 hours per week)"

$choices$`2`$imageDescription
NULL

$choices$`2`$variableName
NULL

$choices$`2`$analyze
[1] TRUE

using fetch_description() with variable naming off:

$QuestionText
[1] "Which of the following best describes your employment status:"

$DataExportTag
[1] "Q1"

$QuestionType
[1] "MC"

$Selector
[1] "SAVR"

$SubSelector
[1] "TX"

$Configuration
$Configuration$QuestionDescriptionOption
[1] "UseText"


$QuestionDescription
[1] "Which of the following best describes your employment status:"

$Choices
$Choices$`1`
$Choices$`1`$Display
[1] "Full-time (greater than 30 hours per week)"


$Choices$`2`
$Choices$`2`$Display
[1] "Part-time (less than 30 hours per week)"



$ChoiceOrder
$ChoiceOrder[[1]]
[1] 1

$ChoiceOrder[[2]]
[1] 2


$Validation
$Validation$Settings
$Validation$Settings$ForceResponse
[1] "OFF"

$Validation$Settings$Type
[1] "None"



$Language
list()

$NextChoiceId
[1] 3

$NextAnswerId
[1] 1

$QuestionID
[1] "QID1"

$DataVisibility
$DataVisibility$Private
[1] FALSE

$DataVisibility$Hidden
[1] FALSE


$RecodeValues
$RecodeValues$`1`
[1] "1"

$RecodeValues$`2`
[1] "2"


$QuestionText_Unsafe
[1] "Which of the following best describes your employment status:"