Improved survey-* plugins

[becky-gilbert]

When overhauling the survey-* plugins, we could do the following:

• Allow different question types on the same page (the survey-form plugin does this, but this version would still be parameterized)
• Allow different pages of questions with back/next navigation, like the instructions plugin
• Add a validation parameter. I assume this would be a function that runs after the 'submit/next' button is pressed and checks whether or not the participant can continue. It would have access to all the elements on the page so could be used to check response values. See Conditional response options for multiple choice question #1206 for example use case.

[becky-gilbert]

Quick note on implementation: maybe add a validation function and validation_message parameter for each question, which can be used in the submit and/or input event handlers to check for response validity. Can override the automatic form validation with form.noValidate = true; which will cause the submit event handler to be called on every submit attempt. Then, check validity and use the setCustomValidity function to set/clear custom validation messages (out-of-flow callouts that usually appear with automatic validation).

Could also make full use of all built-in validation options (min/max, type, regex pattern, see here and here), and pseudo classes :valid, :invalid, :required, :optional, :placeholder-shown (and combine with :focus). Can also use form.reportValidity and form.checkValidty to get/test response validity in the submit and/or input event handlers.

See survey-text-validation plugin for example.

[becky-gilbert]

We could add a tabular formatting option for survey-likert questions - see #1917. I think this would only work when each question uses the same scale. Maybe we could detect whether each question uses the same scale as the previous one, and if so, stack the question, otherwise create a new row with the scale above the question. Or we could make the tabular option a different question type in order to create a different parameterization structure that allows multiple question prompts associated with one scale, something like this:

{question_type: 'survey-likert-table', questions: ['Question 1', 'Question 2', 'Question 3'], scale: ['A lot', 'Some', 'None']}

[jodeleeuw]

I'm starting to think more about this as the next major task.

I think it might make sense to build this plugin a little differently than others. We may want the main plugin file to just contain the basic logic for rendering the form and then create something like a class that each different form element is part of. The class would have a standard set of methods to render, validate, etc. This would allow us to design the survey components modularly and add support for new components relatively easily.

[becky-gilbert]

Yes I was thinking along the same lines. I thought we should have separate functions to display and validate/collect responses for each question type, so that they could be easily modularized and combined arbitrarily into a single form. I wasn't thinking about this in terms of classes but I think that's the same general idea. I can have a go at implementing this with classes but I might need some initial help.

Another tricky part of this would be to allow users to move forward/backward between multiple pages, display previous responses when going backwards, and allow them to be updated. I'm thinking this would have to be implemented in the plugin itself (like instructions), unless you wanted to introduce something to jsPsych core that would make it possible to do this with separate trials, but I'm guessing not?

[jodeleeuw]

Yeah, I think adding that to core would be too difficult at the moment. I could imagine the plugin having a pages option like instructions.

It's not strictly necessary that they be a class. I was thinking a class might be an improvement over a function because then render and validate are separate pieces (and other events too?). But there's probably a clever way to do that with a function based approach too.

[becky-gilbert]

Ok, maybe I should start by mocking up a very basic version that uses functions and just two question types. Then we could look at how to switch to classes, and I could do the remaining question types that way.

Another somewhat tricky thing is that there will be some differences in the required/available parameters across the different question types, and validating this across all question types would clutter up the plugin code. I guess each question type could have it's own check_parameters function. Would the class implementation help with this?

[jodeleeuw]

That sounds perfect.

I was imagining that each component question would have its own validation function that the user could specify. Then when it's time to validate the plugin just runs all the validation functions. I think this could be done with either functions or classes, but classes might be simpler to enforce a standard.

[bjoluc]

I just discovered SurveyJS which looks very promising and may be quite easy to integrate. It solves basically this exact use case (including pagination) and is MIT-licensed!

[jodeleeuw]

That does look like an impressive library that matches pretty closely what we want to do.

My biggest concern is integration time. There's a lot of functionality there! The good news is that the core feature of specifying the survey via JSON matches basically the exact specification we had in mind.

I'm also a bit worried about potential documentation mashups. If we build a "SurveyJS plugin" then there're tricky overlaps between docs and we may not be able to easily support the full range of features that it offers.

I'm really not sure what the right path to take here is.

[jodeleeuw]

PRs that have relevant content:

• survey-multi-choice table #2031
• Randomize options in survey-multi-choice and survey-multi-select #2030
• [FEAT]: add a parameter for tightening the scale in likert survey #644
• Add the scale_height property #1636

[becky-gilbert]

SurveyJS looks amazing. Unfortunately we're on a super tight schedule, and I think it would take me longer to figure out how to use it and get it to work in a simplified (and maybe non-standard?) way inside a plugin package than it would for me to just construct our own (much less fancy!) implementation, which I can do largely by piecing together existing bits of code. I'm sure someone who is experienced with SurveyJS and/or modern web development (modules, classes, Node) could put together a SurveyJS plugin more quickly than I could - if so, please let me know if you're interested!

@jodeleeuw I've started working on a basic/demo implementation of this plugin without SurveyJS, so please let me know if you've decided you do want to use it so that I can switch gears ASAP. We could also wait until I have a very basic non-SurveyJS implementation ready (should be soon), at which point you could decide if this approach seems too complicated and difficult to maintain/extend.

[jodeleeuw]

I'm fully on board with moving ahead without SurveyJS at first.

[becky-gilbert]

Got it, thanks!

[bjoluc]

I was initially thinking of a compromise between both approaches: We can expose whatever API we want to have for the survey plugin and simply use SurveyJS internally, without exposing its API at all. That way, we do not have to reinvent the wheel but easily get the features we want without maintaining them much. Later on, we may as well expose more functionality if we feel like it.

@becky-gilbert I was looking for ready-made solutions like this especially due to the tight schedule, but I do also get that implementing something from scratch may be more comfortable for you. The main benefit of using an existing lib may be that we can reduce testing to the absolute minimum because the lib is already tested extensively. If you're interested (and it doesn't invalidate too much of your so-far work), I'm happy to do all the integration stuff within the next day so you can focus on translating the API. But I'm not offended if you prefer a from-scratch solution!

[becky-gilbert]

@bjoluc

I was initially thinking of a compromise between both approaches: We can expose whatever API we want to have for the survey plugin and simply use SurveyJS internally, without exposing its API at all. That way, we do not have to reinvent the wheel but easily get the features we want without maintaining them much. Later on, we may as well expose more functionality if we feel like it.

Yes this is exactly what I had in mind. But this is also what I meant by possibly having to figure out a simplified/non-standard way of using SurveyJS. I'm guessing the tutorials etc. assume that you're setting up a form using one of a few different standard methods, using their various platform-specific files etc. Whereas we I think we'd just want to use pieces of their API internally to generate (parts of?) the form, so that we can retain control over other aspects of the form/page, including CSS, what data is collected and how it's stored, etc. But I haven't looked too closely at the tutorials and documentation yet, so I'm not sure how difficult this would actually be.

The main benefit of using an existing lib may be that we can reduce testing to the absolute minimum because the lib is already tested extensively.

Totally agree with this,. I'm on board with not re-inventing the wheel, if possible.

If you're interested (and it doesn't invalidate too much of your so-far work), I'm happy to do all the integration stuff within the next day so you can focus on translating the API.

That would be great, thanks! One of the things I wasn't sure about was how to access the SurveyJS library from within a jsPsych plugin. So if you could set up a basic plugin template that provides access to their API and loads a basic style sheet, then I could write the code to translate jsPsych trial parameters into their format go from there. Is that what you were thinking?

I'm still not sure which of these approaches will end up being easier to implement, but since you're offering to help, it's probably worth trying out SurveyJS before deciding.

[bjoluc]

So if you could set up a basic plugin template that provides access to their API and loads a basic style sheet, then I could write the code to translate jsPsych trial parameters into their format go from there. Is that what you were thinking?

100%! I'm currently on it, and it looks like the imperative way (as opposed to the declarative (json) style) in the trial method will look like this (easy to tweak with intellisense because they use typescript):

  const survey = new Survey();
  const page = survey.addNewPage("page1");
  page.addNewQuestion("text", "question1");

  survey.render(display_element);

I'll let you know when something is ready (hopefully in an hour or so).

[bjoluc]

@becky-gilbert Ready! I created a plugin-survey workspace on the plugin-survey branch that has everything now, including a local example.html file to get something up and running quickly. I think the bootstrap theme looks promising, but there's a ton of alternatives and customization options. I added two links to the corresponding docs sections as well. As always, you'll need an npm install in the repo root. So far I love it, only that the plugin bundle becomes ~1.2 Mb now. I'd need to do some research on how to best slim that down. Happy hacking ☺️

[becky-gilbert]

@bjoluc fantastic, thank you!!

[becky-gilbert]

Update: SurveyJS is way easier to use than I'd expected (at least now that @bjoluc has set everything up 😃 ), well-documented, and has all of the features and customization options we need. It seems easy to just implement the features/options that we want to offer, and to keep the parameterizaiton options in our own documentation without referring to SurveyJS docs (which as Josh says, would probably cause problems/confusion). So at this point I'm convinced that this is the best option.

If anyone has concerns (about using SurveyJS or external dependencies in general), please let me know ASAP so that I don't go too far down this track!

[bjoluc]

Awesome! No concerns from my end. I think the bundle size is acceptable for web experiments since it boils down to 276KiB gzipped. That's still not nice, but there seems to be no interest to improve this on their part (surveyjs/survey-library#1362 (comment)) and I think the slightly better loading times are not worth maintaining a fork.