Error message when no input provided for question

[simondate]

Subject of the issue/enhancement/features

Questions (MCQ, GMCQ) don't provide any input when submit is pressed with no answer submitted.
The Matching component does prompt users but only does so by highlighting the dropdown boxes that haven't been selected. Text Input and Slider just takes the default value (empty or 1).

Your environment

• version (AT/Framework) v5.6.0

Note: Proposed solution below, comments welcome.

[moloko]

IIRC it is supposed to highlight the instruction text in red...

[tomgreenfield]

Reference before Vanilla refactor: https://github.com/adaptlearning/adapt-contrib-vanilla/blob/157b98cc45d0febadfa6f5f3a4e07bfded586676/less/core/component.less#L45-L49.

[oliverfoster]

We also have the option to do something different. Like a notify with a warning or similar.
Can confirm: New vanilla doesn't have above styling equivalent. Colour variables are there. Code executes properly. Class is added. Just no styling.

[simondate]

I agree it should be more than just colour as this wouldn't be accessible - https://www.w3.org/TR/UNDERSTANDING-WCAG20/visual-audio-contrast-without-color.html

[moloko]

@oliverfoster switch to having the submit button disabled until the learner has met the canSubmit criteria?

[oliverfoster]

Disabling the button sounds like a plan. The instruction text would have to make it pretty clear each time that an option needs to be selected. Slider might be an issue with any global button disablement, this is as the first option is always a valid selection.

Update: Slider is fine.

[oliverfoster]

What do we think about the above PR?

PR in gif format:

[moloko]

looks good to me. increasingly this seems to be the desired behaviour for many of the clients I look after so it would be really helpful for this to be the default...

[oliverfoster]

QUESTION:

1. Default to have Submit disabled before the conditions are met but add an optional setting to re-enabled.
2. Default to have Submit disabled before the conditions are met and make this behaviour the new unchangeable default.

Note: In case 1 it is possible leave the submit button enabled with no warning - which is the current behaviour.

[moloko]

it's a good question, I'll ask it on the forums and chat and see what people think...

[oliverfoster]

I'll get the prs for matching and textinput done in the meantime.

Below:

[chucklorenz]

👍 for default unchangeable. 2.

[oliverfoster]

https://axesslab.com/disabled-buttons-suck/

I have added this as an alternative.

QUESTION:

1. Default to have Submit disabled before the conditions are met but add an optional setting to re-enabled.
2. Default to have Submit disabled before the conditions are met and make this behaviour the new unchangeable default.
3. Default to have Submit disabled before the conditions are met and have an alternative warning in-case it is needed for accessibility.

Note: In case 1 it is possible leave the submit button enabled with no warning - which is the current behaviour.

course.json: _buttons._incompleteWarning._isEnabled = true (defaults to false)

[paul-mediakitchen]

Thanks for asking for my feedback on this. I would like to discuss this with a colleague who is off work today so I will reply tomorrow if that is OK?

[oliverfoster]

Sure. No worries. 👍 there's no immediate rush, it's been like this for while, a few more days won't hurt.

[paul-mediakitchen]

Still discussing this in-house. One question that has come up that is a concern. If you have a MCQ question where there is more than one answer but the user does not know how many correct answers there are. For example the instruction is "Select one or more answers that match the criteria..."
The submit button would initially be disabled but then the user would see it become enabled as soon as they select one answer. We are concerned the user could view this change in state as an indication that the question is now ready to submit. Whereas the question may in fact require more than one answers to be selected to get the overall question correct.

[moloko]

@paul-mediakitchen do you have a proposed solution to resolve this? Or would leaving in an option to keep the existing 'submit enabled by default' behaviour (combined with a better system for alerting the learner as to why clicking the Submit button has no effect) be sufficient?

[oliverfoster]

I worry the effort involved in designing our way out of this might become somewhat burdensome and be a compromise in all directions - this issue has been around for a while.

Users will click a submit button prematurely because we are mostly not reading the instructions.

The incomplete warning included with Option 3 might help as the Submit button is used to present the instructions to the user in isolation on an invalid selection.

Nothing should stop the mcq being submitted with an incorrect number of selections, as that part of the 'one or many...' use-case.

In absence of the user reading the instructions (a norm) and in keeping the ability to allow the user to select an incorrect number of options (by design), some users will inevitably negatively act upon a button activation with the checkbox-style mcq. Conversely it might also improve the radio-style mcq experience, in that the activation of the submit button is a postive prompt to action.

If it were possible to utilize either the incomplete warning with an always enabled submit button, or just a disabled submit button, but at a per component level rather than just at the global level, that might give us a bit more flexibility.

We could then tailor the behaviour of each question where a given design demands. It would give us two avenues to explore against each of the problems listed and it would mean that Simon's issue and this outstanding bug could be solved, at least in the short-term.

[chucklorenz]

My feeling is that the current strategy of changing the styling of the Instruction is simply too weak and easy to overlook (e.g., sometimes off screen, blends in with theme, learner eyes trained on Submit button). I like the "incomplete" warning/popup Ollie presents in option three. This kind of strong feedback/warning satisfies me to the point that I would leave the Submit button enabled. If we go in this direction, I encourage providing a default text (e.g., "Select an option, then Submit") in case no Instruction was used. I have worked on many courses where no Instruction was provided.

[moloko]

if adding in the new ‘on submit error’ pop up (which i agree is a great improvement on what we were doing before) plus making the default state of the submit button configurable works for everyone then I’m happy to go with that.

my only concern is that making this configurable at component level would make it a total PITA for AAT users who want have all their questions configured differently from whatever the default ends up being. maybe we could have a global setting and default the component level one to ‘inherit’?

[moloko]

equally if people feel that’s just going a bit OTT with yet more configuration options then I’d be happy with leaving it as ‘submit enabled by default’ and, if anyone wants the opposite behaviour, I have a “disableSubmit” extension which I’d be happy to publish for others to use

[moloko]

if we do go down that route then it would be helpful if something could be added to the FW to make it easier for the disableSubmit extension to be notified once the question is in a ‘submittable state’ - at the moment the code to detect that is pretty hacky...

[oliverfoster]

That would be pretty easy to add to the attempts state API pr. It introduces a model function at the end of the submit execution.

https://github.com/adaptlearning/adapt_framework/pull/2748/files#diff-82e5394bd58b8d87f095f6498ab0cb10R215

https://github.com/adaptlearning/adapt_framework/pull/2748/files#diff-d0d36b72b61f2440cb1e0f735ae0cb8dR337

[paul-mediakitchen]

The option to have a global setting to configure the submit button to disabled would be great in addition to being able to set it per question. Having the global toggle means it is not a big deal if the default setting is the opposite of your requirements.

And having a pop up type notification when user selects submit when no answers are selected in the scenario where submit button is not disabled is a must really as if you have not included an instruction there would be no visual indication at all in the current implementation.

I personally prefer the submit button disable functionality to be an official part of Adapt rather than a plugin that Matt has shared with the community.

We are still discussing this but I think as long as we are not forced to have the submit button disabled we should be happy.

[moloko]

I personally prefer the submit button disable functionality to be an official part of Adapt rather than a plugin that Matt has shared with the community.

heh, interesting - after having pondered this a fair bit I'm in favour of not adding more config options (and more settings to test) for what seems like the less-common use-case - and instead delegating that out to a plugin (which would be owned by Kineo not me btw - although we also have the option of making it a non-core Adapt one like Glossary if that's preferred?)

Esp. if something could be added to Adapt to make the disableSubmit code simpler and less hacky than it is now!

[oliverfoster]

List of tasks:

1. Add disabled by default to the framework with a global option to turn it off
2. Make an extension to do the strong feedback at component level with support from the framework

[moloko]

@oliverfoster unless I've misread/understood people's comments, I think we are all onboard with going for leaving submit enabled by default and changing the error-on-submit to a notify popup.

What's up for debate is whether we:

1. add 'submit-initially-disabled' to the framework with a bunch of config options to allow it to be enabled in a way that's flexible and not a PITA to configure in the AAT, or:
2. have an extension that handles that behaviour (thereby obviating the need to add more configuration options) together with some improvements to the FW to allow that extension to be simpler/less hacky

Option 2 is my preference; @paul-mediakitchen has expressed a preference for option 1.

[oliverfoster]

I don't understand how putting the behaviour in an extension obviates the need to add more config as we'd still need component level granularity.

Having an extension to turn off core feedback seems a bit weird.

[moloko]

I don't understand how putting the behaviour in an extension obviates the need to add more config as we'd still need component level granularity.

because if we put the ‘disable submit by default‘ behaviour into an extension the configuration options for it would only be shown in the AAT when the extension is being used in a project i.e. for the majority of users we would not be adding any more config options.

Having an extension to turn off core feedback seems a bit weird.

I’m not sure what you mean by “an extension to turn off core feedback”... unless I’ve missed something this is not something that has been mentioned.

[oliverfoster]

It sounds as though you'd like the strong feedback to be the primary use-case and have disable submit as the secondary in an extension to disable the strong feedback.

I was thinking the disable submit should be the primary use-case and the strong feedback should be the secondary extension as strong feedback is more specialised (checkbox mcqs) and feels more like a feature addition / an extension worthy bit of behaviour.

It's also a bit easier to implement that way round, in my head at least.

I guess an argument could be made that the strong feedback is more accessible and should be the default. But I don't like the idea of adding extra popups by default, especially as we're looking at doing inline feedback for a better mobile experience. Popups on mobile are a bit jarring.

[moloko]

Assuming by 'strong feedback' you mean 'display an error in a notify popup when the learner clicks submit without have made a valid selection' (as shown here) then yes I think I'm right in saying that myself, Chuck and Paul all feel it should be changed to this by default since the current system of showing a 'submission error' is a) currently broken and b) isn't very good even when it is working.

Yes I agree over-use of popups can be jarring but in the case of displaying an error then a popup seems to me to be a pretty normal thing to do.

I was thinking the disable feedback should be the primary use-case

That was my thinking too originally by I think that both Chuck and Paul have made good cases for the submit button to be left enabled by default.

and the strong feedback should be the secondary extension as strong feedback is more specialised (checkbox mcqs) and feels more like a feature addition / extension worthy bit of behaviour.

I'm not sure what you mean by 'more specialised', sorry. Perhaps we could pick this up in the Monday meeting as it feels like we're going round in circles here.

[oliverfoster]

Cool. Sounds like a plan. We've got a clear idea of the solutions.

[moloko]

@simondate / @chucklorenz / @paul-mediakitchen would you like to join the meeting on Monday to talk this through and offer opinions/suggestions? The meeting is normally midday BST but if it would be more convenient to make it later (particularly for Chuck!) then please suggest a time.

[simondate]

Anytime after 2 pm BST on Monday is good for me.

I'm currently helping a client create a theme in Adapt and they've hired an Accessibility testing company to test it, it was actually from their testing that I created this issue in the first place.

I have a meeting with them at 1 pm so I can ask their opinion about this issue and their recommended implementation. Then have it ready to share with you guys.

[moloko]

@simondate that sounds like it could be really useful feedback, thanks!

[chucklorenz]

Thanks for the invite--much appreciated, but I'm pressed for time on Monday. I won't being attending this time. You've validated my main concern that the current strategy that relies on styling the Instruction is too weak a solution. And I echo the general sentiment that the number of config options in the AAT can get burdensome (oh, for a way to hide "advanced" configs...). Seems to me the issue of whether to enable or disable the Submit is a matter of UX in a learning context. I suspect that you folks can represent the needs of learning designers better than I. One last thought: is there benefit from taking a clue from other authoring tools such as Storyline? Seems many designers bring their Storyline experience/expectations to their Adapt projects. Cheers!

[paul-mediakitchen]

@simondate that sounds excellent. In the past we have had content tested by the Digital Accessibility Centre (DAC) - and they produced a very comprehensive report.

[paul-mediakitchen]

BTW happy to join the call today. My colleague Ian Robinson (fellow developer) would like to join too if that is OK? Is it at 2pm?

[moloko]

@paul-mediakitchen yes of course; it'll be 16.00 BST at this url https://hangouts.google.com/call/8g_asKG_3vA2A0BILi_mACEI

[paul-mediakitchen]

Great thanks @moloko

[oliverfoster]

Implemented as discussed. Extension link is included on the pr with a new repo in adaptlearning. I haven't yet registered the plugin and the supported framework version will need bumping once the pr is released.

[oliverfoster]

@simondate this should all be working now with the core components on their master branches, could you check?

git clone https://github.com/adaptlearning/adapt_framework/ ./reset/
cd ./reset/
adapt devinstall && npm install
grunt dev

Install this extension for the alternative https://github.com/adaptlearning/adapt-contrib-instructionError