Implement invisible Recaptcha

[continga]

Pull Request for Issue #14565 and following up #14583 and #16599 et al.

Summary of Changes

This PR implements the possiblity to use the Invisible reCAPTCHA using a separate plugin. Besides implementing the basic functionality to make invisible reCAPTCHA working, there has been made some smaller architectural changes to the whole captcha process (e.g. using Exception instead of JError for errors) and the reCAPTCHA PHP library from Google has been updated, alongside with some smaller changes in the old reCAPTCHA (V1 und V2) plugin.

As this is one of my first PRs for Joomla!, please dont hesitate to make suggestions of how to improve it.

Testing Instructions

1. Create a site- and secret-key at https://www.google.com/recaptcha/admin for an invisible reCAPTCHA (reCAPTCHA V2 keys will not work with Invisible reCAPTCHA)
2. Configure the Invisible reCAPTCHA plugin in the Joomla! backend, entering at least the site- and secret-key
3. Configure the Invisible reCAPTCHA plugin to be used as captcha plugin (System -> Global Configuration -> Default Captcha)
4. Open up a form in the frontend where a captcha is enabled (e.g. by creating a menulink to a contactform)
5. Notice the reCAPTCHA logo on the right side (see screenshot further down) which indicates that the invisible reCAPTCHA has been executed.

We also need to make sure that the reCAPTCHA plugin (V1 and V2) still is working correctly (as there has been made some bigger changes to the code by me), although I have tested this toroughly already, but I would be very glad if other people could take a look at this too, I don't want to take chances that this PR breaks something.

Additionally, as this PR changes the rendering of form fields for the registration form (this was needed as the registration form was using its own rendering of form fields, which caused issues with the invisible reCAPTCHA), we need to make sure the registration form for users is working as expected. So we need to test the registration form in resprect to custom fields of users etc, and whether they get displayed correctly now.

Expected result

[brianteeman]

Because of the keys issue would it not make sense to create this as a new plugin?

[continga]

V1 and V2 keys also were incompatible, and we had them in the same plugin. Additionally, V2 and invisible share very much of the code (e.g. the whole server-side validation), so I don't see any advantage when using a separate plugin. Additionally, the "key issue" is pretty clear on the Google reCAPTCHA admin page, so I think it shouldn't confuse users. I just wrote it down in the testing information to make clear that new keys has to be created to test this.

[brianteeman]

I thought it was a mistake last time as well

[continga]

Well, we could do it as a separate plugin now (it is not very much work to disassemble it), but where do you see the advantage exactly?

[brianteeman]

You would be able to use invisible!e on some forms and V2 on others.
It will also make future updates easier without having to have switches in the code for the old keys

[continga]

Okay, true. Especially the first point is very interesting. I think we should wait for some more feedback, but if the broad agreement is to put it into a separate plugin, I will be very glad to do so.

[810]

i vote for same plugin

[Fedik]

it should be a plugin, there even a pull request for it, already, somewhere here

[paulus103]

Hi This looks a nice feature to add :)

[ghost]

@paulus103 can you please test?

---

_{This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/18146.}

[paulus103]

@franz-wohlkoenig

I have tested this on a live server and it works :)

---

_{This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/18146.}

[ghost]

@paulus103 thanks for Info. Please mark your Test as successfully:

• open Issue Tracker
• Login with your github-Account
• Click on blue "Test this"-Button above Authors-Picture
• mark your Test as successfully
• hit "submit test result"

[paulus103]

I have tested this item ✅ successfully on 259a657

tested successfully on an 3.8.1 old development site :)

---

_{This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/18146.}

[conseilgouz]

What can be done, so this pr will be included in 3.8.3. I was disappointed not seeing it in 3.8.2.

If I can help, let me know,

Pascal

[ghost]

@pmleconte Pull Requests needs 2 successfully Tests. If you can test this PR, next Step is done.

[ghost]

@continga can you please resolve conflicting Files so PR can be tested?

[Fedik]

I would really suggest to make/test a plugin (there even already one #16599)
and invest your time in to a plugin

[ghost]

Status is set on "Needs Review".

[conseilgouz]

I have tested this item ✅ successfully on 259a657

Test ok on 3.8.1 site. Both V2 and invisible captcha are OK.

---

_{This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/18146.}

[coolcat-creations]

Do Custom fields still work with your change?

[continga]

Custom Fields in the registration form? Yes, I think they will still work. To be honest, I think they will work even better than before, as they will be rendered completely using their own mechanics - and not using the "hardcoded" way in the registration view like before.
But, as I said before, I'd like to get some more input to that point

[mbabker]

Then the field renderer of Joomla! is being used to render that form, and the problem is solved for the registration form. I did some quick tests, and I did not see any differences in the generated form when using $field->renderField() instead of the current implementation, but I am unsure whether this would be the way to fix this.

In general I would suggest that unless there is some major need for styling that contradicts what the form fields and their layouts render that $field->renderField() should always be used (or, if you have all the fields enclosed into a fieldset, $form->renderFieldset($fieldset->name);). Not doing so actually makes it harder to use plugins to dynamically alter forms and I'd almost suggest adding a static analysis tool raising warnings about manual rendering of field elements (echo $field->label; or echo $field->input;).

[continga]

I totally agree with you @mbabker and that is also why I want to change this. Good to know you are the same meaning. Let us wait for some more feedback, but you just verified to me that'd be the way to go 🙂

[roland-d]

@continga I also agree with your findings and those of @mbabker . Just like com_contacts we would need a layout override file to account for this code:

<?php if (!$field->required && $field->type !== 'Spacer') : ?>
										<span class="optional"><?php echo JText::_('COM_USERS_OPTIONAL'); ?></span>
<?php endif; ?>

This is also done for com_contacts. That would clean up nicely.

[continga]

I now fixed the label issue, see commit f75d99c6.

Now from my point of view every feedback and review-change-request has been applied. Can you guys maybe take a look again and check whether this is mergable? This PR is pretty old and I am crossing fingers we finally get this thing merged 🙂 Thank you in advance!

[roland-d]

@continga I am sure we will get it merged once all issues are solved :)

When I do a password reset, I see this form:

Same when I do forget username I see this form:

So I think for these 2 pages you still need to add the layout fix.

Looking good:

• New user registration
• Contact form
• Edit article

Those are the only places I can think of and find where the captcha could be shown.

[continga]

Thanks @roland-d I fixed the cases you found.

Additionally I checked the sourcecode for other cases where form fields are being rendered manually and found some additional ones in the com_users login, password reset confirmation and success views. There shouldn't be a captcha there, but just to be sure I replaced the manual rendering with the Form renderer.

As far as I see, the only other places where forms are being rendered manually now are in several com_config views, and in the profile edit view of com_users. Both cases are not easily straight-forward replaceable by the Form renderer, so I left them in their current state, also because they don't display a captcha at all.

Can you check again? ☺️ Thank you very much in advance!

[roland-d]

I have tested this item ✅ successfully on 67ac41a

After applying the patch I setup the ReCaptcha invisible settings in the plugin and after that I tested the following view:

1. User Registration (http://joomla-cms-live.test/index.php?option=com_users&view=registration&Itemid=405)
2. Forgot Password (http://joomla-cms-live.test/index.php?option=com_users&view=reset&Itemid=409)
3. Forgot Username (http://joomla-cms-live.test/index.php?option=com_users&view=remind&Itemid=406)
4. Edit an article (http://joomla-cms-live.test/index.php?option=com_content&view=form&layout=edit&a_id=6&Itemid=257&catid=26)
5. Contact Form (http://joomla-cms-live.test/index.php?option=com_contact&view=contact&id=1&Itemid=229)

All the views show the Captcha logo and allowed me to process the form successfully.

I had a look at the JS callbacks as well they work fine for me.

---

_{This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/18146.}

[MastersOfMedia]

I have tested this item ✅ successfully on 67ac41a

✅ I have tested this item succesfully on a fresh install of Joomla! 3.8.11. On all 5 forms the invisible ReCaptcha badge show correctly and I was able to enter and process the form data correctly. By altering the keys in the plugin settings I was able to determine the validation actually works correctly.

---

_{This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/18146.}

[ghost]

Ready to Commit after two successful tests.

[mbabker]

Merged to 3.9-dev via 9a12f28

[tecpromotion]

has to be PLG_RECAPTCHA_ERROR_CALLBACK_LABEL

[tecpromotion]

hast to be PLG_RECAPTCHA_ERROR_CALLBACK_DESC

[tecpromotion]

two strings are duplicated in the en-GB.plg_captcha_recaptcha.ini file and must be corrected.

[brianteeman]

@tecpromotion
Good spot

as this has been merged already can you do a PR to fix it - or shall I ?

[tecpromotion]

@brianteeman
It would be an honor for me to write the PR. I found the mistake because I'm doing the german translation :)