Add custom field type Hidden #27235

colemanw · 2023-08-31T23:45:40Z

Overview

Adds a "hidden" custom field type which can collect data on forms without being seen (e.g. by storing a value from the url or set by javascript)

Technical Details

Adds a custom field html_type (aka input_type) of "hidden"
Also adds a "hidden" input_type to FormBuilder
This effectively hides the field and the label on any core form, profile, or FormBuilder form.
This improves the flexibility of FormBuilder. FormBuilder already allows input_types to be changed on a per-form basis, and this adds to the available options. So hidden fields can be un-hidden (widget changed to Textfield) and conversely, regular fields can now be changed to hidden.
Profiles aren't very flexible, and don't allow html_types to be changed. A hidden field will be that way on every profile.

civibot · 2023-08-31T23:45:42Z

🤖 Thank you for contributing to CiviCRM! ❤️ We will need to test and review this PR. 👷

Introduction for new contributors...

If this is your first PR, an admin will greenlight automated testing with the command ok to test or add to whitelist.
A series of tests will automatically run. You can see the results at the bottom of this page (if there are any problems, it will include a link to see what went wrong).
A demo site will be built where anyone can try out a version of CiviCRM that includes your changes.
If this process needs to be repeated, an admin will issue the command test this please to rerun tests and build a new demo site.
Before this PR can be merged, it needs to be reviewed. Please keep in mind that reviewers are volunteers, and their response time can vary from a few hours to a few weeks depending on their availability and their knowledge of this particular part of CiviCRM.
A great way to speed up this process is to "trade reviews" with someone - find an open PR that you feel able to review, and leave a comment like "I'm reviewing this now, could you please review mine?" (include a link to yours). You don't have to wait for a response to get started (and you don't have to stop at one!) the more you review, the faster this process goes for everyone 😄
To ensure that you are credited properly in the final release notes, please add yourself to contributor-key.yml
For more information about contributing, see CONTRIBUTING.md.

Quick links for reviewers...

➡️ PR Demo site (can be rebuilt with the command test this please).
📖 Review standards
🗒️ Review template (brief or verbose)

➡️ Online demo of this PR 🔗

larssandergreen · 2023-09-01T17:43:30Z

Comparing to the TrackingFields, I think users will have the reasonable expectation that a hidden custom field won't show up in a emails, but I think with this all the message templates that add all custom fields are going to include these hidden fields.

So if you have a hidden field for tracking where donors clicked, that's going to be sent to donors in their contribution receipt.

MegaphoneJon · 2023-09-01T18:48:53Z

I like this - but I think we need to spec this out a bit before committing to an approach.

I think the appropriate replacement for tracking fields isn't a "hidden" custom field type, but an "is hidden" checkbox on the profile field configuration.

"Hidden" IMO should be an HTML type, not a data type. This allows us to maintain strict data typing in API4, and overall seems more likely to play well with other code.

Do hidden fields appear in SK? I'd lean yes. Should they be editable? I suppose so. But maybe Eileen's is_hidden suggestion is better because we could maintain pseudoconstants for SK editing? I'm undecided.

MegaphoneJon · 2023-09-01T18:50:24Z

Also pinging @twomice since CRM-20922 suggests he has an interest

colemanw · 2023-09-01T19:03:40Z

@MegaphoneJon I've expanded the PR description to clarify that this does add a new html_type not a data_type, and FormBuilder does allow those to be changed on a per-form basis.

MegaphoneJon · 2023-10-13T20:20:44Z

OK, I accidentally got some real-world testing on this because I forgot this is part of the Megaphone Civi flavor.

Hidden fields work as expected in profiles. Yay!
On backend QuickForms of type "Page", the label appears, but the value does not. On QuickForms of type "Form", neither the label nor value appears.

In SearchKit, attempting to use these in the WHERE clause just gives an endless spinning circle. Adding it as a column does not display a result. However, you can use Rewrite on a display to show the "raw" value, not the label, and that works.

I think there's also some interaction here between the former HTML type and the current one. In the examples above, the field has an option_group_id set. I wonder if this should be a disallowed configuration.

colemanw · 2023-10-14T16:41:22Z

@MegaphoneJon thanks for the feedback. I've updated the PR and I think it addresses everything you noted.

colemanw · 2023-10-14T16:42:51Z

CRM/Custom/Page/Field.php

-      switch ($customFieldBAO->data_type) {
-        case "String":
-        case "Int":
-        case "Float":
-        case "Money":
-          // if Multi Select field is selected in custom field
-          if ($customFieldBAO->html_type == 'Text') {
-            $action -= CRM_Core_Action::BROWSE;
-          }
-          break;
-
-        case "ContactReference":
-        case "Memo":
-        case "Date":
-        case "Boolean":
-        case "StateProvince":
-        case "Country":
-        case "File":
-        case "Link":
-          $action -= CRM_Core_Action::BROWSE;
-          break;
+      // Remove link to edit option group if there isn't one
+      if (!$customFieldBAO->option_group_id) {
+        $action -= CRM_Core_Action::BROWSE;


This absolutely takes the prize for the most bizarre waste of 20 lines I've ever seen.

🎉 🎉 Why I'm awarding this one top prize 🏆 🥇 🏆

Switch statement: why even... dear god why?

The code comment made absolutely no sense.

The rest of the switch statement: LOL

None of it had anything to do with the actual criteria that needed to be checked

MegaphoneJon · 2023-10-18T00:41:45Z

Further feedback from user testing:

Can't edit the data in the back end (right now the hidden field is not visible when editing a contribution)
Can't view the hidden field as a searchkit column (the column is just blank)

But also that overall the last round of changes were all successfully tested.

colemanw · 2023-10-18T01:30:34Z

Can't edit the data in the back end (right now the hidden field is not visible when editing a contribution)

I wouldn't expect a hidden field to be editable. I imagine if we changed it to type text then we'd get bug reports that (IMO correctly) complain that hidden fields should be of type hidden not text.

Can't view the hidden field as a searchkit column (the column is just blank)

That does sound like a bug. I'll check...

colemanw · 2023-10-21T22:36:21Z

@MegaphoneJon sorry I'm unable to reproduce the 2nd bug you mentioned. When I create a new hidden custom field and enter some data (via the API since you can't do it in the UI) I'm able to see it both in the UI and in SearchKit:

However, if the field was created prior to that last bugfix (the one that took the prize) then you should check civicrm_custom_field to make sure the field does not have an option_group_id and update it if it does to set that column to null and delete the bogus option group.

civibot bot added the master label Aug 31, 2023

colemanw mentioned this pull request Aug 31, 2023

Smarty notice fix on trackingFields #25614

Closed

colemanw force-pushed the hidden branch from 08d4d81 to ca60ed5 Compare August 31, 2023 23:49

colemanw mentioned this pull request Aug 31, 2023

Delete probably-unused TrackingFields.tpl #27233

Merged

colemanw force-pushed the hidden branch from ca60ed5 to 402acf9 Compare September 1, 2023 18:49

colemanw force-pushed the hidden branch from 402acf9 to 834ed31 Compare September 1, 2023 19:10

colemanw force-pushed the hidden branch from 834ed31 to 8c148d8 Compare October 14, 2023 16:40

colemanw commented Oct 14, 2023

View reviewed changes

Add custom field type Hidden

2bf10c8

colemanw force-pushed the hidden branch from 8c148d8 to 2bf10c8 Compare October 14, 2023 18:11

mattwire merged commit ec33a6d into civicrm:master Oct 16, 2023
3 checks passed

colemanw deleted the hidden branch October 16, 2023 21:32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add custom field type Hidden #27235

Add custom field type Hidden #27235

colemanw commented Aug 31, 2023 •

edited

civibot bot commented Aug 31, 2023

larssandergreen commented Sep 1, 2023

MegaphoneJon commented Sep 1, 2023

MegaphoneJon commented Sep 1, 2023

colemanw commented Sep 1, 2023

MegaphoneJon commented Oct 13, 2023

colemanw commented Oct 14, 2023

colemanw Oct 14, 2023

colemanw Oct 14, 2023

MegaphoneJon commented Oct 18, 2023

colemanw commented Oct 18, 2023 •

edited

colemanw commented Oct 21, 2023

Add custom field type Hidden #27235

Add custom field type Hidden #27235

Conversation

colemanw commented Aug 31, 2023 • edited

Overview

Technical Details

civibot bot commented Aug 31, 2023

larssandergreen commented Sep 1, 2023

MegaphoneJon commented Sep 1, 2023

MegaphoneJon commented Sep 1, 2023

colemanw commented Sep 1, 2023

MegaphoneJon commented Oct 13, 2023

colemanw commented Oct 14, 2023

colemanw Oct 14, 2023

Choose a reason for hiding this comment

colemanw Oct 14, 2023

Choose a reason for hiding this comment

MegaphoneJon commented Oct 18, 2023

colemanw commented Oct 18, 2023 • edited

colemanw commented Oct 21, 2023

colemanw commented Aug 31, 2023 •

edited

colemanw commented Oct 18, 2023 •

edited