Forms, inputs and validation

[stubailo]

Article outline

https://github.com/meteor/guide/blob/master/outlines/forms.md

Major decision points

1. autoform is the best Meteor form solution available today, we're going to make sure it integrates seamlessly with the method solution above
2. CollectionFS is the best file upload solution

---

Old info:

Related PR: #27

Today I got the chance to spend some time talking to @tmeasday, who's visiting from Australia! He's bringing his years of experience writing production apps as part of Percolate Studio and MDG, and his learnings from co-writing Discover Meteor, to the Meteor Guide project. Awesome!

Let's get into what we talked about regarding forms, methods, and user input in general.

Features of a good form solution

1. Form UI - can be expressed in HTML or auto-generated
2. Validation - optionally done in realtime while user types, then is done on submit, then is done inside the method on the server-side
3. Runtime errors from the server - not everything is a validation error, and certainly not all can be caught on the client
4. Methods - you want to control exactly what code is running when you submit a form. Automatic handling via default insert/update can get you to a certain point, but it's hard to go from there

The biggest requirement we identified here is a need for a standardized pattern for different kinds of input errors you might want to display to the user. Then we also came up with some ideas about the code that could be involved:

1. The form, button, or other UI
2. The validation logic for user inputs/schema, which happens to coincide with the arguments of the method so it can be reused to check the method arguments as well
3. Authorization for the method
4. The method body itself

This suggests some idea of a "validated method" - a method that isn't just a single function, but perhaps several that can be run separately. Similar to @SachaG's two-tiered methods, except all of the tiers are conceptually part of the method, and there might be more than two of them.

Here's a diagram that represents the picture I have in my mind. Obviously it's not finished, but it outlines the shape of things that I see:

Action items

1. Designing a standard error format, so that when you throw an error from a method, it's clear what kind of error it is, how it should be displayed to the user, whether it's related to a specific field, etc. This way different packages can show the same kinds of errors. I'd love to see some prior art on this - most validation libraries either throw terrible errors (like check) or have a super custom form integration (like simple-schema).
2. @aldeed's AutoForm is awesome, and has a ton of different plugins and integrations on Atmosphere. We'd like to use that, and work to improve its integration with the validated method pattern. Basically this means making it simpler to work with hooks and supporting the error format from (1) so that it can be used with errors outside of simple-schema.
3. Simple schema is possibly the best JavaScript schema definition and validation library out there. We'd like to deprecate the check package by bringing the best parts of that into simple-schema; in particular that means having a way to just do a one-shot check of an object against a schema without needing to first initialize a validation context. This way it will be much more natural to reuse the schema in the form and the server-side validation.

Notes

• Methods should be idempotent where possible due to reconnect re-runs

[stubailo]

@tmeasday the action items from this should be added as separate issues if we are sure they should actually be done.

[tmeasday]

👍 I've been having second thoughts about deprecating check as being the right call for this iteration of the guides however. What do you think?

[stubailo]

My only reservation is that then you won't be able to easily check stuff like the length of a string in a method argument, right?

[tmeasday]

In a world where the method "checks" form the validate part of the method?

I guess I'm more thinking of trying to re-implement audit-argument-checks based on SS might be non-trivial. I'm not sure what the best approach is if we don't do it though.

[tmeasday]

Proposed outline

1. How to write a method with proper permissions and validation (using methods package "Methods" package #54 and simple-schema)
2. How to call that method from the client side in various ways
3. How to build a simple form that calls the method
4. How to use autoform to build the form + show errors as they appear
5. How to do as-you-type error validation
6. How to handle optimistic form submission + it's implications (see UX chapter).
7. How to build a form automatically from a schema using autoform
8. How to handle file uploads:
   1. Over DDP
   2. Using CFS?
   3. w/ a 3rd party upload solution
   4. manually client side w/ something like S3?

[tmeasday]

@stubailo not sure about

1. Using a method to call external APIs. It feels a little out of place...maybe we should have that APIs chapter after all?
2. Saving intermediate state if a user closes a tab -- do people do this? I've not done it before. Does autoform make it easy?

[stubailo]

1. Yeah I think that external integration article is starting to feel necessary. It could be a short one.
2. GitHub and Asana both do it. It's not easy, I just figured it's something a lot of apps would benefit from. Autoform already persists form content across hot reloads, perhaps it could be extended to do the same across longer periods? Maybe this goes in a section at the bottom like "things we would like to be easy, please build a package for this"

[aldeed]

Wow, I appreciate the love @stubailo. :)

Regarding check/ss, this has been supported for a long time:

check(obj, new SimpleSchema({ ... }))

(And gives more helpful errors than basic check patterns.) But if check method were built into SS, I guess it could be shortened to check(obj, {}) with the SS object instantiation being automatic.

AutoForm thoughts/roadmap:

• There are currently some sticky issues that require some backward-incompatible restructuring, so I'm hoping to release a major version update before the end of this year. This should finally get it to a point that I'm relatively happy with.
• Move bootstrap dependency to a separate package. Use plain templates as default.
• Add a better API for working with form state, etc., like AutoForm.form(id).field('name').isValid / isDirty / hasChanged / etc.
• As part of the one above, input types will now decide when they have changed. Currently the core pkg tries to guess when an input changes using change handler, but it is very buggy on certain types of inputs. This will fix issues with getFieldValue, etc.
• Decouple various pieces so that you can auto-render any component (input, quick field, etc.) without needing an autoForm wrapper or even a schema. This will allow use in places where you just want to render a field, a selection box, etc. using the same template you use elsewhere in the app, but you don't need full validation, submission, or any of that. You can just drop the component in the template and use reactive getFieldValue to adjust view as value changes.

After all that, I'm going to investigate either a release or a port to support React.

[stubailo]

major version update before the end of this year

What does this mean for the huge library of autoform plugins?

Move bootstrap dependency to a separate package. Use plain templates as default.
After all that, I'm going to investigate either a release or a port to support React.

Could this be accomplished as kind of a refactor to move the core logic into a place independent of the view layer somehow? I don't know exactly what that would mean, but it probably boils down to a consistent JS API for the different form components.

[aldeed]

I've attempted several times to figure out a way to separate form rendering, validation, and submission concerns into their own packages, but it's pretty impossible right now. That said, every change I make is with an eye toward that eventual goal.

I don't think there will be huge impacts on the add-on pkgs when releasing 6.0, but any that provide custom input types with tricky values will need to implement the new API for change tracking, basically emitting some custom event when the input value should be considered changed. And they will all have to update their dependencies to || 6.0.0

[tmeasday]

@aldeed I think the love is totally warranted.

I should publish and point you at the react form component lib we built for Galaxy. I'll do that sometime soon. It's in no way as full-featured as autoform[1] but I suspect it has a lot of the same ideas, perhaps leaning more in the direction that we'd love to see autoform go in (and you are probably leaning also by the sounds of it).

[1] I actually don't think it's "correct" in the way it passes data between the form and the inputs, but that's a hugely technical discussion that is out-of-band here.

[tmeasday]

Outline merged https://github.com/meteor/guide/blob/master/outlines/forms.md!

[gwagner57]

I've written a tutorial on Constraint Validation in a Plain JS App, and I'd like to propose taking a similar approach in the Meteor Guide. The most important points are:

• Constraints have to be defined in the model (classes), and not in the UI
• Constraints have to be checked multiple times:
  • in the UI on input/change and before submit
  • in the model (class) code before save
  • possibly also in the DBMS before commit
• Responsive constraint validation in the UI should be done with the HTML5 constraint validation API using the two methods setCustomValidity and checkValidity.

Although I don't know much about Meteor (yet), I'd like to contribute material from my tutorial and co-author a corresponding Meteor validation tutorial.

[merlinstardust]

@tmeasday thanks for pointing me here in reference to files. Is CollectionFS still considered the best solution? My concerns with the package are the number of outstanding issues it has and that it doesn't seem to be maintained.

[tmeasday]

@merlinpatt, honestly, I'm not sure. We wrote this outline a few months ago before the maintainer of CollectionFS decided to move on. What are other people doing?

[merlinstardust]

Looking up "file" or "upload" on Atmosphere returned the below, which I picked because they had the most downloads. I also just put a poll up on the forums about what people are using. I suspect most will say they're still using CollectionFS.

• orionjs:filesystem
• vsivsi:file-collection
• ostrio:files
• tomi:upload-server

[merlinstardust]

I did just realize that with 1.3 having better support for NPM, it might be better to suggest a file upload solution from the NPM community.

[lorensr]

Are there full-stack npm file upload packages? FWIW, I chose https://github.com/CulturalMe/meteor-slingshot and works well for me

[romaroma]

Hi! Was article outline removed / relocated? The link goes to 404 now.

[stubailo]

Here's what it had: https://github.com/meteor/guide/blob/06094c8374a4f5b9035c23be5abdbf15b26e65e0/outlines/forms.md

Not too much stuff unfortunately.

[romaroma]

@stubailo I'd like to contribute if there are some guidelines on proper contribution )

[stubailo]

that would be amazing! first step is writing up an outline, perhaps based on that one, and submitting a PR for it.

[lorensr]

Yeah @romaroma that would be great! We have https://github.com/meteor/guide/blob/master/CONTRIBUTING.md
but it's not relevant for the outline stage – the file to edit for now is: meta/outlines/forms.md

[filipenevola]

I'm closing this issue because it's too old.

We are going to make many updates to the guide in the next weeks and it's better to focus on issues with recent activity. If you think this issue is still relevant please open a new one.