[Form] RFC: Integrate Zend\Filter into Form

[webmozart]

People have voiced the need for form input filtering various times. Filtering could be added to the Form component very easily by creating an (optional) extension that enables the following API:

$builder->add('content', 'textarea', array(
    // a single filter or an array of filters
    'filters' => new StripTags(),
));

The big question is which library to use for filtering. Upon short research, I came upon two existing alternatives:

• Zend\Filter
  • appears mature
  • potentially brings in other ZF2 baggage, which I don't care about once ZF2 supports Composer properly and unless the baggage creates further disadvantages
• DMS-Filter
  • very young
  • supports annotation-based filtering in POPOs

Does anyone know further filtering libraries? Does anyone have experience with Zend\Filter in non-ZF(2) projects?

Comparison Sheet

Zend\Filter	DMS-Filter	Comment
Alnum	Alnum	both support Unicode, Zend has special treatment for Japanese, Chinese and Korean
Alpha
Boolean	Boolean	DMS does a simple cast, Zend's implementation is more complicated, but supports localized "yes" and "no" as well as leaving the original value unchanged unless it evaluates to false (use case?)
Callback
Digits
File\LowerCase
File\Rename
File\UpperCase
	Float
HtmlEntities
Int	Int
PregReplace		supports Unicode
StringToLower	ToLower	both support Unicode
StringToUpper	ToUpper	both support Unicode
StringTrim	Trim	both allow the specification of a character list to be trimmed
StripNewlines	StripNewlines
StripTags	StripTags	Zend allows to specify allowed tags, attributes and comments; DMS only allowed tags (using `strip_tags` internally).
Not a filter, but a transformer
BaseName
Compress		supports BZ2, GZ, LZF, RAR, TAR and ZIP
Decompress		see Compress
Decrypt		supports MCrypt and OpenSSL
Dir
Encrypt		see Decrypt
File\Decrypt		see Decrypt
File\Encrypt		see Decrypt
Null
RealPath
Word\CamelCaseToDash
Word\CamelCaseToSeparator
Word\CamelCaseToUnderscore
Word\DashToCamelCase
Word\DashToSeparator
Word\DashToUnderscore
Word\SeparatorToCamelCase
Word\SeparatorToDash
Word\SeparatorToSeparator
Word\UnderscoreToCamelCase
Word\UnderscoreToDash
Word\UnderscoreToSeparator

[daFish]

That would be a great addition to the form component. The only drawback I see when opting for ZF2 would be the huge list of dependencies on other ZF2 components.

[rdohms]

Also, DMS\Filter can have many more filters, just make a request for them and i'll get on that. I already have a bundle for it and would be glad to fix anything left behind.

Thanks @bschussek

[marijn]

I've got a few questions about filters:

• will they be applied before or after transformations?
• will they be bound to the form layer or the validation layer?
• are they unidirectional or bidirectional?
• does it convert text to lowercase or does it filter the non lowercase values?

[immutef]

👍 for DMS-Filters. It already makes use of Doctrine\Common (which is, in fact, a Symfony2 dependency) and I love the fact that Symfony2 uses/supports small stand-alone-one-task-at-a-time libraries (such like Monolog).

[webmozart]

@marijn Thanks for bringing this up. Filters would use the BIND_NORM_DATA event, i.e. they would be applied on the normalized data during binding. The normalized data would then be transformed back to the client and application format in order to reflect the changes of the filter.

Filters are unidirectional. Bidirectional filters are no filters, but data transformers.

What the filter does depends entirely on its implementation. StripTags for example strips tags away, StringToLower converts to lowercase characters etc.

@rdohms Thanks for the feedback. I will keep it in mind.

[antoinegomez]

Not sure that Zend\Filter has so many dependencies, using a more mature and complete solution is better.

ZF2 will use some SF2 features, so why can't we do the same ? :)

[weierophinney]

I'll clarify a few points about Zend_Filter. ZF2's filters are primarily oriented at normalization -- not validation. As such, they generally transform the input and return the transformed version -- thus "StringToLower" transforms the input to lowercase. Zend\Filter\FilterChain can be of interest for chaining multiple filters, and thus creating an end result that is the product of each filter -- a filter with StringTrim followed by Alpha followed by StringToLower would transform " the WORD" to "theword".

Regarding dependencies -- we plan to go through the package and identify optional dependencies as distinct from required dependencies. In most cases, the only dependencies are optional -- they're necessary if you use a particular filter, but not another. In some cases, dependencies will be disappearing; as an example, because we're rewriting the i18n/l10n layer, and the approach will be quite different, many of the locale-aware filters will either remove that dependency, be moved to a separate component, or make the dependency optional. This work should be largely complete by our upcoming beta4 release (scheduled sometime in the next 4 weeks).

One additional note, Zend\Filter\Input will be removed for beta4, as it is being replaced by a new Zend\InputFilter component (which bridges Zend\Filter and Zend\Validator).

[ericclemmons]

I've been using Zend\Filter successfully for exactly this purpose in both Zend Framework and my Symfony2 projects. The only requirement would be, as @weierophinney mentioned, remove or identify any dependencies and get those straightened out for befa4.

[lsmith77]

@weierophinney indeed .. all we are missing is a solution for normalization/sanitization

DMS filter has the advantage of using similar infrastructure than what we already use in Symfony2 (at least for annotations).

Looking at simply the number of filters provided by https://github.com/rdohms/DMS-Filter/tree/master/Rules and https://github.com/zendframework/zf2/tree/master/library/Zend/Filter its clear that Zend Filter is more mature and I guess if they do refactor it to Zend\InputFilter then at least the broad set of filters will remain and I doubt that there will much instability introduced in the process.

That being said, I don't think its very hard to get DMS up to speed, especially since Zend Framework is BSD licensed, we could in theory just take their filters and adapt them into DMS.

[rdohms]

@lsmith77 indeed. DMS\Filter was modeled after Symfony\Validation so yes it does really look similar.

I did take a look at the ZF Filter list and discussed this with @weierophinney and did indeed note that we follow very different paths. I have a focus on annotations and pure unidirectional filtering, while ZF has filtering/transformation. Since Symfony already has transformation i really focused on the missing filtering step.

Please feel free to open any issues with Filter requests, I will be implementing a few hooks and rules and generally updating the library in the next months.

[webmozart]

@weierophinney Thank you for the clarification. Do you have any plans (and a schedule) to add Composer support to the individual components?

In general, I don't think that the surrounding infrastructure of the filters (annotations, filter chains etc.) is of any importance for us. We just need a set of filters that transform values into filtered values. These filters will be executed sequentially directly by the form extension. I especially don't want to mix the data model with input filtering, i.e., put filter rules into the model via annotations. Doing this violates SoC.

Objectively speaking, Zend\Filter seems far more mature (multibyte support, encryption, compression) and thus is my clear favorite. If it manages to

• reduce dependencies to a minimum
• add composer support

it fulfills all our requirements.

@rdohms I appreciate your offer, but I don't see the point in duplicating efforts unless it makes a big difference for us.

[rdohms]

@bschussek No prob. Just exaplin to me why annotation based filtering violates the SoC and not validation? To me they are 2 parts of the same coin. Having filtering on form level does not also eliminate the need for entity level filtering.

[stof]

@rdohms The validator component is about validating the object, not the form. On the other hand, the filtering described by @bschussek previously would be applied on the normData of the form, not on the appData (i.e. the object), so it is part of the form level, not of the domain object level

[daFish]

@stof But that would leave out filtering of data for scenarios were no form is used with an domain object. This might be necessary and useful.

[stof]

@daFish if you have some stuff needing to be done each time you set a value in an object, the right place to put the logic is the setter.