Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Form] RFC: Integrate Zend\Filter into Form #3973

Closed
webmozart opened this issue Apr 18, 2012 · 50 comments
Closed

[Form] RFC: Integrate Zend\Filter into Form #3973

webmozart opened this issue Apr 18, 2012 · 50 comments
Labels

Comments

@webmozart
Copy link
Contributor

@webmozart webmozart commented Apr 18, 2012

People have voiced the need for form input filtering various times. Filtering could be added to the Form component very easily by creating an (optional) extension that enables the following API:

$builder->add('content', 'textarea', array(
    // a single filter or an array of filters
    'filters' => new StripTags(),
));

The big question is which library to use for filtering. Upon short research, I came upon two existing alternatives:

  • Zend\Filter
    • appears mature
    • potentially brings in other ZF2 baggage, which I don't care about once ZF2 supports Composer properly and unless the baggage creates further disadvantages
  • DMS-Filter
    • very young
    • supports annotation-based filtering in POPOs

Does anyone know further filtering libraries? Does anyone have experience with Zend\Filter in non-ZF(2) projects?

Comparison Sheet

Zend\Filter DMS-Filter Comment
Alnum Alnum both support Unicode, Zend has special treatment for Japanese, Chinese and Korean
Alpha
Boolean Boolean DMS does a simple cast, Zend's implementation is more complicated, but supports localized "yes" and "no" as well as leaving the original value unchanged unless it evaluates to false (use case?)
Callback
Digits
File\LowerCase
File\Rename
File\UpperCase
Float
HtmlEntities
Int Int
PregReplace supports Unicode
StringToLower ToLower both support Unicode
StringToUpper ToUpper both support Unicode
StringTrim Trim both allow the specification of a character list to be trimmed
StripNewlines StripNewlines
StripTags StripTags Zend allows to specify allowed tags, attributes and comments; DMS only allowed tags (using `strip_tags` internally).
Not a filter, but a transformer
BaseName
Compress supports BZ2, GZ, LZF, RAR, TAR and ZIP
Decompress see Compress
Decrypt supports MCrypt and OpenSSL
Dir
Encrypt see Decrypt
File\Decrypt see Decrypt
File\Encrypt see Decrypt
Null
RealPath
Word\CamelCaseToDash
Word\CamelCaseToSeparator
Word\CamelCaseToUnderscore
Word\DashToCamelCase
Word\DashToSeparator
Word\DashToUnderscore
Word\SeparatorToCamelCase
Word\SeparatorToDash
Word\SeparatorToSeparator
Word\UnderscoreToCamelCase
Word\UnderscoreToDash
Word\UnderscoreToSeparator
@daFish
Copy link
Contributor

@daFish daFish commented Apr 18, 2012

That would be a great addition to the form component. The only drawback I see when opting for ZF2 would be the huge list of dependencies on other ZF2 components.

@rdohms
Copy link
Contributor

@rdohms rdohms commented Apr 18, 2012

Also, DMS\Filter can have many more filters, just make a request for them and i'll get on that. I already have a bundle for it and would be glad to fix anything left behind.

Thanks @bschussek

@marijn
Copy link

@marijn marijn commented Apr 18, 2012

I've got a few questions about filters:

  • will they be applied before or after transformations?
  • will they be bound to the form layer or the validation layer?
  • are they unidirectional or bidirectional?
  • does it convert text to lowercase or does it filter the non lowercase values?
@pminnieur
Copy link
Contributor

@pminnieur pminnieur commented Apr 18, 2012

👍 for DMS-Filters. It already makes use of Doctrine\Common (which is, in fact, a Symfony2 dependency) and I love the fact that Symfony2 uses/supports small stand-alone-one-task-at-a-time libraries (such like Monolog).

@webmozart
Copy link
Contributor Author

@webmozart webmozart commented Apr 18, 2012

@marijn Thanks for bringing this up. Filters would use the BIND_NORM_DATA event, i.e. they would be applied on the normalized data during binding. The normalized data would then be transformed back to the client and application format in order to reflect the changes of the filter.

Filters are unidirectional. Bidirectional filters are no filters, but data transformers.

What the filter does depends entirely on its implementation. StripTags for example strips tags away, StringToLower converts to lowercase characters etc.

@rdohms Thanks for the feedback. I will keep it in mind.

@antoinegomez
Copy link

@antoinegomez antoinegomez commented Apr 18, 2012

Not sure that Zend\Filter has so many dependencies, using a more mature and complete solution is better.

ZF2 will use some SF2 features, so why can't we do the same ? :)

@weierophinney
Copy link

@weierophinney weierophinney commented Apr 18, 2012

I'll clarify a few points about Zend_Filter. ZF2's filters are primarily oriented at normalization -- not validation. As such, they generally transform the input and return the transformed version -- thus "StringToLower" transforms the input to lowercase. Zend\Filter\FilterChain can be of interest for chaining multiple filters, and thus creating an end result that is the product of each filter -- a filter with StringTrim followed by Alpha followed by StringToLower would transform " the WORD" to "theword".

Regarding dependencies -- we plan to go through the package and identify optional dependencies as distinct from required dependencies. In most cases, the only dependencies are optional -- they're necessary if you use a particular filter, but not another. In some cases, dependencies will be disappearing; as an example, because we're rewriting the i18n/l10n layer, and the approach will be quite different, many of the locale-aware filters will either remove that dependency, be moved to a separate component, or make the dependency optional. This work should be largely complete by our upcoming beta4 release (scheduled sometime in the next 4 weeks).

One additional note, Zend\Filter\Input will be removed for beta4, as it is being replaced by a new Zend\InputFilter component (which bridges Zend\Filter and Zend\Validator).

@ericclemmons
Copy link
Contributor

@ericclemmons ericclemmons commented Apr 18, 2012

I've been using Zend\Filter successfully for exactly this purpose in both Zend Framework and my Symfony2 projects. The only requirement would be, as @weierophinney mentioned, remove or identify any dependencies and get those straightened out for befa4.

@lsmith77
Copy link
Contributor

@lsmith77 lsmith77 commented Apr 19, 2012

@weierophinney indeed .. all we are missing is a solution for normalization/sanitization

DMS filter has the advantage of using similar infrastructure than what we already use in Symfony2 (at least for annotations).

Looking at simply the number of filters provided by https://github.com/rdohms/DMS-Filter/tree/master/Rules and https://github.com/zendframework/zf2/tree/master/library/Zend/Filter its clear that Zend Filter is more mature and I guess if they do refactor it to Zend\InputFilter then at least the broad set of filters will remain and I doubt that there will much instability introduced in the process.

That being said, I don't think its very hard to get DMS up to speed, especially since Zend Framework is BSD licensed, we could in theory just take their filters and adapt them into DMS.

@rdohms
Copy link
Contributor

@rdohms rdohms commented Apr 19, 2012

@lsmith77 indeed. DMS\Filter was modeled after Symfony\Validation so yes it does really look similar.

I did take a look at the ZF Filter list and discussed this with @weierophinney and did indeed note that we follow very different paths. I have a focus on annotations and pure unidirectional filtering, while ZF has filtering/transformation. Since Symfony already has transformation i really focused on the missing filtering step.

Please feel free to open any issues with Filter requests, I will be implementing a few hooks and rules and generally updating the library in the next months.

@webmozart
Copy link
Contributor Author

@webmozart webmozart commented Apr 19, 2012

@weierophinney Thank you for the clarification. Do you have any plans (and a schedule) to add Composer support to the individual components?

In general, I don't think that the surrounding infrastructure of the filters (annotations, filter chains etc.) is of any importance for us. We just need a set of filters that transform values into filtered values. These filters will be executed sequentially directly by the form extension. I especially don't want to mix the data model with input filtering, i.e., put filter rules into the model via annotations. Doing this violates SoC.

Objectively speaking, Zend\Filter seems far more mature (multibyte support, encryption, compression) and thus is my clear favorite. If it manages to

  • reduce dependencies to a minimum
  • add composer support

it fulfills all our requirements.

@rdohms I appreciate your offer, but I don't see the point in duplicating efforts unless it makes a big difference for us.

@rdohms
Copy link
Contributor

@rdohms rdohms commented Apr 19, 2012

@bschussek No prob. Just exaplin to me why annotation based filtering violates the SoC and not validation? To me they are 2 parts of the same coin. Having filtering on form level does not also eliminate the need for entity level filtering.

@stof
Copy link
Member

@stof stof commented Apr 19, 2012

@rdohms The validator component is about validating the object, not the form. On the other hand, the filtering described by @bschussek previously would be applied on the normData of the form, not on the appData (i.e. the object), so it is part of the form level, not of the domain object level

@daFish
Copy link
Contributor

@daFish daFish commented Apr 19, 2012

@stof But that would leave out filtering of data for scenarios were no form is used with an domain object. This might be necessary and useful.

@stof
Copy link
Member

@stof stof commented Apr 19, 2012

@daFish if you have some stuff needing to be done each time you set a value in an object, the right place to put the logic is the setter.

@daFish
Copy link
Contributor

@daFish daFish commented Apr 19, 2012

@stof The same argument can be applied to validation: Everytime I set a value on my object it should be validated.

@Seldaek
Copy link
Member

@Seldaek Seldaek commented Apr 19, 2012

@stof what about DRY? Some filters have a lot of code, putting such code in every setter is quite a recipe for disaster.

@rdohms
Copy link
Contributor

@rdohms rdohms commented Apr 19, 2012

i have to agree with @daFish the motivation behind filtering in annotations is the same as for validation. So if validation is auto executed by forms, filtering to me is on the same level.

Validation in Symfony forms is not available on the form configuration IIRC, so why would filtering be form specific and not part of the domain?

@marijn
Copy link

@marijn marijn commented Apr 19, 2012

Thus far I agree with @daFish on this.

If Zend\Filter also does transformations than that's an argument against using it. I imagine a lot of newcomers that are familiar with Zend\Filter trying to use it for transformation (which is not the intended use in this implementation context).

@webmozart
Copy link
Contributor Author

@webmozart webmozart commented Apr 19, 2012

We could support filtering both on domain objects and on forms directly (like with validation constraints, although this is only partially finished by now).

@rdohms: What about basing DMS-Filters on Zend\Filter instead of implementing your own rules? This way we could have the best of both worlds.

@rdohms
Copy link
Contributor

@rdohms rdohms commented Apr 19, 2012

@bschussek i had this talk with @weierophinney yesterday. What is your definition of basing them on ZF? Adding ZF as a depedency? or defining the same filters?

I was actually looking into ZF yesterday to separate data transformations from filtering, and most of the actual filters are already implemented in DMS, but i'always open to suggestions and improvements, i have been meaning to leverage the filter_* functions in the filters, whenever it makes sense.

@webmozart
Copy link
Contributor Author

@webmozart webmozart commented Apr 19, 2012

@rdohms I'm speaking about actually using their filters, i.e. anything that implements Zend\Filter\Filter, only from within your walker.

@rdohms
Copy link
Contributor

@rdohms rdohms commented Apr 19, 2012

@bschussek i have a little bit of an issue with that, as the library was meant to have a small footprint. I can however look into making a hook so it can plug ZF in maybe. Will look into the interfaces and see if that possible without creating frankenstein,

@guilhermeblanco
Copy link
Contributor

@guilhermeblanco guilhermeblanco commented Apr 19, 2012

Hi!

I'm using DMS-Filter and I'm pretty happy with its content.
It seems it's only missing 3-5 filters to have the same functionality as Zend\Filter and I after suffering from Zend dependencies for over 2 weeks, I'm more likely to say DMS-Filter is light years easier to be used than actually Zend\Filter.

I'd also attempt as much as I could to bring 20+ dependencies only to be able to have a filtering librabry.
That's exactly why I prefer a lightweight solution and it seems DMS-Filter fits perfectly into this.

Cheers,

@pminnieur
Copy link
Contributor

@pminnieur pminnieur commented Apr 20, 2012

@weierophinney is there an estimation when a first RC is planned or when we could expect a stable ZF2 release?

@webmozart
Copy link
Contributor Author

@webmozart webmozart commented Apr 20, 2012

@guilhermeblanco:

I'm more likely to say DMS-Filter is light years easier to be used than actually Zend\Filter.

Can you elaborate?

It seems it's only missing 3-5 filters to have the same functionality as Zend\Filter

This is wrong. Please see the comparison sheet I added above.

@jalliot
Copy link
Contributor

@jalliot jalliot commented Apr 20, 2012

@bschussek Some of the filters supported by Zend don't make sense for the Form component IMHO (like (Un)Compress, (De)Encrypt f.e. because they have to be bidirectional, all filters related to files (Basename, Realpath, etc.)).
So your comparison sheet should maybe include only what would be of some use for us.

@webmozart
Copy link
Contributor Author

@webmozart webmozart commented Apr 20, 2012

@jalliot you are right about compression and decryption. The file filters on the other hand do well make sense for uploaded files. This way, an uploaded file could be automatically en/decrypted or (de-)compressed. I like this.

Having the compression and encryption filters decoupled from the file specific ones does not hurt, so I won't strike them out above.

@jalliot
Copy link
Contributor

@jalliot jalliot commented Apr 20, 2012

I didn't thought of that... That could be indeed pretty cool if handled correctly

@rdohms
Copy link
Contributor

@rdohms rdohms commented Apr 20, 2012

@bschussek it would be nice to flag what is a filter and what is actually data transformation. Basename for example I consider to be data transformation not filtering. Same with the compress/encrypt stuff. Filtering has a different purpose of avoiding bad data to come through, not of auto-converting data into other formats.

Even Boolean is on a thin line here between transformation and filtering, in my approach i just did a simple cast so that its interpreted as a boolean after filtering.

If we are looking for transformation and filtering in this implementation, then DMS has no way of competing.

@webmozart
Copy link
Contributor Author

@webmozart webmozart commented Apr 20, 2012

@rdohms You raise a valid point. I adjusted the table accordingly.

@rdohms
Copy link
Contributor

@rdohms rdohms commented Apr 20, 2012

@bschussek nice, ok I already did PregReplace this week, have not tagged a new release yet, but i'll take a look into that list and implement a few more. To your point of using ZF in DMS, i'm going to see if its possible to implement a ZendRule, so it would be something like:

@filter\Zend('Zend\Filter[name]', [params])

That way it can leverage the Zend filters if you have them locally. Have not looked if that's possible, but if its interesting to anyone i'll work on it.

@webmozart
Copy link
Contributor Author

@webmozart webmozart commented Apr 20, 2012

To your point of using ZF in DMS, i'm going to see if its possible to implement a ZendRule, so it would be something like:

That's not quite what I meant and probably a waste of time. I was rather speaking about avoiding to duplicate all the filter implementations (and their maintenance).

@rdohms
Copy link
Contributor

@rdohms rdohms commented Apr 20, 2012

@bschussek i know, but i would rather keep a functional DMS library with reasonable support and leverage extra filters without requiring them. That way people like @guilhermeblanco can still use it in a simple format, and people who want to leverage ZF can also do it.

I seriously do not want to make the library depend on ZF, so i would like to keep it optional. Even if its not "selected" for this feature, feels like i'm betraying the reason i made it in the first place. But yeah that a personal opinion.

@guilhermeblanco
Copy link
Contributor

@guilhermeblanco guilhermeblanco commented Apr 20, 2012

@bschussek My idea is pretty simple. When I started to use Zend\Filter, I remember that I had to import around 10 dependencies to get it working isolated.
Then I figured it out I had to write a Service class that allows filtering of not only form related VOs, but also non-form related VOs. That was the exact point I decided to switch to DMS-Filter.

When I made the switch to DMS-Filter, all I had to do is add a simple EventSubscriber (I have to admit it's a very dirty code I wrote in 5min) and add a new instance on every form type.

Of course that would be incredibly easier if form component includes a new event monitoring through a tag. That would simplify my life.
I can even foresee a possible implementation that DMSFilterBundle could do taking advantage of this functionality:

<service id="dms.filter.listener.filter" class="DMS\FilterBundle\Listener\FilterSubscriber">
    <argument type="service" id="dms.filter.service.filter" />
    <tag name="form.event_listener" event="postBind" method="postBind" />
</service>

By doing that, it means Subscriber would be always active, so I wouldn't need to keep instantiating it every time I have a Form Type.

In the case of using the component standalone (for non-form VOs), I imply need to grab the "dms.filter" service.
I can't see a cleaner solution than this enhancement to Form component on Symfony. I'm pretty comfortable with the state of DMS-Filter ATM and I may need for one or two filters in the future.

As people already mentioned, I used it only for input filtering, not for transformation. That can be a good reason why the tool fits so well on my specific environment.

@stof
Copy link
Member

@stof stof commented Apr 20, 2012

@guilhermeblanco forms are not managed by the DIC so using a tag would not work (the DIC cannot hook in your query builder). However, if you want to have a subscriber which is always active, you could create a form type extension applied on form which adds the subscriber for the root (and this extension can be registered through a tag).

@guilhermeblanco
Copy link
Contributor

@guilhermeblanco guilhermeblanco commented Apr 21, 2012

@stof I don't see how it can't be managed.
There're tons of manageable tags that hooks form types, forms and etc already. It doesn't seem that DI depends on Forms, so why not make Form package depending on DI? It doesn't generate any cyclic dependency from my point of view

@stof
Copy link
Member

@stof stof commented Apr 21, 2012

@guilhermeblanco The issue is that you don't have one Form but many (even if your controller only use one form, there is many Form objects as each subform is a Form too). So on which one would you register the listener (each one has its own dispatcher). Tags are allowing to hook into the factory, which is a container-scoped service. I don't see how you could use tags for the forms (keep in mind they are only available at compile-time, not at runtime in the container)

@AurelC2G
Copy link
Contributor

@AurelC2G AurelC2G commented Jun 25, 2012

Any news about this?

@stof
Copy link
Member

@stof stof commented Jun 25, 2012

@AurelC2G as we are in beta now, we won't add any new features for 2.1 but only fix bugs. New features will be for the 2.2 release now.

@AurelC2G
Copy link
Contributor

@AurelC2G AurelC2G commented Jun 25, 2012

I know and I was not expecting anything sooner than 2.2, just wondering whether this subject had been completely forgotten ;)

@rdohms
Copy link
Contributor

@rdohms rdohms commented Jun 25, 2012

@AurelC2G DMS\Filter is now automatically hooking into Symfony Form using the DMSFilterBundle, so its a easy drop in for 2.1 until this is finalized.

@stof
Copy link
Member

@stof stof commented Jun 25, 2012

@AurelC2G it is not forgotten. It is delayed as the priority is the 2.1 stable release currently, not the 2.2 features

@ghost
Copy link

@ghost ghost commented Sep 21, 2012

bump

@chetzof
Copy link

@chetzof chetzof commented May 11, 2013

Hey! Is there any progress on this issue? I suppose it won't be in 2.3 either? :(

@acasademont
Copy link
Contributor

@acasademont acasademont commented Jan 15, 2014

Hi there! This would be nice indeed :)

@rdohms
Copy link
Contributor

@rdohms rdohms commented Jan 15, 2014

dms/dms-filter 2.0 will come with support for embedding Zend\Filter, add to that the DMSFilterBundle and it can auto-filter entities submitted by forms as well as work as described above.

Both 2.0 releases should come out next week, the filter is done the bundle just needs wrapping up to support Symfony 2.3.

I'm open to feedback on anything that's missing and hope that this gives you all a good option to filtering in Symfony.

@gfreeau
Copy link

@gfreeau gfreeau commented May 17, 2014

Symfony needs input filtering to go along with validation.

@rdohms it would be great for DMSFilerBundle to support all standard config formats including yml and xml.

@gfreeau
Copy link

@gfreeau gfreeau commented May 18, 2014

I added xml/yml support to DMS Filter

rdohms/dms-filter#10
rdohms/dms-filter#9

I plan to add support to the sf2 bundle as well.

I noticed there is a lot of cross over between the validator component and this filter component such as loading xml and yml config and metadata.

I think if symfony2 is going to have a filter component some re-factoring could be done to share some code between the two.

@javiereguiluz
Copy link
Member

@javiereguiluz javiereguiluz commented Sep 6, 2014

It's been more than 2 years since this discussion was initiated. Should we definitely close it as won't fix or should we officially add it to the roadmap of the next Symfony version?

@fabpot fabpot closed this Sep 6, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.