Skip to content

JPA @Converter autoApply setting not honoured when set to false #1777

Closed

Description

@KilgallonJ

The system I'm working on has a bunch of legacy data where boolean values have been stored as 'Y' and 'N'. New tables use a BIT column instead and simply store 0 and 1. No table mixes the two approaches.

To support the legacy tables we have the following converter:

@Converter(autoApply = false)
public class BooleanToStringConverter implements AttributeConverter<Boolean, String> {

    private Logger.ALogger Log = Logger.of(BooleanToStringConverter.class);

    @Override
    public String convertToDatabaseColumn(final Boolean attribute) {
        Log.debug("Converting the boolean value {}", attribute);

        if (attribute == null) {
            return "N";
        }

        return attribute ? "Y" : "N";
    }

    @Override
    public Boolean convertToEntityAttribute(final String dbData) {
        return "Y".equalsIgnoreCase(dbData);
    }
}

As this only needs to apply to certain entities the autoApply property has been set to false.

I'm now creating a brand new entity, with a new table. It has two boolean properties, both using the BIT column style instead of Y/N:

@Entity
@Table(name = "MyEntity")
public class MyEntity {

    @Id
    @Column(name = "MyEntityId")
    private Long id;

    @Column(name = "IsClosed")
    private Boolean closed;

    ...
}

Note that I have not applied the @Convert annotation

I have a query that needs to filter out any rows where the entity is closed:

query.where().eq(CLOSED, Boolean.FALSE)

It is at this point that my problem arises. Whenever this query is run I see the log message from the BooleanToStringConverter being written to the logs and indeed, if I dump the actual SQL that was executed from the MySQL database then I can see that the converter did actually get applied to the boolean property, creating the following SQL fragment:

select <columns>
  from MyEntity t0
 where <other predicates>
   and t0.IsClosed = 'N'
 order by <order clause>

This is obviously wrong - the converter shouldn't have been applied, it's not set to be automatic and the closed property isn't annotated with @convert.

I tried to work around this by creating a second converter:

@Converter(autoApply = true)
public class BooleanConverter implements AttributeConverter<Boolean, Boolean> {

    private Logger.ALogger Log = Logger.of(BooleanConverter.class);

    @Override
    public Boolean convertToDatabaseColumn(final Boolean attribute) {
        Log.debug("Processing the value {}.", attribute);
        return attribute;
    }

    @Override
    public Boolean convertToEntityAttribute(final Boolean dbData) {
        return dbData;
    }
}

This resulted in both converters being applied to the property and I see both debug statements appearing in the logs.

2019-07-29 14:19:53,994 [dispatcher-69] DEBUG BooleanConverter     Processing the value false.
2019-07-29 14:19:53,994 [dispatcher-69] DEBUG BooleanToStringConve I'm Converting the boolean value false

Next I tried explicitly setting the converter to use on the entity itself (I hoped this might change the order that the converters were getting applied in so that it'd end up as true/false despite the other converter running):

@Entity
@Table(name = "MyEntity")
public class MyEntity {

    @Id
    @Column(name = "MyEntityId")
    private Long id;

    @Convert(converter = BooleanConverter.class)
    @Column(name = "IsClosed")
    private Boolean closed;

    ...
}

With exactly the same result; both converters are applied to the value sequentially, with the BooleanToStringConverter having the last laugh and mangling the predicate.

I would rather keep the BooleanToStringConverter as it makes dealing with the legacy data a bit less painful, but unless I can figure out why it's being applied when it shouldn't it's looking likely that I'll have to delete it.

I'm using Ebean version 4.1.3 and Play! 2.6.21

Activity

  1. rbygrave commented on Aug 7, 2019

    @rbygrave
    Member

    Hi @KilgallonJ ... just checking that you have a workaround / are not blocked by this. I recall you don't strictly need this now, is that right?

  2. KilgallonJ commented on Aug 7, 2019

    @KilgallonJ
    Author

    That's correct @rbygrave - I removed the offending converter as it turned out to be a pretty straightforward thing to do so it's not causing me any grief at the moment.

  3. rbygrave commented on Aug 19, 2019

    @rbygrave
    Member

    Note that I'm pondering the potential of NOT fixing this issue.

    That is, currently Ebean has pretty much "universal types" meaning that any given scalar type has a single mapping. We sort of let java.util.Date have multiple mappings but that is imo undesirable and not nice (and we should be using the java.time types now anyway).

    For example Boolean has a single mapping to a DB type for a given Database/EbeanServer instance. That is, Ebean doesn't current support mapping Boolean to both Y/N AND also map Boolean to True/False for a given EbeanServer instance (we only allow 1 mapping for Boolean type).

    The nice thing about this approach we have with Ebean (the simplicity we get from this) is that for example, whenever Ebean binds a Boolean it knows how it map it to the DB regardless of the property in the expression. We use this in various places like DTO etc.

    To fix this we lose that simplicity which opens Ebean up for more complexity and potential confusion.

    In short I think we should put this on hold until there is more clear need for mapping a single type specifically for given properties (and that we want to own the downsides of that).

  4. KilgallonJ commented on Aug 19, 2019

    @KilgallonJ
    Author

    From my point of view the main use-cases that I've come across are those cases when you're dealing with inconsistent database representations of the same type of data. It's not strictly limited to boolean values but is often one of the culprits.

    In an ideal world you wouldn't need the converter for something so trivial, but changing the type of an in-use column on a production system isn't done lightly. In my case I actually can fix it at source as a major section of the app is being redesigned from the ground up.

    Being able to "turn off" an attribute converter by specifying autoApply = false allows you to migrate from the old representation to the new one with minimal pain as you can selectively control the conversion of the db values.

    There's also the matter of developer expectation - coming from a Hibernate background I didn't think twice about using autoApply = false because it works as expected there. It's not unreasonable to expect that it would work in another JPA implementation.

  5. rbygrave commented on Aug 19, 2019

    @rbygrave
    Member

    It's not strictly limited to boolean

    Quite right. I was just using it as an example and yes I didn't make that clear.

    allows you to migrate from the old representation to the new one with minimal pain

    Yes and yes this is a nice feature. It just comes at the cost of extra complexity (so we need to make sure we want to pay that complexity cost because everyone pays for it forever).

    developer expectation - coming from a Hibernate background I didn't think twice about using autoApply = false because it works as expected there

    Yes that's true. What we also need to be mindful that Hibernate is approx 7MB and they have approx 2800 open JIRA tickets.

    I still think we should hold on fixing this until there is more demand. We can detect autoApply = false ... throw and exception and point to this ticket.

  6. rbygrave commented on Aug 19, 2019

    @rbygrave
    Member

    It's not unreasonable to expect that it would work in another JPA implementation.

    True as well. Note that this would be only the 2nd JPA mapping feature we don't support per say.

  7. rbygrave commented on Aug 20, 2019

    @rbygrave
    Member

    Ok, closing this. People who desperately need this functionality will need to voice their opinion and push for a re-open etc.

  8. tbee commented on Oct 16, 2020

    @tbee
    Contributor

    EBean is not used as much as JPA implementations, but I prefer its simplicity. But given that it is not used as much, it also has not encountered that many use cases. So I understand that from that it is okay to not support certain functionality. But not being able to specify a specific converter for specific attribute is really a missed functionality.

    For example; I have a DDD project, where EBean takes care of the persistence of some of the entities. One string attribute in these entities needs to be stored encrypted. The domain does not need to know, it is the responsibility of the repository to en-/decrypt just that one column (I cannot hash it, it needs to be reversible). So I cannot use a converter that encrypts all strings, I need one for only that single attribute.

  9. rbygrave commented on Oct 19, 2020

    @rbygrave
    Member

    But given that it is not used as much, it also has not encountered that many use cases.

    Maybe but then again this is the perfect anti-example as Ebean has had built in transparent encryption for more than 10 years now (released in March 2010 Ebean version 2.4.0).

    The customer requesting this feature needed it for an application in the Health sector but I know this has been used in many areas including Financial services. Ebean's encryption feature supports both client side encryption and database side encryption. Hard to say but anecdotal evidence suggests database side encryption is more widely used.

    10 years on and transparent encryption is still not part of JPA. Today as I see it both Hibernate and Eclipselink still have poor approaches/mechanisms for this feature relative to what we have with Ebean (and hence people in the JPA space might have to use AttributeConverter or proprietary ColumnTransformer).

    One string attribute in these entities needs to be stored encrypted

    This is what Ebean's encryption feature is for but we also want to encrypt other types like Dates, Timestamps, enums and Blobs plus we want to support database side encryption so that we can transparently include these encrypted properties in where clauses etc.

    It will be interesting to hear why you think using an AttributeConverter is better than the built in transparent encryption functionality that has been part of Ebean but perhaps you are not aware of Ebean's feature?

    Note that it is possible that you missed Ebean's encryption feature because the documentation on it went awol for a while there.

    Cheers, Rob.

  10. tbee commented on Oct 19, 2020

    @tbee
    Contributor
  11. SchlauFuchs commented on Jun 10, 2021

    @SchlauFuchs

    I just learned that @Convert with a custom converter on a field does not work, because you already have a String to boolean converter registered? And I have a very similar use case to above, trying to map a legacy system. Could one at least point me into the direction how I can sort this the most JPA compatible way?
    @Convert(converter = YNConverter::class) val associated: Boolean

  12. rbygrave commented on Jun 10, 2021

    @rbygrave
    Member

    because you already have a String to boolean converter registered?

    Does that mean the app you are working on has 2 different String to boolean converters? Multiple different ways Booleans are mapped to/from Strings?

    OR are all Boolean's are treated the same way and in fact map to string values like Y/N or T/F ? If so look at DatabaseConfig setDatabaseBooleanTrue(...) and setDatabaseBooleanFalse(...) which is there to handle that case.

  13. rbygrave commented on May 16, 2022

    @rbygrave
    Member

    Unless it is 100% related to autoApply=false ... this should be logged as a separate issue. If we find out it's the same issue we can link it back but if it isn't the exact same issue it just clouds the issue. [edit: and increases the chance that the question/issue is "lost"]

    My take is that your question/issue is NOT the same as this one. Specifically, the app does not need or want to store/map HashAlgorithm more than one way [map it to a varchar or int but no requirement to map it BOTH ways].

    So:

    • Create a new issue/question (I'm fine for people to put questions as github issues these days, another alternative is github discussions https://github.com/ebean-orm/ebean/discussions)
    • Include the enum (I want to see how it's coded ... is there a method that returns the DB value we desire etc)
    • Include an example of how the enum is mapped (Is the JPA standard mapping approach going to work)

    Lets go from there ... it could be just a case of adding a @DbEnumValue https://ebean.io/docs/mapping/extensions/dbenumvalue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions