Repository navigation
Description
The system I'm working on has a bunch of legacy data where boolean values have been stored as 'Y' and 'N'. New tables use a BIT column instead and simply store 0 and 1. No table mixes the two approaches.
To support the legacy tables we have the following converter:
@Converter(autoApply = false)
public class BooleanToStringConverter implements AttributeConverter<Boolean, String> {
private Logger.ALogger Log = Logger.of(BooleanToStringConverter.class);
@Override
public String convertToDatabaseColumn(final Boolean attribute) {
Log.debug("Converting the boolean value {}", attribute);
if (attribute == null) {
return "N";
}
return attribute ? "Y" : "N";
}
@Override
public Boolean convertToEntityAttribute(final String dbData) {
return "Y".equalsIgnoreCase(dbData);
}
}As this only needs to apply to certain entities the autoApply property has been set to false.
I'm now creating a brand new entity, with a new table. It has two boolean properties, both using the BIT column style instead of Y/N:
@Entity
@Table(name = "MyEntity")
public class MyEntity {
@Id
@Column(name = "MyEntityId")
private Long id;
@Column(name = "IsClosed")
private Boolean closed;
...
}Note that I have not applied the @Convert annotation
I have a query that needs to filter out any rows where the entity is closed:
query.where().eq(CLOSED, Boolean.FALSE)
It is at this point that my problem arises. Whenever this query is run I see the log message from the BooleanToStringConverter being written to the logs and indeed, if I dump the actual SQL that was executed from the MySQL database then I can see that the converter did actually get applied to the boolean property, creating the following SQL fragment:
select <columns>
from MyEntity t0
where <other predicates>
and t0.IsClosed = 'N'
order by <order clause>This is obviously wrong - the converter shouldn't have been applied, it's not set to be automatic and the closed property isn't annotated with @convert.
I tried to work around this by creating a second converter:
@Converter(autoApply = true)
public class BooleanConverter implements AttributeConverter<Boolean, Boolean> {
private Logger.ALogger Log = Logger.of(BooleanConverter.class);
@Override
public Boolean convertToDatabaseColumn(final Boolean attribute) {
Log.debug("Processing the value {}.", attribute);
return attribute;
}
@Override
public Boolean convertToEntityAttribute(final Boolean dbData) {
return dbData;
}
}This resulted in both converters being applied to the property and I see both debug statements appearing in the logs.
2019-07-29 14:19:53,994 [dispatcher-69] DEBUG BooleanConverter Processing the value false.
2019-07-29 14:19:53,994 [dispatcher-69] DEBUG BooleanToStringConve I'm Converting the boolean value false
Next I tried explicitly setting the converter to use on the entity itself (I hoped this might change the order that the converters were getting applied in so that it'd end up as true/false despite the other converter running):
@Entity
@Table(name = "MyEntity")
public class MyEntity {
@Id
@Column(name = "MyEntityId")
private Long id;
@Convert(converter = BooleanConverter.class)
@Column(name = "IsClosed")
private Boolean closed;
...
}With exactly the same result; both converters are applied to the value sequentially, with the BooleanToStringConverter having the last laugh and mangling the predicate.
I would rather keep the BooleanToStringConverter as it makes dealing with the legacy data a bit less painful, but unless I can figure out why it's being applied when it shouldn't it's looking likely that I'll have to delete it.
I'm using Ebean version 4.1.3 and Play! 2.6.21
Activity
rbygrave commentedon Aug 7, 2019
Hi @KilgallonJ ... just checking that you have a workaround / are not blocked by this. I recall you don't strictly need this now, is that right?
KilgallonJ commentedon Aug 7, 2019
That's correct @rbygrave - I removed the offending converter as it turned out to be a pretty straightforward thing to do so it's not causing me any grief at the moment.
rbygrave commentedon Aug 19, 2019
Note that I'm pondering the potential of NOT fixing this issue.
That is, currently Ebean has pretty much "universal types" meaning that any given scalar type has a single mapping. We sort of let java.util.Date have multiple mappings but that is imo undesirable and not nice (and we should be using the java.time types now anyway).
For example Boolean has a single mapping to a DB type for a given Database/EbeanServer instance. That is, Ebean doesn't current support mapping Boolean to both Y/N AND also map Boolean to True/False for a given EbeanServer instance (we only allow 1 mapping for Boolean type).
The nice thing about this approach we have with Ebean (the simplicity we get from this) is that for example, whenever Ebean binds a Boolean it knows how it map it to the DB regardless of the property in the expression. We use this in various places like DTO etc.
To fix this we lose that simplicity which opens Ebean up for more complexity and potential confusion.
In short I think we should put this on hold until there is more clear need for mapping a single type specifically for given properties (and that we want to own the downsides of that).
KilgallonJ commentedon Aug 19, 2019
From my point of view the main use-cases that I've come across are those cases when you're dealing with inconsistent database representations of the same type of data. It's not strictly limited to boolean values but is often one of the culprits.
In an ideal world you wouldn't need the converter for something so trivial, but changing the type of an in-use column on a production system isn't done lightly. In my case I actually can fix it at source as a major section of the app is being redesigned from the ground up.
Being able to "turn off" an attribute converter by specifying
autoApply = falseallows you to migrate from the old representation to the new one with minimal pain as you can selectively control the conversion of the db values.There's also the matter of developer expectation - coming from a Hibernate background I didn't think twice about using
autoApply = falsebecause it works as expected there. It's not unreasonable to expect that it would work in another JPA implementation.rbygrave commentedon Aug 19, 2019
Quite right. I was just using it as an example and yes I didn't make that clear.
Yes and yes this is a nice feature. It just comes at the cost of extra complexity (so we need to make sure we want to pay that complexity cost because everyone pays for it forever).
Yes that's true. What we also need to be mindful that Hibernate is approx 7MB and they have approx 2800 open JIRA tickets.
I still think we should hold on fixing this until there is more demand. We can detect
autoApply = false... throw and exception and point to this ticket.rbygrave commentedon Aug 19, 2019
True as well. Note that this would be only the 2nd JPA mapping feature we don't support per say.
rbygrave commentedon Aug 20, 2019
Ok, closing this. People who desperately need this functionality will need to voice their opinion and push for a re-open etc.
tbee commentedon Oct 16, 2020
EBean is not used as much as JPA implementations, but I prefer its simplicity. But given that it is not used as much, it also has not encountered that many use cases. So I understand that from that it is okay to not support certain functionality. But not being able to specify a specific converter for specific attribute is really a missed functionality.
For example; I have a DDD project, where EBean takes care of the persistence of some of the entities. One string attribute in these entities needs to be stored encrypted. The domain does not need to know, it is the responsibility of the repository to en-/decrypt just that one column (I cannot hash it, it needs to be reversible). So I cannot use a converter that encrypts all strings, I need one for only that single attribute.
rbygrave commentedon Oct 19, 2020
Maybe but then again this is the perfect anti-example as Ebean has had built in transparent encryption for more than 10 years now (released in March 2010 Ebean version 2.4.0).
The customer requesting this feature needed it for an application in the Health sector but I know this has been used in many areas including Financial services. Ebean's encryption feature supports both client side encryption and database side encryption. Hard to say but anecdotal evidence suggests database side encryption is more widely used.
10 years on and transparent encryption is still not part of JPA. Today as I see it both Hibernate and Eclipselink still have poor approaches/mechanisms for this feature relative to what we have with Ebean (and hence people in the JPA space might have to use AttributeConverter or proprietary ColumnTransformer).
This is what Ebean's encryption feature is for but we also want to encrypt other types like Dates, Timestamps, enums and Blobs plus we want to support database side encryption so that we can transparently include these encrypted properties in where clauses etc.
It will be interesting to hear why you think using an AttributeConverter is better than the built in transparent encryption functionality that has been part of Ebean but perhaps you are not aware of Ebean's feature?
Note that it is possible that you missed Ebean's encryption feature because the documentation on it went awol for a while there.
Cheers, Rob.
tbee commentedon Oct 19, 2020
SchlauFuchs commentedon Jun 10, 2021
I just learned that
@Convertwith a custom converter on a field does not work, because you already have a String to boolean converter registered? And I have a very similar use case to above, trying to map a legacy system. Could one at least point me into the direction how I can sort this the most JPA compatible way?@Convert(converter = YNConverter::class) val associated: Booleanrbygrave commentedon Jun 10, 2021
Does that mean the app you are working on has 2 different String to boolean converters? Multiple different ways Booleans are mapped to/from Strings?
OR are all Boolean's are treated the same way and in fact map to string values like Y/N or T/F ? If so look at DatabaseConfig setDatabaseBooleanTrue(...) and setDatabaseBooleanFalse(...) which is there to handle that case.
rbygrave commentedon May 16, 2022
Unless it is 100% related to
autoApply=false... this should be logged as a separate issue. If we find out it's the same issue we can link it back but if it isn't the exact same issue it just clouds the issue. [edit: and increases the chance that the question/issue is "lost"]My take is that your question/issue is NOT the same as this one. Specifically, the app does not need or want to store/map HashAlgorithm more than one way [map it to a varchar or int but no requirement to map it BOTH ways].
So:
Lets go from there ... it could be just a case of adding a
@DbEnumValuehttps://ebean.io/docs/mapping/extensions/dbenumvalue