Add support for type converters

[xiprox]

I read a mention of this in #119. I think it's a great idea.

I will be digging the code and seeing if I can help out, but thought I'd open this issue anyway.

Also, thanks a lot for the great library!

[vitusortner]

Hi. That sounds great! If you have questions about the structure of the library, just open a PR of the current state of your development or add them here. You can find a small wiki on the feature in the following.

Type Converters

Type converters allow translating data right before inserting and after reading it in order to make the values storable in the database. For instance, an entity might hold a date in form of a DateTime object. As SQLite/sqlflite doesn't support Dart's DateTime out of the box, there is the need of converting the date into another type that is supported. In that case, it might be an int that represents the date as a timestamp. A type converter should include two simple functions that convert in both directions. When storing the date, functions would be required that translate between DateTime and int. An example can be seen in the following code snippet.

DateTime toDateTime(int milliseconds) {
  return DateTime.fromMillisecondsSinceEpoch(milliseconds);
}

int toInt(DateTime date) {
  return date.millisecondsSinceEpoch;
}

The library has to check whether such a type converter is applied to a specific entity, DAO, DAO function or the whole database. In order a converter is found, the library has to call the previously implemented converter functions whenever a matching type should get inserted, updated or selected.

It's also required to change the field type of the entity when creating the table. In case of the DateTime example, it would be required to change the DateTime column to int.

API

The full API of the feature is visible in the following code snippet. It includes the definition of the converter itself and multiple ways for applying it to either an entity, database, DAO or DAO function.

// converter
class DateTimeConverter {
  @TypeConverter
  DateTime toDateTime(int milliseconds) {
    return DateTime.fromMillisecondsSinceEpoch(milliseconds);
  }

  @TypeConverter
  int toInt(DateTime date) {
    return date.millisecondsSinceEpoch;
  }
}

// converter applied to entity
@TypeConverters([DateTimeConverter])
@entity
class Person {
  @primaryKey final int id;
  final DateTime birthday;
}

// converter applied to database
@TypeConverters([DateTimeConverter])
@Database(version: 1, entities: [Person])
class MyDatabase extends FloorDatabase {
  // dao retrieval
}

// converter applied to DAO
@TypeConverters([DateTimeConverter])
class PersonDao {
  // dao functions
}

// converter applied to specific DAO function
class PersonDao {
  @TypeConverters([DateTimeConverter])
  Future<void> insertPerson(Person person);
}

Room also allows applying converters to just a specific DAO function field. Let's discuss if that's required. More documentation on the feature can be found here and here.

Implementation

This feature requires changes in a couple places. But first, some explanation about the general architecture of the library. The annotation processor of it searches in the user's source code for a class annotated with @Database. In case such an annotated class is found, processors are triggered that collect all required data from the source code and map it into so-called value objects. These objects are the input of the second tier - writers. These take care of generating the actual persistence code.

When implementing the type converter functionality, it's required to check the source code for the @TypeConverters annotation. If such is found, the defined converters can be used. (Depending on the scope mentioned previously.) They might get applied at Entity.getValueMapping() for creating the function that maps entity objects into Maps. For the other way round, the converter might get applied at EntityProcessor._getConstructor().

[MrMonotone]

Type Converters will be really useful for other data types as well. I am looking forward to their implementation.

[dkaera]

@vitusortner I found an alternative approach in dart-lang/json_serializable library. The main idea is using the type adapter as a static method declaration in the field annotation such as
@JsonKey(fromJson: <convertation_to_model_method>, toJson: <convertation_to_primitive_method>)
What do you think about this approach?

[xiprox]

Finally got some time to look into this.

First of all, thank you for the tips. They have been really helpful as I've been studying the code.

Secondly, I have some questons. How would you approach priority? I was thinking dao function > dao > entity > database. What do you think? And how would you implement that? I would really appreciate some pointers here as well.

As for applying converters to specific DAO function fields, I feel like it could be skipped for now and gotten back to later if there is demand. I can't really think of a critical use case right now.

[vitusortner]

@dkaera
The mentioned approach is pretty similar to the ones mentioned.

1. Some methods have to be defined
2. They have to get added to a specific scope they should be applicable to (field, DAO function, etc.)

Once converters have been realized for DAO function, DAO, entity and database scopes, it should be simple to also apply them to the scope of a field.

@xiprox
The mentioned order seems perfect.

1. DAO function
2. DAO
3. Entity
4. Database

[IanDarwin]

Presumably the DateTime converter would ship as part of the library b/c that's such a commonly-used datatype. Maybe a few others too. Having the ability to map arbitrary datatypes used as fields would be definite win. Thanks && hope to see this one soon :-)

[vitusortner]

A quick update: I've started the development of this highly requested feature and will release it as soon as possible.

[jaredgreener]

+1 for this, can't use this library without support for DateTime

[proninyaroslav]

@vitusortner
Hi. Do you have an approximate release date for this feature?

[noordawod]

@vitusortner
Got any time estimation for adding this support?

[IanDarwin]

If you need it now, switch from floor to moor, like I just did. It's a non-trivial switch, but (a) they handle datetime, and enums, and various other things and (b) they already support type converters if you don't like the way they handle e.g., datetime (they store as a Unix time_t, but my existing app database used yyyy-mm-dd, so I just wrote a trivial converter and it's up and running). See https://moor.simonbinder.eu/ or google "flutter moor".

[proninyaroslav]

@IanDarwin
The current solution is not entirely beautiful, but working: use a shadow variable in entity to store data in DB format, and convert it in the getter or initialize a field with the desired type in the constructor.

[IanDarwin]

On Sun, Jul 12, 2020 at 07:42:31AM -0700, Yaroslav Pronin wrote: ***@***.*** The current solution is not entirely beautiful, but working: use a shadow variable in entity to store data in DB format, and convert it in the getter or initialize a field with the desired type in the constructor.

That would have worked, I guess. Too far along to revert from moor to floor; busy working on the rest of the building.

[fnicastri]

@IanDarwin
The current solution is not entirely beautiful, but working: use a shadow variable in entity to store data in DB format, and convert it in the getter or initialize a field with the desired type in the constructor.

This is what we do in our current app.
Not ideal but it works.
About Moor, we discarded it because we don't like the way you need to declare the models.
We love Floor because is a thin layer ;)

[noordawod]

I looked at Moor, and it's too big and too opinionated. The less entanglements the better, and Moor is full of them.

[marcosmko]

Presumably the DateTime converter would ship as part of the library b/c that's such a commonly-used datatype. Maybe a few others too. Having the ability to map arbitrary datatypes used as fields would be definite win. Thanks && hope to see this one soon :-)

Yeah it could come as an out of the box feature.

I would like to suggest to also provide option to transform DateTime to iso8601 or millisecondsSinceEpoch too, where we could config in a build.yaml file 🤔.

Waiting for this feature to come out!

[pablo-johnson]

Hi, is there an ETA for this? thanks for your great work!

[stillie]

Any ETA on the release? I started implementing this library in my one project only to find out later that it doesnt cater for this? Sorry but I am going to have to find another library in the mean time

[vitusortner]

Thanks for your interest in this feature. I'll create a new release ASAP that contains an experimental implementation of type converters. You can follow the progress here #318.