Import/Export Support

[bkerensa]

Feature Request: Just fill in the first two sections below.

Bugs: To help you as fast as possible with an issue or bug please describe your issue and the steps you have taken to reproduce it in as many details as possible.

Thanks for helping us help you :-)

Remove this and above before submitting.

Goal

What do you want to achieve?

We would like to more easily be able to export data from a realm database and import it in. Right now realm does not have any official import/export support like other databases do.

Basically something like this would be nice https://github.com/realm/realm-cocoa-converter or baked right into realm. Right now its hard to get data in and out of realm in formats other than .realm which is proprietary and not very portable.

In order to export more easily to formats like CSV, XLS, PDF we will need this support added

Expected Results

Easy data import/export in multiple formats

[cmelchior]

Yes, this is definitely something we want to add, however we prioritise fixing features that doesn't have work-arounds currently, and import/export is something that should be possible using our existing API's. Although I fully agree it would be nicer if Realm had API's in place for making it easier.

One thing to keep in mind is that Realm is an Object Store which means that the data might not be immediate serialisable into tabular form, e.g. what should we do with circular references. Most likely we will make up "fake" keys for all objects, but it needs to be discussed first.

[zhihuitang]

waiting......

[Bajranghudda1]

Still waiting....
Is any idea, when you are going to add this feature?

And how i can import a big table in CSV file to my .realm database?

[cmelchior]

Sorry we don't have a timeframe yet.

[bkerensa]

@cmelchior We have been successful at exporting the realm database as csv or .realm but we feel like there may be some issues with the workaround as some people report the file is sometimes empty when there should be data.

One thing we have not yet figured out is importing as an example csv into the realm database. If at the very least Realm could produce an example on importing workaround that would probably be very helpful for all the devs using it right now.

[bkerensa]

@cmelchior Any update on this?

[Zhuinden]

I assume you just need to read the CSV into an unmanaged RealmObject and insert that into your DB in a transaction, just like you would with any batch inserts from any data source, including REST APIs.

See https://stackoverflow.com/a/39385985/2413303

[bkerensa]

Would prefer official support like other databases offer

[mauza]

Why is this feature taking so long? Is there a reason not to support it?

[cmelchior]

Mostly because it doesn't have a high priority, compared to other features on our list.

[nojvek]

I think realm should atleast have a documentation page on how one can implement import/export on their own.

This is a much needed feature for any database.

[Zhuinden]

Worst case scenario it is possible using the DynamicRealm API

[damindunc]

any update on this? I need to use this feature in my android application..

[Zhuinden]

Honestly guys, I'm not even sure what you guys need here :D

But supposedly realm studio can import CSV, I'm a bit confused by how to actually set up the schema though with it, seems to think everything is a string

[RohitSurwase]

Any update on this feature? ETA?

[mauza]

I support the backend of a mobile application. The apps teams for android and ios use realm. My interest in this is for when something goes wrong and a user needs to export their data from their phone. I load their exported realm db up in realm browser then manually compare with our data on the backend. I want to be able to automate that task. I'm sure there are many reasons for a programatic export of data from realm. The closed nature of realm and your unwillingness to make something like this a priority makes me want to move away from this technology and recommend other things.

[Zhuinden]

@RohitSurwase what exactly do you need? As I said, people are like +1 but what do they need?

What is their goal? What format to what format? How do they intend to preserve the links of objects without primary keys? How do they intend to portray links in CSV???

[RohitSurwase]

@Zhuinden My requirement is very basic, all I need is .realm file to any lighter readable file format (like .csv). Additionally, even if the exported file could not maintain the relationship without primary IDs, I would manage with that.

[Zhuinden]

Do you need to do this on Android or on Windows or also on Mac?

Although I think on Android that should be easy with the DynamicRealm API, excluding links because that sounds hard

[RohitSurwase]

@Zhuinden For Android. I'll also look into DynamicRealm API.

[Zhuinden]

@RohitSurwase Set<Class<? extends RealmModel>> latestRealmObjectClasses = realmConfiguration.getRealmObjectClasses(); is the magical component, imo.

Actually I started writing it but are you supposed to create a CSV per each table? I assume so because of how you can't really do it any other way, right?

[RohitSurwase]

@Zhuinden Yes, I want to create CSV per each table. Is there any API/function to export a table to CSV in Realm?

[bkerensa]

@cmelchior what would it take to get official import and export support? All major android databases offer it. We made a workaround for export but it isn't pretty and importing is still almost impossible.

[RohitSurwase]

@cmelchior Is that export workaround publicly available? Can you please provide a link to do so? Let me check if it fits my use case.

[bmunkholm]

Realm Studio does have an import/export feature that you can use, but that obviously wouldn't work programmatically in an app, which I assume you are requesting?

This is a feature that is very well suited for anyone to take on by using the public dynamic API zhuinden referred to. It doesn't really require any knowledge of Realm implementation details. It can be implemented using just public APIs. There are however details and compromises that need to be considered about what exactly is needed in particular with relationships.
There are many other features that would be much harder to do for someone not intimately into the codebase, so I think it's better for Realm engineers to focus on those features and ask for help from others on this feature. Alternatively, if this is an important business feature it's always a possibility to fund the development by talking to sales@realm.io.

[Zhuinden]

@bkerensa

All major android databases offer it.

I am not aware of this, can you show an example API?

I mean, Stack Overflow shows that for SQLite, people also wrote their own export function: https://stackoverflow.com/a/47464560/2413303

On that level, it is possible with Realm's APIs.

Importing is tricky because CSV won't show relationships. You have an object graph and not just foreign keys. But that's why there is no overhead of JOIN.

[hernangonzalez]

This is never gonna happen, right? :/

[Zhuinden]

@hernangonzalez probably not, no

I really am not sure how you'd do the relationships in the CSV. Unless you "add foreign keys" to the exported file, but you can't really do that without internals (object id) for relations to RealmObjects that have no primary key.

[Asalle]

seems like realm-cocoa-converter just exports into multiple cvs files (per object). Would that also work for you?