Am I doing things right - loading data from multiple locations

[JerryBels]

Hello !

So I have the need to display an image, which is linked to many other data sources. Basically the image has an uploader, an album, a list of tagged users, and a list of messages.

For example to get the tagged users, I'm forced to make a call for each one of them, since I'm storing only their IDs in the media object and I need their names, genders etc...

A first query would go get the media data, and when the promise is resolved ($loaded), other promises must be fullfilled before returning to the controller the data for the view to display.

$firebaseObject(media).$loaded.then(func(mediaData) {
    $firebaseObject(uploader)...
    $firebaseObject(album)...

    foreach(tagged_users)
        $firebaseObject(this_user)...

    foreach(messages)
        $firebaseObject(this_message)...

    // when all data has been fetched successfully send it over to the controller
    $q.all(promises).then(function(){
        deferred.resolve(allData);
    });
})

It works perfectly well but :

• Is this a good way to do things with Firebase and AngularFire ?
• Performance speaking, all these $firebaseObject instances could be an issue ? Knowing I simplified it, there is a lot more data to fetch actually... Before stable release there was the $off method that could have proved usefull. Maybe should I just use Once from the regular Firebase, skipping AngularFire for the data that doesn't needs to be keep in sync ?
• How will this behave when references will grow to have millions of records ?

Thanks a lot !

[katowulf]

Hello Jerry,

If you are downloading millions of records to the client, something has gone wrong. Generally, any time you start tapping into $loaded you've also done something wrong, as this is a one-time event and only going to work with your initial data (it can't handle any records added/updated/removed after the initial load process completes).

The ideal way to manipulate data in a synchronized object or array is with the $extend method and the $$added/$$updated/$$removed hooks. This is covered in the guide under Extending Factories.

Here's a couple examples of this in action:

http://jsfiddle.net/katowulf/syuzw9k1/
http://jsfiddle.net/katowulf/vspo65qm/

[JerryBels]

Hello Kato !

Of course I will never download millions of records to the client ! The targeted path will contain only what the client need (my firebase is structured for this to be easy).

Thanks for these examples, really helpfull. I think I got how I'm supposed to do these things with AngularFire. Too bad I didn't understand earlier !

But one of my questions remains, the most important one : does all these synchronized objects have a big impact on performances ? Should I worry and use something like .Once() from the regular Firebase as much as possible and keep $firebaseArray and $firebaseObject for calls that really needs to remain synced ?

[jwngr]

The main performance bottleneck you will see if on how much bandwidth you send across the wire, not how many Firebase listeners you have set up. If the data you sync to with $firebaseObject or $firebaseArray rarely changes, then they will have hardly any performance impact. If the data is constantly changing but you don't really care about those changes, then you will be having a negative impact on performance.

[JerryBels]

That seems very clear. Thank you so much !

[JerryBels]

Kato, jwngr,

I understand much better how all of this works now, thanks again.

Just one little more question : looking at this example https://gist.github.com/katowulf/f78d4a224c06a643ddfa I would like to know if there is a way in the $$added method to wait for a response for the userData before returning "record" ? I tried using promises with no luck.

[katowulf]

There's really no reason to wait. Angular and Firebase do a great job of dealing with asynchronous content loading at some time in the future. As you can see in the fiddles I linked, it just works.

[JerryBels]

Yes, but on my app the text appears and then after a few milliseconds the username shows up, it's just ugly.

[katowulf]

A few milliseconds is undetectable to the eye. Looking at the fiddle I provided, you'll note that it loads faster than the page can render. If you are seeing a delay, then you're probably loading too much data in each user record, probably need to flatten data or download a few less records at a time. If that's not sufficient for your needs, then you'll need to roll your own.

[katowulf]

Also, other options are certainly available here, like utilizing ng-show="record.username" or something along these lines, which would prevent displaying the data until the content is fully loaded. But it doesn't seem like that should be necessary since everything is loading in milliseconds.

[JerryBels]

Thanks for the link, I followed these recommandations when I made my database plan though :)

The data is flat, there is no embedded list. The delay is really short, but it makes a little "jump" on the page that doesn't feels great. I will probably use ng-show. Thanks again, as always, you rock.

[jamestalmage]

@katowulf
Even in your fiddle, there is a short but noticeable delay when you hit the run button on JS Fiddle. That said, it's a flawed example of delays because you are setting the dummy data immediately after initializing your listeners. This means the firebase refs are getting dummy data before they ever round-trip to firebase over the network. Change line 89 in your fiddle to this:

 if (false) fb.set({

And the difference is slightly more (but not significant on my machine / connection). Using a tool like Network Link Conditioner and picking a high latency setting (3G / Edge), it finally becomes very pronounced to the point of being a distraction to users.

ng-show is a clever solution, and may end up being the best option. My only other thought would be to allow the $$added (and maybe even$$updated/$$removed) hooks to return a promise.

$$added seems pretty straight forward to implement (just don't insert a new value in the array until the promise resolves). The other two would require a bit of thought (The value on scope has already been updated, so what would "waiting" for the promise to resolve look like?).

[JerryBels]

I also have another issue : the data inserted using $$added is here for old elements, but for new ones it's here for a few milliseconds and disappears...

$$added: function (snap) {
    var record = $firebaseArray.prototype.$$added.call(this, snap);

        $firebaseObject(fbRef("users/" + record.user_id)).$loaded().then(function (userData) {
        record.userData = userData;
    });

    return record;
}

In the HTML I simply display the record like {{record}}, when adding elements the userData can be seen when the new element first pops up but disappears immediately.

[katowulf]

Another approach for normalizing data, which would inherently resolve the issue of loading time, would be to utilize a Firebase.util.NormalizedCollection with a field-based dependency

These can plug directly into $firebaseArray and $firebaseObject.

[JerryBels]

This looks awesome ! This library is officially maintained by Firebase ? Basically, can I use it and be sure it will follow Firbase updates ?

[EDIT 1] I saw here : https://github.com/firebase/firebase-util/blob/master/src/NormalizedCollection/README.md that you warn against using it in production. Is it still relevant or can I use it ? Our app is going to be released soon and that feature would help me finish it much better.

Also, do you have something about my last issue ? How to prevent the $$added data to disappear ?

[EDIT 2] I prefixed the fields with $ and it seems to work... Is this right ?

[JerryBels]

Bump for the last questions

[JerryBels]

Bump again

[jwngr]

Is firebase-util officially maintained by Firebase?

Yup, @katowulf works here at Firebase and is the maintainer. We plan to continue to support and update the library.

Is [firebase-util] ready for production?

It is still in beta, so use at your own risk. There are no known issues that I'm aware of, but it has not been battle-tested like other libraries we offer. If you do find any problems please open up GitHub issues on that repo.

Issues with $$added() data disappearing.

What fields are you prefixing with $? That seems like a hacky way around the problem. In your code sample for $added(), you are setting record.userData asynchronously (the $loaded() won't fire right away since it needs to go to the server), so when you do return record;, the record will not have the userData set. So I would not expect that to work. I'm a bit confused about you saying that the user data flashes there for a second, so maybe if you can provide a full repro (JSFiddle, Plunker, CodePen, etc.), we can use it to debug your problem and offer better feedback.

[JerryBels]

Hey, and thanks for helping !

I will put together a Codepen whenever I have some time, I'm on a rush for the next two weeks but will try.

What I'm currently doing (and that seems to work) is

record.$userData = userData;

And it seems to work. But since you seems surprised by my use of $loaded, I must say I can't see how I'm supposed to get that userData into the record if it's not that way ?

[jason-engage]

Does anyone know if using .once() opens and then closes the connection to firebase? I'd like to maximize the number of connections, and if I don't need a data syncing feature, would using .once() be more efficient than using $firebaseObject or $firebaseArray in terms of # of connections? Or do I manually have to call goOffline() to truly close the connections? Thx

[katowulf]

.once() functions much like this pseudo code:

function once(callback, cancelCallback, context) {
   ref.on(event, function onFn(snapshot) {
      ref.off(event, onFn);
      callback.call(context, snapshot);
   });
});

(Housekeeping: try to start new issues when you have new questions, so these don't turn into encyclopedias; it's a courtesy to others who will end up here searching for answers.)