Deploy API

[ijemmao]

Now that the API has basic functionality to search for Igbo terms using either Igbo or English, it's time to release to the world.

The tools that I'm thinking about using include:

• Heroku -hosting and managing the Igbo Dictionary Node API service
• MongoDB Atlas - host the MongoDB data

What are people's thoughts about these platforms? Is there something else we should consider using that could help down the road with scaling?

[ebubae]

I think this setup definitely works well, but I think it may be pricy at scale.

Atlas is free as long as we're not storing more than 5GB, which shouldn't be a problem when only storing compressed words. That said, after 5GB we're talking $57/mo.

Heroku free tier can definitely work for development, but it may have a similar problem. Scaling up to additional containers as the app grows, you'll need to by container (or dyno? as they call it). I worry about a couple things with this:

• we don't automatically have a system that scales unless you you pay for it, and
• once you pay for it you've paid for it (even if it goes unused)

This is not to mention the API service will probably require it's own DB and infrastructure, which also start around $50.

Personally I suggest we use something like Firebase to host this project. Not only does it make it simpler to have the database and core project run using the same platform, the pricing works very well for growth.

• we could use Cloud Functions to set up serverless endpoints that automatically scale to use while getting the first 2 million calls/month for free (and 40 cents for each million after that)
• we pay per document read (first 50k/day free), and it's relatively easy to set up rate limiting and secure protocols using Security Rules

I think if we use this system earlier on we'll be paying little to nothing, and we'll also have something that scales well without having to change much infrastructure down the line. Especially since our existing solution is using JSON document based databases I think Cloud Firestore is well suited for the task (and really cheap).

[ijemmao]

@ebubae these are solid points. Scaling on these platforms doesn't seem that friendly for small apps that have slow and steady growth.

I like the idea of using Firebase because it's JSON based, and it provides a whole suite of tools that could be helpful with building full stack applications.

The only main concern that I have (which might not even be a concern) is database migrations and building schemas to keep the shape of the data consistent.

I wonder how easy it is to update the shape of objects with in the entire database. Like if I wanted to add a new key to a word object, all word objects should also have the key. Also, would you know of any Firebase schema library like mongoose that can help preserve the shape of the data?

Other than this concern, Firebase seems like a more complete solution in comparison to MongoDB

[pappyJ]

Mongodb would be better because of the flexibility mongoose schemas offer in case of any change in data structure e.g additional information field ... Also is there a similar function like mongodb aggregation in firebase??

[ijemmao]

Yeah thinking about it more, Firebase does offer a lot of services that would be beneficial to the app but it's more of a backend as a service platform, and I think if we want this to grow it would be better for us to own the backend as much as possible.

Complex queries like joining or aggregating data in one call don't seem possible with Firebase, and that's a really big downside since I imagine our data queries getting more complex down the road.

When it comes to costs, I'm still liking Firebase's model way more than Heroku and MongoDB. But, I don't see it so much as a concern right now because of our small scale. I don't see us reaching 5BG anytime soon since the database is only 1MB right now. Also, dyno hours are a solid concern since if the app is running for longer than 18 hours on average per day then we have to start paying, but it looks like we can get around that extending the number of free dyno hours we're initially given

With that being said, though, Firebase does have a nice way of storing non-text based data like images and audio (that I really love), which could be especially handy down the road if we wanted to include audio pronunciations, for example.

@ebubae what do you think?

[ebubae]

The only main concern that I have (which might not even be a concern) is database migrations and building schemas to keep the shape of the data consistent.

I think there's two easy ways to fix this without having to use Mongo Schema's — TypeScript and unit tests. Because Firebase has an Admin SDK you write in Javascript. It's easy to store types and use it across the backend. It's also explicit and defined in code. This makes migrations easier because a broken schema will fail at compile time. Once tests pass, you can be pretty confident in your migration.

Also is there a similar function like mongodb aggregation in firebase

Firebase does have aggregation using compound queries.

Complex queries like joining or aggregating data in one call don't seem possible with Firebase, and that's a really big downside since I imagine our data queries getting more complex down the road.

Joining and aggregating data is very much possible in Firebase.

Although I'm still down to use other systems I think this may be a good option for down the line especially when it's time to build out a web client.

[pappyJ]

@ebubae you're really a firebase fan man!!!

[ebubae]

@pappyJ you're not wrong. I also am not a big fan of having to define a schema when you can do it in Typescript and have essentially language supported schemas.

[pappyJ]

I'm not a lover of typescript either .... I prefer writing good tests ... But whatever z best for d project I'll adapt

[ijemmao]

It's clear that Firebase offers a really strong suite of tools to help build apps, but I think its Realtime Database isn't a significantly better option than MongoDB. I would want to take advantage of Firebase features down the road, but not with a database.

With that being said, I think it's worthwhile sticking with MongoDB because of its robust query language that's made easier to use with Mongoose. As we get more data that's interconnected, we need more freedom and flexibility with the way that we grab deeply nested data. It looks like MongoDB has better performance when it comes to completing queries.

I'm not a fan of Typescript since it makes files harder to read and I haven't experienced real benefits from using it, but that doesn't mean we couldn't start using it through the project. The benefit of relying on creating schemas with Mongoose is that we still get to use MongoDB's power query language.

On top of that, MongoDB makes it immensely easy to test data on both local machines and remote servers with the use of MongoDB Compass and MongoDB Atlas

We would also have more freedom with how we could configure the server on Heroku (or any better cloud platform option). Firebase's Cloud Functions solution feels like a roundabout solution given the fact that Firebase doesn't host servers like Heroku.

At the project's current stage, I don't think we'll see immediate benefits with either option, so I think it's a matter of who likes what more. I'm going with MongoDB.

[pappyJ]

Aye captain

[ebubae]

I'm down with whichever option. I'm always down to learn new tools.

Realtime Database isn't a significantly better option than MongoDB

True I wasn't really considering Realtime Database as the althernative. I was really thinking of using Firestore which has a stupidly easy API to read from client-side or server-side via the admin sdk and perform queries on JSON based document data. Also organizing deeply nested data is easy and they have some pretty clear docs to that end.

MongoDB makes it immensely easy to test data on both local machines and remote servers

Firebase console is the tool for for visualizing data in production and Firebase emulator is for doing it locally.

Firebase doesn't host servers like Heroku.

Firebase does host servers. Cloud Functions is the easy serverless way to use manage these servers in the cloud. If it's not being used, we don't pay, and if the service is being called more, we automatically scale up (and still don't pay until after 2 million invocations/mo. of the functions). If we want to manage those servers, we can just do that in code

That said, these are also relatively new tools and if more people know how to use Compass and Atlas then it may make more sense to use that.

I may be outnumbered in my support for Typescript but I do think strong typing makes things easier to manage (especially when building an API where everyone will ask for type support anyway). Of course, I'll just write my jsdoc if that's what we're settling on 😂

[ijemmao]

Your love for this service is PRESENT 😂 I can see why you ride heavy with Firebase though

Once we get to other features like storing media or authenticating users on the front site we could come back to evaluating Firebase.

But again when it comes to database-related features, I don't see anything extraordinary with Firebase that we can't get right now with MongoDB. I think the unfortunate truth is that we won't know if we made the right or wrong decision until we start hitting bottlenecks.

I may be outnumbered in my support for Typescript but I do think strong typing makes things easier to manage (especially when building an API where everyone will ask for type support anyway)

You actually make a good point since all of my experience with Typescript has been in private repos where we own the code. As much as I dislike Typescript, adding it to an open-source project will serve a lot future contributors down the road.

@ebubae Once you get a chance, feel free to open another ticket so we can discuss the prospects of using Typescript (like how and where we want to use it) in the project. Then that could result in us updating the CONTRIBUTING.md guide

[pappyJ]

But again when it comes to database-related features, I don't see anything extraordinary with Firebase that we can't get right now with MongoDB. I think the unfortunate truth is that we won't know if we made the right or wrong decision until we start hitting bottlenecks

There must be tradeoffs surely

[ijemmao]

Closing this issue out. We are sticking with MongoDB for the database and Heroku for the cloud platform solution.

But we will look to Firebase for other tools like media storage or authentication down the road.