Persistence layer

[Andersos]

The developers are considering using Postgresql or Cassandra for the persistence layer because those databases are in the technology stack of Finn.no.

@jarib is looking into how we can use LevelUP

[jarib]

A quick look through the list of LevelUP modules reveals only https://github.com/calvinmetcalf/SQLdown for PostgreSQL and nothing for Cassandra.

In any case, if we limit our persistence needs to what can be done through the LevelUP API, that could help us keep things simple AND open up for plugging in other storage backends down the road. Any downsides? Especially the people who has worked on the POC should weigh in on this.

[ivarconr]

The database needs for Unleash are really limited. The thing to consider is the availability model we want for the persistence layer. Cassandra allow us to trade consistency for availability, where traditional DBs, like postgres, requires us to go with consistency over availability.

LevelUP + postgres:

• A CP system. With intelligent clients and reasonable defaults this should be acceptable in MVP1.
• Makes it super simple for us to support a wide range of databases, which definitely will make it easier for others to adopt Unleash.
• Swappable database implementation, allows us to use an in-memory database to facilitate testing.

Cassandra on the other hand will give us:

• An AP-system if we want
• A bit more complicated to test as we have to manually start a cassandra-test-server which the test can run against.

I am open for both approaches, and think we should re-evaluate the choice made after MVP1 is delivered and we have to actual production experience.

[michaelsembwever]

A bit more complicated to test as we have to manually start a cassandra-test-server which the test can run against.

For reference on how to test against cassandra – https://github.com/michaelsembwever/cassandra-project/tree/meetup-2014-09-23/

[Andersos]

@jarib We used Any-DB for the POC. It looks like Any-DB is like the abstraction layer LevelUP provides. Any-DB has adapters for the following (MS SQL, MySQL, Postgres, and SQLite3). LevelUP might be a better choice considering it has a bigger community.

[jarib]

@michaelsembwever That link is broken?

Anyone interested in learning more about the level-* ecosystem should watch this talk.

[Andersos]

@ivarconr and I talked about the Nodeschool elective Level Me Up Scotty!. Maybe a good place to start to learn more about LevelUP.

[michaelsembwever]

@jarib fixed link.

[michaelsembwever]

moved from #6

Of course the persistence layer matters in the end, but we want to be less opinionated about the implementation used. … The goal is to not be constrained and force technology on to companies, but instead build a flexible system where the user can plug-in their own persistence layer.

This is something i don't understand, and i get the feeling that the definition of success in open-sourcing unleash hasn't been clearly enough defined.

For FINN open-sourcing unleash should be about commoditising a cost-center. By designing a solution that provides flexibility for any persistence layer you are doing the opposite and committing yourself to having to support all persistence layers. While one might think that providing such flexibility will help in community building, it is a short-sighted approach and the opposite might in fact be true: that people see a product that isn't properly supported against the design it proposes and this discourages community growth.

I would focus on the correct and MVP design for the cost-center at hand.

[Andersos]

1. Unleash should be an AP system (http://en.wikipedia.org/wiki/CAP_theorem)

The design should favor availability and partition tolerance over consistency.

[gardleopard]

We are choosing postgresql for the MVP. The team has experience with postgresql and little to none with cassandra. We want to limit the amount of unknowns in the project, so we can deliver in reasonable time.