Browse files

dependencies in COMPILING

  • Loading branch information...
1 parent 6fdfa24 commit e1cad1cda56e0c7c7a698956774457eb8b1cfabb Romain Slootmaekers committed May 5, 2011
Showing with 36 additions and 20 deletions.
  1. +8 −13 COMPILING
  2. +1 −1 Makefile
  3. +1 −1 doc/introduction.tex
  4. +26 −5 doc/nursery.tex
View
21 COMPILING
@@ -1,17 +1,12 @@
PREREQUISITES:
-ocaml >= 3.11.0 : aptitude
-ocaml-findlib >= 1.2.2 : aptitude
-libounit-ocaml-dev >= 1.0.3 : aptitude
-libreact-ocaml-dev : aptitude
-libtext-ocaml-dev : aptitude
-camlp4-extra : aptitude
-bzip2 : aptitude
-libbz2-dev : aptitude
-libbz2-ocaml-dev : aptitude
-ocaml-lwt = 2.1.1 : darcs get --to-match 'hash 20100825112633-c41ad-c9fab33910d2bb5f4d96ee7807e8c88f26b44739' http://ocsigen.org/darcs/lwt/
- make
- sudo make install
+ocaml 3.12
+findlib 1.2.7
+lwt 2.3.0
+ounit 1.1.0
+react 0.9.2
+camlbz2 0.6.0
+
COMPILING:
@@ -22,7 +17,7 @@ manifesto
ocamlbuild manifest.pdf ==> _build/manifest.pdf
executable
- ocamlbuild arakoon.native ==> arakoon.native
+ ocamlbuild -use-ocamlfind arakoon.native ==> arakoon.native
libraries
ocamlbuild arakoon_client.cma arakoon_client.cmxa ==>
View
2 Makefile
@@ -10,7 +10,7 @@ clean:
ocamlbuild -clean
build:
- ocamlbuild arakoon.byte arakoon.native arakoon_client.cma arakoon_client.cmxa arakoon_client.a manifest.pdf
+ ocamlbuild -use-ocamlfind arakoon.byte arakoon.native arakoon_client.cma arakoon_client.cmxa arakoon_client.a manifest.pdf
test:
./arakoon.native --run-all-tests
View
2 doc/introduction.tex
@@ -55,7 +55,7 @@ \subsection{Isn't this what Keyspace does?}
\item{} abandoned python client. There used to be a pure python client,
but now it's abandoned in favour of a \emph{SWIGed around C} library,
which enforces it's own reconnection strategy.
-\item{} Berkeley DB.
+\item{} Berkeley DB.\footnote{May, 2011, Scalien dropped Keyspace due to \emph{'BerkeleyDB issues'}}
\end{itemize}
It's not that these issues are impossible to address. It's just that when you're fighting these, you don't want to be owner of the problem, not a spectator.
View
31 doc/nursery.tex
@@ -2,14 +2,14 @@ \section{Scaling Arakoon}
We want to be able to use arakoon for increasingly large key-value spaces.
For a single arakoon cluster the capacity is limited by the size of a single disk.
So it is only natural to allow different arakoon clusters to team up.
-A nursery\footnote{after a \emph{a nursery of raccoons}} provides a semi-unified view on a set of arakoon clusters.
+A \emph{nursery}\footnote{after a \emph{a nursery of raccoons}} provides a semi-unified view on a set of arakoon clusters.
Each cluster is uniquely responsible for a prefix range.
\subsection{Limitations}
The simple strategy of mapping a cluster to a range of keys already implies some limitations compared to the single cluster setup.
As a result, applications willing to scale from a single cluster to a nursery need to do some planning.
\subsubsection{impact on sequences}
Sequences are multiple updates that are done atomically.
-Since atomicity can only be achieved inside 1 cluster, this means that all keys for a sequence need to share the same prefix.
+Since atomicity can only be achieved inside 1 cluster\footnote{you could build transactionality across clusters, but it's a can of worms}, this means that all keys for a sequence need to share the same prefix.
\subsubsection{impact on ranges}
Every cluster is responsible for a specific range.
As client range query will only be served by a single cluster, it means that only ranges that are subranges of cluster ranges can be served.
@@ -34,8 +34,29 @@ \subsection{Migrations}
The problem with the migration strategy is that there is a point in time where none of the clusters is serving $[k_e -a, k_e)$, so any request for anything in that range is refused.
\subsection{Client side support}
-\ldots routing tables \ldots how to get them \ldots \\
-It's not really important that the nursery clients have correct routing tables.
-If a client asks something from a cluster that's not able to comply, it will simply refuse. This means that either there is some migration, or that the client has outdated routing information. In that case, it can simply refetch the ranges from the clusters it knows.
+Each client needs to know which cluster is responsible for a certain key(-range).
+This information is kept in a routing table. At construction time, a client fetches this from a designated Arakoon that knows all the clusters in a nursery.
+The privileged clients performing migrations also must update this designated arakoon.
+As far as clients are concerned, it's not really important that the nursery clients have correct routing tables:
+If a client asks something from a cluster that's not able to comply, it will simply refuse.
+This means that either, there is some migration, or that the client has outdated routing information.
+In that case, it can simply refetch the ranges from the clusters it knows, or refetch it from the designated arakoon that keeps this information.
+\subsection{Problems}
+We depend on having a designated arakoon that knows all the clusters in a nursery, and their routing tables.
+So conceptually, we introduced a single point of failure.
+Since this point is in reality an arakoon cluster which is synchronuously replicated, that should not pose big practical problems.
+\paragraph{}
+Having to maintain configuration of (multiple/many?) arakoon clusters on lots of machines will become a significant problem.
+As this information is both crucial, and maintained by humans, which is a recipe for disaster. In time, we should move to service discovery.
+Possible options are:
+\begin{itemize}
+ \item{XMPP disco}
+ \item{DNS-Based Service Discovery}
+ \item{openSLP}
+ \item{Salutation}
+ \item{UPnP}
+ \item{svrloc}
+ \item{\ldots}
+\end{itemize}

0 comments on commit e1cad1c

Please sign in to comment.