Create a INSTALL file explaning how to compile ONLY bucket_engine #1

shakaran opened this Issue Feb 27, 2013 · 11 comments


None yet
3 participants

I am trying to compile the bucket engine independent and I am having some difficulties.

At the moment, I notice that it is needed compile a lot projects of membase as dependency of bucket_engine.

I need install libsnappy-dev package in Ubuntu 13.04 Raring and then I can compile couchstore store project because libcouchstore is a dependency of ep-engine (aka membase engine). For compile ep-engine I need memcached, but compile with engine-pu (proposed update) branch.

After that, It failed in configure because it needs the couchbase server too. But I can't find where is the repository with the source. Only I found the download page with all the sources bundled as dependencies ( but I only want the headers needed by ep_extension.h for compile.

So, I am stuck for compile bucket_engine, because ep-engine needs couchbase-server. I am reporting this issue here, because ep-engine github repo has disabled the issues.

This is the error in configure for ep-engine:

/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I./src  -pipe -I./include -DHAVE_VISIBILITY=1 -fvisibility=hidden -I./src -I./include  -pipe  -O3  -MT src/ep_la-access_scanner.lo -MD -MP -MF src/.deps/ep_la-access_scanner.Tpo -c -o src/ep_la-access_scanner.lo `test -f 'src/' || echo './'`src/
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I./src -pipe -I./include -DHAVE_VISIBILITY=1 -fvisibility=hidden -I./src -I./include -pipe -O3 -MT src/ep_la-access_scanner.lo -MD -MP -MF src/.deps/ep_la-access_scanner.Tpo -c src/  -fPIC -DPIC -o src/.libs/ep_la-access_scanner.o
In file included from src/access_scanner.h:25:0,
                 from src/
src/common.h:118:8: error: 'ALLOCATOR_HOOKS_API' does not name a type
In file included from src/stats.h:30:0,
                 from src/item.h:33,
                 from src/bgfetcher.h:30,
                 from src/ep.h:44,
                 from src/ep_engine.h:35,
                 from src/access_scanner.h:27,
                 from src/
src/memory_tracker.h:61:5: error: 'allocator_stats' does not name a type
In file included from src/ep_engine.h:37:0,
                 from src/access_scanner.h:27,
                 from src/
src/ep_extension.h:43:45: error: 'mc_extension_token_t' has not been declared
src/ep_extension.h:46:44: error: 'mc_extension_token_t' has not been declared
make[1]: *** [src/ep_la-access_scanner.lo] Error 1
make[1]: Leaving directory `/home/shakaran/Documentos/Projects/ep-engine'
make: *** [all] Error 2

This is how I am trying to compile all the dependencies:

# Compile memcached using engine-pu branch (proposed update)
git clone -b engine-pu --single-branch git:// memcached-engine-pu
cd memcached-engine-pu; sh config/; 
.configure; make; make install

# sudo apt-get install libsnappy-dev
git clone -b master --single-branch
cd couchstore; sh config/; 
./configure; make; make install

# Require libcouchstore
git clone -b master --single-branch git://
cd ep-engine; sh config/; 

# Require memcached using engine-pu branch (proposed update) and ep-engine
git clone
cd bucket_engine/
sh config/

Downloading the couchbase server (all bundled):

mkdir -p couchbase-server; cd couchbase-server;
wget -O - | tar xvz
cd couchbase-server_src; 

Then I have all there, but I only want the files needed for ep-engine and not all the plataform.

I am trying to compile this, because I want a memcached with ACL support (for shared hosting enviroment and offer memcached with security for shared hosting users). I read that bucket_engine has integrated as far I know ¿it is possible?

trondn closed this Feb 27, 2013

@trondn I just read that webpage, but it imply download all couch-server package with the versions bundled in the tar.gz. I mean in that tar.gz you ship a exact version of memcache, ep-engine and others. I want compile the current git master of ep-engine from github. It only fails in ep-engine from github (I already note that it works compiling via the couchbase-server). So, could you please give instruction about only compile ep-engine from github?

For example, I see that mc_extension_token_t is not declared, but grepping in couch base server it is declared in couchbase-server_src/memcached/include/memcached/extension.h line 127.

But I have compiled that memcached with engine-pu and it still fails. So, what it is different from github memcache branch engine-pu and the current memcached shipped as bundle in couchbase-server?


avsej commented Feb 27, 2013

It also described in the README file

$ mkdir couchbase
$ cd couchbase
$ repo init -u git:// -m 2.1-unstable.xml
$ repo sync
$ make

It will put server installation into couchbase/install prefix


trondn commented Feb 27, 2013

In order to do that you need the "correct" memcached version with its headers (you'll find that in the bundle we ship).

@trondn I am having the same issue trying this:

./configure --with-memcached=../memcached-engine-pu

It is the right way to correct memcached version with its headers? I don't know if you are refering to other changes in the bundle that I have to perform in my git build.


trondn commented Feb 27, 2013

If you want to use bucket_engine you're going to want to use the corresponding memcached version from our memcached git repository.

Ok, I suppose that bucket_engine is using the memcached original repo. My bad.

I compile now this:

git clone -b engine --single-branch git:// memcached-membase
cd memcached-membase; sh config/;
./configure; make; make install

Then I pass the folder of compilation to ep-engine:

./configure --with-memcached=../memcached-membase

But I still get the same failures, so what I still doing wrong?


trondn commented Feb 27, 2013

Check the include path the compiler is using and see if you have some of the memcached headers earlier in that path.. if so you're getting the wrong header...

We don't really support building this isolated due to a resource issue on our side... The easiest way to build bucket engine is to follow the instruction outlined above

It doesn't seem a memcached headers problem.

I am seeing now the problem. For example for this small change:

git diff ../memcached-membase/include/memcached/extension.h ../couchbase-server/couchbase-server_src/memcached/include/memcached/extension.h 
diff --git a/../memcached-membase/include/memcached/extension.h b/../couchbase-server/couchbase-server_src/memcached/include/memcached/extension.h
index 6ec0040..e807709 100644
--- a/../memcached-membase/include/memcached/extension.h
+++ b/../couchbase-server/couchbase-server_src/memcached/include/memcached/extension.h
@@ -2,7 +2,10 @@

+#ifndef __cplusplus
 #include <stdbool.h>
 #include <stdint.h>
 #include <memcached/engine_common.h>
 #include <memcached/protocol_binary.h>
@@ -121,7 +124,7 @@ extern "C" {
     typedef struct {
         char *value;
         size_t length;
-    } token_t;
+    } mc_extension_token_t;

      * ASCII protocol extensions must provide the following descriptor to
@@ -165,7 +168,7 @@ extern "C" {
         bool (*accept)(const void *cmd_cookie,
                        void *cookie,
                        int argc,
-                       token_t *argv,
+                       mc_extension_token_t *argv,
                        size_t *ndata,
                        char **ptr);

@@ -181,7 +184,7 @@ extern "C" {
         ENGINE_ERROR_CODE (*execute)(const void *cmd_cookie,
                                      const void *cookie,
-                                     int argc, token_t *argv,
+                                     int argc, mc_extension_token_t *argv,
                                      ENGINE_ERROR_CODE (*response_handler)(const void *cookie,
                                                                            int nbytes,
                                                                            const char *dta));

In memcached membase git there are not the same code that you bundled with couchbase-server memcached folder. Why is not the same content? It is not pushed the new content to git memcached repository? It seems that only a bit files differs with small changes.

Now I can compile with:

./configure --with-memcached=../couchbase-server/couchbase-server_src/memcached/

But I would like that memcached membase git repository ships the same changes of couchbase-server.


trondn commented Feb 27, 2013

Are you using the correct branch in the memcached repository ;) we're building multiple versions, and you would have to use the correct manifest file to find the correct version to use


Thanks trondn, picking the master branch of memcache membase I can compile now without problems. Before I was picking the engine branch (as recommended in github readme). Thanks for your support.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment