Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Create a INSTALL file explaning how to compile ONLY bucket_engine #1

Closed
shakaran opened this Issue · 11 comments

3 participants

Ángel Guzmán Maeso Trond Norbye Sergey Avseyev
Ángel Guzmán Maeso

I am trying to compile the bucket engine independent and I am having some difficulties.

At the moment, I notice that it is needed compile a lot projects of membase as dependency of bucket_engine.

I need install libsnappy-dev package in Ubuntu 13.04 Raring and then I can compile couchstore store project because libcouchstore is a dependency of ep-engine (aka membase engine). For compile ep-engine I need memcached, but compile with engine-pu (proposed update) branch.

After that, It failed in configure because it needs the couchbase server too. But I can't find where is the repository with the source. Only I found the download page with all the sources bundled as dependencies (http://packages.couchbase.com/releases/2.0.0/couchbase-server_src-2.0.0.tar.g) but I only want the headers needed by ep_extension.h for compile.

So, I am stuck for compile bucket_engine, because ep-engine needs couchbase-server. I am reporting this issue here, because ep-engine github repo has disabled the issues.

This is the error in configure for ep-engine:

/bin/bash ./libtool  --tag=CXX   --mode=compile g++ -DHAVE_CONFIG_H -I. -I./src  -pipe -I./include -DHAVE_VISIBILITY=1 -fvisibility=hidden -I./src -I./include  -pipe  -O3  -MT src/ep_la-access_scanner.lo -MD -MP -MF src/.deps/ep_la-access_scanner.Tpo -c -o src/ep_la-access_scanner.lo `test -f 'src/access_scanner.cc' || echo './'`src/access_scanner.cc
libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I./src -pipe -I./include -DHAVE_VISIBILITY=1 -fvisibility=hidden -I./src -I./include -pipe -O3 -MT src/ep_la-access_scanner.lo -MD -MP -MF src/.deps/ep_la-access_scanner.Tpo -c src/access_scanner.cc  -fPIC -DPIC -o src/.libs/ep_la-access_scanner.o
In file included from src/access_scanner.h:25:0,
                 from src/access_scanner.cc:22:
src/common.h:118:8: error: 'ALLOCATOR_HOOKS_API' does not name a type
In file included from src/stats.h:30:0,
                 from src/item.h:33,
                 from src/bgfetcher.h:30,
                 from src/ep.h:44,
                 from src/ep_engine.h:35,
                 from src/access_scanner.h:27,
                 from src/access_scanner.cc:22:
src/memory_tracker.h:61:5: error: 'allocator_stats' does not name a type
In file included from src/ep_engine.h:37:0,
                 from src/access_scanner.h:27,
                 from src/access_scanner.cc:22:
src/ep_extension.h:43:45: error: 'mc_extension_token_t' has not been declared
src/ep_extension.h:46:44: error: 'mc_extension_token_t' has not been declared
make[1]: *** [src/ep_la-access_scanner.lo] Error 1
make[1]: Leaving directory `/home/shakaran/Documentos/Projects/ep-engine'
make: *** [all] Error 2

This is how I am trying to compile all the dependencies:

# Compile memcached using engine-pu branch (proposed update)
git clone -b engine-pu --single-branch git://github.com/memcached/memcached.git memcached-engine-pu
cd memcached-engine-pu; sh config/autorun.sh; 
.configure; make; make install

# sudo apt-get install libsnappy-dev
git clone -b master --single-branch https://github.com/couchbaselabs/couchstore
cd couchstore; sh config/autorun.sh; 
./configure; make; make install

# Require libcouchstore
git clone -b master --single-branch git://github.com/membase/ep-engine.git
cd ep-engine; sh config/autorun.sh; 
./configure

# Require memcached using engine-pu branch (proposed update) and ep-engine
git clone https://github.com/membase/bucket_engine.git
cd bucket_engine/
sh config/autorun.sh
./configure

Downloading the couchbase server (all bundled):

mkdir -p couchbase-server; cd couchbase-server;
wget http://packages.couchbase.com/releases/2.0.0/couchbase-server_src-2.0.0.tar.gz -O - | tar xvz
cd couchbase-server_src; 

Then I have all there, but I only want the files needed for ep-engine and not all the plataform.

I am trying to compile this, because I want a memcached with ACL support (for shared hosting enviroment and offer memcached with security for shared hosting users). I read that bucket_engine has integrated as far I know ¿it is possible?

Trond Norbye trondn closed this
Ángel Guzmán Maeso

@trondn I just read that webpage, but it imply download all couch-server package with the versions bundled in the tar.gz. I mean in that tar.gz you ship a exact version of memcache, ep-engine and others. I want compile the current git master of ep-engine from github. It only fails in ep-engine from github (I already note that it works compiling via the couchbase-server). So, could you please give instruction about only compile ep-engine from github?

For example, I see that mc_extension_token_t is not declared, but grepping in couch base server it is declared in couchbase-server_src/memcached/include/memcached/extension.h line 127.

But I have compiled that memcached with engine-pu and it still fails. So, what it is different from github memcache branch engine-pu and the current memcached shipped as bundle in couchbase-server?

Sergey Avseyev
Owner

It also described in the README file https://github.com/couchbase/manifest#readme

$ mkdir couchbase
$ cd couchbase
$ repo init -u git://github.com/couchbase/manifest.git -m 2.1-unstable.xml
$ repo sync
$ make

It will put server installation into couchbase/install prefix

Trond Norbye
Owner

In order to do that you need the "correct" memcached version with its headers (you'll find that in the bundle we ship).

Ángel Guzmán Maeso

@trondn I am having the same issue trying this:

./configure --with-memcached=../memcached-engine-pu
make

It is the right way to correct memcached version with its headers? I don't know if you are refering to other changes in the bundle that I have to perform in my git build.

Trond Norbye
Owner

If you want to use bucket_engine you're going to want to use the corresponding memcached version from our memcached git repository.

Ángel Guzmán Maeso

Ok, I suppose that bucket_engine is using the memcached original repo. My bad.

I compile now this:

git clone -b engine --single-branch git://github.com/membase/memcached.git memcached-membase
cd memcached-membase; sh config/autorun.sh;
./configure; make; make install

Then I pass the folder of compilation to ep-engine:

./configure --with-memcached=../memcached-membase

But I still get the same failures, so what I still doing wrong?

Trond Norbye
Owner

Check the include path the compiler is using and see if you have some of the memcached headers earlier in that path.. if so you're getting the wrong header...

We don't really support building this isolated due to a resource issue on our side... The easiest way to build bucket engine is to follow the instruction outlined above

Ángel Guzmán Maeso

It doesn't seem a memcached headers problem.

I am seeing now the problem. For example for this small change:

git diff ../memcached-membase/include/memcached/extension.h ../couchbase-server/couchbase-server_src/memcached/include/memcached/extension.h 
diff --git a/../memcached-membase/include/memcached/extension.h b/../couchbase-server/couchbase-server_src/memcached/include/memcached/extension.h
index 6ec0040..e807709 100644
--- a/../memcached-membase/include/memcached/extension.h
+++ b/../couchbase-server/couchbase-server_src/memcached/include/memcached/extension.h
@@ -2,7 +2,10 @@
 #ifndef MEMCACHED_EXTENSION_H
 #define MEMCACHED_EXTENSION_H

+#ifndef __cplusplus
 #include <stdbool.h>
+#endif
+
 #include <stdint.h>
 #include <memcached/engine_common.h>
 #include <memcached/protocol_binary.h>
@@ -121,7 +124,7 @@ extern "C" {
     typedef struct {
         char *value;
         size_t length;
-    } token_t;
+    } mc_extension_token_t;

     /**
      * ASCII protocol extensions must provide the following descriptor to
@@ -165,7 +168,7 @@ extern "C" {
         bool (*accept)(const void *cmd_cookie,
                        void *cookie,
                        int argc,
-                       token_t *argv,
+                       mc_extension_token_t *argv,
                        size_t *ndata,
                        char **ptr);

@@ -181,7 +184,7 @@ extern "C" {
          */
         ENGINE_ERROR_CODE (*execute)(const void *cmd_cookie,
                                      const void *cookie,
-                                     int argc, token_t *argv,
+                                     int argc, mc_extension_token_t *argv,
                                      ENGINE_ERROR_CODE (*response_handler)(const void *cookie,
                                                                            int nbytes,
                                                                            const char *dta));

In memcached membase git there are not the same code that you bundled with couchbase-server memcached folder. Why is not the same content? It is not pushed the new content to git memcached repository? It seems that only a bit files differs with small changes.

Now I can compile with:

./configure --with-memcached=../couchbase-server/couchbase-server_src/memcached/

But I would like that memcached membase git repository ships the same changes of couchbase-server.

Trond Norbye
Owner

Are you using the correct branch in the memcached repository ;) we're building multiple versions, and you would have to use the correct manifest file to find the correct version to use

Trond

Ángel Guzmán Maeso

Thanks trondn, picking the master branch of memcache membase I can compile now without problems. Before I was picking the engine branch (as recommended in github readme). Thanks for your support.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.