Every repository with this icon (
Every repository with this icon (
| Description: | Beanstalk is a simple, fast work queue. edit |
-
reeler% git clone git://github.com/kr/beanstalkd.git
reeler% cd beanstalkd
reeler% ./autogen.sh
configure.in:10: installing./install-sh'<br/> configure.in:10: installing./missing'
reeler% ./configure
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... ./install-sh -c -d
checking for gawk... no
checking for mawk... no
checking for nawk... nawk
checking whether make sets $(MAKE)... yes
checking whether to enable maintainer-specific portions of Makefiles... no
checking for style of include used by make... GNU
checking for gcc... gcc
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables...
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking dependency style of gcc... none
checking for supported compiler flags... -Wall -Werror
checking for gcc... (cached) gcc
checking whether we are using the GNU C compiler... (cached) yes
checking whether gcc accepts -g... (cached) yes
checking for gcc option to accept ISO C89... (cached) none needed
checking dependency style of gcc... (cached) none
checking for a BSD-compatible install... /usr/bin/install -c
checking if compiler supports -R... yes
checking libevent install prefix... /usr/local
checking for posix_fallocate... no
checking for fdatasync... no
checking for bind in -lsocket... no
checking for inet_aton in -lnsl... no
checking for event_get_version in -levent... yes
checking for event_reinit in -levent... yes
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for uint16_t... yes
configure: creating ./config.status
config.status: creating Makefile
config.status: creating beanstalkd.spec
config.status: creating config.h
config.status: executing depfiles commands
reeler% make
make all-am
gcc -DHAVE_CONFIG_H -I. -g -O2 -Wall -Werror -I/usr/local/include -c -o beanstalkd.o beanstalkd.c
gcc -DHAVE_CONFIG_H -I. -g -O2 -Wall -Werror -I/usr/local/include -c -o binlog.o binlog.c
gcc -DHAVE_CONFIG_H -I. -g -O2 -Wall -Werror -I/usr/local/include -c -o conn.o conn.c
gcc -DHAVE_CONFIG_H -I. -g -O2 -Wall -Werror -I/usr/local/include -c -o job.o job.c
gcc -DHAVE_CONFIG_H -I. -g -O2 -Wall -Werror -I/usr/local/include -c -o ms.o ms.c
gcc -DHAVE_CONFIG_H -I. -g -O2 -Wall -Werror -I/usr/local/include -c -o net.o net.c
In file included from net.h:22,
from net.c:22:/usr/include/netinet/tcp.h:40: error: expected '=', ',', ';', 'asm' or 'attribute' before 'tcp_seq' /usr/include/netinet/tcp.h:50: error: expected specifier-qualifier-list before 'u_short' /usr/include/netinet/tcp.h:175: error: expected specifier-qualifier-list before 'u_int8_t' *** Error code 1
Stop in /usr/home/peter/beanstalkd.
*** Error code 1Stop in /usr/home/peter/beanstalkd.
reeler%
Comments
-
I get a build error on Mac OS X Snow Leopard where it says it can't find fdatasync.
Output of ./configure:
https://gist.github.com/0290aa815842b888eed4Error upon running make:
cc1: warnings being treated as errors binlog.c: In function ‘binlog_write_job’: binlog.c:491: warning: implicit declaration of function ‘fdatasync’ make[1]: *** [binlog.o] Error 1 make: *** [all] Error 2Using libevent-1.4.12-stable ("make verify" passed).
Comments
To clarify: This was while building beanstalkd-1.4 stable release from http://kr.github.com/beanstalkd/download.html
I worked around this issue by removing the following from config.h after running ./configure, before running make.
/* Define to 1 if you have the `fdatasync' function. */ #define HAVE_FDATASYNC 0Hopefully the real solution is clear to someone :)
Does snow leopard have a man page for fdatasync? Can anyone post or send me the #include lines from the man page?
I get nothing for "man fdatasync" and a Google search appears to say that it's not available on any flavor of Mac OS X.
I couldn't find it in /usr/include/unistd.h or /usr/include/sys/unistd.h
I'm confused by the line in your configure transcript that says:
checking for fdatasync... yesThat indicates that such a function is defined somewhere in your system's header files. I think a vanilla installation of 10.6 (and for sure 10.5) doesn't have fdatasync anywhere.
The build is also getting confused; configure thinks fdatasync is defined somewhere, but then gcc doesn't have a prototype for it, hence the error message.
I can confirm this as well. The only reference to fdatasync in /usr/include is in:
include/sys/syscall.h 227:#define SYS_fdatasync 187On my system, setting HAVE_FDATASYNC to 0 in config.h still results in a failed build:
gcc -DHAVE_CONFIG_H -I. -g -O2 -Wall -Werror -I/sw/local/include -c -o beanstalkd.o beanstalkd.c gcc -DHAVE_CONFIG_H -I. -g -O2 -Wall -Werror -I/sw/local/include -c -o binlog.o binlog.c cc1: warnings being treated as errors binlog.c: In function ‘binlog_write_job’: binlog.c:491: warning: implicit declaration of function ‘fdatasync’ make[1]: *** [binlog.o] Error 1 make: *** [all] Error 2Edit: Of course - removing HAVE_FDATASYNC works. Thanks, mattyoho -- serves me for not doing anything in C for a long while.
robotadam: "On my system, setting HAVE_FDATASYNC to 0 in config.h still results in a failed build:"
This was true for me as well. I commented the def out instead, cleaned and ran make again and was then able to install successfully.
copiousfreetime
Sun Oct 11 14:56:20 -0700 2009
| link
autoconf does not check this existence correctly. I believe I have fixed this.
-
beanstalkd only handles SIGINT and does then a clean shutdown. The following patch adds this behavior for SIGTERM. Tested against version 1.3
Patch:
--- beanstalkd.c.orig 2009-08-15 13:48:46.000000000 +0200 +++ beanstalkd.c 2009-08-15 13:49:11.000000000 +0200 @@ -120,6 +120,10 @@ sa.sa_handler = exit_cleanly; r = sigaction(SIGINT, &sa, 0); if (r == -1) twarn("sigaction(SIGINT)"), exit(111); + + sa.sa_handler = exit_cleanly; + r = sigaction(SIGTERM, &sa, 0); + if (r == -1) twarn("sigaction(SIGTERM)"), exit(111); } /* This is a workaround for a mystifying workaround in libevent's epollComments
-
Should have optional throttling. There should be three modes: always fsync (for best reliability), fsync at most once every N seconds, or never fsync (best performance).
Could be controlled by two options: "-s 0" for always fsync, "-s N" for sync at most once every N seconds, and "-S" for never fsync.
Comments
-
Allow beanstalkd to reserve arbitrary amounts of binlog space
1 comment Created 3 months ago by krCurrently it can only create one extra binlog file on disk. After that it will deny operations that want to reserve space. (Search for "overextended" in binlog.c.) There just needs to be a linked list of future binlog files; currently there is a pointer to (up to) one future file. In practice this only happens when the queue holds more than about 10 MB of jobs.
Comments
-
Currently the peek commands do a job_copy() before calling reply_job() presumably to avoid a race between the job being sent back to the client, and it being DELETEd by another client.
reset_conn() should free the job copy however it currently requires the job.id to be 0 before it will call job_free(). This works fine for the h_conn_timeout() case as in http://github.com/kr/beanstalkd/commit/d77814816f54c48b7bcc85c36bb681fd277d3e65 but it overloads the meaning of job.id which is an essential part of the peek responses.
The result is that we can't set a copied job.id to 0 for peek responses which means it won't get freed by reset_conn(), leading to a fast memory leak if any of the peek commands are used.
I'll submit a patch which adds a new job state to mark a job as copied (JOB_STATE_COPY) but I'm not sure if its the best way to solve this.
http://gist.github.com/139279 will hammer peek-buried sufficiently that you'll see the beanstalkd process size grow very rapidly.
Comments
Thanks for finding this and for the patch. I'm reviewing it now.
-
Currently, if a client reserves a job and then crashes, the job will be put in the ready queue with no indication that it had ever been reserved. This is not good; clients should be able to distinguish such a job from one that has never been reserved.
So we should add a "reserves" counter and/or increment the "timeouts" counter in this situation and/or introduce a separate counter just for this situation.
See http://groups.google.com/group/beanstalk-talk/t/951b6c35752257a4 for more info.
Comments
-
See http://groups.google.com/group/beanstalk-talk/t/8210428dd5f2c98f for details.
Comments
-
One of them breaks on Mac OS.
Comments
-
Avoid stupid off-by-one errors at one-second granularity.
Comments
-
See http://groups.google.com/group/beanstalk-talk/t/e589c79f86f4d0ac for the bug report.
Comments
-
1 comment Created 6 months ago by krcarefully allocate disk space for binlog recordskrx












I found a fix for this. Simply put #include <sys/types.h> under #include <sys/socket.h> in the beanstalkd source file: net.h
Fix build on FreeBSD.
Thanks to Peter Kieltyka http://github.com/pkieltyka for the fix.
Closed by 24aa767.