Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Browse files

initial commit

  • Loading branch information...
commit c62b949db8a6aff2def234d47bc541a4000f1192 0 parents
@ingenthr authored
385 LICENSE.txt
@@ -0,0 +1,385 @@
+Unless otherwise noted, all files in this distribution are released
+under the Common Development and Distribution License (CDDL),
+Version 1.0 only. Exceptions are noted within the associated
+source files.
+
+--------------------------------------------------------------------
+
+
+COMMON DEVELOPMENT AND DISTRIBUTION LICENSE Version 1.0
+
+1. Definitions.
+
+ 1.1. "Contributor" means each individual or entity that creates
+ or contributes to the creation of Modifications.
+
+ 1.2. "Contributor Version" means the combination of the Original
+ Software, prior Modifications used by a Contributor (if any),
+ and the Modifications made by that particular Contributor.
+
+ 1.3. "Covered Software" means (a) the Original Software, or (b)
+ Modifications, or (c) the combination of files containing
+ Original Software with files containing Modifications, in
+ each case including portions thereof.
+
+ 1.4. "Executable" means the Covered Software in any form other
+ than Source Code.
+
+ 1.5. "Initial Developer" means the individual or entity that first
+ makes Original Software available under this License.
+
+ 1.6. "Larger Work" means a work which combines Covered Software or
+ portions thereof with code not governed by the terms of this
+ License.
+
+ 1.7. "License" means this document.
+
+ 1.8. "Licensable" means having the right to grant, to the maximum
+ extent possible, whether at the time of the initial grant or
+ subsequently acquired, any and all of the rights conveyed
+ herein.
+
+ 1.9. "Modifications" means the Source Code and Executable form of
+ any of the following:
+
+ A. Any file that results from an addition to, deletion from or
+ modification of the contents of a file containing Original
+ Software or previous Modifications;
+
+ B. Any new file that contains any part of the Original
+ Software or previous Modifications; or
+
+ C. Any new file that is contributed or otherwise made
+ available under the terms of this License.
+
+ 1.10. "Original Software" means the Source Code and Executable
+ form of computer software code that is originally released
+ under this License.
+
+ 1.11. "Patent Claims" means any patent claim(s), now owned or
+ hereafter acquired, including without limitation, method,
+ process, and apparatus claims, in any patent Licensable by
+ grantor.
+
+ 1.12. "Source Code" means (a) the common form of computer software
+ code in which modifications are made and (b) associated
+ documentation included in or with such code.
+
+ 1.13. "You" (or "Your") means an individual or a legal entity
+ exercising rights under, and complying with all of the terms
+ of, this License. For legal entities, "You" includes any
+ entity which controls, is controlled by, or is under common
+ control with You. For purposes of this definition,
+ "control" means (a) the power, direct or indirect, to cause
+ the direction or management of such entity, whether by
+ contract or otherwise, or (b) ownership of more than fifty
+ percent (50%) of the outstanding shares or beneficial
+ ownership of such entity.
+
+2. License Grants.
+
+ 2.1. The Initial Developer Grant.
+
+ Conditioned upon Your compliance with Section 3.1 below and
+ subject to third party intellectual property claims, the Initial
+ Developer hereby grants You a world-wide, royalty-free,
+ non-exclusive license:
+
+ (a) under intellectual property rights (other than patent or
+ trademark) Licensable by Initial Developer, to use,
+ reproduce, modify, display, perform, sublicense and
+ distribute the Original Software (or portions thereof),
+ with or without Modifications, and/or as part of a Larger
+ Work; and
+
+ (b) under Patent Claims infringed by the making, using or
+ selling of Original Software, to make, have made, use,
+ practice, sell, and offer for sale, and/or otherwise
+ dispose of the Original Software (or portions thereof).
+
+ (c) The licenses granted in Sections 2.1(a) and (b) are
+ effective on the date Initial Developer first distributes
+ or otherwise makes the Original Software available to a
+ third party under the terms of this License.
+
+ (d) Notwithstanding Section 2.1(b) above, no patent license is
+ granted: (1) for code that You delete from the Original
+ Software, or (2) for infringements caused by: (i) the
+ modification of the Original Software, or (ii) the
+ combination of the Original Software with other software
+ or devices.
+
+ 2.2. Contributor Grant.
+
+ Conditioned upon Your compliance with Section 3.1 below and
+ subject to third party intellectual property claims, each
+ Contributor hereby grants You a world-wide, royalty-free,
+ non-exclusive license:
+
+ (a) under intellectual property rights (other than patent or
+ trademark) Licensable by Contributor to use, reproduce,
+ modify, display, perform, sublicense and distribute the
+ Modifications created by such Contributor (or portions
+ thereof), either on an unmodified basis, with other
+ Modifications, as Covered Software and/or as part of a
+ Larger Work; and
+
+ (b) under Patent Claims infringed by the making, using, or
+ selling of Modifications made by that Contributor either
+ alone and/or in combination with its Contributor Version
+ (or portions of such combination), to make, use, sell,
+ offer for sale, have made, and/or otherwise dispose of:
+ (1) Modifications made by that Contributor (or portions
+ thereof); and (2) the combination of Modifications made by
+ that Contributor with its Contributor Version (or portions
+ of such combination).
+
+ (c) The licenses granted in Sections 2.2(a) and 2.2(b) are
+ effective on the date Contributor first distributes or
+ otherwise makes the Modifications available to a third
+ party.
+
+ (d) Notwithstanding Section 2.2(b) above, no patent license is
+ granted: (1) for any code that Contributor has deleted
+ from the Contributor Version; (2) for infringements caused
+ by: (i) third party modifications of Contributor Version,
+ or (ii) the combination of Modifications made by that
+ Contributor with other software (except as part of the
+ Contributor Version) or other devices; or (3) under Patent
+ Claims infringed by Covered Software in the absence of
+ Modifications made by that Contributor.
+
+3. Distribution Obligations.
+
+ 3.1. Availability of Source Code.
+
+ Any Covered Software that You distribute or otherwise make
+ available in Executable form must also be made available in Source
+ Code form and that Source Code form must be distributed only under
+ the terms of this License. You must include a copy of this
+ License with every copy of the Source Code form of the Covered
+ Software You distribute or otherwise make available. You must
+ inform recipients of any such Covered Software in Executable form
+ as to how they can obtain such Covered Software in Source Code
+ form in a reasonable manner on or through a medium customarily
+ used for software exchange.
+
+ 3.2. Modifications.
+
+ The Modifications that You create or to which You contribute are
+ governed by the terms of this License. You represent that You
+ believe Your Modifications are Your original creation(s) and/or
+ You have sufficient rights to grant the rights conveyed by this
+ License.
+
+ 3.3. Required Notices.
+
+ You must include a notice in each of Your Modifications that
+ identifies You as the Contributor of the Modification. You may
+ not remove or alter any copyright, patent or trademark notices
+ contained within the Covered Software, or any notices of licensing
+ or any descriptive text giving attribution to any Contributor or
+ the Initial Developer.
+
+ 3.4. Application of Additional Terms.
+
+ You may not offer or impose any terms on any Covered Software in
+ Source Code form that alters or restricts the applicable version
+ of this License or the recipients' rights hereunder. You may
+ choose to offer, and to charge a fee for, warranty, support,
+ indemnity or liability obligations to one or more recipients of
+ Covered Software. However, you may do so only on Your own behalf,
+ and not on behalf of the Initial Developer or any Contributor.
+ You must make it absolutely clear that any such warranty, support,
+ indemnity or liability obligation is offered by You alone, and You
+ hereby agree to indemnify the Initial Developer and every
+ Contributor for any liability incurred by the Initial Developer or
+ such Contributor as a result of warranty, support, indemnity or
+ liability terms You offer.
+
+ 3.5. Distribution of Executable Versions.
+
+ You may distribute the Executable form of the Covered Software
+ under the terms of this License or under the terms of a license of
+ Your choice, which may contain terms different from this License,
+ provided that You are in compliance with the terms of this License
+ and that the license for the Executable form does not attempt to
+ limit or alter the recipient's rights in the Source Code form from
+ the rights set forth in this License. If You distribute the
+ Covered Software in Executable form under a different license, You
+ must make it absolutely clear that any terms which differ from
+ this License are offered by You alone, not by the Initial
+ Developer or Contributor. You hereby agree to indemnify the
+ Initial Developer and every Contributor for any liability incurred
+ by the Initial Developer or such Contributor as a result of any
+ such terms You offer.
+
+ 3.6. Larger Works.
+
+ You may create a Larger Work by combining Covered Software with
+ other code not governed by the terms of this License and
+ distribute the Larger Work as a single product. In such a case,
+ You must make sure the requirements of this License are fulfilled
+ for the Covered Software.
+
+4. Versions of the License.
+
+ 4.1. New Versions.
+
+ Sun Microsystems, Inc. is the initial license steward and may
+ publish revised and/or new versions of this License from time to
+ time. Each version will be given a distinguishing version number.
+ Except as provided in Section 4.3, no one other than the license
+ steward has the right to modify this License.
+
+ 4.2. Effect of New Versions.
+
+ You may always continue to use, distribute or otherwise make the
+ Covered Software available under the terms of the version of the
+ License under which You originally received the Covered Software.
+ If the Initial Developer includes a notice in the Original
+ Software prohibiting it from being distributed or otherwise made
+ available under any subsequent version of the License, You must
+ distribute and make the Covered Software available under the terms
+ of the version of the License under which You originally received
+ the Covered Software. Otherwise, You may also choose to use,
+ distribute or otherwise make the Covered Software available under
+ the terms of any subsequent version of the License published by
+ the license steward.
+
+ 4.3. Modified Versions.
+
+ When You are an Initial Developer and You want to create a new
+ license for Your Original Software, You may create and use a
+ modified version of this License if You: (a) rename the license
+ and remove any references to the name of the license steward
+ (except to note that the license differs from this License); and
+ (b) otherwise make it clear that the license contains terms which
+ differ from this License.
+
+5. DISCLAIMER OF WARRANTY.
+
+ COVERED SOFTWARE IS PROVIDED UNDER THIS LICENSE ON AN "AS IS"
+ BASIS, WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED,
+ INCLUDING, WITHOUT LIMITATION, WARRANTIES THAT THE COVERED
+ SOFTWARE IS FREE OF DEFECTS, MERCHANTABLE, FIT FOR A PARTICULAR
+ PURPOSE OR NON-INFRINGING. THE ENTIRE RISK AS TO THE QUALITY AND
+ PERFORMANCE OF THE COVERED SOFTWARE IS WITH YOU. SHOULD ANY
+ COVERED SOFTWARE PROVE DEFECTIVE IN ANY RESPECT, YOU (NOT THE
+ INITIAL DEVELOPER OR ANY OTHER CONTRIBUTOR) ASSUME THE COST OF ANY
+ NECESSARY SERVICING, REPAIR OR CORRECTION. THIS DISCLAIMER OF
+ WARRANTY CONSTITUTES AN ESSENTIAL PART OF THIS LICENSE. NO USE OF
+ ANY COVERED SOFTWARE IS AUTHORIZED HEREUNDER EXCEPT UNDER THIS
+ DISCLAIMER.
+
+6. TERMINATION.
+
+ 6.1. This License and the rights granted hereunder will terminate
+ automatically if You fail to comply with terms herein and fail to
+ cure such breach within 30 days of becoming aware of the breach.
+ Provisions which, by their nature, must remain in effect beyond
+ the termination of this License shall survive.
+
+ 6.2. If You assert a patent infringement claim (excluding
+ declaratory judgment actions) against Initial Developer or a
+ Contributor (the Initial Developer or Contributor against whom You
+ assert such claim is referred to as "Participant") alleging that
+ the Participant Software (meaning the Contributor Version where
+ the Participant is a Contributor or the Original Software where
+ the Participant is the Initial Developer) directly or indirectly
+ infringes any patent, then any and all rights granted directly or
+ indirectly to You by such Participant, the Initial Developer (if
+ the Initial Developer is not the Participant) and all Contributors
+ under Sections 2.1 and/or 2.2 of this License shall, upon 60 days
+ notice from Participant terminate prospectively and automatically
+ at the expiration of such 60 day notice period, unless if within
+ such 60 day period You withdraw Your claim with respect to the
+ Participant Software against such Participant either unilaterally
+ or pursuant to a written agreement with Participant.
+
+ 6.3. In the event of termination under Sections 6.1 or 6.2 above,
+ all end user licenses that have been validly granted by You or any
+ distributor hereunder prior to termination (excluding licenses
+ granted to You by any distributor) shall survive termination.
+
+7. LIMITATION OF LIABILITY.
+
+ UNDER NO CIRCUMSTANCES AND UNDER NO LEGAL THEORY, WHETHER TORT
+ (INCLUDING NEGLIGENCE), CONTRACT, OR OTHERWISE, SHALL YOU, THE
+ INITIAL DEVELOPER, ANY OTHER CONTRIBUTOR, OR ANY DISTRIBUTOR OF
+ COVERED SOFTWARE, OR ANY SUPPLIER OF ANY OF SUCH PARTIES, BE
+ LIABLE TO ANY PERSON FOR ANY INDIRECT, SPECIAL, INCIDENTAL, OR
+ CONSEQUENTIAL DAMAGES OF ANY CHARACTER INCLUDING, WITHOUT
+ LIMITATION, DAMAGES FOR LOST PROFITS, LOSS OF GOODWILL, WORK
+ STOPPAGE, COMPUTER FAILURE OR MALFUNCTION, OR ANY AND ALL OTHER
+ COMMERCIAL DAMAGES OR LOSSES, EVEN IF SUCH PARTY SHALL HAVE BEEN
+ INFORMED OF THE POSSIBILITY OF SUCH DAMAGES. THIS LIMITATION OF
+ LIABILITY SHALL NOT APPLY TO LIABILITY FOR DEATH OR PERSONAL
+ INJURY RESULTING FROM SUCH PARTY'S NEGLIGENCE TO THE EXTENT
+ APPLICABLE LAW PROHIBITS SUCH LIMITATION. SOME JURISDICTIONS DO
+ NOT ALLOW THE EXCLUSION OR LIMITATION OF INCIDENTAL OR
+ CONSEQUENTIAL DAMAGES, SO THIS EXCLUSION AND LIMITATION MAY NOT
+ APPLY TO YOU.
+
+8. U.S. GOVERNMENT END USERS.
+
+ The Covered Software is a "commercial item," as that term is
+ defined in 48 C.F.R. 2.101 (Oct. 1995), consisting of "commercial
+ computer software" (as that term is defined at 48
+ C.F.R. 252.227-7014(a)(1)) and "commercial computer software
+ documentation" as such terms are used in 48 C.F.R. 12.212
+ (Sept. 1995). Consistent with 48 C.F.R. 12.212 and 48
+ C.F.R. 227.7202-1 through 227.7202-4 (June 1995), all
+ U.S. Government End Users acquire Covered Software with only those
+ rights set forth herein. This U.S. Government Rights clause is in
+ lieu of, and supersedes, any other FAR, DFAR, or other clause or
+ provision that addresses Government rights in computer software
+ under this License.
+
+9. MISCELLANEOUS.
+
+ This License represents the complete agreement concerning subject
+ matter hereof. If any provision of this License is held to be
+ unenforceable, such provision shall be reformed only to the extent
+ necessary to make it enforceable. This License shall be governed
+ by the law of the jurisdiction specified in a notice contained
+ within the Original Software (except to the extent applicable law,
+ if any, provides otherwise), excluding such jurisdiction's
+ conflict-of-law provisions. Any litigation relating to this
+ License shall be subject to the jurisdiction of the courts located
+ in the jurisdiction and venue specified in a notice contained
+ within the Original Software, with the losing party responsible
+ for costs, including, without limitation, court costs and
+ reasonable attorneys' fees and expenses. The application of the
+ United Nations Convention on Contracts for the International Sale
+ of Goods is expressly excluded. Any law or regulation which
+ provides that the language of a contract shall be construed
+ against the drafter shall not apply to this License. You agree
+ that You alone are responsible for compliance with the United
+ States export administration regulations (and the export control
+ laws and regulation of any other countries) when You use,
+ distribute or otherwise make available any Covered Software.
+
+10. RESPONSIBILITY FOR CLAIMS.
+
+ As between Initial Developer and the Contributors, each party is
+ responsible for claims and damages arising, directly or
+ indirectly, out of its utilization of rights under this License
+ and You agree to work with Initial Developer and Contributors to
+ distribute such responsibility on an equitable basis. Nothing
+ herein is intended or shall be deemed to constitute any admission
+ of liability.
+
+--------------------------------------------------------------------
+
+NOTICE PURSUANT TO SECTION 9 OF THE COMMON DEVELOPMENT AND
+DISTRIBUTION LICENSE (CDDL)
+
+For Covered Software in this distribution, this License shall
+be governed by the laws of the State of California (excluding
+conflict-of-law provisions).
+
+Any litigation relating to this License shall be subject to the
+jurisdiction of the Federal Courts of the Northern District of
+California and the state courts of the State of California, with
+venue lying in Santa Clara County, California.
5 Makefile.am
@@ -0,0 +1,5 @@
+AUTOMAKE_OPTIONS = foreign
+bin_PROGRAMS = testclient
+
+testclient_SOURCES = main.c libmemc.c libmemc.h boxmuller.c metrics.c
+
42 boxmuller.c
@@ -0,0 +1,42 @@
+/* code borrowed from ftp://ftp.taygeta.com/pub/c/boxmuller.c */
+/* boxmuller.c Implements the Polar form of the Box-Muller
+ Transformation
+
+ (c) Copyright 1994, Everett F. Carter Jr.
+ Permission is granted by the author to use
+ this software for any application provided this
+ copyright notice is preserved.
+
+*/
+#include <math.h>
+#include <stdlib.h>
+
+double box_muller(double m, double s) /* normal random variate generator */
+{ /* mean m, standard deviation s */
+ double x1, x2, w, y1;
+ static double y2;
+ static int use_last = 0;
+
+ if (use_last) /* use value from previous call */
+ {
+ y1 = y2;
+ use_last = 0;
+ }
+ else
+ {
+ do {
+ x1 = 2.0 * drand48() - 1.0;
+ x2 = 2.0 * drand48() - 1.0;
+ w = x1 * x1 + x2 * x2;
+ } while ( w >= 1.0 );
+
+ w = sqrt( (-2.0 * log( w ) ) / w );
+ y1 = x1 * w;
+ y2 = x2 * w;
+ use_last = 1;
+ }
+
+ double result = ( m + y1 * s );
+
+ return result;
+}
107 config/bootstrap
@@ -0,0 +1,107 @@
+#!/usr/bin/env bash
+# Taken from lighthttpd server (BSD). Thanks Jan!
+# then taken from libmemcached :)
+# Run this to generate all the initial makefiles, etc.
+
+die() { echo "$@"; exit 1; }
+
+# LIBTOOLIZE=${LIBTOOLIZE:-libtoolize}
+LIBTOOLIZE_FLAGS=" --automake --copy --force"
+# ACLOCAL=${ACLOCAL:-aclocal}
+ACLOCAL_FLAGS="-I m4"
+# AUTOHEADER=${AUTOHEADER:-autoheader}
+# AUTOMAKE=${AUTOMAKE:-automake}
+AUTOMAKE_FLAGS="--add-missing --copy --force"
+# AUTOCONF=${AUTOCONF:-autoconf}
+
+ARGV0=$0
+ARGS="$@"
+
+
+run() {
+ echo "$ARGV0: running \`$@' $ARGS"
+ $@ $ARGS
+}
+
+## jump out if one of the programs returns 'false'
+set -e
+
+## We do not currently support glibtoolize
+if test x$LIBTOOLIZE = x; then
+ if test \! "x`which glibtoolize 2> /dev/null | grep -v '^no'`" = x; then
+ LIBTOOLIZE=glibtoolize
+ elif test \! "x`which libtoolize-1.5 2> /dev/null | grep -v '^no'`" = x; then
+ LIBTOOLIZE=libtoolize-1.5
+ elif test \! "x`which libtoolize 2> /dev/null | grep -v '^no'`" = x; then
+ LIBTOOLIZE=libtoolize
+ else
+ echo "libtoolize 1.5.x wasn't found, exiting"; exit 0
+ fi
+fi
+
+## suse has aclocal and aclocal-1.9
+if test x$ACLOCAL = x; then
+ if test \! "x`which aclocal-1.10 2> /dev/null | grep -v '^no'`" = x; then
+ ACLOCAL=aclocal-1.10
+ elif test \! "x`which aclocal-1.9 2> /dev/null | grep -v '^no'`" = x; then
+ ACLOCAL=aclocal-1.9
+ elif test \! "x`which aclocal19 2> /dev/null | grep -v '^no'`" = x; then
+ ACLOCAL=aclocal19
+ elif test \! "x`which aclocal 2> /dev/null | grep -v '^no'`" = x; then
+ ACLOCAL=aclocal
+ else
+ echo "automake 1.9.x (aclocal) wasn't found, exiting"; exit 0
+ fi
+fi
+
+if test x$AUTOMAKE = x; then
+ if test \! "x`which automake-1.10 2> /dev/null | grep -v '^no'`" = x; then
+ AUTOMAKE=automake-1.10
+ elif test \! "x`which automake-1.9 2> /dev/null | grep -v '^no'`" = x; then
+ AUTOMAKE=automake-1.9
+ elif test \! "x`which automake19 2> /dev/null | grep -v '^no'`" = x; then
+ AUTOMAKE=automake19
+ elif test \! "x`which automake 2> /dev/null | grep -v '^no'`" = x; then
+ AUTOMAKE=automake
+ else
+ echo "automake 1.9.x wasn't found, exiting"; exit 0
+ fi
+fi
+
+
+## macosx has autoconf-2.59 and autoconf-2.60
+if test x$AUTOCONF = x; then
+ if test \! "x`which autoconf-2.59 2> /dev/null | grep -v '^no'`" = x; then
+ AUTOCONF=autoconf-2.59
+ elif test \! "x`which autoconf259 2> /dev/null | grep -v '^no'`" = x; then
+ AUTOCONF=autoconf259
+ elif test \! "x`which autoconf 2> /dev/null | grep -v '^no'`" = x; then
+ AUTOCONF=autoconf
+ else
+ echo "autoconf 2.59+ wasn't found, exiting"; exit 0
+ fi
+fi
+
+if test x$AUTOHEADER = x; then
+ if test \! "x`which autoheader-2.59 2> /dev/null | grep -v '^no'`" = x; then
+ AUTOHEADER=autoheader-2.59
+ elif test \! "x`which autoheader259 2> /dev/null | grep -v '^no'`" = x; then
+ AUTOHEADER=autoheader259
+ elif test \! "x`which autoheader 2> /dev/null | grep -v '^no'`" = x; then
+ AUTOHEADER=autoheader
+ else
+ echo "autoconf 2.59+ (autoheader) wasn't found, exiting"; exit 0
+ fi
+fi
+
+
+# --force means overwrite ltmain.sh script if it already exists
+# run $LIBTOOLIZE $LIBTOOLIZE_FLAGS || die "Can't execute libtoolize"
+
+run $ACLOCAL $ACLOCAL_FLAGS || die "Can't execute aclocal"
+run $AUTOHEADER || die "Can't execute autoheader"
+
+# --add-missing instructs automake to install missing auxiliary files
+# and --force to overwrite them if they already exist
+run $AUTOMAKE $AUTOMAKE_FLAGS || die "Can't execute automake"
+run $AUTOCONF || die "Can't execute autoconf"
97 configure.ac
@@ -0,0 +1,97 @@
+AC_PREREQ(2.52)
+AC_INIT(memcachebench, 0.5.0, ingenthr@cep.net)
+AC_CONFIG_SRCDIR(main.c)
+AM_INIT_AUTOMAKE(AC_PACKAGE_NAME, AC_PACKAGE_VERSION)
+AM_CONFIG_HEADER(config.h)
+
+AC_PROG_CC
+AC_PROG_CC_C99
+AM_PROG_CC_C_O
+AC_PROG_INSTALL
+AC_C_CONST
+
+AC_ARG_ENABLE(64bit,
+ [AS_HELP_STRING([--enable-64bit],[build 64bit version])])
+if test "x$enable_64bit" == "xyes"
+then
+ org_cflags=$CFLAGS
+ CFLAGS=-m64
+ AC_RUN_IFELSE(
+ [AC_LANG_PROGRAM([], [dnl
+return sizeof(void*) == 8 ? 0 : 1;
+ ])
+ ],[
+ CFLAGS="-m64 $org_cflags"
+ ],[
+ AC_MSG_ERROR([Don't know how to build a 64-bit object.])
+ ])
+fi
+
+AC_ARG_WITH(libmemcached,
+ AS_HELP_STRING(--with-libmemcached,[Add support for libmemcached]),
+ [
+ if test "x$withval" = "xno"
+ then
+ without_libmemcached=true
+ else
+ if test "x$withval" != "x"
+ then
+ CFLAGS="-I$withval/include $CFLAGS"
+ CPPFLAGS="-I$withval/include $CPPFLAGS"
+ # TODO: linux should have --rpath instead of -R
+ LDFLAGS="-L$withval/lib -Wl,-R$withval/lib"
+ fi
+ fi
+ ]
+)
+
+AC_ARG_WITH(memcached,
+ AS_HELP_STRING(--with-memcached,[Specify location of memcached installation]),
+ [
+ if test "x$withval" = "xno"
+ then
+ without_memcached=true
+ else
+ if test "x$withval" != "x"
+ then
+ CFLAGS="-I$withval/include $CFLAGS"
+ CPPFLAGS="-I$withval/include $CPPFLAGS"
+ fi
+ fi
+ ]
+)
+
+# Currently I only support GCC and Sun Studio
+if test "$GCC" = "yes"
+then
+ CFLAGS="-Wall -O3 $CFLAGS"
+else
+ CFLAGS="-Xa -xstrconst -mt -xunroll=2 -xprefetch=auto -xprefetch_level=3 $CFLAGS"
+fi
+
+AC_SEARCH_LIBS(socket, socket)
+AC_SEARCH_LIBS(gethostbyname, nsl)
+AC_SEARCH_LIBS(log, m)
+AC_SEARCH_LIBS(sqrt, m)
+AC_SEARCH_LIBS(pthread_create, pthread)
+
+AC_CHECK_HEADER(sys/mman.h, AC_DEFINE(HAVE_MMAN_H, 1, [Define if you have sys/mman.h]))
+AC_CHECK_FUNC(mmap, AC_DEFINE(HAVE_MMAP, 1, [Define if you have mmap]))
+AC_CHECK_FUNC(madvise, AC_DEFINE(HAVE_MADVISE, 1, [Define if you have madvise]))
+AC_CHECK_FUNC(gethrtime, AC_DEFINE(HAVE_GETHRTIME, 1, [Define if you have gethrtime]))
+
+if test "$without_libmemcaced" != "true"
+then
+ AC_CHECK_HEADER(libmemcached/memcached.h,
+ AC_SEARCH_LIBS(memcached_create, memcached,
+ AC_DEFINE(HAVE_LIBMEMCACHED, 1, [Define if you have libmemcached/memcached.h])))
+fi
+
+if test "$without_memcaced" != "true"
+then
+ AC_CHECK_HEADER(memcached/protocol_binary.h,
+ AC_DEFINE(HAVE_PROTOCOL_BINARY, 1, [Define if you have memcached/protocol_binary.h]))
+fi
+
+AC_CONFIG_FILES(Makefile)
+AC_OUTPUT
752 libmemc.c
@@ -0,0 +1,752 @@
+/*
+ * CDDL HEADER START
+ *
+ * The contents of this file are subject to the terms of the
+ * Common Development and Distribution License (the "License").
+ * You may not use this file except in compliance with the License.
+ *
+ * See LICENSE.txt included in this distribution for the specific
+ * language governing permissions and limitations under the License.
+ *
+ * When distributing Covered Code, include this CDDL HEADER in each
+ * file and include the License file at LICENSE.txt.
+ * If applicable, add the following below this CDDL HEADER, with the
+ * fields enclosed by brackets "[]" replaced with your own identifying
+ * information: Portions Copyright [yyyy] [name of copyright owner]
+ *
+ * CDDL HEADER END
+ */
+
+/*
+ * Portions Copyright 2009 Matt Ingenthron
+ */
+
+#include "config.h"
+
+#include "libmemc.h"
+
+#if HAVE_PROTOCOL_BINARY
+#include <memcached/protocol_binary.h>
+#endif
+#include <sys/types.h>
+#include <sys/socket.h>
+#include <netdb.h>
+#include <arpa/inet.h>
+#include <netinet/tcp.h>
+#include <unistd.h>
+#include <errno.h>
+#include <stdlib.h>
+#include <string.h>
+#include <stdio.h>
+#include <assert.h>
+
+struct Server {
+ int sock;
+ struct addrinfo *addrinfo;
+ const char *errmsg;
+ const char *peername;
+ char *buffer;
+ int buffersize;
+};
+
+enum StoreCommand {add, set, replace};
+
+struct Memcache {
+ struct Server** servers;
+ enum Protocol protocol;
+ int no_servers;
+};
+
+static struct Server* server_create(const char *name, in_port_t port);
+static void server_destroy(struct Server *server);
+static int textual_store(struct Server* server, enum StoreCommand cmd,
+ const struct Item *item);
+static int textual_get(struct Server* server, struct Item* item);
+static int binary_store(struct Server* server, enum StoreCommand cmd,
+ const struct Item *item);
+static int binary_get(struct Server* server, struct Item* item);
+static int libmemc_store(struct Memcache* handle, enum StoreCommand cmd, const struct Item *item);
+static struct Server *get_server(struct Memcache *handle, const char *key);
+static int server_connect(struct Server *server);
+
+
+/**
+ * External interface
+ */
+struct Memcache* libmemc_create(enum Protocol protocol) {
+ struct Memcache* ret = calloc(1, sizeof(struct Memcache));
+ if (ret != NULL) {
+ ret->protocol = protocol;
+ }
+ return ret;
+}
+
+void libmemc_destroy(struct Memcache* handle) {
+ for (int ii = 0; ii < handle->no_servers; ++ii) {
+ server_destroy(handle->servers[ii]);
+ }
+ free(handle);
+}
+
+int libmemc_add_server(struct Memcache *handle, const char *host, in_port_t port) {
+ struct Server** servers = calloc(handle->no_servers + 1, sizeof(struct Server));
+ struct Server** old = handle->servers;
+
+ if (servers == 0) {
+ return -1;
+ }
+
+ for (int ii = 0; ii < handle->no_servers; ++ii) {
+ servers[ii] = handle->servers[ii];
+ }
+
+ handle->servers = servers;
+ free(old);
+
+ struct Server *server = server_create(host, port);
+ if (server != NULL) {
+ handle->servers[handle->no_servers++] = server;
+ }
+
+ return 0;
+}
+
+int libmemc_add(struct Memcache *handle, const struct Item *item) {
+ return libmemc_store(handle, add, item);
+}
+
+int libmemc_set(struct Memcache *handle, const struct Item *item) {
+ return libmemc_store(handle, set, item);
+}
+
+int libmemc_replace(struct Memcache *handle, const struct Item *item) {
+ return libmemc_store(handle, replace, item);
+}
+
+int libmemc_get(struct Memcache *handle, struct Item *item) {
+ struct Server* server = get_server(handle, item->key);
+ if (server == NULL) {
+ return -1;
+ } else {
+ if (server->sock == -1) {
+ if (server_connect(server) == -1) {
+ fprintf(stderr, "%s\n", server->errmsg);
+ fflush(stderr);
+ return -1;
+ }
+ }
+
+ if (handle->protocol == Binary) {
+ return binary_get(server, item);
+ } else {
+ return textual_get(server, item);
+ }
+ }
+}
+
+static struct addrinfo *lookuphost(const char *hostname, in_port_t port)
+{
+ struct addrinfo *ai = 0;
+ struct addrinfo hints = {0};
+ char service[NI_MAXSERV];
+ int error;
+
+ hints.ai_flags = AI_PASSIVE|AI_ADDRCONFIG;
+ hints.ai_family = AF_UNSPEC;
+ hints.ai_protocol = IPPROTO_TCP;
+ hints.ai_socktype = SOCK_STREAM;
+
+ (void)snprintf(service, NI_MAXSERV, "%d", port);
+ if ((error = getaddrinfo(hostname, service, &hints, &ai)) != 0) {
+ if (error != EAI_SYSTEM) {
+ fprintf(stderr, "getaddrinfo(): %s\n", gai_strerror(error));
+ } else {
+ perror("getaddrinfo()");
+ }
+ }
+
+ return ai;
+}
+
+int libmemc_connect_server(const char *hostname, in_port_t port)
+{
+ struct addrinfo *ai = lookuphost(hostname, port);
+ int sock = -1;
+ if (ai != NULL) {
+ if ((sock = socket(ai->ai_family, ai->ai_socktype,
+ ai->ai_protocol)) != -1) {
+ if (connect(sock, ai->ai_addr, ai->ai_addrlen) == -1) {
+ fprintf(stderr, "Failed to connect socket: %s\n",
+ strerror(errno));
+ close(sock);
+ sock = -1;
+ }
+ } else {
+ fprintf(stderr, "Failed to create socket: %s\n", strerror(errno));
+ }
+
+ freeaddrinfo(ai);
+ }
+ return sock;
+}
+
+/**
+ * Internal functions used by both protocols
+ */
+static uint32_t simplehash(const char *key) {
+ if (key == 0) {
+ return 0;
+ }
+ uint32_t ret = 0;
+ for (ret = *key; *key != 0; ++key) {
+ ret = (ret << 4) + *key;
+ }
+ return ret;
+}
+
+static struct Server *get_server(struct Memcache *handle, const char *key) {
+ if (handle->no_servers == 1) {
+ return handle->servers[0];
+ } else if (handle->no_servers > 0) {
+ int idx = simplehash(key) % handle->no_servers;
+ return handle->servers[idx];
+ } else {
+ return NULL;
+ }
+}
+
+static int libmemc_store(struct Memcache* handle, enum StoreCommand cmd,
+ const struct Item *item) {
+ struct Server* server = get_server(handle, item->key);
+ if (server == NULL) {
+ return -1;
+ } else {
+ if (server->sock == -1) {
+ if (server_connect(server) == -1) {
+ return -1;
+ }
+ }
+
+ if (handle->protocol == Binary) {
+ return binary_store(server, cmd, item);
+ } else {
+ return textual_store(server, cmd, item);
+ }
+ }
+}
+
+static size_t server_receive(struct Server* server, char* data, size_t size, int line);
+static int server_sendv(struct Server* server, struct iovec *iov, int iovcnt);
+static int server_send(struct Server* server, const void *data, size_t size);
+static int server_connect(struct Server *server);
+static void server_disconnect(struct Server *server);
+
+void server_destroy(struct Server *server) {
+ if (server != NULL) {
+ if (server->sock != -1) {
+ close(server->sock);
+ }
+ free(server->buffer);
+ free(server);
+ }
+}
+
+struct Server* server_create(const char *name, in_port_t port) {
+ struct addrinfo* ai = lookuphost(name, port);
+ struct Server* ret = NULL;
+ if (ai != NULL) {
+ ret = calloc(1, sizeof(struct Server));
+ if (ret != 0) {
+ char buffer[1024];
+ ret->sock = -1;
+ ret->errmsg = 0;
+ ret->addrinfo = ai;
+ sprintf(buffer, "%s:%d", name, port);
+ ret->peername = strdup(buffer);
+ ret->buffer = malloc(65 * 1024);
+ ret->buffersize = 65 * 1024;
+ server_connect(ret);
+ if (ret->buffer == NULL) {
+ server_destroy(ret);
+ ret = 0;
+ }
+ }
+ }
+
+ return ret;
+}
+
+static void server_disconnect(struct Server *server) {
+ if (server->sock != -1) {
+ (void)close(server->sock);
+ server->sock = -1;
+ }
+}
+
+static int server_connect(struct Server *server)
+{
+ int flag = 1;
+
+ if ((server->sock = socket(server->addrinfo->ai_family,
+ server->addrinfo->ai_socktype,
+ server->addrinfo->ai_protocol)) == -1) {
+ char errmsg[1024];
+ sprintf(errmsg, "Failed to create socket: %s", strerror(errno));
+ server->errmsg = strdup(errmsg);
+ return -1;
+ }
+
+ if (setsockopt(server->sock, IPPROTO_TCP, TCP_NODELAY,
+ &flag, sizeof(flag)) == -1) {
+ perror("Failed to set TCP_NODELAY");
+ }
+
+ if (connect(server->sock, server->addrinfo->ai_addr,
+ server->addrinfo->ai_addrlen) == -1) {
+ char errmsg[1024];
+ sprintf(errmsg, "Failed to connect socket: %s", strerror(errno));
+ server->errmsg = strdup(errmsg);
+ server_disconnect(server);
+ return -1;
+ }
+
+ return 0;
+}
+
+static int server_send(struct Server* server, const void *data, size_t size) {
+ size_t offset = 0;
+ do {
+ ssize_t sent = send(server->sock, ((const char*)data) + offset, size - offset, 0);
+ if (sent == -1) {
+ if (errno != EINTR) {
+ char errmsg[1024];
+ sprintf(errmsg, "Failed to send data to server: %s", strerror(errno));
+ server->errmsg = strdup(errmsg);
+ server_disconnect(server);
+ return -1;
+ }
+ } else {
+ offset += sent;
+ }
+ } while (offset < size);
+
+ return 0;
+}
+
+static int server_sendv(struct Server* server, struct iovec *iov, int iovcnt) {
+#ifdef WIN32
+ // @todo I might have a scattered IO function on windows...
+ for (int ii = 0; ii < iovcnt; ++ii) {
+ if (send(server, iov[ii].iov_base, iov[ii].iov_len) != 0) {
+ return -1;
+ }
+ }
+#else
+ // @todo Verify implementation if the writev returns with partitial
+ // writes!
+ size_t size = 0;
+ for (int ii = 0; ii < iovcnt; ++ ii) {
+ size += iov[ii].iov_len;
+ }
+
+ do {
+ ssize_t sent = writev(server->sock, iov, iovcnt);
+ if (sent == -1) {
+ if (errno != EINTR) {
+ char errmsg[1024];
+ sprintf(errmsg, "Failed to send data to server: %s", strerror(errno));
+ server->errmsg = strdup(errmsg);
+ server_disconnect(server);
+ return -1;
+ }
+ } else {
+ if (sent == size) {
+ return 0;
+ }
+
+ for (int ii = 0; ii < iovcnt && sent > 0; ++ii) {
+ if (iov[ii].iov_len < sent) {
+ size -= iov[ii].iov_len;
+ sent -= iov[ii].iov_len;
+ iov[ii].iov_len = 0;
+ } else {
+#ifdef __sun
+ iov[ii].iov_base += sent;
+#else
+ // iov_base is a void pointer...
+ iov[ii].iov_base = ((char*)iov[ii].iov_base) + sent;
+#endif
+ iov[ii].iov_len -= sent;
+ size -= sent;
+ break;
+ }
+ }
+ }
+ } while (size > 0);
+#endif
+ return 0;
+}
+
+static size_t server_receive(struct Server* server, char* data, size_t size, int line) {
+ size_t offset = 0;
+ int stop = 0;
+ do {
+ ssize_t nread = recv(server->sock, data + offset, size - offset, 0);
+ if (nread == -1) {
+ if (errno != EINTR) {
+ char errmsg[1024];
+ sprintf(errmsg, "Failed to receive data from server: %s", strerror(errno));
+ server->errmsg = strdup(errmsg);
+ server_disconnect(server);
+ return -1;
+ }
+ } else {
+ if (line) {
+ if (strchr(data + offset, '\r') != 0) {
+ stop = 1;
+ }
+ }
+ offset += nread;
+ }
+ } while (offset < size && !stop);
+
+ if (line && !stop) {
+ server->errmsg = strdup("Protocol error");
+ server_disconnect(server);
+ return -1;
+ }
+
+ return offset;
+}
+
+
+/* Byte swap a 64-bit number */
+static int64_t swap64(int64_t in) {
+#ifndef __sparc
+ /* Little endian, flip the bytes around until someone makes a faster/better
+ * way to do this. */
+ int64_t rv = 0;
+ int i = 0;
+ for(i = 0; i < 8; i++) {
+ rv = (rv << 8) | (in & 0xff);
+ in >>= 8;
+ }
+ return rv;
+#else
+ /* big-endian machines don't need byte swapping */
+ return in;
+#endif
+}
+
+
+/**
+ * Implementation of the Binary protocol
+ */
+static int binary_get(struct Server* server, struct Item* item)
+{
+#if HAVE_PROTOCOL_BINARY
+ uint16_t keylen = item->keylen;
+ uint32_t bodylen = keylen;
+
+ protocol_binary_request_get request = { .bytes = {0} };
+ request.message.header.request.magic = PROTOCOL_BINARY_REQ;
+ request.message.header.request.opcode = PROTOCOL_BINARY_CMD_GET;
+ request.message.header.request.keylen = htons(keylen);
+ request.message.header.request.datatype = PROTOCOL_BINARY_RAW_BYTES;
+ request.message.header.request.bodylen = htonl(bodylen);
+
+ struct iovec iovec[2];
+ iovec[0].iov_base = (void*)&request;
+ iovec[0].iov_len = sizeof(request);
+ iovec[1].iov_base = (void*)item->key;
+ iovec[1].iov_len = keylen;
+
+ server_sendv(server, iovec, 2);
+
+ protocol_binary_response_set response;
+ size_t nread = server_receive(server, (char*)response.bytes,
+ sizeof(response.bytes), 0);
+ if (nread != sizeof(response)) {
+ server->errmsg = strdup("Protocol error");
+ server_disconnect(server);
+ return -1;
+ }
+
+ bodylen = ntohl(response.message.header.response.bodylen);
+ if (response.message.header.response.status == 0) {
+ if (item->data != NULL) {
+ if ((bodylen-response.message.header.response.extlen) > item->size) {
+ free(item->data);
+ item->data = NULL;
+ }
+ }
+
+ if (item->data == NULL) {
+ item->size = bodylen - response.message.header.response.extlen;
+ item->data = malloc(item->size);
+ if (item->data == NULL) {
+ server->errmsg = strdup("failed to allocate memory\n");
+ server_disconnect(server);
+ return -1;
+ }
+ }
+
+ if (response.message.header.response.extlen != 0) {
+ assert(response.message.header.response.extlen == 4);
+ uint32_t flags;
+ struct iovec iovec[2];
+ iovec[0].iov_base = (void*)&flags;
+ iovec[0].iov_len = sizeof(flags);
+ iovec[1].iov_base = item->data;
+ iovec[1].iov_len = item->size;
+
+ ssize_t nread = readv(server->sock, iovec, 2);
+ if (nread < bodylen) {
+ // partial read.. read the rest!
+ nread -= 4;
+ size_t left = item->size - nread;
+ if (server_receive(server, item->data + nread, left, 0) != left) {
+ abort();
+ }
+ }
+ } else {
+ size_t nread = server_receive(server, item->data, item->size, 0);
+ assert(nread == item->size);
+ }
+
+ item->cas_id = swap64(response.message.header.response.cas);
+ } else {
+ char *buffer = malloc(bodylen + 1);
+ if (buffer == NULL) {
+ server->errmsg = strdup("failed to allocate memory\n");
+ server_disconnect(server);
+ return -1;
+ }
+ buffer[bodylen] = '\0';
+ server_receive(server, buffer, bodylen, 0);
+ server->errmsg = buffer;
+
+ return -1;
+ }
+
+ return 0;
+#endif
+ return -1;
+}
+
+static int binary_store(struct Server* server,
+ enum StoreCommand cmd,
+ const struct Item *item)
+{
+#if HAVE_PROTOCOL_BINARY
+ protocol_binary_request_set request = { .bytes = {0} };
+ request.message.header.request.magic = PROTOCOL_BINARY_REQ;
+
+ switch (cmd) {
+ case add :
+ request.message.header.request.opcode = PROTOCOL_BINARY_CMD_ADD; break;
+ case set :
+ request.message.header.request.opcode = PROTOCOL_BINARY_CMD_SET; break;
+ case replace :
+ request.message.header.request.opcode = PROTOCOL_BINARY_CMD_REPLACE; break;
+ default:
+ abort();
+ }
+
+ uint16_t keylen = item->keylen;
+ request.message.header.request.keylen = htons(keylen);
+ request.message.header.request.extlen = 8;
+ request.message.header.request.datatype = 0;
+ request.message.header.request.reserved = 0;
+ request.message.header.request.bodylen = htonl(keylen + item->size + 8);
+ request.message.header.request.opaque = 0;
+ request.message.header.request.cas = swap64(item->cas_id);
+ request.message.body.flags = 0;
+ request.message.body.expiration = htonl(item->exptime);
+
+ struct iovec iovec[3];
+ iovec[0].iov_base = (void*)&request;
+ iovec[0].iov_len = sizeof(request);
+ iovec[1].iov_base = (void*)item->key;
+ iovec[1].iov_len = keylen;
+ iovec[2].iov_base = item->data;
+ iovec[2].iov_len = item->size;
+
+ server_sendv(server, iovec, 3);
+
+ protocol_binary_response_set response;
+ size_t nread = server_receive(server, (char*)response.bytes,
+ sizeof(response.bytes), 0);
+ if (nread != sizeof(response)) {
+ server->errmsg = strdup("Protocol error");
+ server_disconnect(server);
+ return -1;
+ }
+
+ if (response.message.header.response.status == 0 &&
+ response.message.header.response.bodylen != 0) {
+ server->errmsg = strdup("Unexpected data returned\n");
+ server_disconnect(server);
+ return -1;
+ } else if (response.message.header.response.bodylen != 0) {
+ uint32_t len = ntohl(response.message.header.response.bodylen);
+ char* buffer = malloc(len);
+ if (buffer == 0) {
+ server->errmsg = strdup("failed to allocate memory\n");
+ server_disconnect(server);
+ return -1;
+ }
+
+ size_t nread = server_receive(server, buffer, len, 0);
+ free(buffer);
+ }
+
+ return (response.message.header.response.status == 0) ? 0 : -1;
+#endif
+ return -1;
+}
+
+/**
+ * Implementation of the Textual protocol
+ */
+static int parse_value_line(char *header, uint32_t* flag, size_t* size, char** data) {
+ char *end = strchr(header, ' ');
+ if (end == 0) {
+ return -1;
+ }
+ char *start = end + 1;
+ *flag = (uint32_t)strtoul(start, &end, 10);
+ if (start == end) {
+ return -1;
+ }
+ start = end + 1;
+ *size = (size_t)strtoul(start, &end, 10);
+ if (start == end) {
+ return -1;
+ }
+ if (strstr(end, "\r\n") != end) {
+ return -1;
+ }
+
+ *data = end + 2;
+ return 0;
+}
+
+static int textual_get(struct Server* server, struct Item* item) {
+ uint32_t flag;
+
+ struct iovec iovec[3];
+ iovec[0].iov_base = (char*)"get ";
+ iovec[0].iov_len = 4;
+ iovec[1].iov_base = (char*)item->key;
+ iovec[1].iov_len = item->keylen;
+ iovec[2].iov_base = (char*)"\r\n";
+ iovec[2].iov_len = 2;
+ server_sendv(server, iovec, 3);
+
+ size_t nread = server_receive(server, server->buffer,server->buffersize, 1);
+
+ // Split the header line
+ if (strstr(server->buffer, "VALUE ") == server->buffer) {
+ size_t elemsize;
+ char *ptr;
+
+ if (parse_value_line(server->buffer + 6, &flag, &elemsize, &ptr) == -1){
+ server->errmsg = strdup("Protocol error");
+ server_disconnect(server);
+ return -1;
+ }
+#ifndef __sun
+ typedef size_t ptrdiff_t;
+#endif
+ ptrdiff_t headsize = ptr - server->buffer;
+ size_t chunk = nread - headsize;
+
+ if (chunk < (elemsize + 7)) {
+ // I don't have all of the data.. keep on reading
+ server_receive(server, server->buffer + nread,
+ (elemsize - chunk) + 7, 0);
+ }
+
+ void *result = ptr;
+ if (elemsize > item->size) {
+ if (item->data != 0) {
+ free(item->data);
+ }
+ item->size = elemsize;
+ item->data = malloc(item->size);
+ if (item->data == 0) {
+ item->size = 0;
+ return -1;
+ }
+ }
+
+ memcpy(item->data, result, item->size);
+ return 0;
+ } else if (strstr(server->buffer, "END") == server->buffer) {
+ return -1;
+ }
+
+ abort();
+}
+
+static int textual_store(struct Server* server,
+ enum StoreCommand cmd,
+ const struct Item *item) {
+ static const char* const commands[] = { "add ", "set ", "replace " };
+
+ uint32_t flags = 0;
+ const void *dta = item->data;
+ size_t size = item->size;
+ ssize_t len = sprintf(server->buffer, " %d %ld %ld\r\n",
+ flags, (long)item->exptime, (long)item->size);
+
+ struct iovec iovec[5];
+ iovec[0].iov_base = (char*)commands[cmd];
+ iovec[0].iov_len = strlen(commands[cmd]);
+ iovec[1].iov_base = (char*)item->key;
+ iovec[1].iov_len = item->keylen;
+ iovec[2].iov_base = server->buffer;
+ iovec[2].iov_len = len;
+ iovec[3].iov_base = (char*)dta;
+ iovec[3].iov_len = size;
+ iovec[4].iov_base = (char*)"\r\n";
+ iovec[4].iov_len = 2;
+ server_sendv(server, iovec, 5);
+
+ size_t offset = 0;
+ do {
+ len = recv(server->sock, (void*)(server->buffer + offset),
+ server->buffersize - offset, 0);
+ if (len == -1) {
+ if (errno != EINTR) {
+ char errmsg[1024];
+ sprintf(errmsg, "Failed to receive data from server: %s",
+ strerror(errno));
+ server->errmsg = strdup(errmsg);
+ server_disconnect(server);
+ return -1;
+ }
+ } else if (len == 0) {
+ server->errmsg = strdup("Lost contact with server");
+ server_disconnect(server);
+ return -1;
+ } else {
+ offset += len;
+ if (strchr(server->buffer, '\r') != 0) {
+ if (strstr(server->buffer, "STORED\r\n") == server->buffer) {
+ return 0;
+ } else if (strstr(server->buffer,
+ "NOT_STORED\r\n") == server->buffer) {
+ server->errmsg = strdup("Item NOT stored");
+ return -1;
+ }
+ }
+ }
+ if (offset == server->buffersize) {
+ server->errmsg = strdup("Out of sync with server...");
+ server_disconnect(server);
+ return -1;
+ }
+ } while (1);
+}
59 libmemc.h
@@ -0,0 +1,59 @@
+/*
+ * CDDL HEADER START
+ *
+ * The contents of this file are subject to the terms of the
+ * Common Development and Distribution License (the "License").
+ * You may not use this file except in compliance with the License.
+ *
+ * See LICENSE.txt included in this distribution for the specific
+ * language governing permissions and limitations under the License.
+ *
+ * When distributing Covered Code, include this CDDL HEADER in each
+ * file and include the License file at LICENSE.txt.
+ * If applicable, add the following below this CDDL HEADER, with the
+ * fields enclosed by brackets "[]" replaced with your own identifying
+ * information: Portions Copyright [yyyy] [name of copyright owner]
+ *
+ * CDDL HEADER END
+ */
+
+/*
+ * Copyright 2007 Sun Microsystems, Inc. All rights reserved.
+ * Use is subject to license terms.
+ */
+#ifndef LIBMEMC_LIBMEMC_H
+#define LIBMEMC_LIBMEMC_H
+
+#include <sys/types.h>
+#include <arpa/inet.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+struct Item {
+ uint64_t cas_id;
+ const char *key;
+ int keylen;
+ void *data;
+ size_t size;
+ size_t exptime;
+};
+
+enum Protocol { Binary = 1, Textual = 2 };
+
+struct Memcache* libmemc_create(enum Protocol protocol);
+void libmemc_destroy(struct Memcache* handle);
+int libmemc_add_server(struct Memcache *handle, const char *host, in_port_t port);
+int libmemc_add(struct Memcache *handle, const struct Item *item);
+int libmemc_set(struct Memcache *handle, const struct Item *item);
+int libmemc_replace(struct Memcache *handle, const struct Item *item);
+int libmemc_get(struct Memcache *handle, struct Item *item);
+
+int libmemc_connect_server(const char *hostname, in_port_t port);
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* LIBMEMC_LIBMEMC_H */
1,358 main.c
@@ -0,0 +1,1358 @@
+/*
+ * CDDL HEADER START
+ *
+ * The contents of this file are subject to the terms of the
+ * Common Development and Distribution License (the "License").
+ * You may not use this file except in compliance with the License.
+ *
+ * See LICENSE.txt included in this distribution for the specific
+ * language governing permissions and limitations under the License.
+ *
+ * When distributing Covered Code, include this CDDL HEADER in each
+ * file and include the License file at LICENSE.txt.
+ * If applicable, add the following below this CDDL HEADER, with the
+ * fields enclosed by brackets "[]" replaced with your own identifying
+ * information: Portions Copyright [yyyy] [name of copyright owner]
+ *
+ * CDDL HEADER END
+ */
+
+/*
+ * Portions Copyright 2009 Matt Ingenthron
+ */
+#include "config.h"
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <sys/types.h>
+#include <sys/time.h>
+#include <sys/stat.h>
+#include <sys/socket.h>
+#include <netdb.h>
+#include <arpa/inet.h>
+#include <stdio.h>
+#include <signal.h>
+#include <pthread.h>
+#include <unistd.h>
+#include <stdlib.h>
+#include <errno.h>
+#include <sys/resource.h>
+#include <assert.h>
+#include <string.h>
+#ifdef HAVE_MMAN_H
+#include <sys/mman.h>
+#endif
+
+#ifdef HAVE_LIBMEMCACHED
+#include "libmemcached/memcached.h"
+#endif
+
+#include "libmemc.h"
+#include "metrics.h"
+
+#ifndef MAXINT
+/* I couldn't find MAXINT on my MacOSX box.. I should update this... */
+#define MAXINT (int)(unsigned int)-1
+#endif
+
+
+
+
+struct host {
+ const char *hostname;
+ in_port_t port;
+ struct host *next;
+} *hosts = NULL;
+
+/**
+ * A struct holding the information I would like to measure for each test-run
+ */
+struct report {
+ /** The index in the items array to start at */
+ int offset;
+ /** The number of operations to execute */
+ size_t total;
+ /** The number of set-operation executed */
+ size_t set;
+ /** The total time of all of the set-operations */
+ hrtime_t setDelta;
+ /** The number of get-operations executed */
+ size_t get;
+ /** The total time of all of the get-operations */
+ hrtime_t getDelta;
+ /** The best set operation */
+ hrtime_t bestSet;
+ /** The best get operation */
+ hrtime_t bestGet;
+ /** The worst set operation */
+ hrtime_t worstSet;
+ /** The worst get operation */
+ hrtime_t worstGet;
+};
+
+/**
+ * Each item represented in the cache
+ */
+struct item {
+ /**
+ * The item's key
+ */
+ const char *key;
+ /**
+ * The length of the key to avoid calling strlen all the time
+ */
+ size_t keylen;
+ /**
+ * The size of the data stored in the cache (all data stored in memcached
+ * will start from datablock.data)
+ */
+ size_t size;
+};
+
+/**
+ * The set of data to operate on
+ */
+struct item *dataset;
+
+static void *keyarray;
+static int keylength;
+static long totalkeys;
+
+/**
+ * The datablock to operate with
+ */
+struct datablock {
+ /**
+ * Pointer to the datablock
+ */
+ void *data;
+ /**
+ * The size of the datablock
+ */
+ size_t size;
+ /**
+ * The average size of all the blocks
+ */
+ size_t avg;
+} datablock = {.data = NULL, .size = 4096, .avg = 0};
+
+/**
+ * Set to one if you would like fixed block sizes
+ */
+int use_fixed_block_size = 0;
+
+/**
+ * Set to 1 if you would like the memcached client to connect to multiple
+ * servers.
+ */
+int use_multiple_servers = 0;
+
+/** The number of items to operate on (may be overridden with -i */
+long no_items = 10000;
+/** The number of operations (pr thread) to execute (may be overridden with -c */
+int no_iterations = 10000;
+/** If we should verify the data received. May be overridden with -V */
+int verify_data = 0;
+
+/** The probaility for a set operation */
+int setprc = 33;
+
+/** what kind of benchmark are we running
+ TODO: this shouldn't be a global, or if so shouldn't be done this way */
+int set_bench = 0; /* whether or not this is a simple dataset bench */
+
+double stddev = 0;
+int verbose = 0;
+
+/** TODO: get rid of these after testing */
+double max_result, min_result;
+
+/**
+ * The different client libraries we have support for
+ */
+enum Libraries {
+ LIBMEMC_TEXTUAL = 1,
+ LIBMEMC_BINARY,
+#ifdef HAVE_LIBMEMCACHED
+ LIBMEMCACHED_TEXTUAL,
+ LIBMEMCACHED_BINARY,
+#endif
+ INVALID_LIBRARY
+};
+
+struct memcachelib {
+ int type;
+ void *handle;
+};
+
+/**
+ * The current library in use (all threads must use the same library)
+ */
+int current_memcached_library = LIBMEMC_TEXTUAL;
+
+/**
+ * Print progress information during the test..
+ */
+static int progress = 0;
+
+struct connection {
+ pthread_mutex_t mutex;
+ void *handle;
+};
+
+#ifndef HAVE_GETHRTIME
+static hrtime_t gethrtime() {
+ hrtime_t ret;
+
+#ifdef HAVE_CLOCK_GETTIME
+ struct timespec ts;
+
+ if (clock_gettime(CLOCK_MONOTONIC, &ts) == -1) {
+ return (-1ULL);
+ }
+
+ ret = ts.tv_sec * 1000000000;
+ ret += ts.tv_nsec;
+#else
+ struct timeval tv;
+ if (gettimeofday(&tv, NULL) == -1) {
+ return (-1ULL);
+ }
+
+ ret = tv.tv_sec * 1000000000;
+ ret += tv.tv_usec * 1000;
+#endif
+
+ return ret;
+}
+#endif
+
+extern double box_muller(double m, double s);
+
+/**
+ * Create a handle to a memcached library
+ */
+static void *create_memcached_handle(void) {
+ struct memcachelib* ret = malloc(sizeof (*ret));
+ ret->type = current_memcached_library;
+
+ switch (current_memcached_library) {
+#ifdef HAVE_LIBMEMCACHED
+ case LIBMEMCACHED_TEXTUAL:
+ {
+ memcached_st *memc = memcached_create(NULL);
+ for (struct host *host = hosts; host != NULL; host = host->next) {
+ memcached_server_add(memc, host->hostname, host->port);
+ if (!use_multiple_servers) {
+ break;
+ }
+ }
+ ret->handle = memc;
+ }
+ break;
+ case LIBMEMCACHED_BINARY:
+ {
+ memcached_st *memc = memcached_create(NULL);
+ memcached_behavior_set(memc, MEMCACHED_BEHAVIOR_BINARY_PROTOCOL, 1);
+ for (struct host *host = hosts; host != NULL; host = host->next) {
+ memcached_server_add(memc, host->hostname, host->port);
+ if (!use_multiple_servers) {
+ break;
+ }
+ }
+ ret->handle = memc;
+ }
+ break;
+#endif
+ case LIBMEMC_TEXTUAL:
+ {
+ struct Memcache* memcache = libmemc_create(Textual);
+ for (struct host *host = hosts; host != NULL; host = host->next) {
+ libmemc_add_server(memcache, host->hostname, host->port);
+ if (!use_multiple_servers) {
+ break;
+ }
+ }
+ ret->handle = memcache;
+ }
+ break;
+ case LIBMEMC_BINARY:
+ {
+ struct Memcache* memcache = libmemc_create(Binary);
+ for (struct host *host = hosts; host != NULL; host = host->next) {
+ libmemc_add_server(memcache, host->hostname, host->port);
+ if (!use_multiple_servers) {
+ break;
+ }
+ }
+ ret->handle = memcache;
+ }
+ break;
+ default:
+ abort();
+ }
+
+ return ret;
+}
+
+/**
+ * Release a handle to a memcached library
+ */
+static void release_memcached_handle(void *handle) {
+ struct memcachelib* lib = (struct memcachelib*) handle;
+ switch (lib->type) {
+#ifdef HAVE_LIBMEMCACHED
+ case LIBMEMCACHED_BINARY: /* FALLTHROUGH */
+ case LIBMEMCACHED_TEXTUAL:
+ {
+ memcached_st *memc = lib->handle;
+ memcached_free(memc);
+ }
+ break;
+#endif
+
+ case LIBMEMC_BINARY:
+ case LIBMEMC_TEXTUAL:
+ libmemc_destroy(lib->handle);
+ break;
+
+ default:
+ abort();
+ }
+}
+
+/**
+ * Set a key / value pair on the memcached server
+ * @param handle Thandle to the memcached library to use
+ * @param key The items key
+ * @param keylength The length of the key
+ * @param data The data to set
+ * @param The size of the data to set
+ * @return 0 on success -1 otherwise
+ */
+static inline int memcached_set_wrapper(struct connection *connection,
+ const char *key,
+ int keylength, const void *data, int size) {
+ struct memcachelib* lib = (struct memcachelib*) connection->handle;
+ switch (lib->type) {
+#ifdef HAVE_LIBMEMCACHED
+ case LIBMEMCACHED_BINARY: /* FALLTHROUGH */
+ case LIBMEMCACHED_TEXTUAL:
+ {
+ int rc = memcached_set(lib->handle, key, keylength, data, size, 0, 0);
+ if (rc != MEMCACHED_SUCCESS) {
+ return -1;
+ }
+ }
+ break;
+#endif
+ case LIBMEMC_BINARY:
+ case LIBMEMC_TEXTUAL:
+ {
+ struct Item mitem = {0};
+ mitem.key = key;
+ mitem.keylen = keylength;
+ /* Set will not modify data */
+ mitem.data = (void*) data;
+ mitem.size = size;
+ if (libmemc_set(lib->handle, &mitem) != 0) {
+ return -1;
+ }
+ }
+ break;
+
+ default:
+ abort();
+ }
+ return 0;
+}
+
+/**
+ * Get the value for a key from the memcached server
+ * @param connection the connection to use
+ * @param key The items key
+ * @param keylength The length of the key
+ * @param The size of the data
+ * @return pointer to the data on success, -1 otherwise
+ * TODO: the return of -1 isn't really true
+ */
+static inline void *memcached_get_wrapper(struct connection* connection,
+ const char *key,
+ int keylength, size_t *size) {
+ struct memcachelib* lib = (struct memcachelib*) connection->handle;
+ void *ret = NULL;
+ switch (lib->type) {
+#ifdef HAVE_LIBMEMCACHED
+ case LIBMEMCACHED_BINARY: /* FALLTHROUGH */
+ case LIBMEMCACHED_TEXTUAL:
+ {
+ memcached_return rc;
+ uint32_t flags;
+ ret = memcached_get(lib->handle, key, keylength, size, &flags, &rc);
+ if (rc != MEMCACHED_SUCCESS) {
+ return NULL;
+ }
+ }
+ break;
+#endif
+ case LIBMEMC_BINARY:
+ case LIBMEMC_TEXTUAL:
+ {
+ struct Item mitem = {0};
+ mitem.key = key;
+ mitem.keylen = keylength;
+
+ if (libmemc_get(lib->handle, &mitem) != 0) {
+ return NULL;
+ }
+ *size = mitem.size;
+ ret = mitem.data;
+ }
+ break;
+
+ default:
+ abort();
+ }
+
+ return ret;
+}
+
+
+
+static struct connection* connectionpool;
+static int connection_pool_size = 1;
+static int thread_bind_connection = 0;
+
+static int create_connection_pool(void) {
+ connectionpool = calloc(connection_pool_size, sizeof (struct connection));
+ if (connectionpool == NULL) {
+ return -1;
+ }
+
+ for (int ii = 0; ii < connection_pool_size; ++ii) {
+ if (pthread_mutex_init(&connectionpool[ii].mutex, NULL) != 0) {
+ abort();
+ }
+ if ((connectionpool[ii].handle = create_memcached_handle()) == NULL) {
+ abort();
+ }
+ }
+ return 0;
+}
+
+static void destroy_connection_pool(void) {
+ for (int ii = 0; ii < connection_pool_size; ++ii) {
+ pthread_mutex_destroy(&connectionpool[ii].mutex);
+ release_memcached_handle(connectionpool[ii].handle);
+ }
+
+ free(connectionpool);
+ connectionpool = NULL;
+}
+
+static struct connection *get_connection(void) {
+ if (thread_bind_connection) {
+#ifdef __sun
+ return &connectionpool[pthread_self()];
+#else
+ /* @FIXME!!!! */
+ return &connectionpool[0];
+#endif
+ } else {
+ int index;
+ do {
+ index = random() % connection_pool_size;
+ } while (pthread_mutex_trylock(&connectionpool[index].mutex) != 0);
+
+ return &connectionpool[index];
+ }
+}
+
+static void release_connection(struct connection *connection) {
+ pthread_mutex_unlock(&connection->mutex);
+}
+
+/**
+ * Convert a time (in ns) to a human readable form...
+ * @param time the time in nanoseconds
+ * @param buffer where to store the result
+ * @param size the size of the buffer
+ * @return buffer
+ */
+const char* hrtime2text(hrtime_t time, char *buffer, size_t size) {
+ const char * const extensions[] = {"ns", "µs", "ms", "s" };
+ int id = 0;
+
+ while (time > 9999) {
+ ++id;
+ time /= 1000;
+ if (id > 3) {
+ break;
+ }
+ }
+
+ snprintf(buffer, size, "%d %s", (int) time, extensions[id]);
+ buffer[size - 1] = '\0';
+ return buffer;
+}
+
+/**
+ * Convert a timeval structure to human readable form..
+ * @param val the value to convert
+ * @param buffer where to store the result
+ * @param size the size of the buffer
+ * @return buffer
+ */
+const char* timeval2text(struct timeval* val, char *buffer, size_t size) {
+ snprintf(buffer, size, "%2ld.%06lu", (long) val->tv_sec,
+ (long) val->tv_usec);
+
+ return buffer;
+}
+
+/**
+ * Initialize the dataset to work on
+ * @return 0 if success, -1 if memory allocation fails
+ */
+static int initialize_dataset() {
+ uint64_t total = 0;
+
+ if (datablock.data != NULL) {
+ free(datablock.data);
+ }
+
+ datablock.data = malloc(datablock.size);
+ if (datablock.data == NULL) {
+ fprintf(stderr, "Failed to allocate memory for the datablock\n");
+ return -1;
+ }
+
+ memset(datablock.data, 0xff, datablock.size);
+
+ if (dataset != NULL) {
+ free(dataset);
+ }
+
+ dataset = calloc(no_items, sizeof (struct item));
+ if (dataset == NULL) {
+ fprintf(stderr, "Failed to allocate memory for the dataset\n");
+ return -1;
+ }
+
+ for (long ii = 0; ii < no_items; ++ii) {
+ char buffer[128];
+ sprintf(buffer, "%08ld", ii);
+
+ dataset[ii].key = strdup(buffer);
+ dataset[ii].keylen = strlen(buffer);
+ if (use_fixed_block_size) {
+ dataset[ii].size = datablock.size;
+ total += dataset[ii].size;
+ } else {
+ dataset[ii].size = random() % datablock.size;
+ if (dataset[ii].size == 0) {
+ dataset[ii].size = 1024;
+ }
+ total += dataset[ii].size;
+ }
+ }
+
+ datablock.avg = (size_t) (total / no_items);
+ return 0;
+}
+
+/**
+ * Populate the dataset to the server
+ * @return 0 if success, -1 if an error occurs
+ */
+static int populate_dataset(struct report *rep) {
+ struct connection* connection = get_connection();
+ int end = rep->offset + rep->total;
+ for (int ii = rep->offset; ii < end; ++ii) {
+ if (memcached_set_wrapper(connection,
+ dataset[ii].key, dataset[ii].keylen,
+ datablock.data, dataset[ii].size) != 0) {
+ fprintf(stderr, "Failed to set data!\n");
+ release_connection(connection);
+ return -1;
+ }
+ assert((verify_data) ? dataset[ii].keylen == strlen(dataset[ii].key) : 1);
+ }
+
+ release_connection(connection);
+ return 0;
+}
+
+/**
+ * The threads entry function
+ * @param arg this should be a pointer to where this thread should report
+ * the result
+ * @return arg
+ */
+void *populate_thread_main(void* arg) {
+ populate_dataset((struct report*) arg);
+ return arg;
+}
+
+/**
+ * Populate the data on the servers
+ * @param no_threads the number of theads to use
+ * @return 0 if success, -1 otherwise
+ */
+int populate_data(int no_threads) {
+ if (no_threads > 1) {
+ pthread_t *threads = calloc(sizeof (pthread_t), no_threads);
+ struct report *reports = calloc(sizeof (struct report), no_threads);
+ int perThread = no_items / no_threads;
+ int rest = no_items % no_threads;
+ size_t offset = 0;
+ int ii;
+
+ if (threads == NULL || reports == NULL) {
+ fprintf(stderr, "Failed to allocate memory\n");
+ free(threads);
+ free(reports);
+ return -1;
+ }
+
+ for (ii = 0; ii < no_threads; ++ii) {
+ reports[ii].offset = offset;
+ reports[ii].total = perThread;
+ offset += perThread;
+ if (rest > 0) {
+ --rest;
+ ++reports[ii].total;
+ ++offset;
+ }
+ pthread_create(&threads[ii], 0, populate_thread_main,
+ &reports[ii]);
+ }
+
+ for (ii = 0; ii < no_threads; ++ii) {
+ void *ret;
+ pthread_join(threads[ii], &ret);
+ }
+ free(threads);
+ free(reports);
+ } else {
+ struct report report;
+ report.offset = 0;
+ report.total = no_items;
+
+ if (populate_dataset(&report) == -1) {
+ return 1;
+ }
+ }
+ return 0;
+}
+
+
+struct item get_setval(void) {
+ struct item ret = { .key = NULL, .keylen = 0, .size = 0 };
+
+ if (keyarray != 0) {
+ // Use keys loaded from disk..
+ ret.keylen = keylength;
+ ret.size = 1024; // add random value;
+
+ /* test the box_muller */
+ double m = 6.0; /* this is the mean; stddev set via option -y */
+ assert(totalkeys > 0);
+
+ int median_keys = totalkeys;
+ if (totalkeys % 2 == 1) {
+ median_keys = (totalkeys - 1) / 2;
+ } else {
+ median_keys = totalkeys / 2;
+ }
+
+ double result = box_muller(m, stddev);
+
+ /* the following and setting the mean around 6 and later dividing
+ * by 6 is a hackish way to deal with the outliers
+ */
+ if (result < 0) {
+ if (verbose) {
+ fprintf(stderr, "WARNING: result was %f, result set to 0\n",
+ result);
+ }
+ result = 0;
+ }
+
+ if (result > 12) {
+ if (verbose) {
+ fprintf(stderr, "WARNING: result was %f, set_result set to 12\n",
+ result);
+ }
+ result = 12.0;
+ }
+
+ /* TODO: make sure we don't go above or below the bottom */
+ unsigned int offset = (int)(median_keys * result/6);
+ offset %= totalkeys;
+
+ ret.key = keyarray + ((keylength + 1) * (offset));
+ } else {
+ /* try use the ring... */
+ int offset = random() % no_items;
+ ret.key = dataset[offset].key;
+ ret.keylen = dataset[offset].keylen;
+ ret.size = dataset[offset].size;
+ }
+
+ return ret;
+}
+
+/**
+ * Test the library (perform a number of operations on the server).
+ * @param rep Where to store the result of the test
+ * @return 0 on success, -1 otherwise
+ */
+static int test(struct report *rep) {
+ int ret = 0;
+ struct connection* connection;
+ rep->bestGet = rep->bestSet = 99999999;
+ rep->worstGet = rep->worstSet = 0;
+
+ for (int ii = 0; ii < rep->total; ++ii) {
+ connection = get_connection();
+
+ struct item item = get_setval();
+
+ // Ensure that the item is paged in..
+ for (int ii = 0; ii < item.keylen; ++ii) {
+ if (item.key[ii] == '\0' || item.key[ii] == '\n') {
+ abort();
+ }
+ }
+
+ if (keyarray == NULL && setprc > 0 && (random() % 100) < setprc) { // @todo fixme!!! (we don't do set right now...)
+ hrtime_t delta;
+ hrtime_t start = gethrtime();
+ memcached_set_wrapper(connection, item.key, item.keylen,
+ datablock.data, item.size);
+ delta = gethrtime() - start;
+ if (delta < rep->bestSet) {
+ rep->bestSet = delta;
+ }
+ if (delta > rep->worstSet) {
+ rep->worstSet = delta;
+ }
+ rep->setDelta += delta;
+ ++rep->set;
+ } else {
+ /* go set it from random data */
+ if (verbose) {
+ char buffer[260] = {0};
+ snprintf(buffer, item.keylen+1, "%s", item.key);
+ fprintf(stderr, "CMD: get %s\n", buffer);
+ fflush(stderr);
+ }
+ hrtime_t delta;
+ size_t size;
+ hrtime_t start = gethrtime();
+ void *data = memcached_get_wrapper(connection, item.key,
+ item.keylen, &size);
+ delta = gethrtime() - start;
+ if (delta < rep->bestGet) {
+ rep->bestGet = delta;
+ }
+ if (delta > rep->worstGet) {
+ rep->worstGet = delta;
+ }
+ rep->getDelta += delta;
+ if (data != NULL) {
+ if (size != item.size && keyarray == NULL) {
+ fprintf(stderr,
+ "Incorrect length returned for <%s>. "
+ "Stored %ld got %ld\n",
+ item.key, (long)item.keylen, (long)size);
+ } else if (verify_data && keyarray == NULL &&
+ memcmp(datablock.data, data, item.size) != 0) {
+ fprintf(stderr, "Garbled data for <%s>\n",
+ item.key);
+ }
+ record_tx(TX_GET, delta);
+ free(data);
+ } else {
+ fprintf(stderr, "missing data for <");
+ for (int ii = 0; ii < item.keylen; ++ii) {
+ fprintf(stderr, "%c", item.key[ii]);
+ }
+ fprintf(stderr, ">\n");
+ record_error(TX_GET, delta);
+ }
+ ++rep->get;
+ }
+ release_connection(connection);
+ }
+
+ return ret;
+}
+
+/**
+ * The threads entry function
+ * @param arg this should be a pointer to where this thread should report
+ * the result
+ * @return arg
+ */
+void *test_thread_main(void* arg) {
+ test((struct report*) arg);
+ return arg;
+}
+
+/**
+ * Add a host into the list of memcached servers to use
+ * @param hostname the hostname:port to connect to
+ */
+void add_host(const char *hostname) {
+ struct host *entry = malloc(sizeof (struct host));
+ if (entry == 0) {
+ fprintf(stderr, "Failed to allocate memory for <%s>. Host ignored\n",
+ hostname);
+ fflush(stderr);
+ return;
+ }
+ entry->next = hosts;
+ hosts = entry;
+ entry->hostname = strdup(hostname);
+ char *ptr = strchr(entry->hostname, ':');
+ if (ptr != NULL) {
+ *ptr = '\0';
+ entry->port = atoi(ptr + 1);
+ } else {
+ entry->port = 11211;
+ }
+}
+
+struct addrinfo *lookuphost(const char *hostname, in_port_t port) {
+ struct addrinfo *ai = 0;
+ struct addrinfo hints = {0};
+ char service[NI_MAXSERV];
+ int error;
+
+ hints.ai_flags = AI_PASSIVE | AI_ADDRCONFIG;
+ hints.ai_family = AF_UNSPEC;
+ hints.ai_protocol = IPPROTO_TCP;
+ hints.ai_socktype = SOCK_STREAM;
+
+ (void) snprintf(service, NI_MAXSERV, "%d", port);
+ if ((error = getaddrinfo(hostname, service, &hints, &ai)) != 0) {
+ if (error != EAI_SYSTEM) {
+ fprintf(stderr, "getaddrinfo(): %s\n", gai_strerror(error));
+ } else {
+ perror("getaddrinfo()");
+ }
+ }
+
+ return ai;
+}
+
+static int get_server_rusage(const struct host *entry, struct rusage *rusage) {
+ int sock;
+ int ret = -1;
+ char buffer[8192];
+
+ struct addrinfo* addrinfo = lookuphost(entry->hostname, entry->port);
+ if (addrinfo == NULL) {
+ return -1;
+ }
+
+ memset(rusage, 0, sizeof (*rusage));
+
+ if ((sock = socket(addrinfo->ai_family,
+ addrinfo->ai_socktype,
+ addrinfo->ai_protocol)) != -1) {
+ if (connect(sock, addrinfo->ai_addr, addrinfo->ai_addrlen) != -1) {
+ if (send(sock, "stats\r\n", 7, 0) > 0) {
+ if (recv(sock, buffer, sizeof (buffer), 0) > 0) {
+ char *ptr = strstr(buffer, "rusage_user");
+ if (ptr != NULL) {
+ rusage->ru_utime.tv_sec = atoi(ptr + 12);
+ ptr = strchr(ptr, '.');
+ if (ptr != NULL) {
+ rusage->ru_utime.tv_usec = atoi(ptr + 1);
+ }
+ }
+
+ ptr = strstr(buffer, "rusage_system");
+ if (ptr != NULL) {
+ rusage->ru_stime.tv_sec = atoi(ptr + 14);
+
+ ptr = strchr(ptr, '.');
+ if (ptr != NULL) {
+ rusage->ru_stime.tv_usec = atoi(ptr + 1);
+ }
+ }
+ ret = 0;
+ } else {
+ fprintf(stderr, "Failed to read data: %s\n", strerror(errno));
+ }
+ } else {
+ fprintf(stderr, "Failed to send data: %s\n", strerror(errno));
+ }
+ } else {
+ fprintf(stderr, "Failed to connect socket: %s\n", strerror(errno));
+ }
+
+ close(sock);
+ } else {
+ fprintf(stderr, "Failed to create socket: %s\n", strerror(errno));
+ }
+
+ freeaddrinfo(addrinfo);
+ return ret;
+}
+
+static int load_keys(const char *fname) {
+ struct stat st;
+ if (stat(fname, &st) == -1) {
+ perror("Failed to stat(3c) keyfile");
+ return -1;
+ }
+ FILE *fp = fopen(fname, "r");
+ if (fp == NULL) {
+ perror("Failed to open keyfile");
+ return -1;
+ }
+#ifdef HAVE_MMAP
+ keyarray = mmap((caddr_t) 0, st.st_size, PROT_READ, MAP_PRIVATE, fileno(fp), 0);
+ if (keyarray == NULL) {
+ perror("Failed to memorymap key file");
+ fclose(fp);
+ return -1;
+ }
+#ifdef HAVE_MADVISE
+ (void) madvise(keyarray, st.st_size, MADV_RANDOM);
+#endif
+
+#else
+ keyarray = malloc(st.st_size);
+ size_t offset;
+ ssize_t nr;
+ do {
+ nr = read(fileno(fp), keyarray + offset, st.st_size - offset);
+ if (nr > 0) {
+ offset += nr;
+ }
+ } while (offset < st.st_size);
+#endif
+
+ char *ptr = keyarray;
+ keylength = 0;
+ while (*ptr != '\0' && *ptr != '\n') {
+ ++ptr;
+ ++keylength;
+ if (ptr == (keyarray + st.st_size)) {
+ break;
+ }
+ }
+ totalkeys = st.st_size / (keylength + 1);
+ return 0;
+}
+
+/**
+ * Program entry point
+ * @param argc argument count
+ * @param argv argument vector
+ * @return 0 on success, 1 otherwise
+ */
+int main(int argc, char **argv) {
+ int cmd;
+ int no_threads = 1;
+ int populate = 1;
+ int loop = 0;
+ struct rusage rusage;
+ struct rusage server_start;
+ struct timeval starttime = {0};
+
+ gettimeofday(&starttime, NULL);
+
+ while ((cmd = getopt(argc, argv, "QW:MpL:P:Fm:t:h:i:s:c:VlSvy:xk:")) != EOF) {
+ switch (cmd) {
+ case 'p':
+ progress = 1;
+ break;
+ case 'P':
+ setprc = atoi(optarg);
+ if (setprc > 100) {
+ setprc = 100;
+ } else if (setprc < 0) {
+ setprc = 0;
+ }
+ break;
+ case 't':
+ no_threads = atoi(optarg);
+ break;
+ case 'L':
+ current_memcached_library = atoi(optarg);
+ break;
+ case 'M': use_multiple_servers = 1;
+ break;
+ case 'F': use_fixed_block_size = 1;
+ break;
+ case 'h': add_host(optarg);
+ break;
+ case 'i': no_items = atoi(optarg);
+ break;
+ case 's': srand(atoi(optarg));
+ break;
+ case 'c': no_iterations = atoi(optarg);
+ break;
+ case 'V': verify_data = 1;
+ break;
+ case 'l': loop = 1;
+ break;
+ case 'S': populate = 0;
+ break;
+ case 'v': verbose = 1;
+ break;
+ case 'W': connection_pool_size = atoi(optarg);
+ break;
+ case 'Q': thread_bind_connection = 1;
+ break;
+ case 'm':
+ {
+ int size = atoi(optarg);
+ if (size > 1024 * 1024) {
+ fprintf(stderr, "WARNING: Too big block size %d\n", size);
+ } else {
+ datablock.size = atoi(optarg);
+ }
+ break;
+ }
+ case 'k':
+ if (load_keys(optarg) == -1) {
+ return 1;
+ }
+ break;
+ case 'y':
+ stddev = atof(optarg);
+ break;
+ case 'x':
+ set_bench = 1;
+ populate = 0;
+ break;
+ default:
+ fprintf(stderr, "Usage: test [-h host[:port]] [-t #threads]");
+ fprintf(stderr, " [-T] [-i #items] [-c #iterations] [-v] ");
+ fprintf(stderr, "[-V] [-f dir] [-s seed] [-W size] [-x] [-y stddev] [-k keyfile]\n");
+ fprintf(stderr, "\t-h The hostname:port where the memcached server is running\n");
+ fprintf(stderr, "\t-t The number of threads to use\n");
+ fprintf(stderr, "\t-m The max message size to operate on\n");
+ fprintf(stderr, "\t-F Use fixed message size\n");
+ fprintf(stderr, "\t-i The number of items to operate with\n");
+ fprintf(stderr, "\t-c The number of iteratons each thread should do\n");
+ fprintf(stderr, "\t-l Loop and repeat the test, but print out information for each run\n");
+ fprintf(stderr, "\t-V Verify the retrieved data\n");
+ fprintf(stderr, "\t-v Verbose output\n");
+ fprintf(stderr, "\t-L Use the specified memcached client library\n");
+ fprintf(stderr, "\t-M Use multiple servers (specified with -h)\n");
+ fprintf(stderr, "\t-W connection pool size\n");
+ fprintf(stderr, "\t-s Use the specified seed to initialize the random generator\n");
+ fprintf(stderr, "\t-S Skip the populate of the data\n");
+ fprintf(stderr, "\t-P The probability for a set operation\n");
+ fprintf(stderr, "\t-y Specify standard deviation for -x option test\n");
+ fprintf(stderr, "\t-k The file with keys to be retrieved\n");
+ fprintf(stderr, "\t-x randomly request from a set in a supplied file\n");
+ fprintf(stderr, "\t\t(implies -S, requires -k)\n");
+ return 1;
+ }
+ }
+
+ if (set_bench == 1 && keyarray==NULL) {
+ fprintf(stderr,"-x option requires keyfile (-k)\n");
+ exit(-1);
+ }
+
+ if (connection_pool_size < no_threads) {
+ connection_pool_size = no_threads;
+ }
+
+ {
+ int maxthreads = no_threads;
+ struct rlimit rlim;
+
+ if (maxthreads < connection_pool_size) {
+ maxthreads = connection_pool_size;
+ }
+
+ if (getrlimit(RLIMIT_NOFILE, &rlim) == 0) {