From 8f3ac0d2260dd9b5955ccc1781a830ea8f1a7fbc Mon Sep 17 00:00:00 2001 From: ferg Date: Thu, 13 Oct 2011 00:35:09 +0000 Subject: [PATCH] memcache server functions git-svn-id: svn://svn.caucho.com/resin/trunk@8487 9c94448d-38f1-0310-a231-d98308ff1ebf --- README | 131 +++--- .../distcache/memcache/MemcacheClient.java | 12 +- .../memcache/MemcacheConnection.java | 430 +++++++++++++++++- resin-doc/admin/resin-admin-console.xtp | 41 +- resin-doc/admin/toc.xml | 3 +- resin-doc/changes/changes.xtp | 1 + 6 files changed, 522 insertions(+), 96 deletions(-) diff --git a/README b/README index 56426b964..8b56a0dc1 100644 --- a/README +++ b/README @@ -6,8 +6,10 @@ This is the README file for Resin 4.0(tm) The documentation included with Resin is in XTP format starting at http://localhost:8080/resin-doc/index.xtp. -The full documentation is also available online at -http://www.caucho.com/resin-4.0/ if you have trouble getting started. +For more information about configuring Resin, the full documentation is +also available online at: + + http://www.caucho.com/resin-4.0/ Questions should go to our mailing list or forums. @@ -15,6 +17,7 @@ Questions should go to our mailing list or forums. http://forum.caucho.com -- forum http://bugs.caucho.com -- bug reports + I) Licensing/Support -------------------- @@ -29,18 +32,21 @@ technical support options to fit your support and budget needs. http://www.caucho.com/resin/sales -- licenses and support + II) Preconditions ----------------- -1) Resin needs a Java JDK before it can run. Resin 4.0 works with -JDK 1.6 or greater. + 1) Resin needs a Java JDK before it can run. Resin 4.0 works with + JDK 1.6 or greater. -Some locations for JDKs include: + Some locations for JDKs include: -Oracle: - http://www.oracle.com/technetwork/java/javase/downloads/index.html + Oracle: + http://www.oracle.com/technetwork/java/javase/downloads/index.html -2) On Windows, you'll need an unzip tool available at http://www.winzip.com + 2) On Windows, you'll need an unzip tool available at http://www.winzip.com + + 3) A HTML 5 browser is needed for some features of the /resin-admin tool. III) Getting Started -------------------- @@ -49,67 +55,90 @@ The included HTTP/1.1 server makes it easy to evaluate Resin, JSP and servlets. Just start it up. The default configuration file is found in resin-4.0.x/conf/resin.xml. -1) On Unix, you'll want to build the JNI libraries: + 1) On Unix, you'll want to build the JNI libraries: + + resin-4.0.x> ./configure --prefix=`pwd` + resin-4.0.x> make + resin-4.0.x> make install + + 2) To start the server: + + unix> bin/resin.sh start + + or + + C:\> java -jar lib/resin.jar start + + The server listens to port 8080. Port 8080 is the default specified + in the configuration file resin-4.0.x/conf/resin.xml + + Log files are in resin-4.0.x/log + + 3) Usually, Resin can find the JDK, but you may need to set JAVA_HOME in some + configurations. + + Starting Resin with -verbose is a great way to see what environment + Resin is running. - resin-4.0.x> ./configure --prefix=`pwd` - resin-4.0.x> make - resin-4.0.x> make install + 4) Browse http://localhost:8080 or equivalently http://127.0.0.1:8080. -2) To start the server: + 5) Go through the documentation at http://localhost:8080/resin-doc - unix> java -jar lib/resin.jar start - or +IV) Application Deployment +-------------------------- - C:\> java -jar lib/resin.jar start + 1) You can use the command-line to deploy a .war file: + + unix> bin/resin.sh deploy foo.war - The server listens to port 8080. Port 8080 is the default specified - in the configuration file resin-4.0.x/conf/resin.xml + 2) You can also copy the war file to the webapps directory. -3) Usually, Resin can find the JDK, but you may need to set JAVA_HOME in some - configurations. + To deploy a war file, place the file in resin-4.0.x/webapps/foo.war + Resin will expand the war file, and the application will be available + with the url http://localhost:8080/foo/ - Starting Resin with -verbose is a great way to see what environment - Resin is running. + 3) Resin supports the development of applications without requiring a war + file deployment. Create a directory resin-4.0.x/webapps/bar to + correspond to url http://localhost:8080/bar/. Java source files + placed in resin-4.0.x/webapps/bar/WEB-INF/classes/ are automatically + compiled by Resin. -4) Browse http://localhost:8080 or equivalently http://127.0.0.1:8080. + resin-4.0.x/webapps/bar/index.jsp + --> http://localhost:8080/bar/ + --> http://localhost:8080/bar/index.jsp -5) Go through the documentation at http://localhost:8080/resin-doc + resin-4.0.x/webapps/bar/WEB-INF/web.xml + -- servlet/jsp configuration file -6) For JSP, create test.jsp in resin-4.0.x/webapps/ROOT/test.jsp and browse - http://localhost:8080/test.jsp. + resin-4.0.x/webapps/bar/WEB-INF/resin-web.xml + -- Resin specific configuration - Resin serves files from resin-4.0.x/webapps/ROOT/ in the default - configuration. All files with the extension '.jsp' are interpreted - as jsp files. + resin-4.0.x/webapps/bar/WEB-INF/classes/example/HelloServlet.java + -- java source file, automatically compiled by Resin + + 4) For PHP, create test.php in resin-4.0.x/webapps/ROOT/test.php and browse + http://localhost:8080/test.php. -7) For PHP, create test.php in resin-4.0.x/webapps/ROOT/test.php and browse - http://localhost:8080/test.php. -8) To deploy a war file, place the file in resin-4.0.x/webapps/foo.war - Resin will expand the war file, and the application will be available - with the url http://localhost:8080/foo/ +IV) PDF Reports +--------------- - Resin supports the development of applications without requiring a war - file deployment. Create a directory resin-4.0.x/webapps/bar to - correspond to url http://localhost:8080/bar/. Java source files - placed in resin-4.0.x/webapps/bar/WEB-INF/classes/ are automatically - compiled by Resin. + With Resin-Pro, you can get a PDF snapshot report of the server with the + command-line: - resin-4.0.x/webapps/bar/index.jsp - --> http://localhost:8080/bar/ - --> http://localhost:8080/bar/index.jsp + unix> bin/resin.sh pdf-report - resin-4.0.x/webapps/bar/WEB-INF/web.xml - -- servlet/jsp configuration file + You can also get a watchdog restart report with the -watchdog option + + unix> bin/resin.sh pdf-report -watchdog - resin-4.0.x/webapps/bar/WEB-INF/resin-web.xml - -- Resin specific configuration + +V) /resin-admin browser-based administration +-------------------------------------------- - resin-4.0.x/webapps/bar/WEB-INF/classes/example/HelloServlet.java - -- java source file, automatically compiled by Resin + An administration application is available at -9) An administration application is available at - http://localhost:8080/resin-admin - Instructions are provided for setting a password. + http://localhost:8080/resin-admin + Instructions are provided for setting a password. diff --git a/modules/resin/src/com/caucho/distcache/memcache/MemcacheClient.java b/modules/resin/src/com/caucho/distcache/memcache/MemcacheClient.java index a86fa1a70..cc745304c 100644 --- a/modules/resin/src/com/caucho/distcache/memcache/MemcacheClient.java +++ b/modules/resin/src/com/caucho/distcache/memcache/MemcacheClient.java @@ -341,9 +341,9 @@ public void put(Object key, Object value) throws CacheException WriteStream out = client.getOutputStream(); ReadStream is = client.getInputStream(); - //TempStream ts = serialize(value); - // long length = ts.getLength(); - long length = ((String) value).length(); + TempStream ts = serialize(value); + long length = ts.getLength(); + // long length = ((String) value).length(); out.print("set "); out.print(key); @@ -358,10 +358,10 @@ public void put(Object key, Object value) throws CacheException out.print(length); out.print("\r\n"); - out.print(value); - // ts.writeToStream(out); + //out.print(value); + ts.writeToStream(out); - System.out.println("SET-LEN: " + length); + // System.out.println("SET-LEN: " + length); out.print("\r\n"); out.flush(); diff --git a/modules/resin/src/com/caucho/distcache/memcache/MemcacheConnection.java b/modules/resin/src/com/caucho/distcache/memcache/MemcacheConnection.java index 5a515615d..5af5af786 100644 --- a/modules/resin/src/com/caucho/distcache/memcache/MemcacheConnection.java +++ b/modules/resin/src/com/caucho/distcache/memcache/MemcacheConnection.java @@ -29,6 +29,7 @@ package com.caucho.distcache.memcache; +import java.io.ByteArrayInputStream; import java.io.IOException; import java.io.InputStream; import java.io.OutputStream; @@ -42,6 +43,7 @@ import com.caucho.util.CharBuffer; import com.caucho.util.HashKey; import com.caucho.vfs.ReadStream; +import com.caucho.vfs.TempStream; import com.caucho.vfs.WriteStream; /** @@ -130,8 +132,11 @@ public boolean handleRequest() throws IOException Command command = _commandMap.get(_method); if (command == null) { - System.out.println("unknown command: '" + _method + "' " + command); - return false; + WriteStream out = getWriteStream(); + + out.print("ERROR\r\n"); + + return true; } //return command.execute(this); @@ -420,6 +425,97 @@ public boolean doCommand(MemcacheConnection conn, } } + static class AppendCommand extends StoreCommand { + @Override + public boolean doCommand(MemcacheConnection conn, + String key, + long bytes, + long timeout, + int flags) + throws IOException + { + ClusterCache cache = conn.getCache(); + + ExtCacheEntry entry = cache.getExtCacheEntry(key); + + ReadStream rs = conn.getReadStream(); + + if (entry == null || entry.isValueNull()) { + rs.skip(bytes); + + return false; + } + + TempStream ts = new TempStream(); + + WriteStream os = new WriteStream(ts); + os.setDisableClose(true); + + cache.get(key, os); + + SetInputStream setIs = conn.getSetInputStream(); + + setIs.init(rs, bytes); + + os.writeStream(setIs); + os.setDisableClose(false); + os.close(); + + cache.put(key, + ts.openRead(), + entry.getIdleTimeout(), + entry.getUserFlags()); + + return true; + } + } + + static class PrependCommand extends StoreCommand { + @Override + public boolean doCommand(MemcacheConnection conn, + String key, + long bytes, + long timeout, + int flags) + throws IOException + { + ClusterCache cache = conn.getCache(); + + ExtCacheEntry entry = cache.getExtCacheEntry(key); + + ReadStream rs = conn.getReadStream(); + + if (entry == null || entry.isValueNull()) { + rs.skip(bytes); + + return false; + } + + TempStream ts = new TempStream(); + + WriteStream os = new WriteStream(ts); + os.setDisableClose(true); + + SetInputStream setIs = conn.getSetInputStream(); + + setIs.init(rs, bytes); + + os.writeStream(setIs); + + cache.get(key, os); + + os.setDisableClose(false); + os.close(); + + cache.put(key, + ts.openRead(), + entry.getIdleTimeout(), + entry.getUserFlags()); + + return true; + } + } + static class GetCommand extends Command { @Override public boolean execute(MemcacheConnection conn) @@ -432,7 +528,7 @@ public boolean execute(MemcacheConnection conn) CharBuffer cb = new CharBuffer(); while (readKey(rs, cb)) { - getCache(out, conn.getCache(), cb.toString(), conn); + getCache(out, conn.getCache(), cb.toString(), conn, 0); } int ch = rs.read(); @@ -477,10 +573,11 @@ private boolean readKey(ReadStream rs, CharBuffer cb) return true; } - private void getCache(WriteStream out, - ClusterCache cache, - String key, - MemcacheConnection conn) + protected void getCache(WriteStream out, + ClusterCache cache, + String key, + MemcacheConnection conn, + long hash) throws IOException { ExtCacheEntry entry = cache.getExtCacheEntry(key); @@ -497,6 +594,14 @@ private void getCache(WriteStream out, return; } + HashKey valueKey = entry.getValueHashKey(); + long unique = getCasKey(valueKey); + + if (hash != 0 && hash == unique) { + // get-if-modified + return; + } + out.print("VALUE "); out.print(key); out.print(" "); @@ -505,13 +610,6 @@ private void getCache(WriteStream out, long bytes = entry.getValueLength(); out.print(" "); out.print(bytes); - - HashKey valueKey = entry.getValueHashKey(); - - long unique = 0; - - unique = getCasKey(valueKey); - out.print(" "); out.print(unique); out.print("\r\n"); @@ -525,6 +623,251 @@ private void getCache(WriteStream out, } } + static class GetIfModifiedCommand extends GetCommand { + @Override + public boolean execute(MemcacheConnection conn) + throws IOException + { + ReadStream rs = conn.getReadStream(); + WriteStream out = conn.getWriteStream(); + out.setDisableClose(true); + + CharBuffer cb = new CharBuffer(); + + int ch = 0; + + for (ch = rs.read(); ch >= 0 && ch == ' '; ch = rs.read()) { + } + + for (; ch >= 0 && ch != ' ' && ch != '\n'; ch = rs.read()) { + cb.append((char) ch); + } + + for (; ch == ' '; ch = rs.read()) { + } + + long hash = 0; + + for (; '0' <= ch && ch <= '9'; ch = rs.read()) { + hash = 10 * hash + ch - '0'; + } + + getCache(out, conn.getCache(), cb.toString(), conn, hash); + + for (; ch >= 0 && ch != '\r' && ch != '\n'; ch = rs.read()) { + } + + if (ch == '\r') { + ch = rs.read(); + if (ch != '\n') { + System.out.println("PROTOL: " + ch); + throw new IOException("PROTOCOL: " + ch); + } + } + + out.print("END\r\n"); + out.flush(); + + return true; + } + } + + static class DeleteCommand extends Command { + @Override + public boolean execute(MemcacheConnection conn) + throws IOException + { + ReadStream rs = conn.getReadStream(); + WriteStream out = conn.getWriteStream(); + out.setDisableClose(true); + + boolean isNoReply = false; + + CharBuffer cb = new CharBuffer(); + + int ch = 0; + + for (ch = rs.read(); ch >= 0 && ch == ' '; ch = rs.read()) { + } + + for (; ch >= 0 && ch != ' ' && ch != '\n'; ch = rs.read()) { + cb.append((char) ch); + } + + String key = cb.toString(); + + for (; ch == ' '; ch = rs.read()) { + } + + long time = 0; + + for (; '0' <= ch && ch <= '9'; ch = rs.read()) { + time = 10 * time + ch - '0'; + } + + for (; ch == ' '; ch = rs.read()) { + } + + cb.clear(); + for (; ch >= 0 && ch != ' ' && ch != '\r' && ch != '\n'; ch = rs.read()) { + cb.append((char) ch); + } + + if (cb.length() > 0 && cb.matches("noreply")) + isNoReply = true; + + for (; ch >= 0 && ch != '\r' && ch != '\n'; ch = rs.read()) { + } + + if (ch == '\r') { + ch = rs.read(); + if (ch != '\n') { + System.out.println("PROTOL: " + ch); + throw new IOException("PROTOCOL: " + ch); + } + } + + if (deleteCache(conn.getCache(), time, key)) { + if (! isNoReply) + out.print("DELETED\r\n"); + } + else { + if (! isNoReply) + out.print("NOT_FOUND\r\n"); + } + + out.flush(); + + return true; + } + + protected boolean deleteCache(ClusterCache cache, + long time, + String key) + throws IOException + { + ExtCacheEntry entry = cache.getExtCacheEntry(key); + + cache.remove(key); + + return (entry != null && ! entry.isValueNull()); + } + } + + static class IncrementCommand extends Command { + @Override + public boolean execute(MemcacheConnection conn) + throws IOException + { + ReadStream rs = conn.getReadStream(); + WriteStream out = conn.getWriteStream(); + out.setDisableClose(true); + + boolean isNoReply = false; + + CharBuffer cb = new CharBuffer(); + + int ch = 0; + + for (ch = rs.read(); ch >= 0 && ch == ' '; ch = rs.read()) { + } + + for (; ch >= 0 && ch != ' ' && ch != '\n'; ch = rs.read()) { + cb.append((char) ch); + } + + String key = cb.toString(); + + for (; ch == ' '; ch = rs.read()) { + } + + long delta = 0; + + for (; '0' <= ch && ch <= '9'; ch = rs.read()) { + delta = 10 * delta + ch - '0'; + } + + for (; ch == ' '; ch = rs.read()) { + } + + cb.clear(); + for (; ch >= 0 && ch != ' ' && ch != '\r' && ch != '\n'; ch = rs.read()) { + cb.append((char) ch); + } + + if (cb.length() > 0 && cb.matches("noreply")) + isNoReply = true; + + for (; ch >= 0 && ch != '\r' && ch != '\n'; ch = rs.read()) { + } + + if (ch == '\r') { + ch = rs.read(); + if (ch != '\n') { + System.out.println("PROTOL: " + ch); + throw new IOException("PROTOCOL: " + ch); + } + } + + long value = changeCache(conn.getCache(), key, delta); + + if (isNoReply) { + + } + else if (value == Long.MIN_VALUE) { + out.print("NOT_FOUND\r\n"); + } + else { + out.print("VALUE " + value + "\r\n"); + } + + return true; + } + + protected long changeCache(ClusterCache cache, + String key, + long delta) + throws IOException + { + return incrementCache(cache, key, delta); + } + + protected long incrementCache(ClusterCache cache, + String key, + long delta) + throws IOException + { + ExtCacheEntry entry = cache.getExtCacheEntry(key); + + if (entry == null || entry.isValueNull()) + return Long.MIN_VALUE; + + CounterStream os = new CounterStream(); + + cache.get(key, os); + + long newValue = os.getValue() + delta; + + byte []values = String.valueOf(newValue).getBytes(); + + ByteArrayInputStream bis = new ByteArrayInputStream(values); + + cache.put(key, bis, entry.getIdleTimeout()); + + return newValue; + } + } + + static class DecrementCommand extends IncrementCommand { + protected long changeCache(ClusterCache cache, + String key, + long delta) + throws IOException + { + return incrementCache(cache, key, -delta); + } + } + static class QuitCommand extends Command { @Override public boolean execute(MemcacheConnection conn) @@ -556,6 +899,26 @@ public boolean execute(MemcacheConnection conn) } } + static class VerbosityCommand extends Command { + @Override + public boolean execute(MemcacheConnection conn) + throws IOException + { + ReadStream rs = conn.getReadStream(); + + int ch; + + while ((ch = rs.read()) >= 0 && ch != '\n') { + } + + WriteStream out = conn.getWriteStream(); + + out.print("OK\r\n"); + + return true; + } + } + static class StatsCommand extends Command { @Override public boolean execute(MemcacheConnection conn) @@ -584,12 +947,22 @@ public boolean execute(MemcacheConnection conn) if ("".equals(key)) { out.print("END\r\n"); } + else if ("resin".equals(key)) { + printResinStats(out); + out.print("END\r\n"); + } else { out.print("ERROR\r\n"); } return true; } + + private void printResinStats(WriteStream out) + throws IOException + { + out.print("STAT enable_get_if_modified 1\r\n"); + } } static class SetInputStream extends InputStream { @@ -660,14 +1033,43 @@ public final void write(byte []buffer, int offset, int length) } } + static class CounterStream extends OutputStream { + private int _sign = 1; + private long _value; + + public void write(int ch) + { + if (ch == '-') + _sign = -1; + + if ('0' <= ch && ch <= '9') + _value = 10 * _value + ch - '0'; + } + + public long getValue() + { + return _sign * _value; + } + + public void flush() {} + public void close() {} + } + static { addCommand("add", new AddCommand()); + addCommand("append", new AppendCommand()); addCommand("get", new GetCommand()); addCommand("gets", new GetCommand()); + addCommand("get_if_modified", new GetIfModifiedCommand()); + addCommand("decr", new DecrementCommand()); + addCommand("delete", new DeleteCommand()); + addCommand("incr", new IncrementCommand()); + addCommand("prepend", new PrependCommand()); addCommand("quit", new QuitCommand()); addCommand("replace", new ReplaceCommand()); addCommand("set", new SetCommand()); addCommand("stats", new StatsCommand()); addCommand("version", new VersionCommand()); + addCommand("verbosity", new VerbosityCommand()); } } diff --git a/resin-doc/admin/resin-admin-console.xtp b/resin-doc/admin/resin-admin-console.xtp index ed69e32a0..dbf6c67b3 100644 --- a/resin-doc/admin/resin-admin-console.xtp +++ b/resin-doc/admin/resin-admin-console.xtp @@ -192,39 +192,32 @@ a dedicated profiler available.

-

For security, you will also need to add a <user> to -the <resin:AdminAuthenticator> section of the resin.xml. -The password will be -a MD5 hash. By default, the /resin-admin web-app provides a form -for generating the hash codes. You will need to copy the generated -password into the resin.xml. This guarantees that you have access -to the resin.xml itself to add any users. In other words, the configuration -is very cautious about security issues to enable the administration.

- - +

For advanced users, you can change the standard AdminAuthenticator +to be any of the Resin authenticators with a little extra configuration. +The default authenticator uses an XML file called admin-users.xml to +define the admin users.

+ +

You can create an alternative authenticator by configuring it and +setting its CDI name to be "resinAuth". For example, the following +will configure an XmlAuthenticator.

+ + <resin xmlns="http://caucho.com/ns/resin" + xmlns:ee="urn:java:ee"> xmlns:resin="urn:java:com.caucho.resin"> - <resin:AdminAuthenticator> - <resin:import path="${__DIR__}/admin-users.xml" optional="true"/> - </resin:AdminAuthenticator> + <resin:XmlAuthenticator ee:Named="resinAuth"> + <user name="admin" password="{SSHA}h5QdSulQyqIgYo7BIJ3YfnRSY56kD847"/> + </resin:XmlAuthenticator> + ... + <cluster id=""> <host id=""> - ... - <web-app id="/resin-admin" root-directory="${resin.root}/doc/admin"> - <prologue> - <!-- only allow access from internal IP (10.*,192.168.*,127.*) --> - <resin:set var="resin_admin_external" value="false"/> - <resin:set var="resin_admin_insecure" value="true"/> - </prologue> + ... </web-app> - -</host> -</cluster> -</resin> diff --git a/resin-doc/admin/toc.xml b/resin-doc/admin/toc.xml index 5f64a826e..558bf0f8c 100644 --- a/resin-doc/admin/toc.xml +++ b/resin-doc/admin/toc.xml @@ -3,7 +3,8 @@ + description="Getting Started with Resin"> + diff --git a/resin-doc/changes/changes.xtp b/resin-doc/changes/changes.xtp index ceb41b4f3..8604ffd95 100644 --- a/resin-doc/changes/changes.xtp +++ b/resin-doc/changes/changes.xtp @@ -12,6 +12,7 @@
    +
  • solaris: solaris needs #define _POSIX_PTHREAD_SEMANTICS for proper ctime_r behavior (#4791, rep by Alan Wright)
  • health: added health events and anomaly events (#4724)
  • db-pool: reduced default max-create-connections back to 5 (#4734)
  • db-pool: add min-idle-count to <database> configuration (#4664)