From f373fafc56dc351785e4cec1ca0a51f6e87135c1 Mon Sep 17 00:00:00 2001 From: Daniel Date: Mon, 14 Jun 2021 07:23:50 +0800 Subject: [PATCH 01/25] OfficeFloor avoid writer stream lock (#6623) * Removing Rapidoid as seems dead project (no commit last year) * Downloading from SourceForge * Bump maven-compiler-plugin in /frameworks/Java/officefloor/src Bumps [maven-compiler-plugin](https://github.com/apache/maven-compiler-plugin) from 3.8.0 to 3.8.1. - [Release notes](https://github.com/apache/maven-compiler-plugin/releases) - [Commits](https://github.com/apache/maven-compiler-plugin/compare/maven-compiler-plugin-3.8.0...maven-compiler-plugin-3.8.1) Signed-off-by: dependabot-preview[bot] * Bump maven-shade-plugin in /frameworks/Java/officefloor/src Bumps [maven-shade-plugin](https://github.com/apache/maven-shade-plugin) from 3.2.1 to 3.2.2. - [Release notes](https://github.com/apache/maven-shade-plugin/releases) - [Commits](https://github.com/apache/maven-shade-plugin/compare/maven-shade-plugin-3.2.1...maven-shade-plugin-3.2.2) Signed-off-by: dependabot-preview[bot] * Fixing Raw OfficeFloor * Specifying Spring plugin version * Bump to OfficeFloor 3.21.0 * Bump spring-boot-maven-plugin in /frameworks/Java/officefloor/src Bumps [spring-boot-maven-plugin](https://github.com/spring-projects/spring-boot) from 2.2.4.RELEASE to 2.2.5.RELEASE. - [Release notes](https://github.com/spring-projects/spring-boot/releases) - [Commits](https://github.com/spring-projects/spring-boot/compare/v2.2.4.RELEASE...v2.2.5.RELEASE) Signed-off-by: dependabot-preview[bot] * Bump spring-boot-maven-plugin in /frameworks/Java/officefloor/src Bumps [spring-boot-maven-plugin](https://github.com/spring-projects/spring-boot) from 2.2.5.RELEASE to 2.2.6.RELEASE. - [Release notes](https://github.com/spring-projects/spring-boot/releases) - [Commits](https://github.com/spring-projects/spring-boot/compare/v2.2.5.RELEASE...v2.2.6.RELEASE) Signed-off-by: dependabot-preview[bot] * Bump net.officefloor:bom in /frameworks/Java/officefloor/src Bumps [net.officefloor:bom](https://github.com/officefloor/OfficeFloor) from 3.21.0 to 3.22.0. - [Release notes](https://github.com/officefloor/OfficeFloor/releases) - [Commits](https://github.com/officefloor/OfficeFloor/commits) Signed-off-by: dependabot-preview[bot] * Bump maven-shade-plugin in /frameworks/Java/officefloor/src Bumps [maven-shade-plugin](https://github.com/apache/maven-shade-plugin) from 3.2.2 to 3.2.3. - [Release notes](https://github.com/apache/maven-shade-plugin/releases) - [Commits](https://github.com/apache/maven-shade-plugin/compare/maven-shade-plugin-3.2.2...maven-shade-plugin-3.2.3) Signed-off-by: dependabot-preview[bot] * Bump net.officefloor:bom in /frameworks/Java/officefloor/src Bumps [net.officefloor:bom](https://github.com/officefloor/OfficeFloor) from 3.22.0 to 3.23.0. - [Release notes](https://github.com/officefloor/OfficeFloor/releases) - [Commits](https://github.com/officefloor/OfficeFloor/commits) Signed-off-by: dependabot-preview[bot] * Bump net.officefloor:bom in /frameworks/Java/officefloor/src Bumps [net.officefloor:bom](https://github.com/officefloor/OfficeFloor) from 3.23.0 to 3.24.0. - [Release notes](https://github.com/officefloor/OfficeFloor/releases) - [Commits](https://github.com/officefloor/OfficeFloor/commits) Signed-off-by: dependabot-preview[bot] * Bump spring-boot-maven-plugin in /frameworks/Java/officefloor/src Bumps [spring-boot-maven-plugin](https://github.com/spring-projects/spring-boot) from 2.2.6.RELEASE to 2.2.7.RELEASE. - [Release notes](https://github.com/spring-projects/spring-boot/releases) - [Commits](https://github.com/spring-projects/spring-boot/compare/v2.2.6.RELEASE...v2.2.7.RELEASE) Signed-off-by: dependabot-preview[bot] * Bump spring-boot-maven-plugin in /frameworks/Java/officefloor/src Bumps [spring-boot-maven-plugin](https://github.com/spring-projects/spring-boot) from 2.2.7.RELEASE to 2.3.0.RELEASE. - [Release notes](https://github.com/spring-projects/spring-boot/releases) - [Commits](https://github.com/spring-projects/spring-boot/compare/v2.2.7.RELEASE...v2.3.0.RELEASE) Signed-off-by: dependabot-preview[bot] * Bump net.officefloor:bom in /frameworks/Java/officefloor/src Bumps [net.officefloor:bom](https://github.com/officefloor/OfficeFloor) from 3.24.0 to 3.25.0. - [Release notes](https://github.com/officefloor/OfficeFloor/releases) - [Commits](https://github.com/officefloor/OfficeFloor/commits) Signed-off-by: dependabot-preview[bot] * Bump maven-shade-plugin in /frameworks/Java/officefloor/src Bumps [maven-shade-plugin](https://github.com/apache/maven-shade-plugin) from 3.2.3 to 3.2.4. - [Release notes](https://github.com/apache/maven-shade-plugin/releases) - [Commits](https://github.com/apache/maven-shade-plugin/compare/maven-shade-plugin-3.2.3...maven-shade-plugin-3.2.4) Signed-off-by: dependabot-preview[bot] * Bump spring-boot-maven-plugin in /frameworks/Java/officefloor/src Bumps [spring-boot-maven-plugin](https://github.com/spring-projects/spring-boot) from 2.3.0.RELEASE to 2.3.1.RELEASE. - [Release notes](https://github.com/spring-projects/spring-boot/releases) - [Commits](https://github.com/spring-projects/spring-boot/compare/v2.3.0.RELEASE...v2.3.1.RELEASE) Signed-off-by: dependabot-preview[bot] * Bump spring-boot-maven-plugin in /frameworks/Java/officefloor/src Bumps [spring-boot-maven-plugin](https://github.com/spring-projects/spring-boot) from 2.3.1.RELEASE to 2.3.2.RELEASE. - [Release notes](https://github.com/spring-projects/spring-boot/releases) - [Commits](https://github.com/spring-projects/spring-boot/compare/v2.3.1.RELEASE...v2.3.2.RELEASE) Signed-off-by: dependabot-preview[bot] * Bump net.officefloor:bom in /frameworks/Java/officefloor/src Bumps [net.officefloor:bom](https://github.com/officefloor/OfficeFloor) from 3.25.0 to 3.26.0. - [Release notes](https://github.com/officefloor/OfficeFloor/releases) - [Commits](https://github.com/officefloor/OfficeFloor/compare/release-3.25.0...release-3.26.0) Signed-off-by: dependabot-preview[bot] * Bump spring-boot-maven-plugin in /frameworks/Java/officefloor/src Bumps [spring-boot-maven-plugin](https://github.com/spring-projects/spring-boot) from 2.3.2.RELEASE to 2.3.3.RELEASE. - [Release notes](https://github.com/spring-projects/spring-boot/releases) - [Commits](https://github.com/spring-projects/spring-boot/compare/v2.3.2.RELEASE...v2.3.3.RELEASE) Signed-off-by: dependabot-preview[bot] * Bump net.officefloor:bom in /frameworks/Java/officefloor/src Bumps [net.officefloor:bom](https://github.com/officefloor/OfficeFloor) from 3.26.0 to 3.27.0. - [Release notes](https://github.com/officefloor/OfficeFloor/releases) - [Commits](https://github.com/officefloor/OfficeFloor/commits) Signed-off-by: dependabot-preview[bot] * Bump spring-boot-maven-plugin in /frameworks/Java/officefloor/src Bumps [spring-boot-maven-plugin](https://github.com/spring-projects/spring-boot) from 2.3.3.RELEASE to 2.3.4.RELEASE. - [Release notes](https://github.com/spring-projects/spring-boot/releases) - [Commits](https://github.com/spring-projects/spring-boot/compare/v2.3.3.RELEASE...v2.3.4.RELEASE) Signed-off-by: dependabot-preview[bot] * Bump net.officefloor:bom in /frameworks/Java/officefloor/src Bumps [net.officefloor:bom](https://github.com/officefloor/OfficeFloor) from 3.27.0 to 3.28.0. - [Release notes](https://github.com/officefloor/OfficeFloor/releases) - [Commits](https://github.com/officefloor/OfficeFloor/commits) Signed-off-by: dependabot-preview[bot] * Fixing to run with v3.28.0 * Including Undertow * Using slim docker images * Fixing to add all supported tests * Increasing connection pool size * Write up the 503 errors and why * Fix grammer * Bump spring-boot-maven-plugin in /frameworks/Java/officefloor/src Bumps [spring-boot-maven-plugin](https://github.com/spring-projects/spring-boot) from 2.3.4.RELEASE to 2.3.5.RELEASE. - [Release notes](https://github.com/spring-projects/spring-boot/releases) - [Commits](https://github.com/spring-projects/spring-boot/compare/v2.3.4.RELEASE...v2.3.5.RELEASE) Signed-off-by: dependabot-preview[bot] * Improving chances of full processing Increasing max thread counts to have thread per client. Also, shading jars correctly. Plus adding in DB test for raw * Fixing raw test to have database * Ensure random numbers to avoid cached entities * Ensuring spring random numbers to avoid entity caching * Adding queries for raw * officefloor-raw queries passing * office-raw supporting all requests * Fixed update test to ensure updates occur before responding * Adding officefloor-async This will demonstrate OfficeFloor running as a single thread asynchronous server * Upgrading to OfficeFloor 2.38.1 * Tidy up read me * Appropriate back pressure queue * Handle rate limiting * Providing appropriate throttling * Lowering threading to handle through put * officefloor-raw passing tests * Passing tests * OfficeFloor 3.28.2 * Fixing rate limiting * Bump net.officefloor:bom in /frameworks/Java/officefloor/src Bumps [net.officefloor:bom](https://github.com/officefloor/OfficeFloor) from 3.28.2 to 3.29.0. - [Release notes](https://github.com/officefloor/OfficeFloor/releases) - [Commits](https://github.com/officefloor/OfficeFloor/commits) Signed-off-by: dependabot-preview[bot] * Fixing for bump to 3.30.0 * Increasing max direct memory * Increasing threshold to enable verifications to pass for officefloor-raw * Start script to take available memory into consideration * Max direct memory aware of available memory Also, cleaning writer threads of buffers to avoid leaks. * Avoiding OOM by managing disable/enable reading * DB and Fortune passing * Ensuring free command is available * Removing TODOs * Increasing reactor buffer for multiple queries * Fixing for 3.30.1 * Passing benchmark tests * Avoiding OOM on reactor buffer sizes * Passing validate tests * Using defaults from 3.30.1 * Bump to 512 threads and connections * Reverting dockerfiles to provide appropriate parameters * openjdk:slim for apt-get available * Providing thread affinity to raw * Fixing rate limit throttling * Multiplexing queries over connections per socket * Fixing for large queries in validate * Fixing versions of maven and java * Providing thread affinity of DB connection thread * Fixing for Netty event loop thread * Tidying up compiler warnings * Single db pool to avoid additional threads * Use default LoopResources * Tidying up code for warnings * Further tidy up of code * Revert to latest pull request * Using parallel GC for better throughput * Fix up for thread local buffering improvements * OfficeFloor fortune raw Focus of officefloor-raw is raw performance of the OfficeFloor HTTP server. Using raw fortune to remove mustache overheads. * Tidy up loop * Server name (as per discussions) * Bump spring-boot-maven-plugin in /frameworks/Java/officefloor/src Bumps [spring-boot-maven-plugin](https://github.com/spring-projects/spring-boot) from 2.3.5.RELEASE to 2.4.2. - [Release notes](https://github.com/spring-projects/spring-boot/releases) - [Commits](https://github.com/spring-projects/spring-boot/compare/v2.3.5.RELEASE...v2.4.2) Signed-off-by: dependabot-preview[bot] * Bump net.officefloor:bom in /frameworks/Java/officefloor/src Bumps [net.officefloor:bom](https://github.com/officefloor/OfficeFloor) from 3.30.2 to 3.31.0. - [Release notes](https://github.com/officefloor/OfficeFloor/releases) - [Commits](https://github.com/officefloor/OfficeFloor/commits) Signed-off-by: dependabot-preview[bot] * Bump net.officefloor:bom in /frameworks/Java/officefloor/src Bumps [net.officefloor:bom](https://github.com/officefloor/OfficeFloor) from 3.31.0 to 3.32.0. - [Release notes](https://github.com/officefloor/OfficeFloor/releases) - [Commits](https://github.com/officefloor/OfficeFloor/commits) Signed-off-by: dependabot-preview[bot] * Bump spring-boot-maven-plugin in /frameworks/Java/officefloor/src Bumps [spring-boot-maven-plugin](https://github.com/spring-projects/spring-boot) from 2.4.2 to 2.4.3. - [Release notes](https://github.com/spring-projects/spring-boot/releases) - [Commits](https://github.com/spring-projects/spring-boot/compare/v2.4.2...v2.4.3) Signed-off-by: dependabot-preview[bot] * Bump spring-boot-maven-plugin in /frameworks/Java/officefloor/src Bumps [spring-boot-maven-plugin](https://github.com/spring-projects/spring-boot) from 2.4.3 to 2.4.5. - [Release notes](https://github.com/spring-projects/spring-boot/releases) - [Commits](https://github.com/spring-projects/spring-boot/compare/v2.4.3...v2.4.5) Signed-off-by: dependabot-preview[bot] * Upgrade to GitHub-native Dependabot * Bump net.officefloor:bom in /frameworks/Java/officefloor/src Bumps [net.officefloor:bom](https://github.com/officefloor/OfficeFloor) from 3.32.0 to 3.35.0. - [Release notes](https://github.com/officefloor/OfficeFloor/releases) - [Commits](https://github.com/officefloor/OfficeFloor/commits/release-3.35.0) Signed-off-by: dependabot-preview[bot] * Bump to OfficeFloor 3.35.0 * Allow bump of all versions * Vertx server Also, renaming officefloor-raw to officefloor-r2dbc to make way for officefloor-sqlclient * Rename to R2dbc (from raw) * Bump net.officefloor:bom in /frameworks/Java/officefloor/src Bumps [net.officefloor:bom](https://github.com/officefloor/OfficeFloor) from 3.35.0 to 3.36.0. - [Release notes](https://github.com/officefloor/OfficeFloor/releases) - [Commits](https://github.com/officefloor/OfficeFloor/commits) Signed-off-by: dependabot[bot] * Fixing meta-data * Can not propagate Dependabot to TechEmpower * Abstract WoOF from database driver This will allow introducing Vertx SQL Client * Fixing db port * Providing Vertx SQL Client implementation * Using OfficeFloorVertx for vertx Allows for tests to reset * Fixing link in read me * Reducing repetition in readme * Swapped OfficeFloor async to use Vertx SQL Client * Bump spring-boot-maven-plugin in /frameworks/Java/officefloor/src Bumps [spring-boot-maven-plugin](https://github.com/spring-projects/spring-boot) from 2.4.5 to 2.5.0. - [Release notes](https://github.com/spring-projects/spring-boot/releases) - [Commits](https://github.com/spring-projects/spring-boot/compare/v2.4.5...v2.5.0) Signed-off-by: dependabot[bot] * Increasing SQL Client pool size to 512 Also, fixing update to sort to avoid deadlocks * Removing unnecessary logging * Removing dependabot configuration * Improving performance of Vertx This is by caching queries and using native communication. Also, adding further performance updates to update test by sorting updates * Fixing OfficeFloor-vertx name * Reducing load on DB callback threads * Database worker threads relative to number of socket threads * Further improvements to performance (by less contention) * Avoid lock on write stream * Using Guava * Removing commented out line Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .../officefloor/src/woof_benchmark_woof/pom.xml | 8 ++++++++ .../benchmark/FortunesSendResponse.java | 16 +++++++++++++--- 2 files changed, 21 insertions(+), 3 deletions(-) diff --git a/frameworks/Java/officefloor/src/woof_benchmark_woof/pom.xml b/frameworks/Java/officefloor/src/woof_benchmark_woof/pom.xml index a79ad25450c..5551b3f815c 100644 --- a/frameworks/Java/officefloor/src/woof_benchmark_woof/pom.xml +++ b/frameworks/Java/officefloor/src/woof_benchmark_woof/pom.xml @@ -8,6 +8,9 @@ 1.0.0 woof_benchmark_woof + + 29.0-jre + net.officefloor.web @@ -31,5 +34,10 @@ + + com.google.guava + guava + ${guava-version} + \ No newline at end of file diff --git a/frameworks/Java/officefloor/src/woof_benchmark_woof/src/main/java/net/officefloor/benchmark/FortunesSendResponse.java b/frameworks/Java/officefloor/src/woof_benchmark_woof/src/main/java/net/officefloor/benchmark/FortunesSendResponse.java index 4b4032443c7..71e351839c9 100644 --- a/frameworks/Java/officefloor/src/woof_benchmark_woof/src/main/java/net/officefloor/benchmark/FortunesSendResponse.java +++ b/frameworks/Java/officefloor/src/woof_benchmark_woof/src/main/java/net/officefloor/benchmark/FortunesSendResponse.java @@ -4,6 +4,7 @@ import java.nio.ByteBuffer; import java.nio.channels.CancelledKeyException; import java.nio.channels.ClosedChannelException; +import java.nio.charset.Charset; import java.util.Collections; import java.util.Comparator; import java.util.List; @@ -11,6 +12,8 @@ import org.apache.commons.text.StringEscapeUtils; import com.fasterxml.jackson.databind.ObjectMapper; +import com.google.common.escape.Escaper; +import com.google.common.html.HtmlEscapers; import net.officefloor.server.RequestHandler; import net.officefloor.server.http.HttpHeaderValue; @@ -18,6 +21,7 @@ import net.officefloor.server.http.ServerHttpConnection; import net.officefloor.server.http.impl.ProcessAwareServerHttpConnectionManagedObject; import net.officefloor.server.http.parse.HttpRequestParser; +import net.officefloor.server.stream.ServerOutputStream; import net.officefloor.server.stream.ServerWriter; /** @@ -41,6 +45,10 @@ public class FortunesSendResponse extends AbstractSendResponse { private static final byte[] TEMPLATE_END = "" .getBytes(ServerHttpConnection.DEFAULT_HTTP_ENTITY_CHARSET); + private static final Charset UTF8 = Charset.forName("UTF-8"); + + private static final Escaper HTML_ESCAPER = HtmlEscapers.htmlEscaper(); + private static Comparator SORT_FORTUNE = (a, b) -> a.message.compareTo(b.message); public FortunesSendResponse(RequestHandler requestHandler, @@ -57,15 +65,17 @@ public void sendFortunes(List fortunes) { // Send response HttpResponse response = this.connection.getResponse(); + + // Raw template response.setContentType(TEXT_HTML, null); - ServerWriter writer = response.getEntityWriter(); + ServerOutputStream writer = response.getEntity(); writer.write(TEMPLATE_START); for (Fortune fortune : fortunes) { writer.write(FORTUNE_START); int id = fortune.id; - writer.write(Integer.valueOf(id).toString()); + writer.write(Integer.valueOf(id).toString().getBytes(UTF8)); writer.write(FORTUNE_MIDDLE); - StringEscapeUtils.ESCAPE_HTML4.translate(fortune.message, writer); + writer.write(HTML_ESCAPER.escape(fortune.message).getBytes(UTF8)); writer.write(FORTUNE_END); } writer.write(TEMPLATE_END); From 3946b330ede7665b0ca60cd244f28b9816f3b7e7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E4=B8=89=E5=88=80?= Date: Mon, 14 Jun 2021 07:24:43 +0800 Subject: [PATCH 02/25] rename handle (#6626) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * update smart-servlet to 0.1.3-SNAPSHOT * update aio-enhance to 1.0.3-SNAPSHOT * smart-servlet bugfix * bugfix * update smart-socket to 1.5.6-SNAPSHOT * remove file * update aio-enhance to 1.0.4-SNAPSHOT * 优化代码 * 优化代码 * update smart-socket to 1.5.6 * config threadNum * update smart-socket to 1.5.7-SNAPSHOT * 优化代码 * update smart-socket to 1.5.10-SNAPSHOT * 优化代码 * 优化代码 * 优化代码 --- .../main/java/org/smartboot/http/Bootstrap.java | 16 ++++++++-------- .../smartboot/http/MultipleQueriesHandler.java | 6 +++--- .../org/smartboot/http/SingleQueryHandler.java | 6 +++--- .../java/org/smartboot/http/UpdateHandler.java | 6 +++--- .../java/org/smartboot/servlet/Bootstrap.java | 6 +++--- 5 files changed, 20 insertions(+), 20 deletions(-) diff --git a/frameworks/Java/smart-socket/src/main/java/org/smartboot/http/Bootstrap.java b/frameworks/Java/smart-socket/src/main/java/org/smartboot/http/Bootstrap.java index 85c6d89aaef..267a03dde42 100755 --- a/frameworks/Java/smart-socket/src/main/java/org/smartboot/http/Bootstrap.java +++ b/frameworks/Java/smart-socket/src/main/java/org/smartboot/http/Bootstrap.java @@ -15,8 +15,8 @@ import org.smartboot.http.server.HttpBootstrap; import org.smartboot.http.server.HttpRequest; import org.smartboot.http.server.HttpResponse; -import org.smartboot.http.server.HttpServerHandle; -import org.smartboot.http.server.handle.HttpRouteHandle; +import org.smartboot.http.server.HttpServerHandler; +import org.smartboot.http.server.handler.HttpRouteHandler; import org.smartboot.http.server.impl.Request; import org.smartboot.socket.StateMachineEnum; import org.smartboot.socket.extension.processor.AbstractMessageProcessor; @@ -31,22 +31,22 @@ public class Bootstrap { public static void main(String[] args) { System.setProperty("java.nio.channels.spi.AsynchronousChannelProvider", EnhanceAsynchronousChannelProvider.class.getName()); - HttpRouteHandle routeHandle = new HttpRouteHandle(); + HttpRouteHandler routeHandle = new HttpRouteHandler(); routeHandle - .route("/plaintext", new HttpServerHandle() { + .route("/plaintext", new HttpServerHandler() { @Override - public void doHandle(HttpRequest request, HttpResponse response) throws IOException { + public void handle(HttpRequest request, HttpResponse response) throws IOException { response.setContentLength(body.length); response.setContentType("text/plain; charset=UTF-8"); response.write(body); } }) - .route("/json", new HttpServerHandle() { + .route("/json", new HttpServerHandler() { @Override - public void doHandle(HttpRequest request, HttpResponse response) throws IOException { + public void handle(HttpRequest request, HttpResponse response) throws IOException { response.setContentType("application/json"); JsonUtil.writeJsonBytes(response, new Message("Hello, World!")); @@ -76,7 +76,7 @@ public void stateEvent0(AioSession session, StateMachineEnum stateMachineEnum, T bootstrap.pipeline(routeHandle).setPort(8080).start(); } - private static void initDB(HttpRouteHandle routeHandle) { + private static void initDB(HttpRouteHandler routeHandle) { try { Class.forName("org.postgresql.Driver"); } catch (ClassNotFoundException e) { diff --git a/frameworks/Java/smart-socket/src/main/java/org/smartboot/http/MultipleQueriesHandler.java b/frameworks/Java/smart-socket/src/main/java/org/smartboot/http/MultipleQueriesHandler.java index 0896790de9f..d623f3bf1de 100644 --- a/frameworks/Java/smart-socket/src/main/java/org/smartboot/http/MultipleQueriesHandler.java +++ b/frameworks/Java/smart-socket/src/main/java/org/smartboot/http/MultipleQueriesHandler.java @@ -3,7 +3,7 @@ import org.smartboot.http.common.utils.NumberUtils; import org.smartboot.http.server.HttpRequest; import org.smartboot.http.server.HttpResponse; -import org.smartboot.http.server.HttpServerHandle; +import org.smartboot.http.server.HttpServerHandler; import javax.sql.DataSource; import java.io.IOException; @@ -17,7 +17,7 @@ * @author 三刀 * @version V1.0 , 2020/6/16 */ -public class MultipleQueriesHandler extends HttpServerHandle { +public class MultipleQueriesHandler extends HttpServerHandler { private DataSource dataSource; public MultipleQueriesHandler(DataSource dataSource) { @@ -25,7 +25,7 @@ public MultipleQueriesHandler(DataSource dataSource) { } @Override - public void doHandle(HttpRequest httpRequest, HttpResponse response) throws IOException { + public void handle(HttpRequest httpRequest, HttpResponse response) throws IOException { int queries = Math.min(Math.max(NumberUtils.toInt(httpRequest.getParameter("queries"), 1), 1), 500); World[] worlds = new World[queries]; try (Connection connection = dataSource.getConnection(); diff --git a/frameworks/Java/smart-socket/src/main/java/org/smartboot/http/SingleQueryHandler.java b/frameworks/Java/smart-socket/src/main/java/org/smartboot/http/SingleQueryHandler.java index 4b15f15d516..e72f4b06433 100644 --- a/frameworks/Java/smart-socket/src/main/java/org/smartboot/http/SingleQueryHandler.java +++ b/frameworks/Java/smart-socket/src/main/java/org/smartboot/http/SingleQueryHandler.java @@ -2,7 +2,7 @@ import org.smartboot.http.server.HttpRequest; import org.smartboot.http.server.HttpResponse; -import org.smartboot.http.server.HttpServerHandle; +import org.smartboot.http.server.HttpServerHandler; import javax.sql.DataSource; import java.io.IOException; @@ -16,7 +16,7 @@ * @author 三刀 * @version V1.0 , 2020/6/16 */ -public class SingleQueryHandler extends HttpServerHandle { +public class SingleQueryHandler extends HttpServerHandler { private DataSource dataSource; public SingleQueryHandler(DataSource dataSource) { @@ -24,7 +24,7 @@ public SingleQueryHandler(DataSource dataSource) { } @Override - public void doHandle(HttpRequest httpRequest, HttpResponse response) throws IOException { + public void handle(HttpRequest httpRequest, HttpResponse response) throws IOException { World world = new World(); try (Connection connection = dataSource.getConnection(); PreparedStatement preparedStatement = connection.prepareStatement("SELECT * FROM World WHERE id=?");) { diff --git a/frameworks/Java/smart-socket/src/main/java/org/smartboot/http/UpdateHandler.java b/frameworks/Java/smart-socket/src/main/java/org/smartboot/http/UpdateHandler.java index 51477460746..065bed5a9a6 100644 --- a/frameworks/Java/smart-socket/src/main/java/org/smartboot/http/UpdateHandler.java +++ b/frameworks/Java/smart-socket/src/main/java/org/smartboot/http/UpdateHandler.java @@ -3,7 +3,7 @@ import org.smartboot.http.common.utils.NumberUtils; import org.smartboot.http.server.HttpRequest; import org.smartboot.http.server.HttpResponse; -import org.smartboot.http.server.HttpServerHandle; +import org.smartboot.http.server.HttpServerHandler; import javax.sql.DataSource; import java.io.IOException; @@ -18,7 +18,7 @@ * @author 三刀 * @version V1.0 , 2020/6/16 */ -public class UpdateHandler extends HttpServerHandle { +public class UpdateHandler extends HttpServerHandler { private DataSource dataSource; public UpdateHandler(DataSource dataSource) { @@ -26,7 +26,7 @@ public UpdateHandler(DataSource dataSource) { } @Override - public void doHandle(HttpRequest httpRequest, HttpResponse response) throws IOException { + public void handle(HttpRequest httpRequest, HttpResponse response) throws IOException { int queries = Math.min(Math.max(NumberUtils.toInt(httpRequest.getParameter("queries"), 1), 1), 500); World[] worlds = new World[queries]; StringJoiner updateSql = new StringJoiner( diff --git a/frameworks/Java/smart-socket/src/main/java/org/smartboot/servlet/Bootstrap.java b/frameworks/Java/smart-socket/src/main/java/org/smartboot/servlet/Bootstrap.java index b589d5e1720..3cc870dfbdf 100644 --- a/frameworks/Java/smart-socket/src/main/java/org/smartboot/servlet/Bootstrap.java +++ b/frameworks/Java/smart-socket/src/main/java/org/smartboot/servlet/Bootstrap.java @@ -4,7 +4,7 @@ import org.smartboot.http.server.HttpBootstrap; import org.smartboot.http.server.HttpRequest; import org.smartboot.http.server.HttpResponse; -import org.smartboot.http.server.HttpServerHandle; +import org.smartboot.http.server.HttpServerHandler; import org.smartboot.http.server.impl.Request; import org.smartboot.servlet.conf.ServletInfo; import org.smartboot.socket.StateMachineEnum; @@ -61,9 +61,9 @@ public void stateEvent0(AioSession session, StateMachineEnum stateMachineEnum, T } }); bootstrap.setPort(8080) - .pipeline(new HttpServerHandle() { + .pipeline(new HttpServerHandler() { @Override - public void doHandle(HttpRequest request, HttpResponse response) throws IOException { + public void handle(HttpRequest request, HttpResponse response) throws IOException { containerRuntime.doHandle(request, response); } }) From 75264bedacf1cd54f251c186e591f89007763443 Mon Sep 17 00:00:00 2001 From: Joan Miquel Date: Mon, 14 Jun 2021 18:17:10 +0200 Subject: [PATCH 03/25] [php-ngx] Updateo to Nginx 1.21 (#6630) --- frameworks/PHP/php-ngx/php-ngx-async.dockerfile | 2 +- frameworks/PHP/php-ngx/php-ngx-mysql.dockerfile | 2 +- frameworks/PHP/php-ngx/php-ngx-pgsql.dockerfile | 2 +- frameworks/PHP/php-ngx/php-ngx.dockerfile | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/frameworks/PHP/php-ngx/php-ngx-async.dockerfile b/frameworks/PHP/php-ngx/php-ngx-async.dockerfile index e9552f71d83..8f3c59a3fb6 100644 --- a/frameworks/PHP/php-ngx/php-ngx-async.dockerfile +++ b/frameworks/PHP/php-ngx/php-ngx-async.dockerfile @@ -11,7 +11,7 @@ RUN apt-get update -yqq > /dev/null && \ ADD ./ ./ -ENV NGINX_VERSION 1.20.0 +ENV NGINX_VERSION 1.21.0 RUN git clone -b v0.0.25 --single-branch --depth 1 https://github.com/rryqszq4/ngx_php7.git > /dev/null diff --git a/frameworks/PHP/php-ngx/php-ngx-mysql.dockerfile b/frameworks/PHP/php-ngx/php-ngx-mysql.dockerfile index 6e7d7c778d6..bb3097574a3 100644 --- a/frameworks/PHP/php-ngx/php-ngx-mysql.dockerfile +++ b/frameworks/PHP/php-ngx/php-ngx-mysql.dockerfile @@ -11,7 +11,7 @@ RUN apt-get update -yqq > /dev/null && \ ADD ./ ./ -ENV NGINX_VERSION 1.20.0 +ENV NGINX_VERSION 1.21.0 RUN git clone -b v0.0.25 --single-branch --depth 1 https://github.com/rryqszq4/ngx_php7.git > /dev/null diff --git a/frameworks/PHP/php-ngx/php-ngx-pgsql.dockerfile b/frameworks/PHP/php-ngx/php-ngx-pgsql.dockerfile index 59b545f016e..9ff605eebb0 100644 --- a/frameworks/PHP/php-ngx/php-ngx-pgsql.dockerfile +++ b/frameworks/PHP/php-ngx/php-ngx-pgsql.dockerfile @@ -11,7 +11,7 @@ RUN apt-get update -yqq > /dev/null && \ ADD ./ ./ -ENV NGINX_VERSION 1.20.0 +ENV NGINX_VERSION 1.21.0 RUN git clone -b v0.0.25 --single-branch --depth 1 https://github.com/rryqszq4/ngx_php7.git > /dev/null diff --git a/frameworks/PHP/php-ngx/php-ngx.dockerfile b/frameworks/PHP/php-ngx/php-ngx.dockerfile index 1d2fd369c14..bbfd99d5e58 100644 --- a/frameworks/PHP/php-ngx/php-ngx.dockerfile +++ b/frameworks/PHP/php-ngx/php-ngx.dockerfile @@ -10,7 +10,7 @@ RUN apt-get update -yqq > /dev/null && \ php8.0-cli php8.0-dev libphp8.0-embed php8.0-mysql nginx > /dev/null ADD ./ ./ -ENV NGINX_VERSION 1.20.0 +ENV NGINX_VERSION 1.21.0 RUN git clone -b v0.0.25 --single-branch --depth 1 https://github.com/rryqszq4/ngx_php7.git > /dev/null From e33ae563de65a08e7e8c27b21e08ea24cd50bb16 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 14 Jun 2021 09:17:22 -0700 Subject: [PATCH 04/25] Bump django from 3.1.5 to 3.1.12 in /frameworks/Python/django (#6629) Bumps [django](https://github.com/django/django) from 3.1.5 to 3.1.12. - [Release notes](https://github.com/django/django/releases) - [Commits](https://github.com/django/django/compare/3.1.5...3.1.12) --- updated-dependencies: - dependency-name: django dependency-type: direct:production ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- frameworks/Python/django/requirements.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/frameworks/Python/django/requirements.txt b/frameworks/Python/django/requirements.txt index c627e72faca..4993bf98feb 100644 --- a/frameworks/Python/django/requirements.txt +++ b/frameworks/Python/django/requirements.txt @@ -1,4 +1,4 @@ -Django==3.1.5 +Django==3.1.12 greenlet==0.4.17 gunicorn==20.0.4 meinheld==1.0.2 From 9f7be7fb8dd4b411ad65d0f4f5c336b0234ee289 Mon Sep 17 00:00:00 2001 From: Nino Floris Date: Tue, 15 Jun 2021 21:33:21 +0200 Subject: [PATCH 05/25] Disable log subsystem (#6628) --- toolset/databases/postgres/postgresql.conf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/toolset/databases/postgres/postgresql.conf b/toolset/databases/postgres/postgresql.conf index 390afdddd8b..3e695aab4b0 100644 --- a/toolset/databases/postgres/postgresql.conf +++ b/toolset/databases/postgres/postgresql.conf @@ -294,7 +294,7 @@ max_wal_senders = 0 # max number of walsender processes # - Where to Log - -#log_destination = 'stderr' # Valid values are combinations of +log_destination = '' # Valid values are combinations of # stderr, csvlog, syslog, and eventlog, # depending on platform. csvlog # requires logging_collector to be on. From 11854537a6f69f216555ff47ea42f4275582f65d Mon Sep 17 00:00:00 2001 From: LLT21 <43903768+LLT21@users.noreply.github.com> Date: Wed, 16 Jun 2021 16:27:18 +0200 Subject: [PATCH 06/25] Shorter db loop cycle (0.001 millisecond) + plaintext correction (#6658) Co-authored-by: LLT21 --- .../appmpower/src/Db/PooledConnections.cs | 46 +++++++------------ .../CSharp/appmpower/src/HttpApplication.cs | 6 +-- .../CSharp/appmpower/src/Kestrel/PlainText.cs | 9 ++-- 3 files changed, 23 insertions(+), 38 deletions(-) diff --git a/frameworks/CSharp/appmpower/src/Db/PooledConnections.cs b/frameworks/CSharp/appmpower/src/Db/PooledConnections.cs index 97bae3e9ed8..e157c60d94d 100644 --- a/frameworks/CSharp/appmpower/src/Db/PooledConnections.cs +++ b/frameworks/CSharp/appmpower/src/Db/PooledConnections.cs @@ -7,53 +7,39 @@ namespace appMpower.Db { public static class PooledConnections { - private static int _maxLoops = 999; + private static bool _connectionsCreated = false; private static byte _createdConnections = 0; private static byte _maxConnections = Math.Min((byte)Environment.ProcessorCount, (byte)21); private static ConcurrentStack _stack = new ConcurrentStack(); + private static TimeSpan _timeSpan = new TimeSpan(10); // 10 = 0.001 millisecond public static async Task GetConnection(string connectionString) { - int i = 0; - PooledConnection pooledConnection; + PooledConnection pooledConnection = null; - if (_createdConnections < _maxConnections) + if (_connectionsCreated) + { + while (!_stack.TryPop(out pooledConnection)) + { + await Task.Delay(_timeSpan); + } + + return pooledConnection; + } + else { pooledConnection = new PooledConnection(); pooledConnection.OdbcConnection = new OdbcConnection(connectionString); _createdConnections++; + + if (_createdConnections == _maxConnections) _connectionsCreated = true; + pooledConnection.Number = _createdConnections; pooledConnection.PooledCommands = new ConcurrentDictionary(); //Console.WriteLine("opened connection number: " + pooledConnection.Number); return pooledConnection; } - else - { - while (!_stack.TryPop(out pooledConnection) && i < _maxLoops) - { - if (i < 5) await Task.Delay(1); - else if (i < 10) await Task.Delay(2); - else if (i < 25) await Task.Delay(3); - else if (i < 50) await Task.Delay(4); - else if (i < 100) await Task.Delay(5); - else if (i < 500) await Task.Delay(10); - else await Task.Delay(20); - - i++; - //Console.WriteLine("waiting: " + i); - } - - if (i < _maxLoops) - { - //Console.WriteLine("opened connection number: " + pooledConnection.Number); - return pooledConnection; - } - else - { - throw new Exception("No connections are available"); - } - } } public static void ReleaseConnection(PooledConnection pooledConnection) diff --git a/frameworks/CSharp/appmpower/src/HttpApplication.cs b/frameworks/CSharp/appmpower/src/HttpApplication.cs index ac388f9d9b9..7b72dae1385 100644 --- a/frameworks/CSharp/appmpower/src/HttpApplication.cs +++ b/frameworks/CSharp/appmpower/src/HttpApplication.cs @@ -1,16 +1,16 @@ using System; +using System.Text; using System.Threading.Tasks; using Microsoft.AspNetCore.Hosting.Server; using Microsoft.AspNetCore.Http; using Microsoft.AspNetCore.Http.Features; using appMpower.Kestrel; -using PlatformBenchmarks; namespace appMpower { public class HttpApplication : IHttpApplication { - private readonly static AsciiString _plainText = "Hello, World!"; + public static readonly byte[] _plainText = Encoding.UTF8.GetBytes("Hello, World!"); private readonly static JsonMessageSerializer _jsonMessageSerializer = new JsonMessageSerializer(); private readonly static WorldSerializer _worldSerializer = new WorldSerializer(); @@ -34,7 +34,7 @@ public async Task ProcessRequestAsync(IFeatureCollection featureCollection) if (pathStringLength == 10 && pathStringStart == "p") { - PlainText.Render(httpResponse.Headers, httpResponseBody.Writer, _plainText); + await PlainText.RenderAsync(httpResponse.Headers, httpResponseBody.Writer, _plainText); return; } else if (pathStringLength == 5 && pathStringStart == "j") diff --git a/frameworks/CSharp/appmpower/src/Kestrel/PlainText.cs b/frameworks/CSharp/appmpower/src/Kestrel/PlainText.cs index def7bf9fcf2..38dd3844c72 100644 --- a/frameworks/CSharp/appmpower/src/Kestrel/PlainText.cs +++ b/frameworks/CSharp/appmpower/src/Kestrel/PlainText.cs @@ -1,8 +1,9 @@ +using System; +using System.Threading.Tasks; using System.Collections.Generic; using System.IO.Pipelines; using Microsoft.AspNetCore.Http; using Microsoft.Extensions.Primitives; -using PlatformBenchmarks; namespace appMpower.Kestrel { @@ -13,15 +14,13 @@ public static class PlainText private readonly static KeyValuePair _headerContentType = new KeyValuePair("Content-Type", new StringValues("text/plain")); - public static void Render(IHeaderDictionary headerDictionary, PipeWriter pipeWriter, AsciiString utf8String) + public static async Task RenderAsync(IHeaderDictionary headerDictionary, PipeWriter pipeWriter, ReadOnlyMemory utf8String) { headerDictionary.Add(_headerServer); headerDictionary.Add(_headerContentType); headerDictionary.Add(new KeyValuePair("Content-Length", utf8String.Length.ToString())); - var bufferWriter = new BufferWriter(new WriterAdapter(pipeWriter), 208); - bufferWriter.Write(utf8String); - bufferWriter.Commit(); + return await pipeWriter.WriteAsync(utf8String); } } } \ No newline at end of file From 1e45c83008a2b1b8d44556283dffa3d96de7e650 Mon Sep 17 00:00:00 2001 From: Nate Date: Wed, 16 Jun 2021 13:49:28 -0700 Subject: [PATCH 07/25] Revert "Disable log subsystem (#6628)" (#6661) This reverts commit 9f7be7fb8dd4b411ad65d0f4f5c336b0234ee289. --- toolset/databases/postgres/postgresql.conf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/toolset/databases/postgres/postgresql.conf b/toolset/databases/postgres/postgresql.conf index 3e695aab4b0..390afdddd8b 100644 --- a/toolset/databases/postgres/postgresql.conf +++ b/toolset/databases/postgres/postgresql.conf @@ -294,7 +294,7 @@ max_wal_senders = 0 # max number of walsender processes # - Where to Log - -log_destination = '' # Valid values are combinations of +#log_destination = 'stderr' # Valid values are combinations of # stderr, csvlog, syslog, and eventlog, # depending on platform. csvlog # requires logging_collector to be on. From d02cb6c8453d9c102fb9d669cdb2692a01abbf70 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 16 Jun 2021 19:28:13 -0700 Subject: [PATCH 08/25] Bump undertow-core in /frameworks/Java/undertow-jersey (#6660) Bumps [undertow-core](https://github.com/undertow-io/undertow) from 2.0.21.Final to 2.0.23.Final. - [Release notes](https://github.com/undertow-io/undertow/releases) - [Commits](https://github.com/undertow-io/undertow/compare/2.0.21.Final...2.0.23.Final) --- updated-dependencies: - dependency-name: io.undertow:undertow-core dependency-type: direct:production ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- frameworks/Java/undertow-jersey/pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/frameworks/Java/undertow-jersey/pom.xml b/frameworks/Java/undertow-jersey/pom.xml index aa0933698d3..8b421dbc298 100644 --- a/frameworks/Java/undertow-jersey/pom.xml +++ b/frameworks/Java/undertow-jersey/pom.xml @@ -174,7 +174,7 @@ io.undertow undertow-core - 2.0.21.Final + 2.0.23.Final From 99a9502387d330f0b4f2c067f8b5c11314a82df1 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 16 Jun 2021 19:28:25 -0700 Subject: [PATCH 09/25] Bump undertow-core in /frameworks/Java/light-java (#6659) Bumps [undertow-core](https://github.com/undertow-io/undertow) from 2.0.21.Final to 2.0.23.Final. - [Release notes](https://github.com/undertow-io/undertow/releases) - [Commits](https://github.com/undertow-io/undertow/compare/2.0.21.Final...2.0.23.Final) --- updated-dependencies: - dependency-name: io.undertow:undertow-core dependency-type: direct:production ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- frameworks/Java/light-java/pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/frameworks/Java/light-java/pom.xml b/frameworks/Java/light-java/pom.xml index 0d5743424b5..61f91db74ec 100644 --- a/frameworks/Java/light-java/pom.xml +++ b/frameworks/Java/light-java/pom.xml @@ -25,7 +25,7 @@ 11 2.0.1 1.2.3 - 2.0.21.Final + 2.0.23.Final 3.3.1 8.0.18 42.2.5 From 4cba05ef016897d6be4614ca118b1d1eadc8cf92 Mon Sep 17 00:00:00 2001 From: Sam Bull Date: Mon, 21 Jun 2021 20:58:41 +0100 Subject: [PATCH 10/25] aiohttp improvements (#6663) * Update aiohttp-pg-raw.dockerfile * Update aiohttp.dockerfile * Update fortune.jinja * Update fortune.jinja * Update views.py * Update requirements.txt * Update views.py * Update views.py * Update views.py * Update views.py * Update views.py * Update views.py * Update views.py * Update views.py * Update views.py * Update views.py * Update views.py * Update views.py * Update views.py * Update views.py --- .../Python/aiohttp/aiohttp-pg-raw.dockerfile | 2 +- frameworks/Python/aiohttp/aiohttp.dockerfile | 2 +- .../aiohttp/app/templates/fortune.jinja | 18 ++---- frameworks/Python/aiohttp/app/views.py | 57 +++++++++---------- frameworks/Python/aiohttp/requirements.txt | 12 ++-- 5 files changed, 39 insertions(+), 52 deletions(-) diff --git a/frameworks/Python/aiohttp/aiohttp-pg-raw.dockerfile b/frameworks/Python/aiohttp/aiohttp-pg-raw.dockerfile index f53ad52f1ed..ddcb349006a 100644 --- a/frameworks/Python/aiohttp/aiohttp-pg-raw.dockerfile +++ b/frameworks/Python/aiohttp/aiohttp-pg-raw.dockerfile @@ -4,7 +4,7 @@ ADD ./ /aiohttp WORKDIR aiohttp -RUN pip3 install cython==0.29.13 && \ +RUN pip3 install cython==0.29.23 && \ pip3 install -r /aiohttp/requirements.txt ENV CONNECTION=RAW diff --git a/frameworks/Python/aiohttp/aiohttp.dockerfile b/frameworks/Python/aiohttp/aiohttp.dockerfile index bda2d02426e..b817ec61264 100644 --- a/frameworks/Python/aiohttp/aiohttp.dockerfile +++ b/frameworks/Python/aiohttp/aiohttp.dockerfile @@ -4,7 +4,7 @@ ADD ./ /aiohttp WORKDIR aiohttp -RUN pip3 install cython==0.29.13 && \ +RUN pip3 install cython==0.29.23 && \ pip3 install -r /aiohttp/requirements.txt WORKDIR /aiohttp diff --git a/frameworks/Python/aiohttp/app/templates/fortune.jinja b/frameworks/Python/aiohttp/app/templates/fortune.jinja index aeaec49ccf9..5126edffbf7 100644 --- a/frameworks/Python/aiohttp/app/templates/fortune.jinja +++ b/frameworks/Python/aiohttp/app/templates/fortune.jinja @@ -1,20 +1,10 @@ - -Fortunes - +Fortunes - - - - -{% for fortune in fortunes %} - - - - -{% endfor %} -
idmessage
{{ fortune.id }}{{ fortune.message|escape }}
+idmessage +{% for fortune in fortunes %}{{ fortune.id }}{{ fortune.message|e }} +{% endfor %} diff --git a/frameworks/Python/aiohttp/app/views.py b/frameworks/Python/aiohttp/app/views.py index a47db01fc90..8fd659aa93f 100644 --- a/frameworks/Python/aiohttp/app/views.py +++ b/frameworks/Python/aiohttp/app/views.py @@ -10,9 +10,17 @@ from .models import sa_fortunes, sa_worlds, Fortune, World +ADDITIONAL_FORTUNE_ORM = Fortune(id=0, message='Additional fortune added at request time.') +ADDITIONAL_FORTUNE_ROW = {'id': 0, 'message': 'Additional fortune added at request time.'} +READ_ROW_SQL = 'SELECT "randomnumber", "id" FROM "world" WHERE id = $1' +READ_SELECT_ORM = select(World.randomnumber) +WRITE_ROW_SQL = 'UPDATE "world" SET "randomnumber"=$2 WHERE id=$1' + json_response = partial(json_response, dumps=ujson.dumps) template_path = Path(__file__).parent / 'templates' / 'fortune.jinja' template = jinja2.Template(template_path.read_text()) +sort_fortunes_orm = attrgetter('message') +sort_fortunes_raw = itemgetter('message') def get_num_queries(request): @@ -62,12 +70,11 @@ async def multiple_database_queries_orm(request): num_queries = get_num_queries(request) ids = [randint(1, 10000) for _ in range(num_queries)] - ids.sort() result = [] async with request.app['db_session']() as sess: for id_ in ids: - num = await sess.scalar(select(World.randomnumber).filter_by(id=id_)) + num = await sess.scalar(READ_SELECT_ORM.filter_by(id=id_)) result.append({'id': id_, 'randomNumber': num}) return json_response(result) @@ -79,11 +86,10 @@ async def multiple_database_queries_raw(request): num_queries = get_num_queries(request) ids = [randint(1, 10000) for _ in range(num_queries)] - ids.sort() result = [] async with request.app['pg'].acquire() as conn: - stmt = await conn.prepare('SELECT id,randomnumber FROM world WHERE id = $1') + stmt = await conn.prepare(READ_ROW_SQL) for id_ in ids: result.append({ 'id': id_, @@ -99,8 +105,8 @@ async def fortunes(request): async with request.app['db_session']() as sess: ret = await sess.execute(select(Fortune.id, Fortune.message)) fortunes = ret.all() - fortunes.append(Fortune(id=0, message='Additional fortune added at request time.')) - fortunes.sort(key=attrgetter('message')) + fortunes.append(ADDITIONAL_FORTUNE_ORM) + fortunes.sort(key=sort_fortunes_orm) content = template.render(fortunes=fortunes) return Response(text=content, content_type='text/html') @@ -111,8 +117,8 @@ async def fortunes_raw(request): """ async with request.app['pg'].acquire() as conn: fortunes = await conn.fetch('SELECT * FROM Fortune') - fortunes.append(dict(id=0, message='Additional fortune added at request time.')) - fortunes.sort(key=itemgetter('message')) + fortunes.append(ADDITIONAL_FORTUNE_ROW) + fortunes.sort(key=sort_fortunes_raw) content = template.render(fortunes=fortunes) return Response(text=content, content_type='text/html') @@ -122,42 +128,33 @@ async def updates(request): Test 5 ORM """ num_queries = get_num_queries(request) - result = [] - - ids = [randint(1, 10000) for _ in range(num_queries)] - ids.sort() + updates = [(randint(1, 10000), randint(1, 10000)) for _ in range(num_queries)] + updates.sort() + worlds = [{'id': row_id, 'randomNumber': number} for row_id, number in updates] async with request.app['db_session'].begin() as sess: - for id_ in ids: - rand_new = randint(1, 10000) + for id_, number in updates: world = await sess.get(World, id_, populate_existing=True) - world.randomnumber = rand_new - - result.append({'id': id_, 'randomNumber': rand_new}) - return json_response(result) + world.randomnumber = number + return json_response(worlds) async def updates_raw(request): """ Test 5 RAW """ num_queries = get_num_queries(request) + updates = [(randint(1, 10000), randint(1, 10000)) for _ in range(num_queries)] + updates.sort() + worlds = [{'id': row_id, 'randomNumber': number} for row_id, number in updates] - ids = [randint(1, 10000) for _ in range(num_queries)] - ids.sort() - - result = [] - updates = [] async with request.app['pg'].acquire() as conn: - stmt = await conn.prepare('SELECT id,randomnumber FROM world WHERE id = $1') - for id_ in ids: + stmt = await conn.prepare(READ_ROW_SQL) + for id_, _ in updates: # the result of this is the int previous random number which we don't actually use await stmt.fetchval(id_) - rand_new = randint(1, 10000) - result.append({'id': id_, 'randomNumber': rand_new}) - updates.append((rand_new, id_)) - await conn.executemany('UPDATE world SET randomnumber=$1 WHERE id=$2', updates) + await conn.executemany(WRITE_ROW_SQL, updates) - return json_response(result) + return json_response(worlds) async def plaintext(request): diff --git a/frameworks/Python/aiohttp/requirements.txt b/frameworks/Python/aiohttp/requirements.txt index 7c18972a172..e305581fc51 100644 --- a/frameworks/Python/aiohttp/requirements.txt +++ b/frameworks/Python/aiohttp/requirements.txt @@ -1,9 +1,9 @@ aiohttp==3.7.4 -asyncpg==0.22.0 +asyncpg==0.23.0 cchardet==2.1.7 -gunicorn==20.0.4 -jinja2==2.11.3 +gunicorn==20.1 +jinja2==3.0.1 psycopg2==2.8.6 -SQLAlchemy==1.4.0b3 -ujson==2.0.3 -uvloop==0.14.0 +SQLAlchemy==1.4.18 +ujson==4.0.2 +uvloop==0.15.2 From ff8615a94c2bddbce81adf264d4ffb1e873aaee3 Mon Sep 17 00:00:00 2001 From: Redkale Date: Tue, 22 Jun 2021 03:58:58 +0800 Subject: [PATCH 11/25] Update pom.xml (#6665) --- frameworks/Java/redkale/pom.xml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/frameworks/Java/redkale/pom.xml b/frameworks/Java/redkale/pom.xml index b15f60fe9fe..ef9b543fbd9 100644 --- a/frameworks/Java/redkale/pom.xml +++ b/frameworks/Java/redkale/pom.xml @@ -18,13 +18,13 @@ org.redkale redkale - 2.4.0 + 2.5.0-SNAPSHOT org.redkalex redkale-plugins - 2.4.0 + 2.5.0-SNAPSHOT From b85ed88d3b736845b4b2b7226d39e6a2a76977b6 Mon Sep 17 00:00:00 2001 From: Lukas Eder Date: Tue, 22 Jun 2021 17:31:05 +0200 Subject: [PATCH 12/25] Fixed broken links (#6672) --- frameworks/Java/play2-java/play2-java-jooq-hikaricp/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/frameworks/Java/play2-java/play2-java-jooq-hikaricp/README.md b/frameworks/Java/play2-java/play2-java-jooq-hikaricp/README.md index 9fe94b4dbde..5def045ec51 100644 --- a/frameworks/Java/play2-java/play2-java-jooq-hikaricp/README.md +++ b/frameworks/Java/play2-java/play2-java-jooq-hikaricp/README.md @@ -5,8 +5,8 @@ This is the Play portion of a [benchmarking test suite](../) comparing a variety ### Data-Store/Database Mapping Test * [Database test controller](app/controllers/Application.java) -* [Database World test model](app/models/World.java) -* [Database Fortune test model](app/models/Fortune.java) +* [Database World test model](app/models/tables/World.java) +* [Database Fortune test model](app/models/tables/Fortune.java) ## Infrastructure Software Versions The tests were run with: From 180c2e1fe3e01e103c979794d7c9f051d0fbcf36 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Diogo=20Magalh=C3=A3es=20Martins?= Date: Tue, 22 Jun 2021 12:31:34 -0300 Subject: [PATCH 13/25] Adding Python / async-worker framework (#6602) * Adding async-worker framework tests 1 and 6 * Fixing url path --- frameworks/Python/async-worker/Pipfile | 17 + frameworks/Python/async-worker/Pipfile.lock | 384 ++++++++++++++++++ frameworks/Python/async-worker/README.md | 14 + .../async-worker/async-worker.dockerfile | 19 + .../Python/async-worker/benchmark_config.json | 20 + .../Python/async-worker/src/__init__.py | 0 .../Python/async-worker/src/hello_world.py | 18 + 7 files changed, 472 insertions(+) create mode 100644 frameworks/Python/async-worker/Pipfile create mode 100644 frameworks/Python/async-worker/Pipfile.lock create mode 100644 frameworks/Python/async-worker/README.md create mode 100644 frameworks/Python/async-worker/async-worker.dockerfile create mode 100644 frameworks/Python/async-worker/benchmark_config.json create mode 100644 frameworks/Python/async-worker/src/__init__.py create mode 100644 frameworks/Python/async-worker/src/hello_world.py diff --git a/frameworks/Python/async-worker/Pipfile b/frameworks/Python/async-worker/Pipfile new file mode 100644 index 00000000000..0cf91e337f9 --- /dev/null +++ b/frameworks/Python/async-worker/Pipfile @@ -0,0 +1,17 @@ +[[source]] +url = "https://pypi.org/simple" +verify_ssl = true +name = "pypi" + +[packages] +async-worker = "==0.19.1" +cchardet = "==2.1.7" + +[dev-packages] +black = "*" + +[requires] +python_version = "3.9" + +[pipenv] +allow_prereleases = true diff --git a/frameworks/Python/async-worker/Pipfile.lock b/frameworks/Python/async-worker/Pipfile.lock new file mode 100644 index 00000000000..e1c3bbb5bbc --- /dev/null +++ b/frameworks/Python/async-worker/Pipfile.lock @@ -0,0 +1,384 @@ +{ + "_meta": { + "hash": { + "sha256": "872880755bbf439ce862323482273dd3539044167e90014909fbfe789adaa25a" + }, + "pipfile-spec": 6, + "requires": { + "python_version": "3.9" + }, + "sources": [ + { + "name": "pypi", + "url": "https://pypi.org/simple", + "verify_ssl": true + } + ] + }, + "default": { + "aioamqp": { + "hashes": [ + "sha256:55fa703a70e71bc958ad546b9ee0c68387cab366c82fc44c0742d6ad0303745a", + "sha256:eef5c23a7fedee079d8326406f5c7a5725dfe36c359373da3499fffa16f79915" + ], + "version": "==0.14.0" + }, + "aiohttp": { + "hashes": [ + "sha256:119feb2bd551e58d83d1b38bfa4cb921af8ddedec9fad7183132db334c3133e0", + "sha256:16d0683ef8a6d803207f02b899c928223eb219111bd52420ef3d7a8aa76227b6", + "sha256:2eb3efe243e0f4ecbb654b08444ae6ffab37ac0ef8f69d3a2ffb958905379daf", + "sha256:2ffea7904e70350da429568113ae422c88d2234ae776519549513c8f217f58a9", + "sha256:40bd1b101b71a18a528ffce812cc14ff77d4a2a1272dfb8b11b200967489ef3e", + "sha256:418597633b5cd9639e514b1d748f358832c08cd5d9ef0870026535bd5eaefdd0", + "sha256:481d4b96969fbfdcc3ff35eea5305d8565a8300410d3d269ccac69e7256b1329", + "sha256:4c1bdbfdd231a20eee3e56bd0ac1cd88c4ff41b64ab679ed65b75c9c74b6c5c2", + "sha256:5563ad7fde451b1986d42b9bb9140e2599ecf4f8e42241f6da0d3d624b776f40", + "sha256:58c62152c4c8731a3152e7e650b29ace18304d086cb5552d317a54ff2749d32a", + "sha256:5b50e0b9460100fe05d7472264d1975f21ac007b35dcd6fd50279b72925a27f4", + "sha256:5d84ecc73141d0a0d61ece0742bb7ff5751b0657dab8405f899d3ceb104cc7de", + "sha256:5dde6d24bacac480be03f4f864e9a67faac5032e28841b00533cd168ab39cad9", + "sha256:5e91e927003d1ed9283dee9abcb989334fc8e72cf89ebe94dc3e07e3ff0b11e9", + "sha256:62bc216eafac3204877241569209d9ba6226185aa6d561c19159f2e1cbb6abfb", + "sha256:6c8200abc9dc5f27203986100579fc19ccad7a832c07d2bc151ce4ff17190076", + "sha256:6ca56bdfaf825f4439e9e3673775e1032d8b6ea63b8953d3812c71bd6a8b81de", + "sha256:71680321a8a7176a58dfbc230789790639db78dad61a6e120b39f314f43f1907", + "sha256:7c7820099e8b3171e54e7eedc33e9450afe7cd08172632d32128bd527f8cb77d", + "sha256:7dbd087ff2f4046b9b37ba28ed73f15fd0bc9f4fdc8ef6781913da7f808d9536", + "sha256:822bd4fd21abaa7b28d65fc9871ecabaddc42767884a626317ef5b75c20e8a2d", + "sha256:8ec1a38074f68d66ccb467ed9a673a726bb397142c273f90d4ba954666e87d54", + "sha256:950b7ef08b2afdab2488ee2edaff92a03ca500a48f1e1aaa5900e73d6cf992bc", + "sha256:99c5a5bf7135607959441b7d720d96c8e5c46a1f96e9d6d4c9498be8d5f24212", + "sha256:b84ad94868e1e6a5e30d30ec419956042815dfaea1b1df1cef623e4564c374d9", + "sha256:bc3d14bf71a3fb94e5acf5bbf67331ab335467129af6416a437bd6024e4f743d", + "sha256:c2a80fd9a8d7e41b4e38ea9fe149deed0d6aaede255c497e66b8213274d6d61b", + "sha256:c44d3c82a933c6cbc21039326767e778eface44fca55c65719921c4b9661a3f7", + "sha256:cc31e906be1cc121ee201adbdf844522ea3349600dd0a40366611ca18cd40e81", + "sha256:d5d102e945ecca93bcd9801a7bb2fa703e37ad188a2f81b1e65e4abe4b51b00c", + "sha256:dd7936f2a6daa861143e376b3a1fb56e9b802f4980923594edd9ca5670974895", + "sha256:dee68ec462ff10c1d836c0ea2642116aba6151c6880b688e56b4c0246770f297", + "sha256:e76e78863a4eaec3aee5722d85d04dcbd9844bc6cd3bfa6aa880ff46ad16bfcb", + "sha256:eab51036cac2da8a50d7ff0ea30be47750547c9aa1aa2cf1a1b710a1827e7dbe", + "sha256:f4496d8d04da2e98cc9133e238ccebf6a13ef39a93da2e87146c8c8ac9768242", + "sha256:fbd3b5e18d34683decc00d9a360179ac1e7a320a5fee10ab8053ffd6deab76e0", + "sha256:feb24ff1226beeb056e247cf2e24bba5232519efb5645121c4aea5b6ad74c1f2" + ], + "markers": "python_version >= '3.6'", + "version": "==3.7.4" + }, + "aiologger": { + "hashes": [ + "sha256:5e1493a2c9819a5d751b9e775c79b9afe628387ee80a1d5e91ff719bb2f511b9" + ], + "version": "==0.5.0" + }, + "async-timeout": { + "hashes": [ + "sha256:0c3c816a028d47f659d6ff5c745cb2acf1f966da1fe5c19c77a70282b25f4c5f", + "sha256:4291ca197d287d274d0b6cb5d6f8f8f82d434ed288f962539ff18cc9012f9ea3" + ], + "markers": "python_full_version >= '3.5.3'", + "version": "==3.0.1" + }, + "async-worker": { + "hashes": [ + "sha256:ca60175a50430fd06817163d7edb0d19d0978b54fc65d3df59af3663f2c05ba0" + ], + "index": "pypi", + "version": "==0.19.1" + }, + "attrs": { + "hashes": [ + "sha256:149e90d6d8ac20db7a955ad60cf0e6881a3f20d37096140088356da6c716b0b1", + "sha256:ef6aaac3ca6cd92904cdd0d83f629a15f18053ec84e6432106f7a4d04ae4f5fb" + ], + "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'", + "version": "==21.2.0" + }, + "cached-property": { + "hashes": [ + "sha256:3a026f1a54135677e7da5ce819b0c690f156f37976f3e30c5430740725203d7f", + "sha256:9217a59f14a5682da7c4b8829deadbfc194ac22e9908ccf7c8820234e80a1504" + ], + "version": "==1.5.1" + }, + "cchardet": { + "hashes": [ + "sha256:0b859069bbb9d27c78a2c9eb997e6f4b738db2d7039a03f8792b4058d61d1109", + "sha256:228d2533987c450f39acf7548f474dd6814c446e9d6bd228e8f1d9a2d210f10b", + "sha256:2309ff8fc652b0fc3c0cff5dbb172530c7abb92fe9ba2417c9c0bcf688463c1c", + "sha256:24974b3e40fee9e7557bb352be625c39ec6f50bc2053f44a3d1191db70b51675", + "sha256:273699c4e5cd75377776501b72a7b291a988c6eec259c29505094553ee505597", + "sha256:27a9ba87c9f99e0618e1d3081189b1217a7d110e5c5597b0b7b7c3fedd1c340a", + "sha256:302aa443ae2526755d412c9631136bdcd1374acd08e34f527447f06f3c2ddb98", + "sha256:45456c59ec349b29628a3c6bfb86d818ec3a6fbb7eb72de4ff3bd4713681c0e3", + "sha256:48ba829badef61441e08805cfa474ccd2774be2ff44b34898f5854168c596d4d", + "sha256:50ad671e8d6c886496db62c3bd68b8d55060688c655873aa4ce25ca6105409a1", + "sha256:54341e7e1ba9dc0add4c9d23b48d3a94e2733065c13920e85895f944596f6150", + "sha256:54d0b26fd0cd4099f08fb9c167600f3e83619abefeaa68ad823cc8ac1f7bcc0c", + "sha256:5a25f9577e9bebe1a085eec2d6fdd72b7a9dd680811bba652ea6090fb2ff472f", + "sha256:6b6397d8a32b976a333bdae060febd39ad5479817fabf489e5596a588ad05133", + "sha256:70eeae8aaf61192e9b247cf28969faef00578becd2602526ecd8ae7600d25e0e", + "sha256:80e6faae75ecb9be04a7b258dc4750d459529debb6b8dee024745b7b5a949a34", + "sha256:90086e5645f8a1801350f4cc6cb5d5bf12d3fa943811bb08667744ec1ecc9ccd", + "sha256:a39526c1c526843965cec589a6f6b7c2ab07e3e56dc09a7f77a2be6a6afa4636", + "sha256:b154effa12886e9c18555dfc41a110f601f08d69a71809c8d908be4b1ab7314f", + "sha256:b59ddc615883835e03c26f81d5fc3671fab2d32035c87f50862de0da7d7db535", + "sha256:bd7f262f41fd9caf5a5f09207a55861a67af6ad5c66612043ed0f81c58cdf376", + "sha256:c428b6336545053c2589f6caf24ea32276c6664cb86db817e03a94c60afa0eaf", + "sha256:c6f70139aaf47ffb94d89db603af849b82efdf756f187cdd3e566e30976c519f", + "sha256:c96aee9ebd1147400e608a3eff97c44f49811f8904e5a43069d55603ac4d8c97", + "sha256:ec3eb5a9c475208cf52423524dcaf713c394393e18902e861f983c38eeb77f18", + "sha256:eee4f5403dc3a37a1ca9ab87db32b48dc7e190ef84601068f45397144427cc5e", + "sha256:f16517f3697569822c6d09671217fdeab61dfebc7acb5068634d6b0728b86c0b", + "sha256:f86e0566cb61dc4397297696a4a1b30f6391b50bc52b4f073507a48466b6255a", + "sha256:fdac1e4366d0579fff056d1280b8dc6348be964fda8ebb627c0269e097ab37fa" + ], + "index": "pypi", + "version": "==2.1.7" + }, + "chardet": { + "hashes": [ + "sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae", + "sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691" + ], + "version": "==3.0.4" + }, + "idna": { + "hashes": [ + "sha256:5205d03e7bcbb919cc9c19885f9920d622ca52448306f2377daede5cf3faac16", + "sha256:c5b02147e01ea9920e6b0a3f1f7bb833612d507592c837a6c49552768f4054e1" + ], + "markers": "python_version >= '3.4'", + "version": "==3.1" + }, + "multidict": { + "hashes": [ + "sha256:018132dbd8688c7a69ad89c4a3f39ea2f9f33302ebe567a879da8f4ca73f0d0a", + "sha256:051012ccee979b2b06be928a6150d237aec75dd6bf2d1eeeb190baf2b05abc93", + "sha256:05c20b68e512166fddba59a918773ba002fdd77800cad9f55b59790030bab632", + "sha256:07b42215124aedecc6083f1ce6b7e5ec5b50047afa701f3442054373a6deb656", + "sha256:0e3c84e6c67eba89c2dbcee08504ba8644ab4284863452450520dad8f1e89b79", + "sha256:0e929169f9c090dae0646a011c8b058e5e5fb391466016b39d21745b48817fd7", + "sha256:1ab820665e67373de5802acae069a6a05567ae234ddb129f31d290fc3d1aa56d", + "sha256:25b4e5f22d3a37ddf3effc0710ba692cfc792c2b9edfb9c05aefe823256e84d5", + "sha256:2e68965192c4ea61fff1b81c14ff712fc7dc15d2bd120602e4a3494ea6584224", + "sha256:2f1a132f1c88724674271d636e6b7351477c27722f2ed789f719f9e3545a3d26", + "sha256:37e5438e1c78931df5d3c0c78ae049092877e5e9c02dd1ff5abb9cf27a5914ea", + "sha256:3a041b76d13706b7fff23b9fc83117c7b8fe8d5fe9e6be45eee72b9baa75f348", + "sha256:3a4f32116f8f72ecf2a29dabfb27b23ab7cdc0ba807e8459e59a93a9be9506f6", + "sha256:46c73e09ad374a6d876c599f2328161bcd95e280f84d2060cf57991dec5cfe76", + "sha256:46dd362c2f045095c920162e9307de5ffd0a1bfbba0a6e990b344366f55a30c1", + "sha256:4b186eb7d6ae7c06eb4392411189469e6a820da81447f46c0072a41c748ab73f", + "sha256:54fd1e83a184e19c598d5e70ba508196fd0bbdd676ce159feb412a4a6664f952", + "sha256:585fd452dd7782130d112f7ddf3473ffdd521414674c33876187e101b588738a", + "sha256:5cf3443199b83ed9e955f511b5b241fd3ae004e3cb81c58ec10f4fe47c7dce37", + "sha256:6a4d5ce640e37b0efcc8441caeea8f43a06addace2335bd11151bc02d2ee31f9", + "sha256:7df80d07818b385f3129180369079bd6934cf70469f99daaebfac89dca288359", + "sha256:806068d4f86cb06af37cd65821554f98240a19ce646d3cd24e1c33587f313eb8", + "sha256:830f57206cc96ed0ccf68304141fec9481a096c4d2e2831f311bde1c404401da", + "sha256:929006d3c2d923788ba153ad0de8ed2e5ed39fdbe8e7be21e2f22ed06c6783d3", + "sha256:9436dc58c123f07b230383083855593550c4d301d2532045a17ccf6eca505f6d", + "sha256:9dd6e9b1a913d096ac95d0399bd737e00f2af1e1594a787e00f7975778c8b2bf", + "sha256:ace010325c787c378afd7f7c1ac66b26313b3344628652eacd149bdd23c68841", + "sha256:b47a43177a5e65b771b80db71e7be76c0ba23cc8aa73eeeb089ed5219cdbe27d", + "sha256:b797515be8743b771aa868f83563f789bbd4b236659ba52243b735d80b29ed93", + "sha256:b7993704f1a4b204e71debe6095150d43b2ee6150fa4f44d6d966ec356a8d61f", + "sha256:d5c65bdf4484872c4af3150aeebe101ba560dcfb34488d9a8ff8dbcd21079647", + "sha256:d81eddcb12d608cc08081fa88d046c78afb1bf8107e6feab5d43503fea74a635", + "sha256:dc862056f76443a0db4509116c5cd480fe1b6a2d45512a653f9a855cc0517456", + "sha256:ecc771ab628ea281517e24fd2c52e8f31c41e66652d07599ad8818abaad38cda", + "sha256:f200755768dc19c6f4e2b672421e0ebb3dd54c38d5a4f262b872d8cfcc9e93b5", + "sha256:f21756997ad8ef815d8ef3d34edd98804ab5ea337feedcd62fb52d22bf531281", + "sha256:fc13a9524bc18b6fb6e0dbec3533ba0496bbed167c56d0aabefd965584557d80" + ], + "markers": "python_version >= '3.6'", + "version": "==5.1.0" + }, + "pamqp": { + "hashes": [ + "sha256:2f81b5c186f668a67f165193925b6bfd83db4363a6222f599517f29ecee60b02", + "sha256:5cd0f5a85e89f20d5f8e19285a1507788031cfca4a9ea6f067e3cf18f5e294e8" + ], + "version": "==2.3.0" + }, + "prometheus-client": { + "hashes": [ + "sha256:71cd24a2b3eb335cb800c7159f423df1bd4dcd5171b234be15e3f31ec9f622da" + ], + "version": "==0.7.1" + }, + "pydantic": { + "hashes": [ + "sha256:0b71ca069c16470cb00be0acaf0657eb74cbc4ff5f11b42e79647f170956cda3", + "sha256:12ed0b175bba65e29dfc5859cd539d3512f58bb776bf620a3d3338501fd0f389", + "sha256:22fe5756c6c57279234e4c4027a3549507aca29e9ee832d6aa39c367cb43c99f", + "sha256:26821f61623b01d618bd8b3243f790ac8bd7ae31b388c0e41aa586002cf350eb", + "sha256:2bc9e9f5d91a29dec53346efc5c719d82297885d89c8a62b971492fba222c68d", + "sha256:42b8fb1e4e4783c4aa31df44b64714f96aa4deeacbacf3713a8a238ee7df3b2b", + "sha256:4a83d24bcf9ce8e6fa55c379bba1359461eedb85721bfb3151e240871e2b13a8", + "sha256:5759a4b276bda5ac2360f00e9b1e711aaac51fabd155b422d27f3339710f4264", + "sha256:77e04800d19acc2a8fbb95fe3d47ff397ce137aa5a2b32cc23a87bac70dda343", + "sha256:865410a6df71fb60294887770d19c67d499689f7ce64245182653952cdbd4183", + "sha256:91baec8ed771d4c53d71ef549d8e36b0f92a31c32296062d562d1d7074dd1d6e", + "sha256:999cc108933425752e45d1bf2f57d3cf091f2a5e8b9b8afab5b8872d2cc7645f", + "sha256:a0ff36e3f929d76b91d1624c6673dbdc1407358700d117bb7f29d5696c52d288", + "sha256:a989924324513215ad2b2cfd187426e6372f76f507b17361142c0b792294960c", + "sha256:ad2fae68e185cfae5b6d83e7915352ff0b6e5fa84d84bc6a94c3e2de58327114", + "sha256:b4e03c84f4e96e3880c9d34508cccbd0f0df6e7dc14b17f960ea8c71448823a3", + "sha256:c26d380af3e9a8eb9abe3b6337cea28f057b5425330817c918cf74d0a0a2303d", + "sha256:c8a3600435b83a4f28f5379f3bb574576521180f691828268268e9f172f1b1eb", + "sha256:ccc2ab0a240d01847f3d5f0f9e1582d450a2fc3389db33a7af8e7447b205a935", + "sha256:d361d181a3fb53ebfdc2fb1e3ca55a6b2ad717578a5e119c99641afd11b31a47", + "sha256:d5aeab86837f8799df0d84bec1190e6cc0062d5c5374636b5599234f2b39fe0a", + "sha256:edf37d30ea60179ef067add9772cf42299ea6cd490b3c94335a68f1021944ac4" + ], + "markers": "python_full_version >= '3.6.1'", + "version": "==1.8" + }, + "typing-extensions": { + "hashes": [ + "sha256:0ac0f89795dd19de6b97debb0c6af1c70987fd80a2d62d1958f7e56fcc31b497", + "sha256:50b6f157849174217d0656f99dc82fe932884fb250826c18350e159ec6cdf342", + "sha256:779383f6086d90c99ae41cf0ff39aac8a7937a9283ce0a414e5dd782f4c94a84" + ], + "version": "==3.10.0.0" + }, + "yarl": { + "hashes": [ + "sha256:00d7ad91b6583602eb9c1d085a2cf281ada267e9a197e8b7cae487dadbfa293e", + "sha256:0355a701b3998dcd832d0dc47cc5dedf3874f966ac7f870e0f3a6788d802d434", + "sha256:15263c3b0b47968c1d90daa89f21fcc889bb4b1aac5555580d74565de6836366", + "sha256:2ce4c621d21326a4a5500c25031e102af589edb50c09b321049e388b3934eec3", + "sha256:31ede6e8c4329fb81c86706ba8f6bf661a924b53ba191b27aa5fcee5714d18ec", + "sha256:324ba3d3c6fee56e2e0b0d09bf5c73824b9f08234339d2b788af65e60040c959", + "sha256:329412812ecfc94a57cd37c9d547579510a9e83c516bc069470db5f75684629e", + "sha256:4736eaee5626db8d9cda9eb5282028cc834e2aeb194e0d8b50217d707e98bb5c", + "sha256:4953fb0b4fdb7e08b2f3b3be80a00d28c5c8a2056bb066169de00e6501b986b6", + "sha256:4c5bcfc3ed226bf6419f7a33982fb4b8ec2e45785a0561eb99274ebbf09fdd6a", + "sha256:547f7665ad50fa8563150ed079f8e805e63dd85def6674c97efd78eed6c224a6", + "sha256:5b883e458058f8d6099e4420f0cc2567989032b5f34b271c0827de9f1079a424", + "sha256:63f90b20ca654b3ecc7a8d62c03ffa46999595f0167d6450fa8383bab252987e", + "sha256:68dc568889b1c13f1e4745c96b931cc94fdd0defe92a72c2b8ce01091b22e35f", + "sha256:69ee97c71fee1f63d04c945f56d5d726483c4762845400a6795a3b75d56b6c50", + "sha256:6d6283d8e0631b617edf0fd726353cb76630b83a089a40933043894e7f6721e2", + "sha256:72a660bdd24497e3e84f5519e57a9ee9220b6f3ac4d45056961bf22838ce20cc", + "sha256:73494d5b71099ae8cb8754f1df131c11d433b387efab7b51849e7e1e851f07a4", + "sha256:7356644cbed76119d0b6bd32ffba704d30d747e0c217109d7979a7bc36c4d970", + "sha256:8a9066529240171b68893d60dca86a763eae2139dd42f42106b03cf4b426bf10", + "sha256:8aa3decd5e0e852dc68335abf5478a518b41bf2ab2f330fe44916399efedfae0", + "sha256:97b5bdc450d63c3ba30a127d018b866ea94e65655efaf889ebeabc20f7d12406", + "sha256:9ede61b0854e267fd565e7527e2f2eb3ef8858b301319be0604177690e1a3896", + "sha256:b2e9a456c121e26d13c29251f8267541bd75e6a1ccf9e859179701c36a078643", + "sha256:b5dfc9a40c198334f4f3f55880ecf910adebdcb2a0b9a9c23c9345faa9185721", + "sha256:bafb450deef6861815ed579c7a6113a879a6ef58aed4c3a4be54400ae8871478", + "sha256:c49ff66d479d38ab863c50f7bb27dee97c6627c5fe60697de15529da9c3de724", + "sha256:ce3beb46a72d9f2190f9e1027886bfc513702d748047b548b05dab7dfb584d2e", + "sha256:d26608cf178efb8faa5ff0f2d2e77c208f471c5a3709e577a7b3fd0445703ac8", + "sha256:d597767fcd2c3dc49d6eea360c458b65643d1e4dbed91361cf5e36e53c1f8c96", + "sha256:d5c32c82990e4ac4d8150fd7652b972216b204de4e83a122546dce571c1bdf25", + "sha256:d8d07d102f17b68966e2de0e07bfd6e139c7c02ef06d3a0f8d2f0f055e13bb76", + "sha256:e46fba844f4895b36f4c398c5af062a9808d1f26b2999c58909517384d5deda2", + "sha256:e6b5460dc5ad42ad2b36cca524491dfcaffbfd9c8df50508bddc354e787b8dc2", + "sha256:f040bcc6725c821a4c0665f3aa96a4d0805a7aaf2caf266d256b8ed71b9f041c", + "sha256:f0b059678fd549c66b89bed03efcabb009075bd131c248ecdf087bdb6faba24a", + "sha256:fcbb48a93e8699eae920f8d92f7160c03567b421bc17362a9ffbbd706a816f71" + ], + "markers": "python_version >= '3.6'", + "version": "==1.6.3" + } + }, + "develop": { + "appdirs": { + "hashes": [ + "sha256:7d5d0167b2b1ba821647616af46a749d1c653740dd0d2415100fe26e27afdf41", + "sha256:a841dacd6b99318a741b166adb07e19ee71a274450e68237b4650ca1055ab128" + ], + "version": "==1.4.4" + }, + "black": { + "hashes": [ + "sha256:23695358dbcb3deafe7f0a3ad89feee5999a46be5fec21f4f1d108be0bcdb3b1", + "sha256:8a60071a0043876a4ae96e6c69bd3a127dad2c1ca7c8083573eb82f92705d008" + ], + "index": "pypi", + "version": "==21.5b1" + }, + "click": { + "hashes": [ + "sha256:8c04c11192119b1ef78ea049e0a6f0463e4c48ef00a30160c704337586f3ad7a", + "sha256:fba402a4a47334742d782209a7c79bc448911afe1149d07bdabdf480b3e2f4b6" + ], + "markers": "python_version >= '3.6'", + "version": "==8.0.1" + }, + "mypy-extensions": { + "hashes": [ + "sha256:090fedd75945a69ae91ce1303b5824f428daf5a028d2f6ab8a299250a846f15d", + "sha256:2d82818f5bb3e369420cb3c4060a7970edba416647068eb4c5343488a6c604a8" + ], + "version": "==0.4.3" + }, + "pathspec": { + "hashes": [ + "sha256:86379d6b86d75816baba717e64b1a3a3469deb93bb76d613c9ce79edc5cb68fd", + "sha256:aa0cb481c4041bf52ffa7b0d8fa6cd3e88a2ca4879c533c9153882ee2556790d" + ], + "version": "==0.8.1" + }, + "regex": { + "hashes": [ + "sha256:01afaf2ec48e196ba91b37451aa353cb7eda77efe518e481707e0515025f0cd5", + "sha256:11d773d75fa650cd36f68d7ca936e3c7afaae41b863b8c387a22aaa78d3c5c79", + "sha256:18c071c3eb09c30a264879f0d310d37fe5d3a3111662438889ae2eb6fc570c31", + "sha256:1e1c20e29358165242928c2de1482fb2cf4ea54a6a6dea2bd7a0e0d8ee321500", + "sha256:281d2fd05555079448537fe108d79eb031b403dac622621c78944c235f3fcf11", + "sha256:314d66636c494ed9c148a42731b3834496cc9a2c4251b1661e40936814542b14", + "sha256:32e65442138b7b76dd8173ffa2cf67356b7bc1768851dded39a7a13bf9223da3", + "sha256:339456e7d8c06dd36a22e451d58ef72cef293112b559010db3d054d5560ef439", + "sha256:3916d08be28a1149fb97f7728fca1f7c15d309a9f9682d89d79db75d5e52091c", + "sha256:3a9cd17e6e5c7eb328517969e0cb0c3d31fd329298dd0c04af99ebf42e904f82", + "sha256:47bf5bf60cf04d72bf6055ae5927a0bd9016096bf3d742fa50d9bf9f45aa0711", + "sha256:4c46e22a0933dd783467cf32b3516299fb98cfebd895817d685130cc50cd1093", + "sha256:4c557a7b470908b1712fe27fb1ef20772b78079808c87d20a90d051660b1d69a", + "sha256:52ba3d3f9b942c49d7e4bc105bb28551c44065f139a65062ab7912bef10c9afb", + "sha256:563085e55b0d4fb8f746f6a335893bda5c2cef43b2f0258fe1020ab1dd874df8", + "sha256:598585c9f0af8374c28edd609eb291b5726d7cbce16be6a8b95aa074d252ee17", + "sha256:619d71c59a78b84d7f18891fe914446d07edd48dc8328c8e149cbe0929b4e000", + "sha256:67bdb9702427ceddc6ef3dc382455e90f785af4c13d495f9626861763ee13f9d", + "sha256:6d1b01031dedf2503631d0903cb563743f397ccaf6607a5e3b19a3d76fc10480", + "sha256:741a9647fcf2e45f3a1cf0e24f5e17febf3efe8d4ba1281dcc3aa0459ef424dc", + "sha256:7c2a1af393fcc09e898beba5dd59196edaa3116191cc7257f9224beaed3e1aa0", + "sha256:7d9884d86dd4dd489e981d94a65cd30d6f07203d90e98f6f657f05170f6324c9", + "sha256:90f11ff637fe8798933fb29f5ae1148c978cccb0452005bf4c69e13db951e765", + "sha256:919859aa909429fb5aa9cf8807f6045592c85ef56fdd30a9a3747e513db2536e", + "sha256:96fcd1888ab4d03adfc9303a7b3c0bd78c5412b2bfbe76db5b56d9eae004907a", + "sha256:97f29f57d5b84e73fbaf99ab3e26134e6687348e95ef6b48cfd2c06807005a07", + "sha256:980d7be47c84979d9136328d882f67ec5e50008681d94ecc8afa8a65ed1f4a6f", + "sha256:a91aa8619b23b79bcbeb37abe286f2f408d2f2d6f29a17237afda55bb54e7aac", + "sha256:ade17eb5d643b7fead300a1641e9f45401c98eee23763e9ed66a43f92f20b4a7", + "sha256:b9c3db21af35e3b3c05764461b262d6f05bbca08a71a7849fd79d47ba7bc33ed", + "sha256:bd28bc2e3a772acbb07787c6308e00d9626ff89e3bfcdebe87fa5afbfdedf968", + "sha256:bf5824bfac591ddb2c1f0a5f4ab72da28994548c708d2191e3b87dd207eb3ad7", + "sha256:c0502c0fadef0d23b128605d69b58edb2c681c25d44574fc673b0e52dce71ee2", + "sha256:c38c71df845e2aabb7fb0b920d11a1b5ac8526005e533a8920aea97efb8ec6a4", + "sha256:ce15b6d103daff8e9fee13cf7f0add05245a05d866e73926c358e871221eae87", + "sha256:d3029c340cfbb3ac0a71798100ccc13b97dddf373a4ae56b6a72cf70dfd53bc8", + "sha256:e512d8ef5ad7b898cdb2d8ee1cb09a8339e4f8be706d27eaa180c2f177248a10", + "sha256:e8e5b509d5c2ff12f8418006d5a90e9436766133b564db0abaec92fd27fcee29", + "sha256:ee54ff27bf0afaf4c3b3a62bcd016c12c3fdb4ec4f413391a90bd38bc3624605", + "sha256:fa4537fb4a98fe8fde99626e4681cc644bdcf2a795038533f9f711513a862ae6", + "sha256:fd45ff9293d9274c5008a2054ecef86a9bfe819a67c7be1afb65e69b405b3042" + ], + "version": "==2021.4.4" + }, + "toml": { + "hashes": [ + "sha256:806143ae5bfb6a3c6e736a764057db0e6a0e05e338b5630894a5f779cabb4f9b", + "sha256:b3bda1d108d5dd99f4a20d24d9c348e91c4db7ab1b749200bded2f839ccbe68f" + ], + "markers": "python_version >= '2.6' and python_version not in '3.0, 3.1, 3.2, 3.3'", + "version": "==0.10.2" + } + } +} diff --git a/frameworks/Python/async-worker/README.md b/frameworks/Python/async-worker/README.md new file mode 100644 index 00000000000..fbdcb7e215a --- /dev/null +++ b/frameworks/Python/async-worker/README.md @@ -0,0 +1,14 @@ +# [async-worker](https://github.com/async-worker/async-worker) Benchmark Test + +### Server +Server is exposed at `:8080` + +## Test URLs + +### Test 1: JSON Encoding + + http://localhost:8080/json + +### Test 6: Plaintext + + http://localhost:8080/plaintext diff --git a/frameworks/Python/async-worker/async-worker.dockerfile b/frameworks/Python/async-worker/async-worker.dockerfile new file mode 100644 index 00000000000..2023fb97fb9 --- /dev/null +++ b/frameworks/Python/async-worker/async-worker.dockerfile @@ -0,0 +1,19 @@ +FROM python:3.9-alpine + +LABEL description="Image used to run async-worker benchmark tests." +LABEL version='0.1' + +ENV ASYNCWORKER_HTTP_HOST=0.0.0.0 +ENV ASYNCWORKER_HTTP_PORT=8080 + +WORKDIR /app + +COPY /src /app +COPY Pipfile /app +COPY Pipfile.lock /app + +RUN apk add --virtual .deps gcc g++ make openssl-dev libxml2 libffi-dev && \ + pip install pipenv && \ + pipenv install --system --ignore-pipfile + +CMD ["python", "./hello_world.py"] diff --git a/frameworks/Python/async-worker/benchmark_config.json b/frameworks/Python/async-worker/benchmark_config.json new file mode 100644 index 00000000000..f27f1806581 --- /dev/null +++ b/frameworks/Python/async-worker/benchmark_config.json @@ -0,0 +1,20 @@ +{ + "framework": "async-worker", + "tests": [{ + "default": { + "json_url": "/json", + "plaintext_url": "/plaintext", + "port": 8080, + "approach": "Realistic", + "classification": "Micro", + "framework": "async-worker", + "language": "Python", + "flavor": "Python3", + "orm": "Raw", + "platform": "asyncio", + "os": "Linux", + "display_name": "async-worker", + "versus": "aiohttp" + } + }] +} diff --git a/frameworks/Python/async-worker/src/__init__.py b/frameworks/Python/async-worker/src/__init__.py new file mode 100644 index 00000000000..e69de29bb2d diff --git a/frameworks/Python/async-worker/src/hello_world.py b/frameworks/Python/async-worker/src/hello_world.py new file mode 100644 index 00000000000..ca785cd583b --- /dev/null +++ b/frameworks/Python/async-worker/src/hello_world.py @@ -0,0 +1,18 @@ +from aiohttp import web +from asyncworker import App + +app = App() + + +@app.http.get(["/plaintext"]) +async def handler(request: web.Request) -> web.Response: + return web.Response(body="Hello, World!") + + +@app.http.get(["/json"]) +async def handler(request: web.Request) -> web.Response: + return web.json_response({"message": "Hello, World!"}) + + +if __name__ == "__main__": + app.run() From b10ea2627edd44fb976cee20a9969e12a9c25bb4 Mon Sep 17 00:00:00 2001 From: Daniel Date: Tue, 22 Jun 2021 23:32:30 +0800 Subject: [PATCH 14/25] OfficeFloor supporting Cache Test (#6670) * Removing Rapidoid as seems dead project (no commit last year) * Downloading from SourceForge * Bump maven-compiler-plugin in /frameworks/Java/officefloor/src Bumps [maven-compiler-plugin](https://github.com/apache/maven-compiler-plugin) from 3.8.0 to 3.8.1. - [Release notes](https://github.com/apache/maven-compiler-plugin/releases) - [Commits](https://github.com/apache/maven-compiler-plugin/compare/maven-compiler-plugin-3.8.0...maven-compiler-plugin-3.8.1) Signed-off-by: dependabot-preview[bot] * Bump maven-shade-plugin in /frameworks/Java/officefloor/src Bumps [maven-shade-plugin](https://github.com/apache/maven-shade-plugin) from 3.2.1 to 3.2.2. - [Release notes](https://github.com/apache/maven-shade-plugin/releases) - [Commits](https://github.com/apache/maven-shade-plugin/compare/maven-shade-plugin-3.2.1...maven-shade-plugin-3.2.2) Signed-off-by: dependabot-preview[bot] * Fixing Raw OfficeFloor * Specifying Spring plugin version * Bump to OfficeFloor 3.21.0 * Bump spring-boot-maven-plugin in /frameworks/Java/officefloor/src Bumps [spring-boot-maven-plugin](https://github.com/spring-projects/spring-boot) from 2.2.4.RELEASE to 2.2.5.RELEASE. - [Release notes](https://github.com/spring-projects/spring-boot/releases) - [Commits](https://github.com/spring-projects/spring-boot/compare/v2.2.4.RELEASE...v2.2.5.RELEASE) Signed-off-by: dependabot-preview[bot] * Bump spring-boot-maven-plugin in /frameworks/Java/officefloor/src Bumps [spring-boot-maven-plugin](https://github.com/spring-projects/spring-boot) from 2.2.5.RELEASE to 2.2.6.RELEASE. - [Release notes](https://github.com/spring-projects/spring-boot/releases) - [Commits](https://github.com/spring-projects/spring-boot/compare/v2.2.5.RELEASE...v2.2.6.RELEASE) Signed-off-by: dependabot-preview[bot] * Bump net.officefloor:bom in /frameworks/Java/officefloor/src Bumps [net.officefloor:bom](https://github.com/officefloor/OfficeFloor) from 3.21.0 to 3.22.0. - [Release notes](https://github.com/officefloor/OfficeFloor/releases) - [Commits](https://github.com/officefloor/OfficeFloor/commits) Signed-off-by: dependabot-preview[bot] * Bump maven-shade-plugin in /frameworks/Java/officefloor/src Bumps [maven-shade-plugin](https://github.com/apache/maven-shade-plugin) from 3.2.2 to 3.2.3. - [Release notes](https://github.com/apache/maven-shade-plugin/releases) - [Commits](https://github.com/apache/maven-shade-plugin/compare/maven-shade-plugin-3.2.2...maven-shade-plugin-3.2.3) Signed-off-by: dependabot-preview[bot] * Bump net.officefloor:bom in /frameworks/Java/officefloor/src Bumps [net.officefloor:bom](https://github.com/officefloor/OfficeFloor) from 3.22.0 to 3.23.0. - [Release notes](https://github.com/officefloor/OfficeFloor/releases) - [Commits](https://github.com/officefloor/OfficeFloor/commits) Signed-off-by: dependabot-preview[bot] * Bump net.officefloor:bom in /frameworks/Java/officefloor/src Bumps [net.officefloor:bom](https://github.com/officefloor/OfficeFloor) from 3.23.0 to 3.24.0. - [Release notes](https://github.com/officefloor/OfficeFloor/releases) - [Commits](https://github.com/officefloor/OfficeFloor/commits) Signed-off-by: dependabot-preview[bot] * Bump spring-boot-maven-plugin in /frameworks/Java/officefloor/src Bumps [spring-boot-maven-plugin](https://github.com/spring-projects/spring-boot) from 2.2.6.RELEASE to 2.2.7.RELEASE. - [Release notes](https://github.com/spring-projects/spring-boot/releases) - [Commits](https://github.com/spring-projects/spring-boot/compare/v2.2.6.RELEASE...v2.2.7.RELEASE) Signed-off-by: dependabot-preview[bot] * Bump spring-boot-maven-plugin in /frameworks/Java/officefloor/src Bumps [spring-boot-maven-plugin](https://github.com/spring-projects/spring-boot) from 2.2.7.RELEASE to 2.3.0.RELEASE. - [Release notes](https://github.com/spring-projects/spring-boot/releases) - [Commits](https://github.com/spring-projects/spring-boot/compare/v2.2.7.RELEASE...v2.3.0.RELEASE) Signed-off-by: dependabot-preview[bot] * Bump net.officefloor:bom in /frameworks/Java/officefloor/src Bumps [net.officefloor:bom](https://github.com/officefloor/OfficeFloor) from 3.24.0 to 3.25.0. - [Release notes](https://github.com/officefloor/OfficeFloor/releases) - [Commits](https://github.com/officefloor/OfficeFloor/commits) Signed-off-by: dependabot-preview[bot] * Bump maven-shade-plugin in /frameworks/Java/officefloor/src Bumps [maven-shade-plugin](https://github.com/apache/maven-shade-plugin) from 3.2.3 to 3.2.4. - [Release notes](https://github.com/apache/maven-shade-plugin/releases) - [Commits](https://github.com/apache/maven-shade-plugin/compare/maven-shade-plugin-3.2.3...maven-shade-plugin-3.2.4) Signed-off-by: dependabot-preview[bot] * Bump spring-boot-maven-plugin in /frameworks/Java/officefloor/src Bumps [spring-boot-maven-plugin](https://github.com/spring-projects/spring-boot) from 2.3.0.RELEASE to 2.3.1.RELEASE. - [Release notes](https://github.com/spring-projects/spring-boot/releases) - [Commits](https://github.com/spring-projects/spring-boot/compare/v2.3.0.RELEASE...v2.3.1.RELEASE) Signed-off-by: dependabot-preview[bot] * Bump spring-boot-maven-plugin in /frameworks/Java/officefloor/src Bumps [spring-boot-maven-plugin](https://github.com/spring-projects/spring-boot) from 2.3.1.RELEASE to 2.3.2.RELEASE. - [Release notes](https://github.com/spring-projects/spring-boot/releases) - [Commits](https://github.com/spring-projects/spring-boot/compare/v2.3.1.RELEASE...v2.3.2.RELEASE) Signed-off-by: dependabot-preview[bot] * Bump net.officefloor:bom in /frameworks/Java/officefloor/src Bumps [net.officefloor:bom](https://github.com/officefloor/OfficeFloor) from 3.25.0 to 3.26.0. - [Release notes](https://github.com/officefloor/OfficeFloor/releases) - [Commits](https://github.com/officefloor/OfficeFloor/compare/release-3.25.0...release-3.26.0) Signed-off-by: dependabot-preview[bot] * Bump spring-boot-maven-plugin in /frameworks/Java/officefloor/src Bumps [spring-boot-maven-plugin](https://github.com/spring-projects/spring-boot) from 2.3.2.RELEASE to 2.3.3.RELEASE. - [Release notes](https://github.com/spring-projects/spring-boot/releases) - [Commits](https://github.com/spring-projects/spring-boot/compare/v2.3.2.RELEASE...v2.3.3.RELEASE) Signed-off-by: dependabot-preview[bot] * Bump net.officefloor:bom in /frameworks/Java/officefloor/src Bumps [net.officefloor:bom](https://github.com/officefloor/OfficeFloor) from 3.26.0 to 3.27.0. - [Release notes](https://github.com/officefloor/OfficeFloor/releases) - [Commits](https://github.com/officefloor/OfficeFloor/commits) Signed-off-by: dependabot-preview[bot] * Bump spring-boot-maven-plugin in /frameworks/Java/officefloor/src Bumps [spring-boot-maven-plugin](https://github.com/spring-projects/spring-boot) from 2.3.3.RELEASE to 2.3.4.RELEASE. - [Release notes](https://github.com/spring-projects/spring-boot/releases) - [Commits](https://github.com/spring-projects/spring-boot/compare/v2.3.3.RELEASE...v2.3.4.RELEASE) Signed-off-by: dependabot-preview[bot] * Bump net.officefloor:bom in /frameworks/Java/officefloor/src Bumps [net.officefloor:bom](https://github.com/officefloor/OfficeFloor) from 3.27.0 to 3.28.0. - [Release notes](https://github.com/officefloor/OfficeFloor/releases) - [Commits](https://github.com/officefloor/OfficeFloor/commits) Signed-off-by: dependabot-preview[bot] * Fixing to run with v3.28.0 * Including Undertow * Using slim docker images * Fixing to add all supported tests * Increasing connection pool size * Write up the 503 errors and why * Fix grammer * Bump spring-boot-maven-plugin in /frameworks/Java/officefloor/src Bumps [spring-boot-maven-plugin](https://github.com/spring-projects/spring-boot) from 2.3.4.RELEASE to 2.3.5.RELEASE. - [Release notes](https://github.com/spring-projects/spring-boot/releases) - [Commits](https://github.com/spring-projects/spring-boot/compare/v2.3.4.RELEASE...v2.3.5.RELEASE) Signed-off-by: dependabot-preview[bot] * Improving chances of full processing Increasing max thread counts to have thread per client. Also, shading jars correctly. Plus adding in DB test for raw * Fixing raw test to have database * Ensure random numbers to avoid cached entities * Ensuring spring random numbers to avoid entity caching * Adding queries for raw * officefloor-raw queries passing * office-raw supporting all requests * Fixed update test to ensure updates occur before responding * Adding officefloor-async This will demonstrate OfficeFloor running as a single thread asynchronous server * Upgrading to OfficeFloor 2.38.1 * Tidy up read me * Appropriate back pressure queue * Handle rate limiting * Providing appropriate throttling * Lowering threading to handle through put * officefloor-raw passing tests * Passing tests * OfficeFloor 3.28.2 * Fixing rate limiting * Bump net.officefloor:bom in /frameworks/Java/officefloor/src Bumps [net.officefloor:bom](https://github.com/officefloor/OfficeFloor) from 3.28.2 to 3.29.0. - [Release notes](https://github.com/officefloor/OfficeFloor/releases) - [Commits](https://github.com/officefloor/OfficeFloor/commits) Signed-off-by: dependabot-preview[bot] * Fixing for bump to 3.30.0 * Increasing max direct memory * Increasing threshold to enable verifications to pass for officefloor-raw * Start script to take available memory into consideration * Max direct memory aware of available memory Also, cleaning writer threads of buffers to avoid leaks. * Avoiding OOM by managing disable/enable reading * DB and Fortune passing * Ensuring free command is available * Removing TODOs * Increasing reactor buffer for multiple queries * Fixing for 3.30.1 * Passing benchmark tests * Avoiding OOM on reactor buffer sizes * Passing validate tests * Using defaults from 3.30.1 * Bump to 512 threads and connections * Reverting dockerfiles to provide appropriate parameters * openjdk:slim for apt-get available * Providing thread affinity to raw * Fixing rate limit throttling * Multiplexing queries over connections per socket * Fixing for large queries in validate * Fixing versions of maven and java * Providing thread affinity of DB connection thread * Fixing for Netty event loop thread * Tidying up compiler warnings * Single db pool to avoid additional threads * Use default LoopResources * Tidying up code for warnings * Further tidy up of code * Revert to latest pull request * Using parallel GC for better throughput * Fix up for thread local buffering improvements * OfficeFloor fortune raw Focus of officefloor-raw is raw performance of the OfficeFloor HTTP server. Using raw fortune to remove mustache overheads. * Tidy up loop * Server name (as per discussions) * Bump spring-boot-maven-plugin in /frameworks/Java/officefloor/src Bumps [spring-boot-maven-plugin](https://github.com/spring-projects/spring-boot) from 2.3.5.RELEASE to 2.4.2. - [Release notes](https://github.com/spring-projects/spring-boot/releases) - [Commits](https://github.com/spring-projects/spring-boot/compare/v2.3.5.RELEASE...v2.4.2) Signed-off-by: dependabot-preview[bot] * Bump net.officefloor:bom in /frameworks/Java/officefloor/src Bumps [net.officefloor:bom](https://github.com/officefloor/OfficeFloor) from 3.30.2 to 3.31.0. - [Release notes](https://github.com/officefloor/OfficeFloor/releases) - [Commits](https://github.com/officefloor/OfficeFloor/commits) Signed-off-by: dependabot-preview[bot] * Bump net.officefloor:bom in /frameworks/Java/officefloor/src Bumps [net.officefloor:bom](https://github.com/officefloor/OfficeFloor) from 3.31.0 to 3.32.0. - [Release notes](https://github.com/officefloor/OfficeFloor/releases) - [Commits](https://github.com/officefloor/OfficeFloor/commits) Signed-off-by: dependabot-preview[bot] * Bump spring-boot-maven-plugin in /frameworks/Java/officefloor/src Bumps [spring-boot-maven-plugin](https://github.com/spring-projects/spring-boot) from 2.4.2 to 2.4.3. - [Release notes](https://github.com/spring-projects/spring-boot/releases) - [Commits](https://github.com/spring-projects/spring-boot/compare/v2.4.2...v2.4.3) Signed-off-by: dependabot-preview[bot] * Bump spring-boot-maven-plugin in /frameworks/Java/officefloor/src Bumps [spring-boot-maven-plugin](https://github.com/spring-projects/spring-boot) from 2.4.3 to 2.4.5. - [Release notes](https://github.com/spring-projects/spring-boot/releases) - [Commits](https://github.com/spring-projects/spring-boot/compare/v2.4.3...v2.4.5) Signed-off-by: dependabot-preview[bot] * Upgrade to GitHub-native Dependabot * Bump net.officefloor:bom in /frameworks/Java/officefloor/src Bumps [net.officefloor:bom](https://github.com/officefloor/OfficeFloor) from 3.32.0 to 3.35.0. - [Release notes](https://github.com/officefloor/OfficeFloor/releases) - [Commits](https://github.com/officefloor/OfficeFloor/commits/release-3.35.0) Signed-off-by: dependabot-preview[bot] * Bump to OfficeFloor 3.35.0 * Allow bump of all versions * Vertx server Also, renaming officefloor-raw to officefloor-r2dbc to make way for officefloor-sqlclient * Rename to R2dbc (from raw) * Bump net.officefloor:bom in /frameworks/Java/officefloor/src Bumps [net.officefloor:bom](https://github.com/officefloor/OfficeFloor) from 3.35.0 to 3.36.0. - [Release notes](https://github.com/officefloor/OfficeFloor/releases) - [Commits](https://github.com/officefloor/OfficeFloor/commits) Signed-off-by: dependabot[bot] * Fixing meta-data * Can not propagate Dependabot to TechEmpower * Abstract WoOF from database driver This will allow introducing Vertx SQL Client * Fixing db port * Providing Vertx SQL Client implementation * Using OfficeFloorVertx for vertx Allows for tests to reset * Fixing link in read me * Reducing repetition in readme * Swapped OfficeFloor async to use Vertx SQL Client * Bump spring-boot-maven-plugin in /frameworks/Java/officefloor/src Bumps [spring-boot-maven-plugin](https://github.com/spring-projects/spring-boot) from 2.4.5 to 2.5.0. - [Release notes](https://github.com/spring-projects/spring-boot/releases) - [Commits](https://github.com/spring-projects/spring-boot/compare/v2.4.5...v2.5.0) Signed-off-by: dependabot[bot] * Increasing SQL Client pool size to 512 Also, fixing update to sort to avoid deadlocks * Removing unnecessary logging * Removing dependabot configuration * Improving performance of Vertx This is by caching queries and using native communication. Also, adding further performance updates to update test by sorting updates * Fixing OfficeFloor-vertx name * Reducing load on DB callback threads * Database worker threads relative to number of socket threads * Further improvements to performance (by less contention) * Avoid lock on write stream * Using Guava * Removing commented out line * Providing Cache implementation * Re-enabling thread affinity * Making R2DBC more resilient * Adding cached query configuration entries * Caching runs off World table * Only require World able Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .../Java/officefloor/benchmark_config.json | 8 ++ frameworks/Java/officefloor/config.toml | 8 ++ frameworks/Java/officefloor/src/pom.xml | 7 +- .../officefloor/src/woof_benchmark/pom.xml | 4 + .../officefloor/benchmark/CachedWorld.java | 11 +++ .../benchmark/CachedWorldDataRetriever.java | 39 ++++++++ .../java/net/officefloor/benchmark/Logic.java | 23 +++++ .../src/main/resources/application.objects | 3 + .../src/main/resources/application.woof | 8 +- .../src/woof_benchmark_micro/pom.xml | 4 + .../officefloor/benchmark/CachedWorld.java | 13 +++ .../benchmark/CachedWorldDataRetriever.java | 39 ++++++++ .../java/net/officefloor/benchmark/Logic.java | 23 +++++ .../src/main/resources/application.objects | 3 + .../src/main/resources/application.woof | 8 +- .../src/woof_benchmark_r2dbc/pom.xml | 4 + .../benchmark/R2dbcOfficeFloorMain.java | 90 +++++++++++++++++-- .../src/woof_benchmark_spring/pom.xml | 4 + .../officefloor/benchmark/CachedWorld.java | 11 +++ .../benchmark/CachedWorldDataRetriever.java | 29 ++++++ .../net/officefloor/benchmark/Fortune.java | 3 - .../java/net/officefloor/benchmark/Logic.java | 23 +++++ .../src/main/resources/application.objects | 3 + .../src/main/resources/application.woof | 6 ++ .../benchmark/SqlClientOfficeFloorMain.java | 7 +- .../src/woof_benchmark_woof/pom.xml | 4 - .../benchmark/CachedSendResponse.java | 42 +++++++++ .../officefloor/benchmark/CachedWorld.java | 18 ++++ .../benchmark/DatabaseOperations.java | 5 ++ .../benchmark/DatabaseOperationsFactory.java | 3 +- .../benchmark/FortunesSendResponse.java | 3 - .../net/officefloor/benchmark/RawWoof.java | 13 ++- 32 files changed, 445 insertions(+), 24 deletions(-) create mode 100644 frameworks/Java/officefloor/src/woof_benchmark/src/main/java/net/officefloor/benchmark/CachedWorld.java create mode 100644 frameworks/Java/officefloor/src/woof_benchmark/src/main/java/net/officefloor/benchmark/CachedWorldDataRetriever.java create mode 100644 frameworks/Java/officefloor/src/woof_benchmark_micro/src/main/java/net/officefloor/benchmark/CachedWorld.java create mode 100644 frameworks/Java/officefloor/src/woof_benchmark_micro/src/main/java/net/officefloor/benchmark/CachedWorldDataRetriever.java create mode 100644 frameworks/Java/officefloor/src/woof_benchmark_spring/src/main/java/net/officefloor/benchmark/CachedWorld.java create mode 100644 frameworks/Java/officefloor/src/woof_benchmark_spring/src/main/java/net/officefloor/benchmark/CachedWorldDataRetriever.java create mode 100644 frameworks/Java/officefloor/src/woof_benchmark_woof/src/main/java/net/officefloor/benchmark/CachedSendResponse.java create mode 100644 frameworks/Java/officefloor/src/woof_benchmark_woof/src/main/java/net/officefloor/benchmark/CachedWorld.java diff --git a/frameworks/Java/officefloor/benchmark_config.json b/frameworks/Java/officefloor/benchmark_config.json index ffadb687d77..81e280fbbc9 100755 --- a/frameworks/Java/officefloor/benchmark_config.json +++ b/frameworks/Java/officefloor/benchmark_config.json @@ -7,6 +7,7 @@ "plaintext_url": "/plaintext", "db_url": "/db", "query_url": "/queries?queries=", + "cached_query_url": "/cached-worlds?count=", "fortune_url": "/fortunes", "update_url": "/update?queries=", "port": 8080, @@ -30,6 +31,7 @@ "plaintext_url": "/plaintext", "db_url": "/db", "query_url": "/queries?queries=", + "cached_query_url": "/cached-worlds?count=", "fortune_url": "/fortunes", "update_url": "/update?queries=", "port": 8080, @@ -97,6 +99,7 @@ "plaintext_url": "/plaintext", "db_url": "/db", "query_url": "/queries?queries=", + "cached_query_url": "/cached-worlds?count=", "fortune_url": "/fortunes", "update_url": "/update?queries=", "port": 8080, @@ -120,6 +123,7 @@ "plaintext_url": "/plaintext", "db_url": "/db", "query_url": "/queries?queries=", + "cached_query_url": "/cached-worlds?count=", "fortune_url": "/fortunes", "update_url": "/update?queries=", "port": 8080, @@ -143,6 +147,7 @@ "plaintext_url": "/plaintext", "db_url": "/db", "query_url": "/queries?queries=", + "cached_query_url": "/cached-worlds?count=", "fortune_url": "/fortunes", "update_url": "/update?queries=", "port": 8080, @@ -166,6 +171,7 @@ "plaintext_url": "/plaintext", "db_url": "/db", "query_url": "/queries?queries=", + "cached_query_url": "/cached-worlds?count=", "fortune_url": "/fortunes", "update_url": "/update?queries=", "port": 8080, @@ -189,6 +195,7 @@ "plaintext_url": "/plaintext", "db_url": "/db", "query_url": "/queries?queries=", + "cached_query_url": "/cached-worlds?count=", "fortune_url": "/fortunes", "update_url": "/update?queries=", "port": 8080, @@ -212,6 +219,7 @@ "plaintext_url": "/plaintext", "db_url": "/db", "query_url": "/queries?queries=", + "cached_query_url": "/cached-worlds?count=", "fortune_url": "/fortunes", "update_url": "/update?queries=", "port": 8080, diff --git a/frameworks/Java/officefloor/config.toml b/frameworks/Java/officefloor/config.toml index 327f2eb9082..293356d2b5a 100644 --- a/frameworks/Java/officefloor/config.toml +++ b/frameworks/Java/officefloor/config.toml @@ -6,6 +6,7 @@ urls.plaintext = "/plaintext" urls.json = "/json" urls.db = "/db" urls.query = "/queries?queries=" +urls.cached_query = "/cached-worlds?count=" urls.update = "/update?queries=" urls.fortune = "/fortunes" approach = "Realistic" @@ -23,6 +24,7 @@ urls.plaintext = "/plaintext" urls.json = "/json" urls.db = "/db" urls.query = "/queries?queries=" +urls.cached_query = "/cached-worlds?count=" urls.update = "/update?queries=" urls.fortune = "/fortunes" approach = "Realistic" @@ -40,6 +42,7 @@ urls.plaintext = "/plaintext" urls.json = "/json" urls.db = "/db" urls.query = "/queries?queries=" +urls.cached_query = "/cached-worlds?count=" urls.update = "/update?queries=" urls.fortune = "/fortunes" approach = "Realistic" @@ -57,6 +60,7 @@ urls.plaintext = "/plaintext" urls.json = "/json" urls.db = "/db" urls.query = "/queries?queries=" +urls.cached_query = "/cached-worlds?count=" urls.update = "/update?queries=" urls.fortune = "/fortunes" approach = "Realistic" @@ -74,6 +78,7 @@ urls.plaintext = "/plaintext" urls.json = "/json" urls.db = "/db" urls.query = "/queries?queries=" +urls.cached_query = "/cached-worlds?count=" urls.update = "/update?queries=" urls.fortune = "/fortunes" approach = "Realistic" @@ -108,6 +113,7 @@ urls.plaintext = "/plaintext" urls.json = "/json" urls.db = "/db" urls.query = "/queries?queries=" +urls.cached_query = "/cached-worlds?count=" urls.update = "/update?queries=" urls.fortune = "/fortunes" approach = "Realistic" @@ -125,6 +131,7 @@ urls.plaintext = "/plaintext" urls.json = "/json" urls.db = "/db" urls.query = "/queries?queries=" +urls.cached_query = "/cached-worlds?count=" urls.update = "/update?queries=" urls.fortune = "/fortunes" approach = "Realistic" @@ -142,6 +149,7 @@ urls.plaintext = "/plaintext" urls.json = "/json" urls.db = "/db" urls.query = "/queries?queries=" +urls.cached_query = "/cached-worlds?count=" urls.update = "/update?queries=" urls.fortune = "/fortunes" approach = "Realistic" diff --git a/frameworks/Java/officefloor/src/pom.xml b/frameworks/Java/officefloor/src/pom.xml index 2fe053ccf6a..d3dae1ef9d9 100644 --- a/frameworks/Java/officefloor/src/pom.xml +++ b/frameworks/Java/officefloor/src/pom.xml @@ -42,7 +42,7 @@ net.officefloor bom - 3.36.0 + 3.37.0 pom import @@ -66,6 +66,11 @@ compiler 0.9.7 + + com.google.guava + guava + 29.0-jre + diff --git a/frameworks/Java/officefloor/src/woof_benchmark/pom.xml b/frameworks/Java/officefloor/src/woof_benchmark/pom.xml index b1a59593297..9a33534eca6 100644 --- a/frameworks/Java/officefloor/src/woof_benchmark/pom.xml +++ b/frameworks/Java/officefloor/src/woof_benchmark/pom.xml @@ -29,6 +29,10 @@ net.officefloor.persistence officejpa_hibernate + + net.officefloor.cache + officecache_constant + org.projectlombok lombok diff --git a/frameworks/Java/officefloor/src/woof_benchmark/src/main/java/net/officefloor/benchmark/CachedWorld.java b/frameworks/Java/officefloor/src/woof_benchmark/src/main/java/net/officefloor/benchmark/CachedWorld.java new file mode 100644 index 00000000000..90d67f57e93 --- /dev/null +++ b/frameworks/Java/officefloor/src/woof_benchmark/src/main/java/net/officefloor/benchmark/CachedWorld.java @@ -0,0 +1,11 @@ +package net.officefloor.benchmark; + +import lombok.Data; + +@Data +public class CachedWorld { + + private final int id; + + private final int randomNumber; +} \ No newline at end of file diff --git a/frameworks/Java/officefloor/src/woof_benchmark/src/main/java/net/officefloor/benchmark/CachedWorldDataRetriever.java b/frameworks/Java/officefloor/src/woof_benchmark/src/main/java/net/officefloor/benchmark/CachedWorldDataRetriever.java new file mode 100644 index 00000000000..747518d8019 --- /dev/null +++ b/frameworks/Java/officefloor/src/woof_benchmark/src/main/java/net/officefloor/benchmark/CachedWorldDataRetriever.java @@ -0,0 +1,39 @@ +package net.officefloor.benchmark; + +import java.sql.Connection; +import java.sql.ResultSet; +import java.util.HashMap; +import java.util.Map; + +import javax.sql.DataSource; + +import net.officefloor.cache.Cache; +import net.officefloor.cache.constant.ConstantCacheDataRetriever; +import net.officefloor.plugin.clazz.Dependency; + +/** + * {@link ConstantCacheDataRetriever} containing polling logic to refresh the + * {@link Cache}. + * + * @author Daniel Sagenschneider + */ +public class CachedWorldDataRetriever implements ConstantCacheDataRetriever { + + private @Dependency DataSource dataSource; + + @Override + public Map getData() throws Exception { + try (Connection connection = this.dataSource.getConnection()) { + Map data = new HashMap<>(); + ResultSet resultSet = connection.prepareStatement("SELECT id, randomNumber FROM World") + .executeQuery(); + while (resultSet.next()) { + int id = resultSet.getInt("id"); + int randomNumber = resultSet.getInt("randomNumber"); + data.put(id, new CachedWorld(id, randomNumber)); + } + return data; + } + } + +} \ No newline at end of file diff --git a/frameworks/Java/officefloor/src/woof_benchmark/src/main/java/net/officefloor/benchmark/Logic.java b/frameworks/Java/officefloor/src/woof_benchmark/src/main/java/net/officefloor/benchmark/Logic.java index 0939c7938b8..10f6fb6fcfc 100644 --- a/frameworks/Java/officefloor/src/woof_benchmark/src/main/java/net/officefloor/benchmark/Logic.java +++ b/frameworks/Java/officefloor/src/woof_benchmark/src/main/java/net/officefloor/benchmark/Logic.java @@ -7,6 +7,7 @@ import javax.persistence.EntityManager; import lombok.Data; +import net.officefloor.cache.Cache; import net.officefloor.web.HttpQueryParameter; import net.officefloor.web.ObjectResponse; @@ -67,6 +68,28 @@ public void queries(@HttpQueryParameter("queries") String queries, EntityManager response.send(worlds); } + // ========== CACHED ================== + + public void cached(@HttpQueryParameter("count") String queries, Cache cache, + ObjectResponse response) { + int count = getQueryCount(queries); + + // Set up for unique numbers + ThreadLocalRandom random = ThreadLocalRandom.current(); + + // Obtain the list of cached worlds + CachedWorld[] worlds = new CachedWorld[count]; + for (int i = 0; i < worlds.length; i++) { + + // Obtain unique identifier + int randomNumber = random.nextInt(1, 10001); + + // Obtain the cached world + worlds[i] = cache.get(randomNumber); + } + response.send(worlds); + } + // =========== UPDATES =================== public void update(@HttpQueryParameter("queries") String queries, EntityManager entityManager, diff --git a/frameworks/Java/officefloor/src/woof_benchmark/src/main/resources/application.objects b/frameworks/Java/officefloor/src/woof_benchmark/src/main/resources/application.objects index 5e669eac5d7..ee94d79bae7 100644 --- a/frameworks/Java/officefloor/src/woof_benchmark/src/main/resources/application.objects +++ b/frameworks/Java/officefloor/src/woof_benchmark/src/main/resources/application.objects @@ -5,6 +5,9 @@ + + + diff --git a/frameworks/Java/officefloor/src/woof_benchmark/src/main/resources/application.woof b/frameworks/Java/officefloor/src/woof_benchmark/src/main/resources/application.woof index 3991b490fbb..7c8495809ad 100644 --- a/frameworks/Java/officefloor/src/woof_benchmark/src/main/resources/application.woof +++ b/frameworks/Java/officefloor/src/woof_benchmark/src/main/resources/application.woof @@ -12,6 +12,9 @@
+ +
+ @@ -23,14 +26,15 @@
+ - -
+ + diff --git a/frameworks/Java/officefloor/src/woof_benchmark_micro/pom.xml b/frameworks/Java/officefloor/src/woof_benchmark_micro/pom.xml index e3c73bd5dea..50202cfb4ef 100644 --- a/frameworks/Java/officefloor/src/woof_benchmark_micro/pom.xml +++ b/frameworks/Java/officefloor/src/woof_benchmark_micro/pom.xml @@ -25,6 +25,10 @@ net.officefloor.persistence officejdbc_hikari + + net.officefloor.cache + officecache_constant + org.projectlombok lombok diff --git a/frameworks/Java/officefloor/src/woof_benchmark_micro/src/main/java/net/officefloor/benchmark/CachedWorld.java b/frameworks/Java/officefloor/src/woof_benchmark_micro/src/main/java/net/officefloor/benchmark/CachedWorld.java new file mode 100644 index 00000000000..cdc7199c157 --- /dev/null +++ b/frameworks/Java/officefloor/src/woof_benchmark_micro/src/main/java/net/officefloor/benchmark/CachedWorld.java @@ -0,0 +1,13 @@ +package net.officefloor.benchmark; + +import lombok.AllArgsConstructor; +import lombok.Data; + +@Data +@AllArgsConstructor +public class CachedWorld { + + private int id; + + private int randomNumber; +} \ No newline at end of file diff --git a/frameworks/Java/officefloor/src/woof_benchmark_micro/src/main/java/net/officefloor/benchmark/CachedWorldDataRetriever.java b/frameworks/Java/officefloor/src/woof_benchmark_micro/src/main/java/net/officefloor/benchmark/CachedWorldDataRetriever.java new file mode 100644 index 00000000000..747518d8019 --- /dev/null +++ b/frameworks/Java/officefloor/src/woof_benchmark_micro/src/main/java/net/officefloor/benchmark/CachedWorldDataRetriever.java @@ -0,0 +1,39 @@ +package net.officefloor.benchmark; + +import java.sql.Connection; +import java.sql.ResultSet; +import java.util.HashMap; +import java.util.Map; + +import javax.sql.DataSource; + +import net.officefloor.cache.Cache; +import net.officefloor.cache.constant.ConstantCacheDataRetriever; +import net.officefloor.plugin.clazz.Dependency; + +/** + * {@link ConstantCacheDataRetriever} containing polling logic to refresh the + * {@link Cache}. + * + * @author Daniel Sagenschneider + */ +public class CachedWorldDataRetriever implements ConstantCacheDataRetriever { + + private @Dependency DataSource dataSource; + + @Override + public Map getData() throws Exception { + try (Connection connection = this.dataSource.getConnection()) { + Map data = new HashMap<>(); + ResultSet resultSet = connection.prepareStatement("SELECT id, randomNumber FROM World") + .executeQuery(); + while (resultSet.next()) { + int id = resultSet.getInt("id"); + int randomNumber = resultSet.getInt("randomNumber"); + data.put(id, new CachedWorld(id, randomNumber)); + } + return data; + } + } + +} \ No newline at end of file diff --git a/frameworks/Java/officefloor/src/woof_benchmark_micro/src/main/java/net/officefloor/benchmark/Logic.java b/frameworks/Java/officefloor/src/woof_benchmark_micro/src/main/java/net/officefloor/benchmark/Logic.java index c1d1ff491ae..3c5a0aacca2 100644 --- a/frameworks/Java/officefloor/src/woof_benchmark_micro/src/main/java/net/officefloor/benchmark/Logic.java +++ b/frameworks/Java/officefloor/src/woof_benchmark_micro/src/main/java/net/officefloor/benchmark/Logic.java @@ -16,6 +16,7 @@ import org.apache.commons.text.StringEscapeUtils; import lombok.Data; +import net.officefloor.cache.Cache; import net.officefloor.server.http.HttpHeaderValue; import net.officefloor.server.http.HttpResponse; import net.officefloor.server.http.ServerHttpConnection; @@ -77,6 +78,28 @@ public void queries(@HttpQueryParameter("queries") String queries, DataSource da } } + // ========== CACHED ================== + + public void cached(@HttpQueryParameter("count") String queries, Cache cache, + ObjectResponse response) { + int count = getQueryCount(queries); + + // Set up for unique numbers + ThreadLocalRandom random = ThreadLocalRandom.current(); + + // Obtain the list of cached worlds + CachedWorld[] worlds = new CachedWorld[count]; + for (int i = 0; i < worlds.length; i++) { + + // Obtain unique identifier + int randomNumber = random.nextInt(1, 10001); + + // Obtain the cached world + worlds[i] = cache.get(randomNumber); + } + response.send(worlds); + } + // =========== UPDATES =================== public void update(@HttpQueryParameter("queries") String queries, DataSource dataSource, diff --git a/frameworks/Java/officefloor/src/woof_benchmark_micro/src/main/resources/application.objects b/frameworks/Java/officefloor/src/woof_benchmark_micro/src/main/resources/application.objects index 1eb4f03d16d..c6ec0d60479 100644 --- a/frameworks/Java/officefloor/src/woof_benchmark_micro/src/main/resources/application.objects +++ b/frameworks/Java/officefloor/src/woof_benchmark_micro/src/main/resources/application.objects @@ -4,4 +4,7 @@ + + + \ No newline at end of file diff --git a/frameworks/Java/officefloor/src/woof_benchmark_micro/src/main/resources/application.woof b/frameworks/Java/officefloor/src/woof_benchmark_micro/src/main/resources/application.woof index 5ffbfcbe7c2..ac9ca670a1d 100644 --- a/frameworks/Java/officefloor/src/woof_benchmark_micro/src/main/resources/application.woof +++ b/frameworks/Java/officefloor/src/woof_benchmark_micro/src/main/resources/application.woof @@ -1,5 +1,8 @@ + +
+
@@ -24,15 +27,16 @@
+ - -
+ + diff --git a/frameworks/Java/officefloor/src/woof_benchmark_r2dbc/pom.xml b/frameworks/Java/officefloor/src/woof_benchmark_r2dbc/pom.xml index 66233b0787a..d1957c1a03a 100644 --- a/frameworks/Java/officefloor/src/woof_benchmark_r2dbc/pom.xml +++ b/frameworks/Java/officefloor/src/woof_benchmark_r2dbc/pom.xml @@ -21,6 +21,10 @@ io.r2dbc r2dbc-pool + + net.officefloor.cache + officecache_constant + diff --git a/frameworks/Java/officefloor/src/woof_benchmark_r2dbc/src/main/java/net/officefloor/benchmark/R2dbcOfficeFloorMain.java b/frameworks/Java/officefloor/src/woof_benchmark_r2dbc/src/main/java/net/officefloor/benchmark/R2dbcOfficeFloorMain.java index 00709063d96..ab6af70f454 100644 --- a/frameworks/Java/officefloor/src/woof_benchmark_r2dbc/src/main/java/net/officefloor/benchmark/R2dbcOfficeFloorMain.java +++ b/frameworks/Java/officefloor/src/woof_benchmark_r2dbc/src/main/java/net/officefloor/benchmark/R2dbcOfficeFloorMain.java @@ -1,16 +1,25 @@ package net.officefloor.benchmark; import java.util.Collections; +import java.util.HashMap; +import java.util.Map; import java.util.concurrent.Executor; import java.util.concurrent.ThreadLocalRandom; import io.netty.channel.unix.Socket; import io.r2dbc.pool.PoolingConnectionFactoryProvider; +import io.r2dbc.postgresql.api.PostgresqlException; import io.r2dbc.spi.Batch; import io.r2dbc.spi.Connection; import io.r2dbc.spi.ConnectionFactories; import io.r2dbc.spi.ConnectionFactory; import io.r2dbc.spi.ConnectionFactoryOptions; +import net.officefloor.cache.Cache; +import net.officefloor.cache.constant.ConstantCacheManagedObjectSource; +import net.officefloor.frame.api.managedobject.ManagedObject; +import net.officefloor.frame.util.ManagedObjectSourceStandAlone; +import net.officefloor.frame.util.ManagedObjectUserStandAlone; +import net.officefloor.plugin.managedobject.poll.StatePollContext; import net.officefloor.server.RequestHandler; import net.officefloor.server.http.parse.HttpRequestParser; import reactor.core.publisher.Flux; @@ -38,7 +47,7 @@ public class R2dbcOfficeFloorMain implements DatabaseOperations { /** * Run application. */ - public static void main(String[] args) throws Exception { + public static void main(String[] args) throws Throwable { // Increase the buffer size (note: too high and cause OOM issues) System.setProperty("reactor.bufferSize.small", String.valueOf(QUERY_BUFFER_SIZE)); @@ -58,6 +67,11 @@ public static void main(String[] args) throws Exception { */ private final ThreadLocal threadLocalConnections; + /** + * {@link Cache} of {@link CachedWorld}. + */ + private final Cache cache; + /** * Instantiate. * @@ -68,8 +82,9 @@ public static void main(String[] args) throws Exception { * @param username Username. * @param password Password. */ + @SuppressWarnings("unchecked") public R2dbcOfficeFloorMain(int socketCount, String server, int port, String database, String username, - String password) { + String password) throws Throwable { // Must have enough connection capacity for initial load (+1 for rounding) int requiredConnectionsPerSocket = (QUERY_LOAD_CAPACITY / (socketCount * QUERY_BUFFER_SIZE)) + 1; @@ -99,6 +114,48 @@ protected Connection[] initialValue() { return connections; } }; + + // Provide the cache + ManagedObjectSourceStandAlone source = new ManagedObjectSourceStandAlone(); + source.registerInvokeProcessServicer(0, (processIndex, parameter, managedObject) -> { + + // Poll database for cached data + StatePollContext> pollContext = (StatePollContext>) parameter; + Map data = new HashMap<>(); + try { + Flux.from(connectionFactory.create()).flatMap((connection) -> { + return Flux.from(connection.createStatement("SELECT ID, RANDOMNUMBER FROM WORLD").execute()) + .flatMap(result -> Flux.from(result.map((row, metadata) -> { + Integer id = row.get(0, Integer.class); + Integer randomNumber = row.get(1, Integer.class); + CachedWorld cachedWorld = new CachedWorld(id, randomNumber); + data.put(id, cachedWorld); + return cachedWorld; + }))).last().flatMap(ignore -> Mono.from(connection.close())); + }).blockLast(); + pollContext.setNextState(data, -1, null); + } catch (Exception ex) { + } + }); + ManagedObject cacheMo = new ManagedObjectUserStandAlone() + .sourceManagedObject(source.loadManagedObjectSource(ConstantCacheManagedObjectSource.class)); + this.cache = (Cache) cacheMo.getObject(); + } + + public void sendDatabaseError(Throwable failure, AbstractSendResponse response) { + + // Handle issue of prepared statement not found + // (seems unsafe memory issue in R2DBC that occurs during start then stops) + if (failure instanceof PostgresqlException) { + PostgresqlException postgresqlException = (PostgresqlException) failure; + if ("26000".equals(postgresqlException.getErrorDetails().getCode())) { + // Prepared statement not existing + response.sendError(503); // consider overloaded in connection setup during warm up + } + } + + // Just send the failure + response.sendError(failure); } /* @@ -137,7 +194,7 @@ public void db(DbSendResponse sender) { }))).publishOn(conn.writeScheduler).subscribe(world -> { sender.sendDb(world); }, error -> { - sender.sendError(error); + this.sendDatabaseError(error, sender); }, () -> { conn.processed(1); }); @@ -164,7 +221,7 @@ public void queries(int queryCount, QueriesSendResponse sender) { }))).collectList().publishOn(conn.writeScheduler).subscribe(worlds -> { sender.sendQueries(worlds.toArray(World[]::new)); }, error -> { - sender.sendError(error); + this.sendDatabaseError(error, sender); }, () -> { conn.processed(queryCount); }); @@ -189,7 +246,7 @@ public void fortunes(FortunesSendResponse sender) { }))).collectList().publishOn(conn.writeScheduler).subscribe(fortunes -> { sender.sendFortunes(fortunes); }, error -> { - sender.sendError(error); + this.sendDatabaseError(error, sender); }, () -> { conn.processed(1); }); @@ -233,12 +290,33 @@ public void update(int queryCount, UpdateSendResponse sender) { }).publishOn(conn.writeScheduler).subscribe(worlds -> { sender.sendUpdate(worlds.toArray(World[]::new)); }, error -> { - sender.sendError(error); + this.sendDatabaseError(error, sender); }, () -> { conn.processed(executeQueryCount); }); } + @Override + public void cached(int queryCount, CachedSendResponse sender) { + + // Set up for unique numbers + ThreadLocalRandom random = ThreadLocalRandom.current(); + + // Obtain the list of cached worlds + CachedWorld[] cachedWorlds = new CachedWorld[queryCount]; + for (int i = 0; i < cachedWorlds.length; i++) { + + // Obtain unique identifier + int randomNumber = random.nextInt(1, 10001); + + // Obtain the cached world + cachedWorlds[i] = cache.get(randomNumber); + } + + // Send cached worlds + sender.sendCached(cachedWorlds); + } + private static class RateLimit { private final RateLimitedConnection[] rateLimitedConnections; diff --git a/frameworks/Java/officefloor/src/woof_benchmark_spring/pom.xml b/frameworks/Java/officefloor/src/woof_benchmark_spring/pom.xml index 19052724d81..8ffb5502d7e 100644 --- a/frameworks/Java/officefloor/src/woof_benchmark_spring/pom.xml +++ b/frameworks/Java/officefloor/src/woof_benchmark_spring/pom.xml @@ -28,6 +28,10 @@ net.officefloor.persistence officejdbc_postgresql + + net.officefloor.cache + officecache_constant + org.projectlombok lombok diff --git a/frameworks/Java/officefloor/src/woof_benchmark_spring/src/main/java/net/officefloor/benchmark/CachedWorld.java b/frameworks/Java/officefloor/src/woof_benchmark_spring/src/main/java/net/officefloor/benchmark/CachedWorld.java new file mode 100644 index 00000000000..90d67f57e93 --- /dev/null +++ b/frameworks/Java/officefloor/src/woof_benchmark_spring/src/main/java/net/officefloor/benchmark/CachedWorld.java @@ -0,0 +1,11 @@ +package net.officefloor.benchmark; + +import lombok.Data; + +@Data +public class CachedWorld { + + private final int id; + + private final int randomNumber; +} \ No newline at end of file diff --git a/frameworks/Java/officefloor/src/woof_benchmark_spring/src/main/java/net/officefloor/benchmark/CachedWorldDataRetriever.java b/frameworks/Java/officefloor/src/woof_benchmark_spring/src/main/java/net/officefloor/benchmark/CachedWorldDataRetriever.java new file mode 100644 index 00000000000..fce86fb64e1 --- /dev/null +++ b/frameworks/Java/officefloor/src/woof_benchmark_spring/src/main/java/net/officefloor/benchmark/CachedWorldDataRetriever.java @@ -0,0 +1,29 @@ +package net.officefloor.benchmark; + +import java.util.HashMap; +import java.util.Map; + +import net.officefloor.cache.Cache; +import net.officefloor.cache.constant.ConstantCacheDataRetriever; +import net.officefloor.plugin.clazz.Dependency; + +/** + * {@link ConstantCacheDataRetriever} containing polling logic to refresh the + * {@link Cache}. + * + * @author Daniel Sagenschneider + */ +public class CachedWorldDataRetriever implements ConstantCacheDataRetriever { + + private @Dependency WorldRepository repository; + + @Override + public Map getData() throws Exception { + Map data = new HashMap<>(); + for (World world : this.repository.findAll()) { + data.put(world.getId(), new CachedWorld(world.getId(), world.getRandomNumber())); + } + return data; + } + +} \ No newline at end of file diff --git a/frameworks/Java/officefloor/src/woof_benchmark_spring/src/main/java/net/officefloor/benchmark/Fortune.java b/frameworks/Java/officefloor/src/woof_benchmark_spring/src/main/java/net/officefloor/benchmark/Fortune.java index 63d1f46f8f6..f54662f4d42 100644 --- a/frameworks/Java/officefloor/src/woof_benchmark_spring/src/main/java/net/officefloor/benchmark/Fortune.java +++ b/frameworks/Java/officefloor/src/woof_benchmark_spring/src/main/java/net/officefloor/benchmark/Fortune.java @@ -18,7 +18,4 @@ public class Fortune { private String message; - public String getMessage() { - return this.message; - } } diff --git a/frameworks/Java/officefloor/src/woof_benchmark_spring/src/main/java/net/officefloor/benchmark/Logic.java b/frameworks/Java/officefloor/src/woof_benchmark_spring/src/main/java/net/officefloor/benchmark/Logic.java index b293138fbac..e8107c2939f 100644 --- a/frameworks/Java/officefloor/src/woof_benchmark_spring/src/main/java/net/officefloor/benchmark/Logic.java +++ b/frameworks/Java/officefloor/src/woof_benchmark_spring/src/main/java/net/officefloor/benchmark/Logic.java @@ -5,6 +5,7 @@ import java.util.concurrent.ThreadLocalRandom; import lombok.Data; +import net.officefloor.cache.Cache; import net.officefloor.web.HttpQueryParameter; import net.officefloor.web.ObjectResponse; @@ -65,6 +66,28 @@ public void queries(@HttpQueryParameter("queries") String queries, WorldReposito response.send(worlds); } + // ========== CACHED ================== + + public void cached(@HttpQueryParameter("count") String queries, Cache cache, + ObjectResponse response) { + int count = getQueryCount(queries); + + // Set up for unique numbers + ThreadLocalRandom random = ThreadLocalRandom.current(); + + // Obtain the list of cached worlds + CachedWorld[] worlds = new CachedWorld[count]; + for (int i = 0; i < worlds.length; i++) { + + // Obtain unique identifier + int randomNumber = random.nextInt(1, 10001); + + // Obtain the cached world + worlds[i] = cache.get(randomNumber); + } + response.send(worlds); + } + // =========== UPDATES =================== public void update(@HttpQueryParameter("queries") String queries, WorldRepository repository, diff --git a/frameworks/Java/officefloor/src/woof_benchmark_spring/src/main/resources/application.objects b/frameworks/Java/officefloor/src/woof_benchmark_spring/src/main/resources/application.objects index 1107ee509a8..7ae29e4f234 100644 --- a/frameworks/Java/officefloor/src/woof_benchmark_spring/src/main/resources/application.objects +++ b/frameworks/Java/officefloor/src/woof_benchmark_spring/src/main/resources/application.objects @@ -4,4 +4,7 @@ + + + \ No newline at end of file diff --git a/frameworks/Java/officefloor/src/woof_benchmark_spring/src/main/resources/application.woof b/frameworks/Java/officefloor/src/woof_benchmark_spring/src/main/resources/application.woof index bc105d33c66..6fa74e2fc87 100644 --- a/frameworks/Java/officefloor/src/woof_benchmark_spring/src/main/resources/application.woof +++ b/frameworks/Java/officefloor/src/woof_benchmark_spring/src/main/resources/application.woof @@ -1,5 +1,8 @@ + +
+
@@ -23,12 +26,15 @@
+
+ + diff --git a/frameworks/Java/officefloor/src/woof_benchmark_sqlclient/src/main/java/net/officefloor/benchmark/SqlClientOfficeFloorMain.java b/frameworks/Java/officefloor/src/woof_benchmark_sqlclient/src/main/java/net/officefloor/benchmark/SqlClientOfficeFloorMain.java index 4b39c99c94a..53454a12c51 100644 --- a/frameworks/Java/officefloor/src/woof_benchmark_sqlclient/src/main/java/net/officefloor/benchmark/SqlClientOfficeFloorMain.java +++ b/frameworks/Java/officefloor/src/woof_benchmark_sqlclient/src/main/java/net/officefloor/benchmark/SqlClientOfficeFloorMain.java @@ -29,7 +29,7 @@ public class SqlClientOfficeFloorMain implements DatabaseOperations { /** * Run application. */ - public static void main(String[] args) throws Exception { + public static void main(String[] args) throws Throwable { RawWoof.run(args, (socketCount, server, port, database, username, password) -> new SqlClientOfficeFloorMain(socketCount, server, port, database, username, password)); } @@ -201,4 +201,9 @@ public void update(int queryCount, UpdateSendResponse sender) { } } + @Override + public void cached(int updateCount, CachedSendResponse sender) { + throw new UnsupportedOperationException("/cached-worlds test not supported"); + } + } \ No newline at end of file diff --git a/frameworks/Java/officefloor/src/woof_benchmark_woof/pom.xml b/frameworks/Java/officefloor/src/woof_benchmark_woof/pom.xml index 5551b3f815c..bf53e3a003a 100644 --- a/frameworks/Java/officefloor/src/woof_benchmark_woof/pom.xml +++ b/frameworks/Java/officefloor/src/woof_benchmark_woof/pom.xml @@ -8,9 +8,6 @@ 1.0.0 woof_benchmark_woof - - 29.0-jre - net.officefloor.web @@ -37,7 +34,6 @@ com.google.guava guava - ${guava-version} \ No newline at end of file diff --git a/frameworks/Java/officefloor/src/woof_benchmark_woof/src/main/java/net/officefloor/benchmark/CachedSendResponse.java b/frameworks/Java/officefloor/src/woof_benchmark_woof/src/main/java/net/officefloor/benchmark/CachedSendResponse.java new file mode 100644 index 00000000000..314a87dec71 --- /dev/null +++ b/frameworks/Java/officefloor/src/woof_benchmark_woof/src/main/java/net/officefloor/benchmark/CachedSendResponse.java @@ -0,0 +1,42 @@ +package net.officefloor.benchmark; + +import java.io.IOException; +import java.nio.ByteBuffer; +import java.nio.channels.CancelledKeyException; +import java.nio.channels.ClosedChannelException; + +import com.fasterxml.jackson.databind.ObjectMapper; + +import net.officefloor.server.RequestHandler; +import net.officefloor.server.http.HttpResponse; +import net.officefloor.server.http.impl.ProcessAwareServerHttpConnectionManagedObject; +import net.officefloor.server.http.parse.HttpRequestParser; + +/** + * Sends the Queries response. + * + * @author Daniel Sagenschneider + */ +public class CachedSendResponse extends AbstractSendResponse { + + public CachedSendResponse(RequestHandler requestHandler, + ProcessAwareServerHttpConnectionManagedObject connection, ObjectMapper objectMapper) { + super(requestHandler, connection, objectMapper); + } + + public void sendCached(CachedWorld[] cachedWorlds) { + this.requestHandler.execute(() -> { + try { + HttpResponse response = this.connection.getResponse(); + response.setContentType(APPLICATION_JSON, null); + this.objectMapper.writeValue(response.getEntityWriter(), cachedWorlds); + send(this.connection); + } catch (CancelledKeyException | ClosedChannelException ex) { + // Ignore as disconnecting client + } catch (IOException ex) { + ex.printStackTrace(); + } + }); + } + +} \ No newline at end of file diff --git a/frameworks/Java/officefloor/src/woof_benchmark_woof/src/main/java/net/officefloor/benchmark/CachedWorld.java b/frameworks/Java/officefloor/src/woof_benchmark_woof/src/main/java/net/officefloor/benchmark/CachedWorld.java new file mode 100644 index 00000000000..e27acf4bdb5 --- /dev/null +++ b/frameworks/Java/officefloor/src/woof_benchmark_woof/src/main/java/net/officefloor/benchmark/CachedWorld.java @@ -0,0 +1,18 @@ +package net.officefloor.benchmark; + +/** + * Cached world response. + * + * @author Daniel Sagenschneider + */ +public class CachedWorld { + + public final int id; + + public final int randomNumber; + + public CachedWorld(int id, int randomNumber) { + this.id = id; + this.randomNumber = randomNumber; + } +} \ No newline at end of file diff --git a/frameworks/Java/officefloor/src/woof_benchmark_woof/src/main/java/net/officefloor/benchmark/DatabaseOperations.java b/frameworks/Java/officefloor/src/woof_benchmark_woof/src/main/java/net/officefloor/benchmark/DatabaseOperations.java index 74a2a5c8aa9..d574672446b 100644 --- a/frameworks/Java/officefloor/src/woof_benchmark_woof/src/main/java/net/officefloor/benchmark/DatabaseOperations.java +++ b/frameworks/Java/officefloor/src/woof_benchmark_woof/src/main/java/net/officefloor/benchmark/DatabaseOperations.java @@ -38,4 +38,9 @@ public interface DatabaseOperations { */ void update(int updateCount, UpdateSendResponse sender); + /** + * Undertakes the cached. + */ + void cached(int queryCount, CachedSendResponse sender); + } diff --git a/frameworks/Java/officefloor/src/woof_benchmark_woof/src/main/java/net/officefloor/benchmark/DatabaseOperationsFactory.java b/frameworks/Java/officefloor/src/woof_benchmark_woof/src/main/java/net/officefloor/benchmark/DatabaseOperationsFactory.java index 5695948bc07..1d09f35b714 100644 --- a/frameworks/Java/officefloor/src/woof_benchmark_woof/src/main/java/net/officefloor/benchmark/DatabaseOperationsFactory.java +++ b/frameworks/Java/officefloor/src/woof_benchmark_woof/src/main/java/net/officefloor/benchmark/DatabaseOperationsFactory.java @@ -19,8 +19,9 @@ public interface DatabaseOperationsFactory { * @param username Username. * @param password Password. * @return {@link DatabaseOperations}. + * @throws Throwable If fails to create {@link DatabaseOperations}. */ DatabaseOperations createDatabaseOperations(int socketCount, String server, int port, String database, - String username, String password); + String username, String password) throws Throwable; } \ No newline at end of file diff --git a/frameworks/Java/officefloor/src/woof_benchmark_woof/src/main/java/net/officefloor/benchmark/FortunesSendResponse.java b/frameworks/Java/officefloor/src/woof_benchmark_woof/src/main/java/net/officefloor/benchmark/FortunesSendResponse.java index 71e351839c9..e6301dd32e0 100644 --- a/frameworks/Java/officefloor/src/woof_benchmark_woof/src/main/java/net/officefloor/benchmark/FortunesSendResponse.java +++ b/frameworks/Java/officefloor/src/woof_benchmark_woof/src/main/java/net/officefloor/benchmark/FortunesSendResponse.java @@ -9,8 +9,6 @@ import java.util.Comparator; import java.util.List; -import org.apache.commons.text.StringEscapeUtils; - import com.fasterxml.jackson.databind.ObjectMapper; import com.google.common.escape.Escaper; import com.google.common.html.HtmlEscapers; @@ -22,7 +20,6 @@ import net.officefloor.server.http.impl.ProcessAwareServerHttpConnectionManagedObject; import net.officefloor.server.http.parse.HttpRequestParser; import net.officefloor.server.stream.ServerOutputStream; -import net.officefloor.server.stream.ServerWriter; /** * Sends the Fortunes response. diff --git a/frameworks/Java/officefloor/src/woof_benchmark_woof/src/main/java/net/officefloor/benchmark/RawWoof.java b/frameworks/Java/officefloor/src/woof_benchmark_woof/src/main/java/net/officefloor/benchmark/RawWoof.java index 818c7623105..68fcc1245f3 100644 --- a/frameworks/Java/officefloor/src/woof_benchmark_woof/src/main/java/net/officefloor/benchmark/RawWoof.java +++ b/frameworks/Java/officefloor/src/woof_benchmark_woof/src/main/java/net/officefloor/benchmark/RawWoof.java @@ -79,7 +79,7 @@ public abstract class RawWoof { * @param args Command line arguments. * @param operationsFactory {@link DatabaseOperationsFactory}. */ - public static void run(String[] args, DatabaseOperationsFactory operationsFactory) throws Exception { + public static void run(String[] args, DatabaseOperationsFactory operationsFactory) throws Throwable { // Obtain the port from properties int port = args.length > 0 ? Integer.parseInt(args[0]) : 8080; @@ -147,6 +147,8 @@ private static class RawHttpServicerFactory extends AbstractHttpServicerFactory private static final String UPDATE_PATH_PREFIX = "/update?queries="; + private static final String CACHED_PATH_PREFIX = "/cached-worlds?count="; + /** * Date {@link HttpHeaderValue}. */ @@ -298,6 +300,15 @@ protected ProcessManager service(ProcessAwareServerHttpConnectionManagedObject Date: Tue, 22 Jun 2021 10:35:56 -0500 Subject: [PATCH 15/25] Add TypeScript Deno + MongoDB Raw Test Cases (#6445) * Update standard library version from `v0.50.0` to `0.87.0` * Update standard library version from `v0.50.0` to `0.87.0` * Refactor Deno test cases (1) * Refactor Deno test cases(2) * Refactor Deno test cases (3) * Fix Dockerfile * Update Deno `mongo` to 0.22.0 * Fix Deno `dockerfile` * Update Deno Docker * Fix Deno `dyn_date` * Fix Deno `dyn_date` * Fix Deno `dyn_date` * Fix Deno `export` * Fix Deno `dyn_date` * Add Deno `mongodb-raw` Test Cases * Add Deno `mongodb-raw` Test Cases * Fix Deno `main.ts` import error * Fix Number Resolve * Fix `resolveQueryNumber`&`generateFortunes` * *Debug* Updates Error * Fix fillArray() * Update Deno version * Update MongoClient Version * Update Deno Http Version * Back Mongo Client Version * Change Deno std version * Change Deno std Version * Fix Types & Delete Debug Log * Fix Dockerfile * Fix Dockerfile * Update build.yml * Update build.yml --- .../TypeScript/deno/benchmark_config.json | 61 ++++++++++------ frameworks/TypeScript/deno/config.toml | 18 +++++ frameworks/TypeScript/deno/deno.dockerfile | 2 +- .../TypeScript/deno/deno.mongoraw.dockerfile | 18 +++++ .../{_db_helpers.ts => database.ts} | 50 +++++++++---- .../src/_handlers/mongodb-raw/fortunes.ts | 2 +- .../src/_handlers/mongodb-raw/handlers.ts | 12 ++++ .../_handlers/mongodb-raw/multiple-queries.ts | 6 +- .../src/_handlers/mongodb-raw/single-query.ts | 2 +- .../deno/src/_handlers/mongodb-raw/updates.ts | 6 +- frameworks/TypeScript/deno/src/depends.ts | 4 +- frameworks/TypeScript/deno/src/handlers.ts | 9 +-- .../TypeScript/deno/src/main.mongoraw.ts | 22 ++++++ frameworks/TypeScript/deno/src/main.ts | 8 +-- frameworks/TypeScript/deno/src/tsconfig.json | 70 +++++++++++++++++++ 15 files changed, 238 insertions(+), 52 deletions(-) create mode 100644 frameworks/TypeScript/deno/deno.mongoraw.dockerfile rename frameworks/TypeScript/deno/src/_handlers/mongodb-raw/{_db_helpers.ts => database.ts} (62%) create mode 100644 frameworks/TypeScript/deno/src/_handlers/mongodb-raw/handlers.ts create mode 100644 frameworks/TypeScript/deno/src/main.mongoraw.ts create mode 100644 frameworks/TypeScript/deno/src/tsconfig.json diff --git a/frameworks/TypeScript/deno/benchmark_config.json b/frameworks/TypeScript/deno/benchmark_config.json index 9594233eb98..a1335f06c95 100644 --- a/frameworks/TypeScript/deno/benchmark_config.json +++ b/frameworks/TypeScript/deno/benchmark_config.json @@ -1,21 +1,42 @@ { - "framework": "deno", - "tests": [{ - "default": { - "json_url": "/json", - "plaintext_url": "/plaintext", - "port": 8080, - "approach": "Realistic", - "classification": "Platform", - "framework": "None", - "language": "TypeScript", - "flavor": "deno", - "platform": "deno", - "webserver": "None", - "os": "Linux", - "database_os": "Linux", - "display_name": "deno", - "versus": "nodejs" - } - }] -} + "framework": "deno", + "tests": [{ + "default": { + "json_url": "/json", + "plaintext_url": "/plaintext", + "port": 8080, + "approach": "Realistic", + "classification": "Platform", + "language": "TypeScript", + "flavor": "deno", + "platform": "deno", + "webserver": "None", + "os": "Linux", + "database_os": "Linux", + "display_name": "deno", + "versus": "nodejs" + }, + "mongodb-raw": { + "dockerfile": "deno.mongoraw.dockerfile", + "json_url": "/json", + "plaintext_url": "/plaintext", + "db_url": "/db", + "query_url": "/queries?queries=", + "update_url": "/updates?queries=", + "fortune_url": "/fortunes", + "port": 8080, + "approach": "Realistic", + "classification": "Platform", + "database": "MongoDB", + "orm": "Raw", + "language": "TypeScript", + "flavor": "deno", + "platform": "deno", + "webserver": "None", + "os": "Linux", + "database_os": "Linux", + "display_name": "deno", + "versus": "nodejs" + } + }] +} \ No newline at end of file diff --git a/frameworks/TypeScript/deno/config.toml b/frameworks/TypeScript/deno/config.toml index a2382d14449..86d75ad7aad 100644 --- a/frameworks/TypeScript/deno/config.toml +++ b/frameworks/TypeScript/deno/config.toml @@ -17,3 +17,21 @@ orm = "Raw" platform = "deno" webserver = "None" versus = "nodejs" + +[mongodb] +urls.plaintext = "/plaintext" +urls.json = "/json" +urls.db = "/db" +urls.query = "/queries?queries=" +urls.update = "/updates?queries=" +urls.fortune = "/fortunes" +urls.cached_query = "/cached?queries=" +approach = "Realistic" +classification = "Platform" +database = "MongoDB" +database_os = "Linux" +os = "Linux" +orm = "Raw" +platform = "nodejs" +webserver = "None" +versus = "nodejs" diff --git a/frameworks/TypeScript/deno/deno.dockerfile b/frameworks/TypeScript/deno/deno.dockerfile index 44f6b8e6fd1..49de0d52b00 100644 --- a/frameworks/TypeScript/deno/deno.dockerfile +++ b/frameworks/TypeScript/deno/deno.dockerfile @@ -1,4 +1,4 @@ -FROM hayd/alpine-deno:1.8.1 +FROM hayd/alpine-deno:latest EXPOSE 8080 diff --git a/frameworks/TypeScript/deno/deno.mongoraw.dockerfile b/frameworks/TypeScript/deno/deno.mongoraw.dockerfile new file mode 100644 index 00000000000..9f8a28fad08 --- /dev/null +++ b/frameworks/TypeScript/deno/deno.mongoraw.dockerfile @@ -0,0 +1,18 @@ +FROM hayd/alpine-deno:latest + + +EXPOSE 8080 + +WORKDIR /app + +USER deno +ENV DATABASE mongodb + +COPY ./src . + +RUN deno cache main.mongoraw.ts + +EXPOSE 8080 + +CMD ["run", "--allow-net", "main.mongoraw.ts"] + diff --git a/frameworks/TypeScript/deno/src/_handlers/mongodb-raw/_db_helpers.ts b/frameworks/TypeScript/deno/src/_handlers/mongodb-raw/database.ts similarity index 62% rename from frameworks/TypeScript/deno/src/_handlers/mongodb-raw/_db_helpers.ts rename to frameworks/TypeScript/deno/src/_handlers/mongodb-raw/database.ts index ac132d8ec1a..507c5124a12 100644 --- a/frameworks/TypeScript/deno/src/_handlers/mongodb-raw/_db_helpers.ts +++ b/frameworks/TypeScript/deno/src/_handlers/mongodb-raw/database.ts @@ -10,9 +10,9 @@ export let Fortune = client.database(dbName).collection("fortune"); export const randomNumber = (): number => Math.floor(Math.random() * 10000) + 1; -export const fillArray = async (v: T, l: number) => { +export const fillArrayWithFn = async(v: Function, l: number) => { let o = []; - for (let i = 0; i < l; i += 1) o.push(v); + for (let i = 0; i < l; i += 1) o.push(await v()); return o; }; @@ -21,6 +21,17 @@ export interface FortuneData { message: string; _id?: unknown; } +export const htmlEncodeByRegExp = (str: string) => { + let s = ""; + if (str.length == 0) return ""; + s = str.replaceAll(/&/g, "&"); + s = s.replaceAll(//g, ">"); + // s = s.replaceAll(/ /g, " "); + s = s.replaceAll(/\'/g, "'"); + s = s.replaceAll(/\"/g, '"'); + return s; +}; export const _fortunes_head = [ "", "", @@ -34,18 +45,19 @@ export const _fortunes_head = [ ].join(""); export const _fortunes_end = ["", "", ""].join(""); export const _fortunes_com = ["", "", "", ""]; -export const generateFortunes = (input: FortuneData[]): string => { +export const generateFortunes = (input: FortuneData[]) => { let f = input - .map( - (v) => - _fortunes_com[0] + - +_fortunes_com[2] + - v.id + - _fortunes_com[3] + - _fortunes_com[2] + - v.message + - _fortunes_com[3] + - _fortunes_com[1] + .map((v) => + [ + _fortunes_com[0], + _fortunes_com[2], + v.id.toString(), + _fortunes_com[3], + _fortunes_com[2], + htmlEncodeByRegExp(v.message), + _fortunes_com[3], + _fortunes_com[1], + ].join("") ) .join(""); @@ -85,3 +97,15 @@ export const additionalFortune = { id: 0, message: "Additional fortune added at request time.", }; + +export const resolveQueryNumber = (s: string) => { + let r: number; + if (/^\d+$/.test(s)) { + r = Number(s); + if (r > 500) r = 500; + if (r < 1) r = 1; + } else { + r = 1; + } + return r; +}; diff --git a/frameworks/TypeScript/deno/src/_handlers/mongodb-raw/fortunes.ts b/frameworks/TypeScript/deno/src/_handlers/mongodb-raw/fortunes.ts index 22839297688..dcca21f523b 100644 --- a/frameworks/TypeScript/deno/src/_handlers/mongodb-raw/fortunes.ts +++ b/frameworks/TypeScript/deno/src/_handlers/mongodb-raw/fortunes.ts @@ -4,7 +4,7 @@ import { additionalFortune, generateFortunes, FortuneData, -} from "./_db_helpers.ts"; +} from "./database.ts"; export const headers = new Headers([ ["server", SERVER], diff --git a/frameworks/TypeScript/deno/src/_handlers/mongodb-raw/handlers.ts b/frameworks/TypeScript/deno/src/_handlers/mongodb-raw/handlers.ts new file mode 100644 index 00000000000..c6f5ddd4511 --- /dev/null +++ b/frameworks/TypeScript/deno/src/_handlers/mongodb-raw/handlers.ts @@ -0,0 +1,12 @@ +import { Handlers } from "./../../handlers.ts"; +import FortunesHandler from "./fortunes.ts"; +import MultipleQueriesHandler from "./multiple-queries.ts"; +import SingleQueryHandler from "./single-query.ts"; +import UpdatesHandler from "./updates.ts"; + +export const MongoRawHandlers = { + "/db": SingleQueryHandler, + "/queries": MultipleQueriesHandler, + "/updates": UpdatesHandler, + "/fortunes": FortunesHandler, +} as Handlers; diff --git a/frameworks/TypeScript/deno/src/_handlers/mongodb-raw/multiple-queries.ts b/frameworks/TypeScript/deno/src/_handlers/mongodb-raw/multiple-queries.ts index 30537e601e6..143ba899913 100644 --- a/frameworks/TypeScript/deno/src/_handlers/mongodb-raw/multiple-queries.ts +++ b/frameworks/TypeScript/deno/src/_handlers/mongodb-raw/multiple-queries.ts @@ -1,5 +1,5 @@ import { ServerRequest, SERVER, dyn_date, MIME_JSON } from "../../depends.ts"; -import { randomWorld, fillArray } from "./_db_helpers.ts"; +import { randomWorld, fillArrayWithFn, resolveQueryNumber } from "./database.ts"; export const headers = new Headers([ ["server", SERVER], @@ -8,8 +8,8 @@ export const headers = new Headers([ export default async (req: ServerRequest): Promise => { const u = new URL(req.url, "http://deno"); - const l = Number(u.searchParams.get("queries")); - const rnd = await Promise.all(await fillArray(randomWorld(), l)); + const l = resolveQueryNumber(u.searchParams.get("queries") ?? "1"); + const rnd = await Promise.all(await fillArrayWithFn(() => randomWorld(), l)); headers.set("date", dyn_date()); req.respond({ headers, diff --git a/frameworks/TypeScript/deno/src/_handlers/mongodb-raw/single-query.ts b/frameworks/TypeScript/deno/src/_handlers/mongodb-raw/single-query.ts index 5d25c48cc2a..f9a1c2ebdfb 100644 --- a/frameworks/TypeScript/deno/src/_handlers/mongodb-raw/single-query.ts +++ b/frameworks/TypeScript/deno/src/_handlers/mongodb-raw/single-query.ts @@ -1,5 +1,5 @@ import { ServerRequest, SERVER, dyn_date, MIME_JSON } from "../../depends.ts"; -import { randomWorld } from "./_db_helpers.ts"; +import { randomWorld } from "./database.ts"; export const headers = new Headers([ ["server", SERVER], diff --git a/frameworks/TypeScript/deno/src/_handlers/mongodb-raw/updates.ts b/frameworks/TypeScript/deno/src/_handlers/mongodb-raw/updates.ts index 3629c95d8b7..bd7e720eb09 100644 --- a/frameworks/TypeScript/deno/src/_handlers/mongodb-raw/updates.ts +++ b/frameworks/TypeScript/deno/src/_handlers/mongodb-raw/updates.ts @@ -1,5 +1,5 @@ import { ServerRequest, SERVER, dyn_date, MIME_JSON } from "../../depends.ts"; -import { updateQuery, fillArray } from "./_db_helpers.ts"; +import { updateQuery, fillArrayWithFn, resolveQueryNumber } from "./database.ts"; export const headers = new Headers([ ["server", SERVER], @@ -8,8 +8,8 @@ export const headers = new Headers([ export default async (req: ServerRequest): Promise => { const u = new URL(req.url, "http://deno"); - const l = Number(u.searchParams.get("queries")); - const rnd = await Promise.all(await fillArray(updateQuery(), l)); + const l = resolveQueryNumber(u.searchParams.get("queries") ?? "1"); + const rnd = await Promise.all(await fillArrayWithFn(() => updateQuery(), l)); headers.set("date", dyn_date()); req.respond({ headers, diff --git a/frameworks/TypeScript/deno/src/depends.ts b/frameworks/TypeScript/deno/src/depends.ts index cd9cf9bac30..f20354e3d29 100644 --- a/frameworks/TypeScript/deno/src/depends.ts +++ b/frameworks/TypeScript/deno/src/depends.ts @@ -1,9 +1,9 @@ -export type { Response } from "https://deno.land/std@0.87.0/http/server.ts"; +export type { Response } from "https://deno.land/std@0.96.0/http/server.ts"; export { ServerRequest, serve, -} from "https://deno.land/std@0.87.0/http/server.ts"; +} from "https://deno.land/std@0.96.0/http/server.ts"; export { MongoClient, Bson } from "https://deno.land/x/mongo@v0.22.0/mod.ts"; diff --git a/frameworks/TypeScript/deno/src/handlers.ts b/frameworks/TypeScript/deno/src/handlers.ts index b95f9aee40c..bb217650841 100644 --- a/frameworks/TypeScript/deno/src/handlers.ts +++ b/frameworks/TypeScript/deno/src/handlers.ts @@ -2,15 +2,16 @@ import type { ServerRequest } from "./depends.ts"; import JSONHandler from "./_handlers/json.ts"; import PlaintextHandler from "./_handlers/plaintext.ts"; -interface Handler { +export interface Handler { (request: ServerRequest): Promise; } -interface Handlers { +export interface Handlers { [index: string]: Handler; } -export const handlers: Handlers = { +export default { "/json": JSONHandler, "/plaintext": PlaintextHandler, -}; +} as Handlers; + diff --git a/frameworks/TypeScript/deno/src/main.mongoraw.ts b/frameworks/TypeScript/deno/src/main.mongoraw.ts new file mode 100644 index 00000000000..a7df8c2dfa6 --- /dev/null +++ b/frameworks/TypeScript/deno/src/main.mongoraw.ts @@ -0,0 +1,22 @@ +import { serve } from "https://deno.land/std@0.96.0/http/server.ts"; +import DefaultHandlers from "./handlers.ts"; +import { MongoRawHandlers } from "./_handlers/mongodb-raw/handlers.ts"; +const handlers = { + ...DefaultHandlers, + ...MongoRawHandlers, +}; + +for await (const req of serve("0.0.0.0:8080")) { + const url = new URL(req.url, "http://deno"); + if (handlers[url.pathname] != undefined) { + handlers[url.pathname](req as any).catch((e) => { + console.error(e); + Deno.exit(9); + }); + } else { + req.respond({ + body: "404 Not Found", + }); + } + continue; +} diff --git a/frameworks/TypeScript/deno/src/main.ts b/frameworks/TypeScript/deno/src/main.ts index fcb0bb562de..9b52d044fcc 100644 --- a/frameworks/TypeScript/deno/src/main.ts +++ b/frameworks/TypeScript/deno/src/main.ts @@ -1,8 +1,8 @@ -import { serve } from "https://deno.land/std@0.87.0/http/server.ts"; -import { handlers } from "./handlers.ts"; +import { serve } from "https://deno.land/std@0.96.0/http/server.ts"; +import Handlers from "./handlers.ts"; for await (const req of serve("0.0.0.0:8080")) { - if (handlers[req.url] != undefined) { - handlers[req.url](req).catch((e) => { + if (Handlers[req.url] != undefined) { + Handlers[req.url](req as any).catch((e) => { console.error(e); Deno.exit(9); }); diff --git a/frameworks/TypeScript/deno/src/tsconfig.json b/frameworks/TypeScript/deno/src/tsconfig.json new file mode 100644 index 00000000000..acba78e692b --- /dev/null +++ b/frameworks/TypeScript/deno/src/tsconfig.json @@ -0,0 +1,70 @@ +{ + "compilerOptions": { + /* Visit https://aka.ms/tsconfig.json to read more about this file */ + + /* Basic Options */ + // "incremental": true, /* Enable incremental compilation */ + "target": "esnext", /* Specify ECMAScript target version: 'ES3' (default), 'ES5', 'ES2015', 'ES2016', 'ES2017', 'ES2018', 'ES2019', 'ES2020', or 'ESNEXT'. */ + "module": "esnext", /* Specify module code generation: 'none', 'commonjs', 'amd', 'system', 'umd', 'es2015', 'es2020', or 'ESNext'. */ + // "lib": [], /* Specify library files to be included in the compilation. */ + // "allowJs": true, /* Allow javascript files to be compiled. */ + // "checkJs": true, /* Report errors in .js files. */ + // "jsx": "preserve", /* Specify JSX code generation: 'preserve', 'react-native', or 'react'. */ + // "declaration": true, /* Generates corresponding '.d.ts' file. */ + // "declarationMap": true, /* Generates a sourcemap for each corresponding '.d.ts' file. */ + // "sourceMap": true, /* Generates corresponding '.map' file. */ + // "outFile": "./", /* Concatenate and emit output to single file. */ + // "outDir": "./", /* Redirect output structure to the directory. */ + // "rootDir": "./", /* Specify the root directory of input files. Use to control the output directory structure with --outDir. */ + // "composite": true, /* Enable project compilation */ + // "tsBuildInfoFile": "./", /* Specify file to store incremental compilation information */ + // "removeComments": true, /* Do not emit comments to output. */ + // "noEmit": true, /* Do not emit outputs. */ + // "importHelpers": true, /* Import emit helpers from 'tslib'. */ + // "downlevelIteration": true, /* Provide full support for iterables in 'for-of', spread, and destructuring when targeting 'ES5' or 'ES3'. */ + // "isolatedModules": true, /* Transpile each file as a separate module (similar to 'ts.transpileModule'). */ + + /* Strict Type-Checking Options */ + "strict": false, /* Enable all strict type-checking options. */ + "noImplicitAny": false, /* Raise error on expressions and declarations with an implied 'any' type. */ + // "strictNullChecks": true, /* Enable strict null checks. */ + "strictFunctionTypes": false, /* Enable strict checking of function types. */ + // "strictBindCallApply": true, /* Enable strict 'bind', 'call', and 'apply' methods on functions. */ + // "strictPropertyInitialization": true, /* Enable strict checking of property initialization in classes. */ + // "noImplicitThis": true, /* Raise error on 'this' expressions with an implied 'any' type. */ + // "alwaysStrict": true, /* Parse in strict mode and emit "use strict" for each source file. */ + + /* Additional Checks */ + // "noUnusedLocals": true, /* Report errors on unused locals. */ + // "noUnusedParameters": true, /* Report errors on unused parameters. */ + // "noImplicitReturns": true, /* Report error when not all code paths in function return a value. */ + // "noFallthroughCasesInSwitch": true, /* Report errors for fallthrough cases in switch statement. */ + // "noUncheckedIndexedAccess": true, /* Include 'undefined' in index signature results */ + + /* Module Resolution Options */ + // "moduleResolution": "node", /* Specify module resolution strategy: 'node' (Node.js) or 'classic' (TypeScript pre-1.6). */ + // "baseUrl": "./", /* Base directory to resolve non-absolute module names. */ + // "paths": {}, /* A series of entries which re-map imports to lookup locations relative to the 'baseUrl'. */ + // "rootDirs": [], /* List of root folders whose combined content represents the structure of the project at runtime. */ + // "typeRoots": [], /* List of folders to include type definitions from. */ + // "types": [], /* Type declaration files to be included in compilation. */ + // "allowSyntheticDefaultImports": true, /* Allow default imports from modules with no default export. This does not affect code emit, just typechecking. */ + "esModuleInterop": true, /* Enables emit interoperability between CommonJS and ES Modules via creation of namespace objects for all imports. Implies 'allowSyntheticDefaultImports'. */ + // "preserveSymlinks": true, /* Do not resolve the real path of symlinks. */ + // "allowUmdGlobalAccess": true, /* Allow accessing UMD globals from modules. */ + + /* Source Map Options */ + // "sourceRoot": "", /* Specify the location where debugger should locate TypeScript files instead of source locations. */ + // "mapRoot": "", /* Specify the location where debugger should locate map files instead of generated locations. */ + // "inlineSourceMap": true, /* Emit a single file with source maps instead of having a separate file. */ + // "inlineSources": true, /* Emit the source alongside the sourcemaps within a single file; requires '--inlineSourceMap' or '--sourceMap' to be set. */ + + /* Experimental Options */ + "experimentalDecorators": true, /* Enables experimental support for ES7 decorators. */ + "emitDecoratorMetadata": true, /* Enables experimental support for emitting type metadata for decorators. */ + + /* Advanced Options */ + "skipLibCheck": true, /* Skip type checking of declaration files. */ + "forceConsistentCasingInFileNames": true /* Disallow inconsistently-cased references to the same file. */ + } +} From 06aafa9eabd680008eeb6cb266963b24326ec678 Mon Sep 17 00:00:00 2001 From: Henry Date: Tue, 22 Jun 2021 23:36:29 +0800 Subject: [PATCH 16/25] [c#/beetlex] update BeetleX.Fasthttpapi 1.9.6, optimize platform code (#6589) * update beetlex 1.4.3 update beetlex 1.4.3 * docker add COMPlus_ReadyToRun variable update beetlex * update beetlex, enabled thread queue * beetlex framework add db and queries cases * add db code * change result json data * update query url * beetlex framework add fortunes cases * change Content-Type * add beetlex core cases * fix queries cases * update config * change try readline * update benchmark config * Update README.md * Update README.md * change versus property * beetlex-core update .net core to v3.0 * change beetlex-core project file * beetlex update raw db class * beetlex update raw db * beetlex debug plaintext * change debug docker file * update beetlex to 1.4.0 * update * beetlex update core 3.1 * [c#/beetlex] add updates cases * [c#/beetlex] change Server: TFB, change custom connection pool, add update docker * fix errors * change pool init * change connection pool maxsize * fix fortunes errors * clear DBRaw _connectionString value. * [c#beetlex] change update dbconnection pool size * [c#/beetlex] udpate spanjson to v3.0.1, Npgsql v5.0.0 * [c#/beetlex] add caching sample * set connectionstring multiplexing * remove connection multiplexing setting * [c#/beetlex]change NpgsqlParameter to NpgsqlParameter * [c#/beetlex] update dbraw * [c#/beetlex] change connection string * [c#/beetlex] add fortunes cases to core-updb * update beetlex 1.5.6 * update 5.0.0-alpha1 * update docker file * Enabled IOQueues * Set IOQueues debug mode * update * [c#/beetlex] udpate to v1.6.0.1-beta * update pg drive * [c#/beetlex] update to beetlex v1.6.3 and support pipelining * set options * [c#/beetlex] Optimized actions * [c#/beetlex] update plaintext * Bump ServiceStack in /frameworks/CSharp/servicestack/src/SelfHost Bumps [ServiceStack](https://github.com/ServiceStack/ServiceStack) from 3.9.59 to 5.9.2. - [Release notes](https://github.com/ServiceStack/ServiceStack/releases) - [Commits](https://github.com/ServiceStack/ServiceStack/commits/v5.9.2) Signed-off-by: dependabot[bot] * [c#/beetlex] change plaintext buffer size, optimize fortunes. * [c#/beetlex] update v1.6.5,use System.Text.Json * change docker files * update beetlex 1.6.5.2 * [c#/beetlex] update v1.6.5.3-beta * [c#/beetlex] update 1.6.5.36-beta * update * update json * [c#/beetlex] update * update beetlex 1.6.5.38 beta * update * add array buffer * [c#/beetlex] updates * update * update * [c#/beetlex] update to 1.6.5.4.1 * change server tag * Bump bottle from 0.12.13 to 0.12.19 in /frameworks/Python/bottle Bumps [bottle](https://github.com/bottlepy/bottle) from 0.12.13 to 0.12.19. - [Release notes](https://github.com/bottlepy/bottle/releases) - [Changelog](https://github.com/bottlepy/bottle/blob/master/docs/changelog.rst) - [Commits](https://github.com/bottlepy/bottle/compare/0.12.13...0.12.19) Signed-off-by: dependabot[bot] * [c#/beetlex] remove debug docker file * update benchmark_config * [c#/beetlex] update fasthttpapi 1.9.6, optimize platform code * update connectionstring * update * update * update Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .../beetlex/Benchmarks/Benchmarks.csproj | 2 +- .../CSharp/beetlex/Benchmarks/Program.cs | 17 +-- .../beetlex/PlatformBenchmarks/AsciiString.cs | 102 +++++------------- .../beetlex/PlatformBenchmarks/GMTDate.cs | 36 ++++--- .../{Caching.cs => HttpHandler.Caching.cs} | 6 ++ .../beetlex/PlatformBenchmarks/HttpHandler.cs | 100 +++++++---------- .../{db.cs => HttpHandler.db.cs} | 6 +- .../PlatformBenchmarks/HttpHandler.default.cs | 30 ++++++ .../{fortunes.cs => HttpHandler.fortunes.cs} | 8 +- .../{json.cs => HttpHandler.json.cs} | 9 +- ...{plaintext.cs => HttpHandler.plaintext.cs} | 6 +- .../{queries.cs => HttpHandler.queries.cs} | 4 + .../{updates.cs => HttpHandler.updates.cs} | 5 + .../beetlex/PlatformBenchmarks/HttpServer.cs | 2 +- .../beetlex/PlatformBenchmarks/HttpToken.cs | 4 +- 15 files changed, 165 insertions(+), 172 deletions(-) rename frameworks/CSharp/beetlex/PlatformBenchmarks/{Caching.cs => HttpHandler.Caching.cs} (81%) rename frameworks/CSharp/beetlex/PlatformBenchmarks/{db.cs => HttpHandler.db.cs} (75%) create mode 100644 frameworks/CSharp/beetlex/PlatformBenchmarks/HttpHandler.default.cs rename frameworks/CSharp/beetlex/PlatformBenchmarks/{fortunes.cs => HttpHandler.fortunes.cs} (94%) rename frameworks/CSharp/beetlex/PlatformBenchmarks/{json.cs => HttpHandler.json.cs} (76%) rename frameworks/CSharp/beetlex/PlatformBenchmarks/{plaintext.cs => HttpHandler.plaintext.cs} (78%) rename frameworks/CSharp/beetlex/PlatformBenchmarks/{queries.cs => HttpHandler.queries.cs} (82%) rename frameworks/CSharp/beetlex/PlatformBenchmarks/{updates.cs => HttpHandler.updates.cs} (83%) diff --git a/frameworks/CSharp/beetlex/Benchmarks/Benchmarks.csproj b/frameworks/CSharp/beetlex/Benchmarks/Benchmarks.csproj index 507b1f9f18f..01cb00297fd 100644 --- a/frameworks/CSharp/beetlex/Benchmarks/Benchmarks.csproj +++ b/frameworks/CSharp/beetlex/Benchmarks/Benchmarks.csproj @@ -6,7 +6,7 @@ true - + diff --git a/frameworks/CSharp/beetlex/Benchmarks/Program.cs b/frameworks/CSharp/beetlex/Benchmarks/Program.cs index 3ec7a3b3d43..05ba5ed2db8 100644 --- a/frameworks/CSharp/beetlex/Benchmarks/Program.cs +++ b/frameworks/CSharp/beetlex/Benchmarks/Program.cs @@ -84,33 +84,34 @@ public class BeetleXHttpServer : IHostedService private HttpApiServer mApiServer; + private System.Threading.Tasks.TaskCompletionSource mComplete = new TaskCompletionSource(); + public async virtual Task StartAsync(CancellationToken cancellationToken) { + + plaintextResult = new StringBytes(_helloWorldPayload); mApiServer = new HttpApiServer(); mApiServer.Options.Port = 8080; mApiServer.Options.BufferPoolMaxMemory = 500; mApiServer.Options.MaxConnections = 100000; mApiServer.Options.Statistical = false; - mApiServer.Options.UrlIgnoreCase = false; mApiServer.Options.LogLevel = BeetleX.EventArgs.LogType.Error; mApiServer.Options.LogToConsole = true; - mApiServer.Options.PrivateBufferPool = true; mApiServer.Register(typeof(Program).Assembly); HeaderTypeFactory.SERVAR_HEADER_BYTES = Encoding.ASCII.GetBytes("Server: TFB\r\n"); mApiServer.HttpConnected += (o, e) => { e.Session["DB"] = new RawDb(new ConcurrentRandom(), Npgsql.NpgsqlFactory.Instance); }; + mApiServer.Started += (o, e) => + { + mComplete.TrySetResult(new object()); + }; mApiServer.Open(); RawDb._connectionString = "Server=tfb-database;Database=hello_world;User Id=benchmarkdbuser;Password=benchmarkdbpass;Maximum Pool Size=256;NoResetOnClose=true;Enlist=false;Max Auto Prepare=4;Multiplexing=true;Write Coalescing Delay Us=500;Write Coalescing Buffer Threshold Bytes=1000"; //RawDb._connectionString = "Server=192.168.2.19;Database=hello_world;User Id=benchmarkdbuser;Password=benchmarkdbpass;Maximum Pool Size=256;NoResetOnClose=true;Enlist=false;Max Auto Prepare=3"; - System.Net.Http.HttpClient client = new System.Net.Http.HttpClient(); - var response = await client.GetAsync("http://localhost:8080/json"); - mApiServer.BaseServer.Log(LogType.Info, null, $"Get josn {response.StatusCode}"); - response = await client.GetAsync("http://localhost:8080/plaintext"); - mApiServer.BaseServer.Log(LogType.Info, null, $"Get plaintext {response.StatusCode}"); - + await mComplete.Task; } public virtual Task StopAsync(CancellationToken cancellationToken) diff --git a/frameworks/CSharp/beetlex/PlatformBenchmarks/AsciiString.cs b/frameworks/CSharp/beetlex/PlatformBenchmarks/AsciiString.cs index 835957622e8..002fc07c4ea 100644 --- a/frameworks/CSharp/beetlex/PlatformBenchmarks/AsciiString.cs +++ b/frameworks/CSharp/beetlex/PlatformBenchmarks/AsciiString.cs @@ -8,96 +8,52 @@ namespace PlatformBenchmarks { private readonly byte[] _data; - public byte[] Data => _data; - - public int Length => _data.Length; - - public AsciiString(string s) - { - _data = Encoding.ASCII.GetBytes(s); - } + public AsciiString(string s) => _data = Encoding.ASCII.GetBytes(s); - public ReadOnlySpan AsSpan() - { - return _data; - } + private AsciiString(byte[] b) => _data = b; - public static implicit operator ReadOnlySpan(AsciiString str) - { - return str._data; - } - - public static implicit operator byte[](AsciiString str) - { - return str._data; - } - - public static implicit operator AsciiString(string str) - { - return new AsciiString(str); - } + public int Length => _data.Length; - public override string ToString() - { - return Encoding.ASCII.GetString(_data); - } + public byte[] Data => _data; - public int CopyTo(byte[] source,int offset) - { - var len = Length; - Buffer.BlockCopy(_data, 0, source, offset, len); - return len; - } + public ReadOnlySpan AsSpan() => _data; - public static explicit operator string(AsciiString str) - { - return str.ToString(); - } + public static implicit operator ReadOnlySpan(AsciiString str) => str._data; + public static implicit operator byte[](AsciiString str) => str._data; - public bool Equals(AsciiString other) - { - if (_data != other._data) - { - return SequenceEqual(_data, other._data); - } - return true; - } + public static implicit operator AsciiString(string str) => new AsciiString(str); - private bool SequenceEqual(byte[] data1, byte[] data2) - { - return new Span(data1).SequenceEqual(data2); - } + public override string ToString() => Encoding.ASCII.GetString(_data); + public static explicit operator string(AsciiString str) => str.ToString(); - public static bool operator ==(AsciiString a, AsciiString b) - { - return a.Equals(b); - } + public bool Equals(AsciiString other) => ReferenceEquals(_data, other._data) || SequenceEqual(_data, other._data); + private bool SequenceEqual(byte[] data1, byte[] data2) => new Span(data1).SequenceEqual(data2); - public static bool operator !=(AsciiString a, AsciiString b) - { - return !a.Equals(b); - } + public static bool operator ==(AsciiString a, AsciiString b) => a.Equals(b); + public static bool operator !=(AsciiString a, AsciiString b) => !a.Equals(b); + public override bool Equals(object other) => (other is AsciiString) && Equals((AsciiString)other); - public override bool Equals(object other) + public static AsciiString operator +(AsciiString a, AsciiString b) { - if (other is AsciiString) - { - return Equals((AsciiString)other); - } - return false; + var result = new byte[a.Length + b.Length]; + a._data.CopyTo(result, 0); + b._data.CopyTo(result, a.Length); + return new AsciiString(result); } public override int GetHashCode() { - byte[] data = _data; - int hash3 = 5381; - int hash2 = hash3; - byte[] array = data; - foreach (int b in array) + // Copied from x64 version of string.GetLegacyNonRandomizedHashCode() + // https://github.com/dotnet/coreclr/blob/master/src/mscorlib/src/System/String.Comparison.cs + var data = _data; + int hash1 = 5381; + int hash2 = hash1; + foreach (int b in data) { - hash3 = (((hash3 << 5) + hash3) ^ b); + hash1 = ((hash1 << 5) + hash1) ^ b; } - return hash3 + hash2 * 1566083941; + return hash1 + (hash2 * 1566083941); } + } } diff --git a/frameworks/CSharp/beetlex/PlatformBenchmarks/GMTDate.cs b/frameworks/CSharp/beetlex/PlatformBenchmarks/GMTDate.cs index 8cba0e9ae9c..ac9266097ba 100644 --- a/frameworks/CSharp/beetlex/PlatformBenchmarks/GMTDate.cs +++ b/frameworks/CSharp/beetlex/PlatformBenchmarks/GMTDate.cs @@ -1,4 +1,5 @@ -using System; +using BeetleX.Buffers; +using System; using System.Collections.Generic; using System.Text; using System.Threading; @@ -86,19 +87,25 @@ public GMTDate() private void Init() { - DATE = GetData(inLine: true); + DATE = GetData(); mUpdateTime = new Timer(delegate { - DATE = GetData(inLine: true); + DATE = GetData(); }, null, 1000, 1000); } - private ArraySegment GetData(bool inLine = false) + private ArraySegment GetData() { - return GetData(DateTime.Now, inLine); + return GetData(DateTime.Now); } - private ArraySegment GetData(DateTime date, bool inLine = false) + public void Write(PipeStream stream) + { + var data = DATE; + stream.Write(data.Array, 0, data.Count); + } + + private ArraySegment GetData(DateTime date) { date = date.ToUniversalTime(); int offset13 = 0; @@ -172,13 +179,16 @@ private ArraySegment GetData(DateTime date, bool inLine = false) offset13++; buffer[offset13] = sub8[2]; offset13++; - if (inLine) - { - buffer[offset13] = _r; - offset13++; - buffer[offset13] = _n; - offset13++; - } + + buffer[offset13] = _r; + offset13++; + buffer[offset13] = _n; + offset13++; + buffer[offset13] = _r; + offset13++; + buffer[offset13] = _n; + offset13++; + return new ArraySegment(GTM_BUFFER, 0, offset13); } } diff --git a/frameworks/CSharp/beetlex/PlatformBenchmarks/Caching.cs b/frameworks/CSharp/beetlex/PlatformBenchmarks/HttpHandler.Caching.cs similarity index 81% rename from frameworks/CSharp/beetlex/PlatformBenchmarks/Caching.cs rename to frameworks/CSharp/beetlex/PlatformBenchmarks/HttpHandler.Caching.cs index c22891da742..13d89c6fc97 100644 --- a/frameworks/CSharp/beetlex/PlatformBenchmarks/Caching.cs +++ b/frameworks/CSharp/beetlex/PlatformBenchmarks/HttpHandler.Caching.cs @@ -10,6 +10,8 @@ namespace PlatformBenchmarks { public partial class HttpHandler { + + public async Task caching(string queryString, PipeStream stream, HttpToken token, ISession session) { int count = 1; @@ -31,6 +33,10 @@ public async Task caching(string queryString, PipeStream stream, HttpToken token try { var data = await token.Db.LoadCachedQueries(count); + stream.Write(_jsonResultPreamble.Data, 0, _jsonResultPreamble.Length); + token.ContentLength = stream.Allocate(HttpHandler._LengthSize); + GMTDate.Default.Write(stream); + token.ContentPostion = stream.CacheLength; await JsonSerializer.NonGeneric.Utf8.SerializeAsync(data, stream); } catch (Exception e_) diff --git a/frameworks/CSharp/beetlex/PlatformBenchmarks/HttpHandler.cs b/frameworks/CSharp/beetlex/PlatformBenchmarks/HttpHandler.cs index 1c820fbcc0a..8e5b65d9c79 100644 --- a/frameworks/CSharp/beetlex/PlatformBenchmarks/HttpHandler.cs +++ b/frameworks/CSharp/beetlex/PlatformBenchmarks/HttpHandler.cs @@ -11,11 +11,11 @@ namespace PlatformBenchmarks { public partial class HttpHandler : ServerHandlerBase { - private static AsciiString _line = new AsciiString("\r\n"); + private static readonly AsciiString _line = new AsciiString("\r\n"); - private static AsciiString _2line = new AsciiString("\r\n\r\n"); + private static readonly AsciiString _2line = new AsciiString("\r\n\r\n"); - private static AsciiString _httpsuccess = new AsciiString("HTTP/1.1 200 OK\r\n"); + private static readonly AsciiString _httpsuccess = new AsciiString("HTTP/1.1 200 OK\r\n"); private static readonly AsciiString _headerServer = "Server: B\r\n"; @@ -29,12 +29,6 @@ public partial class HttpHandler : ServerHandlerBase private static readonly AsciiString _headerContentTypeJson = "Content-Type: application/json\r\n"; - private static readonly AsciiString _textResultHeader = _httpsuccess + _headerServer.ToString() + _headerContentTypeText.ToString(); - - private static readonly AsciiString _jsonResultHeader = _httpsuccess + _headerServer.ToString() + _headerContentTypeJson.ToString(); - - private static readonly AsciiString _htmlResultHeader = _httpsuccess + _headerServer.ToString() + _headerContentTypeHtml.ToString(); - private static readonly AsciiString _path_Json = "/json"; private static readonly AsciiString _path_Db = "/db"; @@ -51,9 +45,41 @@ public partial class HttpHandler : ServerHandlerBase private static readonly AsciiString _cached_worlds = "/cached-worlds"; + private readonly static uint _jsonPayloadSize = (uint)System.Text.Json.JsonSerializer.SerializeToUtf8Bytes(new JsonMessage { message = "Hello, World!" }, SerializerOptions).Length; + + + + private readonly static AsciiString _jsonPreamble = + _httpsuccess + + _headerContentTypeJson + + _headerServer + + _headerContentLength + _jsonPayloadSize.ToString() + _line; + + private readonly static AsciiString _plaintextPreamble = + _httpsuccess + + _headerContentTypeText + + _headerServer + + _headerContentLength + _result_plaintext.Length.ToString() + _line; + + + private readonly static AsciiString _jsonResultPreamble = + _httpsuccess + + _headerContentTypeJson + + _headerServer + + _headerContentLength; + + private readonly static AsciiString _HtmlResultPreamble = + _httpsuccess + + _headerContentTypeHtml + + _headerServer + + _headerContentLength; + + + + private static byte _Space = 32; - public const int _LengthSize = 6; + public const int _LengthSize = 8; private static byte _question = 63; @@ -64,13 +90,7 @@ public HttpHandler() private BeetleX.Dispatchs.DispatchCenter RequestDispatchs; - public Task Default(PipeStream stream, HttpToken token, ISession session) - { - stream.Write(" beetlex server
"); - stream.Write("path not found!"); - OnCompleted(stream, session, token); - return Task.CompletedTask; - } + public override void Connected(IServer server, ConnectedEventArgs e) { @@ -235,7 +255,6 @@ private void AnalysisAction(RequestData requestData) public virtual async Task OnStartRequest(RequestData data, ISession session, HttpToken token, PipeStream stream) { - OnWriteHeader(stream, token, data.Action); ActionType type = data.Action; if (type == ActionType.Plaintext) { @@ -272,51 +291,6 @@ public virtual async Task OnStartRequest(RequestData data, ISession session, Htt } - private void OnWriteHeader(PipeStream stream, HttpToken token, ActionType type) - { - switch (type) - { - case ActionType.Caching: - case ActionType.Json: - case ActionType.Queries: - case ActionType.Db: - case ActionType.Updates: - stream.Write(_jsonResultHeader.Data, 0, _jsonResultHeader.Length); - break; - case ActionType.Plaintext: - stream.Write(_textResultHeader.Data, 0, _textResultHeader.Length); - break; - default: - stream.Write(_htmlResultHeader.Data, 0, _htmlResultHeader.Length); - break; - } - - ArraySegment date = GMTDate.Default.DATE; - stream.Write(date.Array, date.Offset, date.Count); - OnWriteContentLength(stream, token); - } - - private void OnWriteContentLength(PipeStream stream, HttpToken token) - { - var action = token.CurrentRequest.Action; - if (action == ActionType.Json) - { - stream.Write(_jsonPreamble.Data, 0, _jsonPreamble.Length); - stream.Write(_2line, 0, 4); - } - else if (action == ActionType.Plaintext) - { - stream.Write(_plaintextPreamble.Data, 0, _plaintextPreamble.Length); - stream.Write(_2line, 0, 4); - } - else - { - stream.Write(_headerContentLength.Data, 0, _headerContentLength.Length); - token.ContentLength = stream.Allocate(HttpHandler._LengthSize); - stream.Write(_2line, 0, 4); - token.ContentPostion = stream.CacheLength; - } - } private void OnCompleted(PipeStream stream, ISession session, HttpToken token) { diff --git a/frameworks/CSharp/beetlex/PlatformBenchmarks/db.cs b/frameworks/CSharp/beetlex/PlatformBenchmarks/HttpHandler.db.cs similarity index 75% rename from frameworks/CSharp/beetlex/PlatformBenchmarks/db.cs rename to frameworks/CSharp/beetlex/PlatformBenchmarks/HttpHandler.db.cs index 237475452da..a6f6c8f2291 100644 --- a/frameworks/CSharp/beetlex/PlatformBenchmarks/db.cs +++ b/frameworks/CSharp/beetlex/PlatformBenchmarks/HttpHandler.db.cs @@ -11,12 +11,16 @@ namespace PlatformBenchmarks public partial class HttpHandler { + public async ValueTask db(PipeStream stream, HttpToken token, ISession session) { try { var data = await token.Db.LoadSingleQueryRow(); - + stream.Write(_jsonResultPreamble.Data, 0, _jsonResultPreamble.Length); + token.ContentLength = stream.Allocate(HttpHandler._LengthSize); + GMTDate.Default.Write(stream); + token.ContentPostion = stream.CacheLength; System.Text.Json.JsonSerializer.Serialize(GetUtf8JsonWriter(stream, token), data, SerializerOptions); } catch (Exception e_) diff --git a/frameworks/CSharp/beetlex/PlatformBenchmarks/HttpHandler.default.cs b/frameworks/CSharp/beetlex/PlatformBenchmarks/HttpHandler.default.cs new file mode 100644 index 00000000000..acbaff5b164 --- /dev/null +++ b/frameworks/CSharp/beetlex/PlatformBenchmarks/HttpHandler.default.cs @@ -0,0 +1,30 @@ +using BeetleX; +using BeetleX.Buffers; +using System; +using System.Collections.Generic; +using System.Linq; +using System.Text; +using System.Threading.Tasks; + +namespace PlatformBenchmarks +{ + public partial class HttpHandler + { + private readonly static AsciiString _defaultPreamble = + _httpsuccess.ToString() + + _headerContentTypeJson.ToString() + + _headerServer.ToString(); + + public Task Default(PipeStream stream, HttpToken token, ISession session) + { + stream.Write(_defaultPreamble.Data, 0, _defaultPreamble.Length); + token.ContentLength = stream.Allocate(HttpHandler._LengthSize); + GMTDate.Default.Write(stream); + token.ContentPostion = stream.CacheLength; + stream.Write(" beetlex server
"); + stream.Write("path not found!"); + OnCompleted(stream, session, token); + return Task.CompletedTask; + } + } +} diff --git a/frameworks/CSharp/beetlex/PlatformBenchmarks/fortunes.cs b/frameworks/CSharp/beetlex/PlatformBenchmarks/HttpHandler.fortunes.cs similarity index 94% rename from frameworks/CSharp/beetlex/PlatformBenchmarks/fortunes.cs rename to frameworks/CSharp/beetlex/PlatformBenchmarks/HttpHandler.fortunes.cs index 0fda0ba0e5f..55250710b35 100644 --- a/frameworks/CSharp/beetlex/PlatformBenchmarks/fortunes.cs +++ b/frameworks/CSharp/beetlex/PlatformBenchmarks/HttpHandler.fortunes.cs @@ -19,9 +19,6 @@ public partial class HttpHandler private readonly static AsciiString _fortunesRowEnd = ""; private readonly static AsciiString _fortunesTableEnd = ""; - [ThreadStatic] - private static char[] mHtmlEncodeBuffer; - protected HtmlEncoder HtmlEncoder { get; } = CreateHtmlEncoder(); private static HtmlEncoder CreateHtmlEncoder() @@ -38,6 +35,11 @@ public async Task fortunes(PipeStream stream, HttpToken token, ISession session) var data = await token.Db.LoadFortunesRows(); + stream.Write(_HtmlResultPreamble.Data, 0, _HtmlResultPreamble.Length); + token.ContentLength = stream.Allocate(HttpHandler._LengthSize); + GMTDate.Default.Write(stream); + token.ContentPostion = stream.CacheLength; + var html = token.GetHtmlBufferWriter(); html.Reset(); html.Write(_fortunesTableStart.Data, 0, _fortunesTableStart.Length); diff --git a/frameworks/CSharp/beetlex/PlatformBenchmarks/json.cs b/frameworks/CSharp/beetlex/PlatformBenchmarks/HttpHandler.json.cs similarity index 76% rename from frameworks/CSharp/beetlex/PlatformBenchmarks/json.cs rename to frameworks/CSharp/beetlex/PlatformBenchmarks/HttpHandler.json.cs index 366346e750d..1577248fa81 100644 --- a/frameworks/CSharp/beetlex/PlatformBenchmarks/json.cs +++ b/frameworks/CSharp/beetlex/PlatformBenchmarks/HttpHandler.json.cs @@ -11,10 +11,7 @@ namespace PlatformBenchmarks { public partial class HttpHandler { - private readonly static uint _jsonPayloadSize = (uint)System.Text.Json.JsonSerializer.SerializeToUtf8Bytes(new JsonMessage { message = "Hello, World!" }, SerializerOptions).Length; - - private readonly static AsciiString _jsonPreamble = - _headerContentLength + _jsonPayloadSize.ToString(); + private static readonly JsonSerializerOptions SerializerOptions = new JsonSerializerOptions(); @@ -32,7 +29,9 @@ private static Utf8JsonWriter GetUtf8JsonWriter(PipeStream stream, HttpToken tok public ValueTask Json(PipeStream stream, HttpToken token, ISession session) { - System.Text.Json.JsonSerializer.Serialize(GetUtf8JsonWriter(stream, token), new JsonMessage { message = "Hello, World!" }, SerializerOptions); + stream.Write(_jsonPreamble.Data, 0, _jsonPreamble.Length); + GMTDate.Default.Write(stream); + System.Text.Json.JsonSerializer.Serialize(GetUtf8JsonWriter(stream, token), new JsonMessage { message = "Hello, World!" }, SerializerOptions); OnCompleted(stream, session, token); return ValueTask.CompletedTask; } diff --git a/frameworks/CSharp/beetlex/PlatformBenchmarks/plaintext.cs b/frameworks/CSharp/beetlex/PlatformBenchmarks/HttpHandler.plaintext.cs similarity index 78% rename from frameworks/CSharp/beetlex/PlatformBenchmarks/plaintext.cs rename to frameworks/CSharp/beetlex/PlatformBenchmarks/HttpHandler.plaintext.cs index b38581ee21b..83741829b84 100644 --- a/frameworks/CSharp/beetlex/PlatformBenchmarks/plaintext.cs +++ b/frameworks/CSharp/beetlex/PlatformBenchmarks/HttpHandler.plaintext.cs @@ -9,12 +9,12 @@ namespace PlatformBenchmarks { public partial class HttpHandler { - private readonly static AsciiString _plaintextPreamble = - - _headerContentLength + _result_plaintext.Length.ToString(); + public ValueTask Plaintext(PipeStream stream, HttpToken token, ISession session) { + stream.Write(_plaintextPreamble.Data, 0, _plaintextPreamble.Length); + GMTDate.Default.Write(stream); stream.Write(_result_plaintext.Data, 0, _result_plaintext.Length); OnCompleted(stream, session, token); return ValueTask.CompletedTask; diff --git a/frameworks/CSharp/beetlex/PlatformBenchmarks/queries.cs b/frameworks/CSharp/beetlex/PlatformBenchmarks/HttpHandler.queries.cs similarity index 82% rename from frameworks/CSharp/beetlex/PlatformBenchmarks/queries.cs rename to frameworks/CSharp/beetlex/PlatformBenchmarks/HttpHandler.queries.cs index 341b3830933..8027191667c 100644 --- a/frameworks/CSharp/beetlex/PlatformBenchmarks/queries.cs +++ b/frameworks/CSharp/beetlex/PlatformBenchmarks/HttpHandler.queries.cs @@ -31,6 +31,10 @@ public async ValueTask queries(string queryString, PipeStream stream, HttpToken try { var data = await token.Db.LoadMultipleQueriesRows(count); + stream.Write(_jsonResultPreamble.Data, 0, _jsonResultPreamble.Length); + token.ContentLength = stream.Allocate(HttpHandler._LengthSize); + GMTDate.Default.Write(stream); + token.ContentPostion = stream.CacheLength; System.Text.Json.JsonSerializer.Serialize(GetUtf8JsonWriter(stream, token), data, SerializerOptions); } catch (Exception e_) diff --git a/frameworks/CSharp/beetlex/PlatformBenchmarks/updates.cs b/frameworks/CSharp/beetlex/PlatformBenchmarks/HttpHandler.updates.cs similarity index 83% rename from frameworks/CSharp/beetlex/PlatformBenchmarks/updates.cs rename to frameworks/CSharp/beetlex/PlatformBenchmarks/HttpHandler.updates.cs index 4ddff6e53cf..8f3e6d84815 100644 --- a/frameworks/CSharp/beetlex/PlatformBenchmarks/updates.cs +++ b/frameworks/CSharp/beetlex/PlatformBenchmarks/HttpHandler.updates.cs @@ -31,6 +31,11 @@ public async ValueTask updates(string queryString, PipeStream stream, HttpToken try { var data = await token.Db.LoadMultipleUpdatesRows(count); + + stream.Write(_jsonResultPreamble.Data, 0, _jsonResultPreamble.Length); + token.ContentLength = stream.Allocate(HttpHandler._LengthSize); + GMTDate.Default.Write(stream); + token.ContentPostion = stream.CacheLength; System.Text.Json.JsonSerializer.Serialize(GetUtf8JsonWriter(stream, token), data, SerializerOptions); } catch (Exception e_) diff --git a/frameworks/CSharp/beetlex/PlatformBenchmarks/HttpServer.cs b/frameworks/CSharp/beetlex/PlatformBenchmarks/HttpServer.cs index bc7ecc5a134..422da242a7a 100644 --- a/frameworks/CSharp/beetlex/PlatformBenchmarks/HttpServer.cs +++ b/frameworks/CSharp/beetlex/PlatformBenchmarks/HttpServer.cs @@ -21,7 +21,7 @@ public virtual Task StartAsync(CancellationToken cancellationToken) serverOptions.DefaultListen.Port = 8080; serverOptions.Statistical = false; serverOptions.BufferPoolMaxMemory = 1000; - serverOptions.BufferPoolSize = 1024 * 16; + serverOptions.BufferPoolSize = 1024 * 24; ApiServer = SocketFactory.CreateTcpServer(serverOptions); ApiServer.Open(); if (!Program.UpDB) diff --git a/frameworks/CSharp/beetlex/PlatformBenchmarks/HttpToken.cs b/frameworks/CSharp/beetlex/PlatformBenchmarks/HttpToken.cs index 187b4e70378..3d85f8426e5 100644 --- a/frameworks/CSharp/beetlex/PlatformBenchmarks/HttpToken.cs +++ b/frameworks/CSharp/beetlex/PlatformBenchmarks/HttpToken.cs @@ -41,10 +41,12 @@ public HtmlBufferWriter GetHtmlBufferWriter() public byte[] GetLengthBuffer(string length) { Encoding.ASCII.GetBytes(length, 0, length.Length, mLengthBuffer, 0); - for (int i = length.Length; i < HttpHandler._LengthSize; i++) + for (int i = length.Length; i < mLengthBuffer.Length; i++) { mLengthBuffer[i] = 32; } + mLengthBuffer[6] = (byte)'\r'; + mLengthBuffer[7] = (byte)'\n'; return mLengthBuffer; } From 635fd78b0fc794aeb11f678000af3060930bfea8 Mon Sep 17 00:00:00 2001 From: Ruben Oanta Date: Wed, 23 Jun 2021 11:11:38 -0600 Subject: [PATCH 17/25] bump (#6676) --- frameworks/Scala/finagle/build.sbt | 2 +- frameworks/Scala/finatra/build.sbt | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/frameworks/Scala/finagle/build.sbt b/frameworks/Scala/finagle/build.sbt index 084370990e8..bdacc76d420 100644 --- a/frameworks/Scala/finagle/build.sbt +++ b/frameworks/Scala/finagle/build.sbt @@ -1,4 +1,4 @@ -lazy val finagleVersion = "21.5.0" +lazy val finagleVersion = "21.6.0" name := "finagle-benchmark" scalaVersion := "2.12.12" diff --git a/frameworks/Scala/finatra/build.sbt b/frameworks/Scala/finatra/build.sbt index f9ca0163835..ed7efe17907 100644 --- a/frameworks/Scala/finatra/build.sbt +++ b/frameworks/Scala/finatra/build.sbt @@ -1,4 +1,4 @@ -lazy val finatraVersion = "21.5.0" +lazy val finatraVersion = "21.6.0" name := "techempower-benchmarks-finatra" organization := "com.twitter" From 53c547aacf52c74e285c4b094c3c741bf14d97db Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 23 Jun 2021 13:48:23 -0700 Subject: [PATCH 18/25] Bump jetty-server in /frameworks/Java/jetty (#6677) Bumps [jetty-server](https://github.com/eclipse/jetty.project) from 9.4.35.v20201120 to 9.4.41.v20210516. - [Release notes](https://github.com/eclipse/jetty.project/releases) - [Commits](https://github.com/eclipse/jetty.project/compare/jetty-9.4.35.v20201120...jetty-9.4.41.v20210516) --- updated-dependencies: - dependency-name: org.eclipse.jetty:jetty-server dependency-type: direct:production ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- frameworks/Java/jetty/pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/frameworks/Java/jetty/pom.xml b/frameworks/Java/jetty/pom.xml index 0446fe6ebd4..c860b1d4bf4 100644 --- a/frameworks/Java/jetty/pom.xml +++ b/frameworks/Java/jetty/pom.xml @@ -11,7 +11,7 @@ UTF-8 11 11 - 9.4.35.v20201120 + 9.4.41.v20210516 hello.handler.HelloWebServer From 543ca7e584f91afe33023e9c127158ed88f7beae Mon Sep 17 00:00:00 2001 From: Joan Miquel Date: Fri, 25 Jun 2021 19:23:01 +0200 Subject: [PATCH 19/25] Revert to Ubuntu 20.10 (#6678) Till #6591 is fixed. --- frameworks/C/nginx/nginx.dockerfile | 2 +- frameworks/PHP/amp/amp.dockerfile | 2 +- frameworks/PHP/reactphp/reactphp.dockerfile | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/frameworks/C/nginx/nginx.dockerfile b/frameworks/C/nginx/nginx.dockerfile index e86d42ff5b6..8371ba02741 100644 --- a/frameworks/C/nginx/nginx.dockerfile +++ b/frameworks/C/nginx/nginx.dockerfile @@ -1,4 +1,4 @@ -FROM ubuntu:21.04 +FROM ubuntu:20.10 ARG DEBIAN_FRONTEND=noninteractive diff --git a/frameworks/PHP/amp/amp.dockerfile b/frameworks/PHP/amp/amp.dockerfile index c26bd5c312f..ef79892bd0e 100644 --- a/frameworks/PHP/amp/amp.dockerfile +++ b/frameworks/PHP/amp/amp.dockerfile @@ -1,4 +1,4 @@ -FROM ubuntu:21.04 +FROM ubuntu:20.10 ARG DEBIAN_FRONTEND=noninteractive diff --git a/frameworks/PHP/reactphp/reactphp.dockerfile b/frameworks/PHP/reactphp/reactphp.dockerfile index eea73e1d220..500d08828e2 100644 --- a/frameworks/PHP/reactphp/reactphp.dockerfile +++ b/frameworks/PHP/reactphp/reactphp.dockerfile @@ -1,4 +1,4 @@ -FROM ubuntu:21.04 +FROM ubuntu:20.10 ARG DEBIAN_FRONTEND=noninteractive From ac2aa08d7cdd3857b2f8fc99a2f190da468a13b6 Mon Sep 17 00:00:00 2001 From: JaeHyeok Kim Date: Sun, 27 Jun 2021 07:38:51 +0900 Subject: [PATCH 20/25] Update node version for nodejs (#6680) --- frameworks/JavaScript/nodejs/README.md | 2 +- frameworks/JavaScript/nodejs/nodejs.dockerfile | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/frameworks/JavaScript/nodejs/README.md b/frameworks/JavaScript/nodejs/README.md index 19f9da5503f..a355e602c81 100644 --- a/frameworks/JavaScript/nodejs/README.md +++ b/frameworks/JavaScript/nodejs/README.md @@ -16,7 +16,7 @@ The server is currently in Alpha state, but aims to be a drop-in replacement for ## Infrastructure Software Versions The tests were run with: -* [Node.js v14.16.1](http://nodejs.org/) +* [Node.js v14.17.1](http://nodejs.org/) * [Node MySQL 2.16.0](https://github.com/felixge/node-mysql/) * [Sequelize 5.15.1](https://github.com/sequelize/sequelize) diff --git a/frameworks/JavaScript/nodejs/nodejs.dockerfile b/frameworks/JavaScript/nodejs/nodejs.dockerfile index 0b68f8f1d9c..210e5fecd9d 100644 --- a/frameworks/JavaScript/nodejs/nodejs.dockerfile +++ b/frameworks/JavaScript/nodejs/nodejs.dockerfile @@ -1,4 +1,4 @@ -FROM node:14.16.1-slim +FROM node:14.17.1-slim ARG TFB_TEST_NAME From 980efc7766b48c27081c515dbdb6a0d4769ba540 Mon Sep 17 00:00:00 2001 From: LLT21 <43903768+LLT21@users.noreply.github.com> Date: Sun, 27 Jun 2021 00:40:30 +0200 Subject: [PATCH 21/25] Replaced Task.Delay by asynchronous callback (#6679) * Replaced Task.Delay by asynchronous callback in PooledConnections * Added pipeWriter.FlushAsync to avoid blocking Co-authored-by: LLT21 --- .../appmpower/src/Db/PooledConnections.cs | 25 ++++++++++++++++--- .../CSharp/appmpower/src/Kestrel/PlainText.cs | 5 +++- 2 files changed, 25 insertions(+), 5 deletions(-) diff --git a/frameworks/CSharp/appmpower/src/Db/PooledConnections.cs b/frameworks/CSharp/appmpower/src/Db/PooledConnections.cs index e157c60d94d..4b0fa504200 100644 --- a/frameworks/CSharp/appmpower/src/Db/PooledConnections.cs +++ b/frameworks/CSharp/appmpower/src/Db/PooledConnections.cs @@ -11,7 +11,7 @@ public static class PooledConnections private static byte _createdConnections = 0; private static byte _maxConnections = Math.Min((byte)Environment.ProcessorCount, (byte)21); private static ConcurrentStack _stack = new ConcurrentStack(); - private static TimeSpan _timeSpan = new TimeSpan(10); // 10 = 0.001 millisecond + private static ConcurrentQueue> _waitingQueue = new ConcurrentQueue>(); public static async Task GetConnection(string connectionString) { @@ -19,12 +19,13 @@ public static async Task GetConnection(string connectionString if (_connectionsCreated) { - while (!_stack.TryPop(out pooledConnection)) + if (!_stack.TryPop(out pooledConnection)) { - await Task.Delay(_timeSpan); + pooledConnection = await GetPooledConnectionAsync(); } return pooledConnection; + } else { @@ -42,15 +43,31 @@ public static async Task GetConnection(string connectionString } } + public static Task GetPooledConnectionAsync() + { + var taskCompletionSource = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); + + _waitingQueue.Enqueue(taskCompletionSource); + return taskCompletionSource.Task; + } + public static void ReleaseConnection(PooledConnection pooledConnection) { + TaskCompletionSource taskCompletionSource; PooledConnection stackedConnection = new PooledConnection(); stackedConnection.OdbcConnection = pooledConnection.OdbcConnection; stackedConnection.Number = pooledConnection.Number; stackedConnection.PooledCommands = pooledConnection.PooledCommands; - _stack.Push(stackedConnection); + if (_waitingQueue.TryDequeue(out taskCompletionSource)) + { + taskCompletionSource.SetResult(stackedConnection); + } + else + { + _stack.Push(stackedConnection); + } } } } \ No newline at end of file diff --git a/frameworks/CSharp/appmpower/src/Kestrel/PlainText.cs b/frameworks/CSharp/appmpower/src/Kestrel/PlainText.cs index 38dd3844c72..e784c27375d 100644 --- a/frameworks/CSharp/appmpower/src/Kestrel/PlainText.cs +++ b/frameworks/CSharp/appmpower/src/Kestrel/PlainText.cs @@ -20,7 +20,10 @@ public static async Task RenderAsync(IHeaderDictionary headerDictio headerDictionary.Add(_headerContentType); headerDictionary.Add(new KeyValuePair("Content-Length", utf8String.Length.ToString())); - return await pipeWriter.WriteAsync(utf8String); + var result = await pipeWriter.WriteAsync(utf8String); + await pipeWriter.FlushAsync(); + + return result; } } } \ No newline at end of file From 80ff402b9a190d5586ce21de8ca61d565930300a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Robin=20Bj=C3=B6rklin?= Date: Sun, 27 Jun 2021 19:23:45 -0700 Subject: [PATCH 22/25] fix: ocaml webmachine & bump to 4.12 (#6681) --- frameworks/OCaml/webmachine/haproxy.cfg | 2 - frameworks/OCaml/webmachine/src/dune-project | 20 +++---- frameworks/OCaml/webmachine/src/lib.opam | 2 +- frameworks/OCaml/webmachine/src/src/bin/dune | 3 +- .../OCaml/webmachine/src/src/bin/tfb.ml | 2 +- frameworks/OCaml/webmachine/src/tfb.opam | 18 +++---- frameworks/OCaml/webmachine/start-servers.sh | 7 ++- .../webmachine/webmachine-flambda.dockerfile | 38 +++++++++---- .../webmachine/webmachine-haproxy.dockerfile | 54 +++++++++++++------ .../OCaml/webmachine/webmachine.dockerfile | 38 +++++++++---- 10 files changed, 124 insertions(+), 60 deletions(-) diff --git a/frameworks/OCaml/webmachine/haproxy.cfg b/frameworks/OCaml/webmachine/haproxy.cfg index 007d2fb3178..fdaa63b629f 100644 --- a/frameworks/OCaml/webmachine/haproxy.cfg +++ b/frameworks/OCaml/webmachine/haproxy.cfg @@ -4,14 +4,12 @@ global maxconn 32768 user haproxy group haproxy - nbthread 2 defaults mode tcp log global option dontlognull option http-server-close - option forwardfor except 127.0.0.0/8 option redispatch retries 3 timeout http-request 10s diff --git a/frameworks/OCaml/webmachine/src/dune-project b/frameworks/OCaml/webmachine/src/dune-project index 69353a60e5c..661d1e52345 100644 --- a/frameworks/OCaml/webmachine/src/dune-project +++ b/frameworks/OCaml/webmachine/src/dune-project @@ -11,21 +11,21 @@ (name tfb) (synopsis "Rudimentary implementation of the Tech Empower Benchmark suite") (depends - (dune (>= 2.7.1)) - (webmachine (>= 0.6.2)) - (lwt (>= 5.3.0)) - (conf-libev (>= 4-11)) - (caqti (>= 1.2.3)) - (caqti-lwt (>= 1.2.0)) - (caqti-driver-postgresql (>= 1.2.4)) - (cohttp-lwt-unix (>= 2.5.4)) - (lwt_ppx (>= 2.0.1)))) + (dune (>= 2.8.5)) + (webmachine (>= 0.7.0)) + (lwt (>= 5.4.1)) + (conf-libev (>= 4-12)) + (caqti (>= 1.6.0)) + (caqti-lwt (>= 1.6.0)) + (caqti-driver-postgresql (>= 1.6.0)) + (cohttp-lwt-unix (>= 4.0.0)) + (lwt_ppx (>= 2.0.2)))) (package (name lib) (synopsis "WIP library") (depends - (dune (>= 2.7.1)) + (dune (>= 2.8.5)) (biniou (>= 1.2.1)) (yojson (>= 1.7.0)) (atd (>= 2.2.1)) diff --git a/frameworks/OCaml/webmachine/src/lib.opam b/frameworks/OCaml/webmachine/src/lib.opam index dabb857fca0..33e1ae6314d 100644 --- a/frameworks/OCaml/webmachine/src/lib.opam +++ b/frameworks/OCaml/webmachine/src/lib.opam @@ -6,7 +6,7 @@ license: "MIT" homepage: "https://github.com/TechEmpower/FrameworkBenchmarks" bug-reports: "https://github.com/TechEmpower/FrameworkBenchmarks/issues" depends: [ - "dune" {>= "2.7" & >= "2.7.1"} + "dune" {>= "2.7" & >= "2.8.5"} "biniou" {>= "1.2.1"} "yojson" {>= "1.7.0"} "atd" {>= "2.2.1"} diff --git a/frameworks/OCaml/webmachine/src/src/bin/dune b/frameworks/OCaml/webmachine/src/src/bin/dune index efed07f29ba..ccf1f4dfeea 100644 --- a/frameworks/OCaml/webmachine/src/src/bin/dune +++ b/frameworks/OCaml/webmachine/src/src/bin/dune @@ -1,6 +1,5 @@ (executable - (libraries webmachine lwt cohttp-lwt-unix ptime ptime.clock ptime.clock.os - caqti caqti-lwt caqti-driver-postgresql yojson atdgen-runtime lib) + (libraries webmachine lwt cohttp-lwt-unix caqti caqti-lwt caqti-driver-postgresql yojson atdgen-runtime lib) (preprocess (pps lwt_ppx)) (public_name tfb) (package tfb) diff --git a/frameworks/OCaml/webmachine/src/src/bin/tfb.ml b/frameworks/OCaml/webmachine/src/src/bin/tfb.ml index 8c20490ce8d..4cdd891a077 100644 --- a/frameworks/OCaml/webmachine/src/src/bin/tfb.ml +++ b/frameworks/OCaml/webmachine/src/src/bin/tfb.ml @@ -166,7 +166,7 @@ let main () = (* https://github.com/mirage/ocaml-cohttp/issues/328#issuecomment-222583580 *) Lwt_io.set_default_buffer_size 0x10000; let nproc = - match Sys.getenv "CORE_COUNT" with + match Sys.getenv "APP_INSTANCES" with | x -> int_of_string x | exception Not_found -> Unix.open_process_in "getconf _NPROCESSORS_ONLN" diff --git a/frameworks/OCaml/webmachine/src/tfb.opam b/frameworks/OCaml/webmachine/src/tfb.opam index 9df63e30315..d587a0d7eba 100644 --- a/frameworks/OCaml/webmachine/src/tfb.opam +++ b/frameworks/OCaml/webmachine/src/tfb.opam @@ -6,15 +6,15 @@ license: "MIT" homepage: "https://github.com/TechEmpower/FrameworkBenchmarks" bug-reports: "https://github.com/TechEmpower/FrameworkBenchmarks/issues" depends: [ - "dune" {>= "2.7" & >= "2.7.1"} - "webmachine" {>= "0.6.2"} - "lwt" {>= "5.3.0"} - "conf-libev" {>= "4-11"} - "caqti" {>= "1.2.3"} - "caqti-lwt" {>= "1.2.0"} - "caqti-driver-postgresql" {>= "1.2.4"} - "cohttp-lwt-unix" {>= "2.5.4"} - "lwt_ppx" {>= "2.0.1"} + "dune" {>= "2.7" & >= "2.8.5"} + "webmachine" {>= "0.7.0"} + "lwt" {>= "5.4.1"} + "conf-libev" {>= "4-12"} + "caqti" {>= "1.6.0"} + "caqti-lwt" {>= "1.6.0"} + "caqti-driver-postgresql" {>= "1.6.0"} + "cohttp-lwt-unix" {>= "4.0.0"} + "lwt_ppx" {>= "2.0.2"} "odoc" {with-doc} ] build: [ diff --git a/frameworks/OCaml/webmachine/start-servers.sh b/frameworks/OCaml/webmachine/start-servers.sh index 5984d945be7..eabea023097 100644 --- a/frameworks/OCaml/webmachine/start-servers.sh +++ b/frameworks/OCaml/webmachine/start-servers.sh @@ -1,10 +1,13 @@ #!/bin/sh +CMD=$1 CPU_COUNT=$(nproc) P=9000 END=$(($P+$CPU_COUNT)) while [ $P -lt $END ]; do - PORT=$P /webmachine/_build/default/src/bin/tfb.exe & - let P=P+1 + PORT=$P $CMD & + P=$((P+1)) done +sleep 2 +exec /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid diff --git a/frameworks/OCaml/webmachine/webmachine-flambda.dockerfile b/frameworks/OCaml/webmachine/webmachine-flambda.dockerfile index 29d5e1a469e..ab7e4b90fc2 100644 --- a/frameworks/OCaml/webmachine/webmachine-flambda.dockerfile +++ b/frameworks/OCaml/webmachine/webmachine-flambda.dockerfile @@ -1,6 +1,6 @@ -FROM ocurrent/opam:debian-10-ocaml-4.11-flambda +FROM ocaml/opam:debian-10-ocaml-4.12-flambda -ENV DIR webmachine +ENV DIR project # https://blog.packagecloud.io/eng/2017/02/21/set-environment-variable-save-thousands-of-system-calls/ ENV TZ :/etc/localtime @@ -9,19 +9,39 @@ ENV TZ :/etc/localtime # https://blog.janestreet.com/memory-allocator-showdown/ ENV OCAMLRUNPARAM a=2,o=240 -RUN sudo apt-get install -y libpq-dev libev-dev pkg-config m4 +ENV PKGS="\ +atdgen>=2.2.1 \ +atdgen-runtime>=2.2.1 \ +caqti>=1.6.0 \ +caqti-driver-postgresql>=1.6.0 \ +caqti-lwt>=1.6.0 \ +cohttp-lwt-unix>=4.0.0 \ +conf-libev>=4-12 \ +dune>=2.8.5 \ +httpaf>=0.7.1 \ +httpaf-lwt-unix>=0.7.1 \ +lwt>=5.4.1 \ +lwt_ppx>=2.0.2 \ +opium>=0.20.0 \ +ppx_deriving_yojson>=3.6.1 \ +ppx_rapper>=3.0.0 \ +tiny_httpd>=0.8 \ +tyxml>=4.5.0 \ +webmachine>=0.7.0 \ +yojson>=1.7.0 \ +" + +RUN \ + opam update && \ + opam depext $PKGS && \ + opam install $PKGS WORKDIR /${DIR} -COPY src/tfb.opam src/Makefile /${DIR}/ -COPY src/lib.opam src/Makefile /${DIR}/ - -RUN make install - COPY ./src /${DIR} RUN sudo chown -R opam: . && make build EXPOSE 8080 -ENTRYPOINT _build/default/src/bin/tfb.exe +ENTRYPOINT ["_build/default/src/bin/tfb.exe"] diff --git a/frameworks/OCaml/webmachine/webmachine-haproxy.dockerfile b/frameworks/OCaml/webmachine/webmachine-haproxy.dockerfile index 9e2ae9fe941..fdcc275d827 100644 --- a/frameworks/OCaml/webmachine/webmachine-haproxy.dockerfile +++ b/frameworks/OCaml/webmachine/webmachine-haproxy.dockerfile @@ -1,6 +1,6 @@ -FROM ocurrent/opam:fedora-32-ocaml-4.11 +FROM ocaml/opam:debian-10-ocaml-4.12 -ENV DIR webmachine +ENV DIR project # https://blog.packagecloud.io/eng/2017/02/21/set-environment-variable-save-thousands-of-system-calls/ ENV TZ :/etc/localtime @@ -8,28 +8,52 @@ ENV TZ :/etc/localtime # https://linux.die.net/man/1/ocamlrun # https://blog.janestreet.com/memory-allocator-showdown/ ENV OCAMLRUNPARAM a=2,o=240 -# This makes the program only spawn one child process to serve requests -ENV CORE_COUNT 1 -RUN sudo dnf install --assumeyes diffutils postgresql-devel libev-devel +ENV PKGS="\ +atdgen>=2.2.1 \ +atdgen-runtime>=2.2.1 \ +caqti>=1.6.0 \ +caqti-driver-postgresql>=1.6.0 \ +caqti-lwt>=1.6.0 \ +cohttp-lwt-unix>=4.0.0 \ +conf-libev>=4-12 \ +dune>=2.8.5 \ +httpaf>=0.7.1 \ +httpaf-lwt-unix>=0.7.1 \ +lwt>=5.4.1 \ +lwt_ppx>=2.0.2 \ +opium>=0.20.0 \ +ppx_deriving_yojson>=3.6.1 \ +ppx_rapper>=3.0.0 \ +tiny_httpd>=0.8 \ +tyxml>=4.5.0 \ +webmachine>=0.7.0 \ +yojson>=1.7.0 \ +" + +RUN \ + opam update && \ + opam depext $PKGS && \ + opam install $PKGS WORKDIR /${DIR} -COPY src/tfb.opam src/Makefile /${DIR}/ -COPY src/lib.opam src/Makefile /${DIR}/ - -RUN make install - COPY ./src /${DIR} RUN sudo chown -R opam: . && make build +EXPOSE 8080 + # try to keep everything above here in sync with other dockerfiles in project for more efficent use of docker build cache -RUN sudo dnf install --assumeyes haproxy + +# This makes the program only spawn one child process to serve requests +ENV APP_INSTANCES 1 + +USER root + +RUN apt-get update && apt-get install -y haproxy COPY haproxy.cfg /etc/haproxy/haproxy.cfg COPY start-servers.sh ./start-servers.sh -RUN sudo chown -R opam: . && chmod +x ./start-servers.sh - -EXPOSE 8080 +RUN chown -R opam: . && chmod +x ./start-servers.sh -ENTRYPOINT ./start-servers.sh && sudo /usr/sbin/haproxy -W -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid +ENTRYPOINT ["/project/start-servers.sh", "_build/default/src/bin/tfb.exe"] diff --git a/frameworks/OCaml/webmachine/webmachine.dockerfile b/frameworks/OCaml/webmachine/webmachine.dockerfile index a4099d4f406..7d9fe1ca489 100644 --- a/frameworks/OCaml/webmachine/webmachine.dockerfile +++ b/frameworks/OCaml/webmachine/webmachine.dockerfile @@ -1,6 +1,6 @@ -FROM ocurrent/opam:fedora-32-ocaml-4.11 +FROM ocaml/opam:debian-10-ocaml-4.12 -ENV DIR webmachine +ENV DIR project # https://blog.packagecloud.io/eng/2017/02/21/set-environment-variable-save-thousands-of-system-calls/ ENV TZ :/etc/localtime @@ -9,19 +9,39 @@ ENV TZ :/etc/localtime # https://blog.janestreet.com/memory-allocator-showdown/ ENV OCAMLRUNPARAM a=2,o=240 -RUN sudo dnf install --assumeyes diffutils postgresql-devel libev-devel +ENV PKGS="\ +atdgen>=2.2.1 \ +atdgen-runtime>=2.2.1 \ +caqti>=1.6.0 \ +caqti-driver-postgresql>=1.6.0 \ +caqti-lwt>=1.6.0 \ +cohttp-lwt-unix>=4.0.0 \ +conf-libev>=4-12 \ +dune>=2.8.5 \ +httpaf>=0.7.1 \ +httpaf-lwt-unix>=0.7.1 \ +lwt>=5.4.1 \ +lwt_ppx>=2.0.2 \ +opium>=0.20.0 \ +ppx_deriving_yojson>=3.6.1 \ +ppx_rapper>=3.0.0 \ +tiny_httpd>=0.8 \ +tyxml>=4.5.0 \ +webmachine>=0.7.0 \ +yojson>=1.7.0 \ +" + +RUN \ + opam update && \ + opam depext $PKGS && \ + opam install $PKGS WORKDIR /${DIR} -COPY src/tfb.opam src/Makefile /${DIR}/ -COPY src/lib.opam src/Makefile /${DIR}/ - -RUN make install - COPY ./src /${DIR} RUN sudo chown -R opam: . && make build EXPOSE 8080 -ENTRYPOINT _build/default/src/bin/tfb.exe +ENTRYPOINT ["_build/default/src/bin/tfb.exe"] From 94f74ee1f34e9b73dc4927e1b80ff12305e553fd Mon Sep 17 00:00:00 2001 From: longzl Date: Mon, 28 Jun 2021 10:29:59 +0800 Subject: [PATCH 23/25] add java isocket-nio to test,thanks (#6674) * add isocket-nio * remove global message and instance message every request --- frameworks/Java/isocket-nio/README.md | 23 ++++ .../Java/isocket-nio/benchmark_config.json | 26 ++++ frameworks/Java/isocket-nio/config.toml | 15 +++ .../Java/isocket-nio/isocket-nio.dockerfile | 13 ++ frameworks/Java/isocket-nio/pom.xml | 126 ++++++++++++++++++ .../java/cn/ibaijia/tfb/HttpBootstrap.java | 28 ++++ .../main/java/cn/ibaijia/tfb/http/Header.java | 12 ++ .../java/cn/ibaijia/tfb/http/HttpEntity.java | 10 ++ .../ibaijia/tfb/http/HttpRequestEntity.java | 99 ++++++++++++++ .../ibaijia/tfb/http/HttpResponseEntity.java | 54 ++++++++ .../cn/ibaijia/tfb/processor/Message.java | 10 ++ .../tfb/processor/PlanTextProcessor.java | 68 ++++++++++ .../tfb/protocol/SimpleHttpProtocol.java | 100 ++++++++++++++ .../isocket-nio/src/main/resources/log4j2.xml | 35 +++++ 14 files changed, 619 insertions(+) create mode 100644 frameworks/Java/isocket-nio/README.md create mode 100644 frameworks/Java/isocket-nio/benchmark_config.json create mode 100644 frameworks/Java/isocket-nio/config.toml create mode 100644 frameworks/Java/isocket-nio/isocket-nio.dockerfile create mode 100644 frameworks/Java/isocket-nio/pom.xml create mode 100644 frameworks/Java/isocket-nio/src/main/java/cn/ibaijia/tfb/HttpBootstrap.java create mode 100644 frameworks/Java/isocket-nio/src/main/java/cn/ibaijia/tfb/http/Header.java create mode 100644 frameworks/Java/isocket-nio/src/main/java/cn/ibaijia/tfb/http/HttpEntity.java create mode 100644 frameworks/Java/isocket-nio/src/main/java/cn/ibaijia/tfb/http/HttpRequestEntity.java create mode 100644 frameworks/Java/isocket-nio/src/main/java/cn/ibaijia/tfb/http/HttpResponseEntity.java create mode 100644 frameworks/Java/isocket-nio/src/main/java/cn/ibaijia/tfb/processor/Message.java create mode 100644 frameworks/Java/isocket-nio/src/main/java/cn/ibaijia/tfb/processor/PlanTextProcessor.java create mode 100644 frameworks/Java/isocket-nio/src/main/java/cn/ibaijia/tfb/protocol/SimpleHttpProtocol.java create mode 100644 frameworks/Java/isocket-nio/src/main/resources/log4j2.xml diff --git a/frameworks/Java/isocket-nio/README.md b/frameworks/Java/isocket-nio/README.md new file mode 100644 index 00000000000..45cf1da6454 --- /dev/null +++ b/frameworks/Java/isocket-nio/README.md @@ -0,0 +1,23 @@ +# isocket-nio Benchmarking Test + + +This is the isocket-nio portion of a [benchmarking test suite](../) comparing a variety of web development platforms. + +### JSON Encoding Test +* [Plaintext test source](src/main/java/cn/ibaijia/tfb/HttpBootstrap.java) +* [JSON test source](src/main/java/cn/ibaijia/tfb/HttpBootstrap.java) + +## Versions + +* [Java OpenJDK 1.8](http://openjdk.java.net/) +* [isocket-nio 1.0.2](https://gitee.com/longzhili2005/soe) + +## Test URLs + +### JSON Encoding Test + + http://localhost:8080/json + +### Plaintext Encoding Test + + http://localhost:8080/plaintext \ No newline at end of file diff --git a/frameworks/Java/isocket-nio/benchmark_config.json b/frameworks/Java/isocket-nio/benchmark_config.json new file mode 100644 index 00000000000..4e56cc1fd40 --- /dev/null +++ b/frameworks/Java/isocket-nio/benchmark_config.json @@ -0,0 +1,26 @@ +{ + "framework": "isocket-nio", + "tests": [ + { + "default": { + "json_url": "/json", + "plaintext_url": "/plaintext", + "port": 8080, + "approach": "Realistic", + "classification": "Platform", + "database": "None", + "framework": "isocket-nio", + "language": "Java", + "flavor": "None", + "orm": "Raw", + "platform": "isocket-nio", + "webserver": "None", + "os": "Linux", + "database_os": "Linux", + "display_name": "isocket-nio", + "notes": "", + "versus": "isocket-nio" + } + } + ] +} diff --git a/frameworks/Java/isocket-nio/config.toml b/frameworks/Java/isocket-nio/config.toml new file mode 100644 index 00000000000..b676376c31e --- /dev/null +++ b/frameworks/Java/isocket-nio/config.toml @@ -0,0 +1,15 @@ +[framework] +name = "isocket-nio" + +[main] +urls.plaintext = "/plaintext" +urls.json = "/json" +approach = "Realistic" +classification = "Platform" +database = "None" +database_os = "Linux" +os = "Linux" +orm = "Raw" +platform = "None" +webserver = "None" +versus = "isocket-nio" diff --git a/frameworks/Java/isocket-nio/isocket-nio.dockerfile b/frameworks/Java/isocket-nio/isocket-nio.dockerfile new file mode 100644 index 00000000000..25c8d3dcd38 --- /dev/null +++ b/frameworks/Java/isocket-nio/isocket-nio.dockerfile @@ -0,0 +1,13 @@ +FROM maven:3.6.1-jdk-11-slim as maven +WORKDIR /isocket +COPY pom.xml pom.xml +COPY src src +RUN mvn clean compile assembly:single -q + +FROM openjdk:11.0.3-jdk-slim +WORKDIR /isocket +COPY --from=maven /isocket/target/isocket-nio-benchmark-1.0-jar-with-dependencies.jar app.jar + +EXPOSE 8080 + +CMD ["java", "-server", "-XX:+UseNUMA", "-XX:+UseParallelGC", "-XX:+AggressiveOpts", "-cp", "app.jar", "cn.ibaijia.tfb.HttpBootstrap"] diff --git a/frameworks/Java/isocket-nio/pom.xml b/frameworks/Java/isocket-nio/pom.xml new file mode 100644 index 00000000000..2c825867dff --- /dev/null +++ b/frameworks/Java/isocket-nio/pom.xml @@ -0,0 +1,126 @@ + + + 4.0.0 + cn.ibaijia + isocket-nio-benchmark + 1.0 + jar + + 1.7.7 + 1.2.17 + 2.7 + 1.0.3-SNAPSHOT + 1.2.44 + + + + + ossrh + https://oss.sonatype.org/content/repositories/snapshots + + true + always + + + + + + + cn.ibaijia + isocket-nio + ${isocket.version} + + + + com.alibaba + fastjson + ${fastjson.version} + + + + + org.slf4j + slf4j-api + ${slf4j.version} + + + + org.apache.logging.log4j + log4j-slf4j-impl + ${log4j2.version} + + + org.apache.logging.log4j + log4j-api + ${log4j2.version} + + + org.apache.logging.log4j + log4j-core + ${log4j2.version} + + + com.lmax + disruptor + 3.2.0 + + + + + + + central + Central Repository + https://repo.maven.apache.org/maven2 + + + sonatype-nexus-snapshots + Sonatype Nexus Snapshots + https://oss.sonatype.org/content/repositories/snapshots + + false + + + true + always + + + + + + + org.apache.maven.plugins + maven-compiler-plugin + + true + true + true + -nowarn + 1.7 + 1.7 + UTF-8 + false + + + + maven-assembly-plugin + 3.1.0 + + + jar-with-dependencies + + + + + make-assembly + package + + single + + + + + + + diff --git a/frameworks/Java/isocket-nio/src/main/java/cn/ibaijia/tfb/HttpBootstrap.java b/frameworks/Java/isocket-nio/src/main/java/cn/ibaijia/tfb/HttpBootstrap.java new file mode 100644 index 00000000000..b50ee91c3b7 --- /dev/null +++ b/frameworks/Java/isocket-nio/src/main/java/cn/ibaijia/tfb/HttpBootstrap.java @@ -0,0 +1,28 @@ +package cn.ibaijia.tfb; + +import cn.ibaijia.isocket.Server; +import cn.ibaijia.isocket.listener.SessionProcessErrorListener; +import cn.ibaijia.isocket.session.Session; +import cn.ibaijia.tfb.processor.PlanTextProcessor; +import cn.ibaijia.tfb.protocol.SimpleHttpProtocol; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +public class HttpBootstrap { + private static final Logger logger = LoggerFactory.getLogger(HttpBootstrap.class); + + public static void main(String[] args) { + Server server = new Server("0.0.0.0", 8080); + server.addProtocol(new SimpleHttpProtocol()); + server.setProcessor(new PlanTextProcessor()); + server.setSessionProcessErrorListener(new SessionProcessErrorListener() { + @Override + public void run(Session session, Object o, Throwable throwable) { + logger.error("session on process error.", throwable); + } + }); +// server.setThreadNumber(16); + server.start(); + } + +} diff --git a/frameworks/Java/isocket-nio/src/main/java/cn/ibaijia/tfb/http/Header.java b/frameworks/Java/isocket-nio/src/main/java/cn/ibaijia/tfb/http/Header.java new file mode 100644 index 00000000000..1a69f89cb6f --- /dev/null +++ b/frameworks/Java/isocket-nio/src/main/java/cn/ibaijia/tfb/http/Header.java @@ -0,0 +1,12 @@ +package cn.ibaijia.tfb.http; + +public class Header { + + public String name; + public String value; + + public Header(String name, String value) { + this.name = name; + this.value = value; + } +} diff --git a/frameworks/Java/isocket-nio/src/main/java/cn/ibaijia/tfb/http/HttpEntity.java b/frameworks/Java/isocket-nio/src/main/java/cn/ibaijia/tfb/http/HttpEntity.java new file mode 100644 index 00000000000..695050f12de --- /dev/null +++ b/frameworks/Java/isocket-nio/src/main/java/cn/ibaijia/tfb/http/HttpEntity.java @@ -0,0 +1,10 @@ +package cn.ibaijia.tfb.http; + +public abstract class HttpEntity { + + public String charset = "UTF-8"; + + public abstract String getHeader(String name); + public abstract void setHeader(String name,String value); + +} diff --git a/frameworks/Java/isocket-nio/src/main/java/cn/ibaijia/tfb/http/HttpRequestEntity.java b/frameworks/Java/isocket-nio/src/main/java/cn/ibaijia/tfb/http/HttpRequestEntity.java new file mode 100644 index 00000000000..fd5e683388e --- /dev/null +++ b/frameworks/Java/isocket-nio/src/main/java/cn/ibaijia/tfb/http/HttpRequestEntity.java @@ -0,0 +1,99 @@ +package cn.ibaijia.tfb.http; + +import java.nio.ByteBuffer; +import java.util.HashMap; +import java.util.Map; + +public class HttpRequestEntity extends HttpEntity { + + public ByteBuffer headerBuffer = ByteBuffer.allocate(4 * 1024); + public ByteBuffer bodyBuffer = null; + public boolean chunked = false; + public int contentLength = -1; + public int crlfNum = 0; + + private static final String CONTENT_LENGTH = "CONTENT-LENGTH"; + private static final String TRANSFER_ENCODING = "TRANSFER-ENCODING"; + private static final String CHUNKED = "CHUNKED"; + + //请求行 + private String headLine; + public String method; + public String url; + public String protocol; + + //请求体 + public String body; + //第一次 请求header时解析 第一行不要 + private String[] headersArr; + private Map headers; + + public boolean headerComplete() { + return crlfNum == 4; + } + + public void processHeader() { + headerBuffer.flip(); + byte[] bytes = new byte[headerBuffer.remaining()]; + headerBuffer.get(bytes); + String str = new String(bytes); + headersArr = str.split("\r\n"); + if (headersArr.length > 0) { + headLine = headersArr[0]; + String[] arr = headLine.split(" "); + method = arr[0]; + url = arr[1]; + protocol = arr[2]; + // + String lengthStr = getHeader(CONTENT_LENGTH); + contentLength = lengthStr == null ? 0 : Integer.valueOf(lengthStr); + chunked = CHUNKED.equalsIgnoreCase(getHeader(TRANSFER_ENCODING)); + if (chunked) { + //TODO + throw new RuntimeException("not support chunked"); + } + if (contentLength > 0) { + bodyBuffer = ByteBuffer.allocate(contentLength); + } + } + } + + private Map getHeaders() { + if (headers == null) { + headers = new HashMap<>(); + boolean firstLine = true; + for (String header : headersArr) { + if (firstLine) { + continue; + } + String[] arr = header.split(":", 2); + headers.put(arr[0], arr[1]); + } + } + return headers; + } + + public String getHeader(String name) { + return getHeaders().get(name); + } + + @Override + public void setHeader(String name, String value) { + // TODO + } + + public void processBody() { + bodyBuffer.flip(); + byte[] bytes = new byte[bodyBuffer.remaining()]; + bodyBuffer.get(bytes); + body = new String(bytes); + } + + public boolean complete() { + if (contentLength == 0) { + return true; + } else { + return body != null; + } + } +} diff --git a/frameworks/Java/isocket-nio/src/main/java/cn/ibaijia/tfb/http/HttpResponseEntity.java b/frameworks/Java/isocket-nio/src/main/java/cn/ibaijia/tfb/http/HttpResponseEntity.java new file mode 100644 index 00000000000..699cfc8b797 --- /dev/null +++ b/frameworks/Java/isocket-nio/src/main/java/cn/ibaijia/tfb/http/HttpResponseEntity.java @@ -0,0 +1,54 @@ +package cn.ibaijia.tfb.http; + +import java.nio.ByteBuffer; +import java.util.HashMap; +import java.util.Map; + +public class HttpResponseEntity extends HttpEntity { + public static final String protocol = "HTTP/1.1"; + public int statusCode = 200; + public String status = "OK"; + + //响应体 + public String body = ""; + + //请求头 或者 响应头 + public Map headers = new HashMap<>(); + + @Override + public String getHeader(String name) { + return headers.get(name); + } + + @Override + public void setHeader(String name, String value) { + headers.put(name, value); + } + + public int getStatusCode() { + return statusCode; + } + + public void setStatusCode(int statusCode) { + this.statusCode = statusCode; + } + + public String getStatus() { + return status; + } + + public void setStatus(String status) { + this.status = status; + } + + public ByteBuffer toBuffer() { + StringBuilder sb = new StringBuilder(); + sb.append(protocol).append(" ").append(statusCode).append(" ").append(status).append("\r\n"); + sb.append("Content-Length:" + body.length()).append("\r\n"); + for (Map.Entry header : headers.entrySet()) { + sb.append(header.getKey()).append(":").append(header.getValue()).append("\r\n"); + } + sb.append("\r\n").append(body); + return ByteBuffer.wrap(sb.toString().getBytes()); + } +} diff --git a/frameworks/Java/isocket-nio/src/main/java/cn/ibaijia/tfb/processor/Message.java b/frameworks/Java/isocket-nio/src/main/java/cn/ibaijia/tfb/processor/Message.java new file mode 100644 index 00000000000..410b0721237 --- /dev/null +++ b/frameworks/Java/isocket-nio/src/main/java/cn/ibaijia/tfb/processor/Message.java @@ -0,0 +1,10 @@ +package cn.ibaijia.tfb.processor; + +public class Message { + + public String message; + + public Message(String message) { + this.message = message; + } +} diff --git a/frameworks/Java/isocket-nio/src/main/java/cn/ibaijia/tfb/processor/PlanTextProcessor.java b/frameworks/Java/isocket-nio/src/main/java/cn/ibaijia/tfb/processor/PlanTextProcessor.java new file mode 100644 index 00000000000..3df85480d88 --- /dev/null +++ b/frameworks/Java/isocket-nio/src/main/java/cn/ibaijia/tfb/processor/PlanTextProcessor.java @@ -0,0 +1,68 @@ +package cn.ibaijia.tfb.processor; + +import cn.ibaijia.isocket.processor.Processor; +import cn.ibaijia.isocket.session.Session; +import cn.ibaijia.tfb.http.HttpEntity; +import cn.ibaijia.tfb.http.HttpRequestEntity; +import cn.ibaijia.tfb.http.HttpResponseEntity; +import cn.ibaijia.tfb.protocol.SimpleHttpProtocol; +import com.alibaba.fastjson.JSON; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.text.SimpleDateFormat; +import java.util.Date; +import java.util.Locale; + +public class PlanTextProcessor implements Processor { + private static final Logger logger = LoggerFactory.getLogger(PlanTextProcessor.class); + private static final SimpleDateFormat dateFormat = new SimpleDateFormat("E, dd MMM yyyy HH:mm:ss z", Locale.ENGLISH); + private static final String SERVER_NAME = "isocket-nio-tfb"; + + @Override + public boolean process(Session session, HttpEntity httpEntity) { + HttpRequestEntity httpRequestEntity = (HttpRequestEntity) httpEntity; + logger.trace("url:{}", httpRequestEntity.url); + if (httpRequestEntity.url.contains("/plaintext")) { + HttpResponseEntity httpResponseEntity = new HttpResponseEntity(); + httpResponseEntity.setHeader("Content-Type", "text/plain; charset=UTF-8"); + httpResponseEntity.setHeader("Server", SERVER_NAME); + httpResponseEntity.setHeader("Date", dateFormat.format(new Date())); + httpResponseEntity.body = "Hello, World!"; + session.write(httpResponseEntity); + } else if (httpRequestEntity.url.contains("/json")) { + HttpResponseEntity httpResponseEntity = new HttpResponseEntity(); + httpResponseEntity.setHeader("Content-Type", "application/json; charset=UTF-8"); + httpResponseEntity.setHeader("Server", SERVER_NAME); + httpResponseEntity.setHeader("Date", dateFormat.format(new Date())); + httpResponseEntity.body = JSON.toJSONString(new Message("Hello, World!")); + session.write(httpResponseEntity); + } else { + HttpResponseEntity httpResponseEntity = new HttpResponseEntity(); + httpResponseEntity.setHeader("Content-Type", "text/plain"); + httpResponseEntity.setHeader("Server", SERVER_NAME); + httpResponseEntity.setHeader("Date", dateFormat.format(new Date())); + httpResponseEntity.body = "hi"; + session.write(httpResponseEntity); + } +// String connection = httpRequestEntity.getHeader("Connection"); +// logger.trace("Connection:{}", connection); +// if (connection == null || "close".equals(connection)) { +// session.close(false); //TODO +// } + return true; + } + + @Override + public void processError(Session session, HttpEntity httpEntity, Throwable throwable) { + logger.error("processError:", throwable); + HttpResponseEntity httpResponseEntity = new HttpResponseEntity(); + httpResponseEntity.setHeader("Content-Type", "text/plain"); + httpResponseEntity.setHeader("Server", SERVER_NAME); + httpResponseEntity.setHeader("Date", dateFormat.format(new Date())); + httpResponseEntity.body = "hi"; + httpResponseEntity.statusCode = 500; + httpResponseEntity.status = "Internal Server Error"; + session.write(httpResponseEntity); + } +} diff --git a/frameworks/Java/isocket-nio/src/main/java/cn/ibaijia/tfb/protocol/SimpleHttpProtocol.java b/frameworks/Java/isocket-nio/src/main/java/cn/ibaijia/tfb/protocol/SimpleHttpProtocol.java new file mode 100644 index 00000000000..05d7061669e --- /dev/null +++ b/frameworks/Java/isocket-nio/src/main/java/cn/ibaijia/tfb/protocol/SimpleHttpProtocol.java @@ -0,0 +1,100 @@ +package cn.ibaijia.tfb.protocol; + +import cn.ibaijia.isocket.Server; +import cn.ibaijia.isocket.protocol.Protocol; +import cn.ibaijia.isocket.session.Session; +import cn.ibaijia.tfb.http.HttpEntity; +import cn.ibaijia.tfb.http.HttpRequestEntity; +import cn.ibaijia.tfb.http.HttpResponseEntity; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.nio.ByteBuffer; + +public class SimpleHttpProtocol implements Protocol { + + private static final Logger logger = LoggerFactory.getLogger(SimpleHttpProtocol.class); + + private static final byte CRLF13 = (byte) 13; // \r + private static final byte CRLF10 = (byte) 10; // \n + + private static final String httpEntityKey = "httpEntity"; + + /** + * 解析HTTP请求 + * + * @param byteBuffer + * @param session + * @return + */ + @Override + public HttpEntity decode(ByteBuffer byteBuffer, Session session) { + HttpRequestEntity httpEntity = (HttpRequestEntity) session.getAttribute(httpEntityKey); + if (httpEntity == null) { + httpEntity = new HttpRequestEntity(); + session.setAttribute(httpEntityKey, httpEntity); + } + + if (byteBuffer.hasRemaining() && !httpEntity.headerComplete()) { //解析header + readHeader(byteBuffer, session, httpEntity); + } + + if (httpEntity.bodyBuffer != null && byteBuffer.hasRemaining() && httpEntity.headerComplete()) {// 解析body + readBody(byteBuffer, session, httpEntity); + } + + if (httpEntity.complete()) { + session.setAttribute(httpEntityKey, null); + return httpEntity; + } + + return null; + } + + private void readHeader(ByteBuffer byteBuffer, Session session, HttpRequestEntity httpEntity) { + try { + while (byteBuffer.hasRemaining()) { + byte b = byteBuffer.get(); + if (b == CRLF10 || b == CRLF13) { + httpEntity.crlfNum++; + } else { + httpEntity.crlfNum = 0; + } + httpEntity.headerBuffer.put(b); + if (httpEntity.crlfNum == 4) { + //处理header + httpEntity.processHeader(); + break; + } + } + } catch (Exception e) { + logger.error("readHeader error.", e); + } + } + + private void readBody(ByteBuffer byteBuffer, Session session, HttpRequestEntity httpEntity) { + try { + if (httpEntity.bodyBuffer.hasRemaining()) { + if (byteBuffer.remaining() <= httpEntity.bodyBuffer.remaining()) { + httpEntity.bodyBuffer.put(byteBuffer); + } else { + byte[] bytes = new byte[httpEntity.bodyBuffer.remaining()]; + byteBuffer.get(bytes); + httpEntity.bodyBuffer.put(bytes); + } + } + if (!httpEntity.bodyBuffer.hasRemaining()) { + httpEntity.processBody(); + } + } catch (Exception e) { + logger.error("readHeader error.", e); + } + } + + @Override + public ByteBuffer encode(HttpEntity httpEntity, Session session) { + HttpResponseEntity httpResponseEntity = (HttpResponseEntity) httpEntity; + + return httpResponseEntity.toBuffer(); + } +} diff --git a/frameworks/Java/isocket-nio/src/main/resources/log4j2.xml b/frameworks/Java/isocket-nio/src/main/resources/log4j2.xml new file mode 100644 index 00000000000..4a318937d6e --- /dev/null +++ b/frameworks/Java/isocket-nio/src/main/resources/log4j2.xml @@ -0,0 +1,35 @@ + + + + isocket-nio-tfb + + + + + + + + [%d{MM-dd HH:mm:ss}] %p [%c][%t] - %m%n + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file From 4ee6fb946b15d275bd2c71a71c7b9210c15e0bc4 Mon Sep 17 00:00:00 2001 From: Nikolay Kim Date: Mon, 28 Jun 2021 08:30:09 +0600 Subject: [PATCH 24/25] upgrade ntex to 0.4 (#6682) * upgrade ntex * bump version --- frameworks/Rust/ntex/Cargo.toml | 7 +++---- frameworks/Rust/ntex/ntex-db.dockerfile | 2 +- frameworks/Rust/ntex/ntex-raw.dockerfile | 2 +- frameworks/Rust/ntex/ntex.dockerfile | 2 +- frameworks/Rust/ntex/src/db.rs | 21 ++++++++++++++------- frameworks/Rust/ntex/src/main.rs | 3 +-- frameworks/Rust/ntex/src/main_db.rs | 11 +++++++---- frameworks/Rust/ntex/src/main_raw.rs | 13 +++++++------ frameworks/Rust/ntex/src/utils.rs | 15 ++++++++++++++- 9 files changed, 49 insertions(+), 27 deletions(-) diff --git a/frameworks/Rust/ntex/Cargo.toml b/frameworks/Rust/ntex/Cargo.toml index 8b92b1ba84f..6a6d297d7fc 100755 --- a/frameworks/Rust/ntex/Cargo.toml +++ b/frameworks/Rust/ntex/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ntex" -version = "0.2.0" +version = "0.2.1" edition = "2018" [[bin]] @@ -16,13 +16,12 @@ name = "ntex-raw" path = "src/main_raw.rs" [dependencies] -ntex = "0.3.17" +ntex = "0.4.0-b.1" mimalloc = { version = "0.1.25", default-features = false } snmalloc-rs = { version = "0.2.26", features = ["1mib", "native-cpu"] } yarte = { version = "0.15", features = ["bytes-buf", "json"] } env_logger = "0.8" nanorand = { version = "0.5", default-features = false, features = ["std", "wyrand"] } -bytes = "1.0" atoi = "0.4" num_cpus = "1.13" futures = "0.3" @@ -32,7 +31,7 @@ simd-json = "0.4.6" simd-json-derive = "0.2.2" serde = { version = "1.0", features = ["derive"] } log = { version = "0.4", features = ["release_max_level_off"] } -tokio = "1" +tokio = { version = "1", default-features = false } tokio-postgres = { git="https://github.com/fafhrd91/postgres.git" } [profile.release] diff --git a/frameworks/Rust/ntex/ntex-db.dockerfile b/frameworks/Rust/ntex/ntex-db.dockerfile index e9d361c6b6c..860fd5c5699 100644 --- a/frameworks/Rust/ntex/ntex-db.dockerfile +++ b/frameworks/Rust/ntex/ntex-db.dockerfile @@ -1,4 +1,4 @@ -FROM rust:1.52.1 +FROM rust:1.53.0 # Disable simd at jsonescape # ENV CARGO_CFG_JSONESCAPE_DISABLE_AUTO_SIMD= diff --git a/frameworks/Rust/ntex/ntex-raw.dockerfile b/frameworks/Rust/ntex/ntex-raw.dockerfile index 3a4b2aace37..487789ead91 100644 --- a/frameworks/Rust/ntex/ntex-raw.dockerfile +++ b/frameworks/Rust/ntex/ntex-raw.dockerfile @@ -1,4 +1,4 @@ -FROM rust:1.52.1 +FROM rust:1.53.0 # Disable simd at jsonescape # ENV CARGO_CFG_JSONESCAPE_DISABLE_AUTO_SIMD= diff --git a/frameworks/Rust/ntex/ntex.dockerfile b/frameworks/Rust/ntex/ntex.dockerfile index 4d9a95643aa..6529cdf3a5b 100644 --- a/frameworks/Rust/ntex/ntex.dockerfile +++ b/frameworks/Rust/ntex/ntex.dockerfile @@ -1,4 +1,4 @@ -FROM rust:1.52.1 +FROM rust:1.53.0 # Disable simd at jsonescape # ENV CARGO_CFG_JSONESCAPE_DISABLE_AUTO_SIMD= diff --git a/frameworks/Rust/ntex/src/db.rs b/frameworks/Rust/ntex/src/db.rs index 56a3ca67f59..41677d59fe5 100644 --- a/frameworks/Rust/ntex/src/db.rs +++ b/frameworks/Rust/ntex/src/db.rs @@ -1,14 +1,16 @@ use std::{borrow::Cow, cell::RefCell, fmt::Write as FmtWrite}; -use bytes::{Bytes, BytesMut}; use futures::{Future, FutureExt}; use nanorand::{WyRand, RNG}; +use ntex::util::{Bytes, BytesMut}; use smallvec::SmallVec; use tokio_postgres::types::ToSql; use tokio_postgres::{connect, Client, NoTls, Statement}; use yarte::{ywrite_html, Serialize}; -#[derive(Copy, Clone, Serialize, Debug)] +use crate::utils::Writer; + +#[derive(Copy, Clone, Serialize, Debug, serde::Serialize)] pub struct World { pub id: i32, pub randomnumber: i32, @@ -72,11 +74,16 @@ impl PgConnection { let random_id = (self.rng.borrow_mut().generate::() % 10_000 + 1) as i32; self.cl.query(&self.world, &[&random_id]).map(|rows| { let rows = rows.unwrap(); - World { - id: rows[0].get(0), - randomnumber: rows[0].get(1), - } - .to_bytes::(40) + let mut body = BytesMut::new(); + simd_json::to_writer( + Writer(&mut body), + &World { + id: rows[0].get(0), + randomnumber: rows[0].get(1), + }, + ) + .unwrap(); + body.freeze() }) } diff --git a/frameworks/Rust/ntex/src/main.rs b/frameworks/Rust/ntex/src/main.rs index e6bdb80023d..860f77116c5 100644 --- a/frameworks/Rust/ntex/src/main.rs +++ b/frameworks/Rust/ntex/src/main.rs @@ -1,9 +1,8 @@ #[global_allocator] static GLOBAL: mimalloc::MiMalloc = mimalloc::MiMalloc; -use bytes::Bytes; use ntex::http::header::{HeaderValue, CONTENT_TYPE, SERVER}; -use ntex::{http, web}; +use ntex::{http, util::Bytes, web}; use yarte::Serialize; mod utils; diff --git a/frameworks/Rust/ntex/src/main_db.rs b/frameworks/Rust/ntex/src/main_db.rs index 5117eaf1e21..d459831cd48 100644 --- a/frameworks/Rust/ntex/src/main_db.rs +++ b/frameworks/Rust/ntex/src/main_db.rs @@ -3,13 +3,12 @@ static GLOBAL: mimalloc::MiMalloc = mimalloc::MiMalloc; use std::{pin::Pin, task::Context, task::Poll}; -use bytes::BytesMut; use futures::future::{ok, Future, FutureExt}; use ntex::http::header::{HeaderValue, CONTENT_TYPE, SERVER}; use ntex::http::{HttpService, KeepAlive, Request, Response}; use ntex::service::{Service, ServiceFactory}; +use ntex::util::BytesMut; use ntex::web::{Error, HttpResponse}; -use yarte::Serialize; mod db; mod utils; @@ -48,20 +47,24 @@ impl Service for App { self.0 .get_worlds(utils::get_query_param(req.uri().query())) .map(|worlds| { + let mut body = BytesMut::with_capacity(35 * worlds.len()); + let _ = simd_json::to_writer(crate::utils::Writer(&mut body), &worlds); Ok(HttpResponse::Ok() .header(SERVER, HeaderValue::from_static("N")) .header(CONTENT_TYPE, HeaderValue::from_static("application/json")) - .body(worlds.to_bytes::(35 * worlds.len()))) + .body(body.freeze())) }), ), "/update" => Box::pin( self.0 .update(utils::get_query_param(req.uri().query())) .map(|worlds| { + let mut body = BytesMut::with_capacity(35 * worlds.len()); + let _ = simd_json::to_writer(crate::utils::Writer(&mut body), &worlds); Ok(HttpResponse::Ok() .header(SERVER, HeaderValue::from_static("N")) .header(CONTENT_TYPE, HeaderValue::from_static("application/json")) - .body(worlds.to_bytes::(35 * worlds.len()))) + .body(body.freeze())) }), ), _ => Box::pin(ok(Response::new(http::StatusCode::NOT_FOUND))), diff --git a/frameworks/Rust/ntex/src/main_raw.rs b/frameworks/Rust/ntex/src/main_raw.rs index 27b3ee140f4..7ba563018e8 100644 --- a/frameworks/Rust/ntex/src/main_raw.rs +++ b/frameworks/Rust/ntex/src/main_raw.rs @@ -7,7 +7,6 @@ use ntex::fn_service; use ntex::framed::{ReadTask, State, WriteTask}; use ntex::http::h1; use ntex::rt::net::TcpStream; -use yarte::Serialize; mod utils; @@ -19,7 +18,7 @@ const HTTPNFOUND: &[u8] = b"HTTP/1.1 400 OK\r\n"; const HDR_SERVER: &[u8] = b"Server: N\r\n"; const BODY: &[u8] = b"Hello, World!"; -#[derive(Serialize)] +#[derive(serde::Serialize)] pub struct Message { pub message: &'static str, } @@ -55,10 +54,12 @@ impl Future for App { "/json" => { buf.extend_from_slice(JSON); this.codec.set_date_header(buf); - Message { - message: "Hello, World!", - } - .to_bytes_mut(buf); + let _ = simd_json::to_writer( + crate::utils::Writer(buf), + &Message { + message: "Hello, World!", + }, + ); } "/plaintext" => { buf.extend_from_slice(PLAIN); diff --git a/frameworks/Rust/ntex/src/utils.rs b/frameworks/Rust/ntex/src/utils.rs index b12131b6011..612fd14f9a6 100644 --- a/frameworks/Rust/ntex/src/utils.rs +++ b/frameworks/Rust/ntex/src/utils.rs @@ -1,7 +1,8 @@ #![allow(dead_code)] -use std::cmp; +use std::{cmp, io}; use atoi::FromRadix10; +use ntex::util::{BufMut, BytesMut}; pub const SIZE: usize = 27; @@ -14,3 +15,15 @@ pub fn get_query_param(query: Option<&str>) -> u16 { }; cmp::min(500, cmp::max(1, q)) } + +pub struct Writer<'a>(pub &'a mut BytesMut); + +impl<'a> io::Write for Writer<'a> { + fn write(&mut self, buf: &[u8]) -> io::Result { + self.0.put_slice(buf); + Ok(buf.len()) + } + fn flush(&mut self) -> io::Result<()> { + Ok(()) + } +} From 9ebb1b3160d96d99a3ccb76d06432cc92a99ad9b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E8=B4=9D=E5=85=8B=E8=A1=97=E7=9A=84=E5=A4=A9=E6=89=8D?= <39583360+yuyenews@users.noreply.github.com> Date: Mon, 28 Jun 2021 10:30:27 +0800 Subject: [PATCH 25/25] update magician framework (#6683) * test martian * martian * martian * add readme.md * edit dockerfile * edit dockerfile * edit dockerfile * test martian * test martian * test martian * test martian * test martian * test martian * test martian * add martian * update martian * update martian * update martian * update martian * update martian * update martian * update martian * update martian * update martian * update martian * update martian * update martian * update martian * update martian * update martian * update martian * update martian * update martian * update martian * add magician framework * add magician framework * add magician framework * add magician framework * add magician framework * add magician framework * add magician framework * update martian * update magician framework * update magician framework --- frameworks/Java/magician-io/pom.xml | 2 +- .../Java/magician-io/src/main/java/com/test/io/Start.java | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/frameworks/Java/magician-io/pom.xml b/frameworks/Java/magician-io/pom.xml index 67b182e239f..d765944f05d 100644 --- a/frameworks/Java/magician-io/pom.xml +++ b/frameworks/Java/magician-io/pom.xml @@ -18,7 +18,7 @@ com.github.yuyenews Magician - 1.1.10 + 1.1.11 diff --git a/frameworks/Java/magician-io/src/main/java/com/test/io/Start.java b/frameworks/Java/magician-io/src/main/java/com/test/io/Start.java index 6770037f765..3f760fbc502 100644 --- a/frameworks/Java/magician-io/src/main/java/com/test/io/Start.java +++ b/frameworks/Java/magician-io/src/main/java/com/test/io/Start.java @@ -17,7 +17,7 @@ public static void main(String[] args) { try { EventGroup ioEventGroup = new EventGroup(1, Executors.newCachedThreadPool()); - EventGroup workerEventGroup = new EventGroup(20, Executors.newCachedThreadPool()); + EventGroup workerEventGroup = new EventGroup(10, Executors.newCachedThreadPool()); Magician.createTCPServer(ioEventGroup, workerEventGroup) .handler("/json", new JsonHandler())