Skip to content

Commit

Permalink
Merge pull request #7 from abdelaziz-mahdy/updated-data-json
Browse files Browse the repository at this point in the history
Migrating to serverpod v2 and updating data scripts and using readme template generator
  • Loading branch information
abdelaziz-mahdy committed May 28, 2024
2 parents fa9fff6 + ed2c11e commit e817afa
Show file tree
Hide file tree
Showing 87 changed files with 4,035 additions and 4,999 deletions.
84 changes: 59 additions & 25 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
# Backend Benchmark Repository

## Table of Contents

- [Introduction](#introduction)
- [Prerequisites](#prerequisites)
- [Completed Benchmarks](#completed-benchmarks)
Expand All @@ -13,77 +12,112 @@
- [Rust](#rust)
- [Testing Tool: Locust](#testing-tool-locust)
- [Benchmark Visualization](#benchmark-visualization)
- [Database Endpoints](#database-endpoints)
- [Static Endpoints](#static-endpoints)
- [Database Endpoints](#database-endpoints)
- [Static Endpoints](#static-endpoints)

## Introduction

This repository serves as a comprehensive resource for comparing backend technologies based on speed and load capabilities. Our benchmarks focus on write and read operations, as well as requests to static endpoints, reflecting real-world usage scenarios to assist developers and decision-makers in choosing the most suitable backend framework for their needs.

## Prerequisites

- Docker installed on the system to run the benchmarks.
- The `scripts/start_tests.sh` script is used to launch each test and create the graphs.

## Completed Benchmarks

Benchmarks are categorized into:

1. **Database Tests (`db_test`):** Involving database operations such as read and write requests.
2. **Static Endpoint Tests (`no_db_test`):** Involving requests to static endpoints without database interaction.

### Python

- **Django** (Sync and Async)
- Connection Pooling with PgBouncer.
- **Django** (Sync and Async) - Connection Pooling with PgBouncer.

### Dart

- **Serverpod**

### JavaScript/TypeScript

- **Express** (Node and Bun)

### C#

- **.Net Core**

### Go

### Rust

## Testing Tool: Locust

- **Configuration:**
- Users: 10000
- Spawn Rate: 10 users/second
- Test Duration: 1000 seconds

## Benchmark Visualization

Visual comparisons for database endpoints and static endpoints are provided to showcase performance differences across technologies.

# Database Endpoints

## Comparison Graph with db endpoints

![Comparison Graph](comparison_graph_db_test.png?v=1716754402)
![Comparison Graph](comparison_graph_db_test.png?v=1716893694)

## Detailed Graphs for each backend
- **go mux db_test**
![go mux db_test Benchmark Graph](backends/go/mux/tests/results/db_test/graph.png?v=1716893694)

- **c_sharp dot net db_test**
![c_sharp dot net db_test Benchmark Graph](backends/c_sharp/dot-net/tests/results/db_test/graph.png?v=1716893694)

- **python django sync db_test**
![python django sync db_test Benchmark Graph](backends/python/django-sync/tests/results/db_test/graph.png?v=1716893694)

- **python fast api db_test**
![python fast api db_test Benchmark Graph](backends/python/fast-api/tests/results/db_test/graph.png?v=1716893694)

- **python django async db_test**
![python django async db_test Benchmark Graph](backends/python/django-async/tests/results/db_test/graph.png?v=1716893694)

- **dart server pod db_test**
![dart server pod db_test Benchmark Graph](backends/dart/server-pod/tests/results/db_test/graph.png?v=1716893694)

- **rust actix web db_test**
![rust actix web db_test Benchmark Graph](backends/rust/actix-web/tests/results/db_test/graph.png?v=1716893694)

- **javascript express bun db_test**
![javascript express bun db_test Benchmark Graph](backends/javascript/express-bun/tests/results/db_test/graph.png?v=1716893694)

- **javascript express node db_test**
![javascript express node db_test Benchmark Graph](backends/javascript/express-node/tests/results/db_test/graph.png?v=1716893694)


| Attribute | Django Async Backend | Django Sync Backend | Dart Serverpod Backend | Express Bun Backend | Express Node Backend | C# .NET Backend | Go Mux Backend |
| --------------- | ------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ |
| Benchmark Graph | ![Django Async Backend Benchmark Graph](/backends/python/django-async/tests/results/db_test/graph.png?v=1716754402) | ![Django Sync Backend Benchmark Graph](/backends/python/django-sync/tests/results/db_test/graph.png?v=1716754402) | ![Dart Serverpod Backend Benchmark Graph](/backends/dart/server-pod/tests/results/db_test/graph.png?v=1716754402) | ![Express Bun Backend Benchmark Graph](/backends/javascript/express-bun/tests/results/db_test/graph.png?v=1716754402) | ![Express Node Backend Benchmark Graph](/backends/javascript/express-node/tests/results/db_test/graph.png?v=1716754402) | ![C# .NET Backend Benchmark Graph](/backends/c_sharp/dot-net/tests/results/db_test/graph.png?v=1716754402) | ![Go Mux Backend Benchmark Graph](/backends/go/mux/tests/results/db_test/graph.png?v=1716754402) |

# Static Endpoints

## Comparison Graph with static endpoints

![Comparison Graph](comparison_graph_no_db_test.png?v=1716754402)
![Comparison Graph](comparison_graph_no_db_test.png?v=1716893694)

## Detailed Graphs for each backend
- **go mux no_db_test**
![go mux no_db_test Benchmark Graph](backends/go/mux/tests/results/no_db_test/graph.png?v=1716893694)

- **c_sharp dot net no_db_test**
![c_sharp dot net no_db_test Benchmark Graph](backends/c_sharp/dot-net/tests/results/no_db_test/graph.png?v=1716893694)

- **python django sync no_db_test**
![python django sync no_db_test Benchmark Graph](backends/python/django-sync/tests/results/no_db_test/graph.png?v=1716893694)

- **python fast api no_db_test**
![python fast api no_db_test Benchmark Graph](backends/python/fast-api/tests/results/no_db_test/graph.png?v=1716893694)

- **python django async no_db_test**
![python django async no_db_test Benchmark Graph](backends/python/django-async/tests/results/no_db_test/graph.png?v=1716893694)

- **dart server pod no_db_test**
![dart server pod no_db_test Benchmark Graph](backends/dart/server-pod/tests/results/no_db_test/graph.png?v=1716893694)

- **rust actix web no_db_test**
![rust actix web no_db_test Benchmark Graph](backends/rust/actix-web/tests/results/no_db_test/graph.png?v=1716893694)

- **javascript express bun no_db_test**
![javascript express bun no_db_test Benchmark Graph](backends/javascript/express-bun/tests/results/no_db_test/graph.png?v=1716893694)

- **javascript express node no_db_test**
![javascript express node no_db_test Benchmark Graph](backends/javascript/express-node/tests/results/no_db_test/graph.png?v=1716893694)


| Attribute | Django Async Backend | Django Sync Backend | Dart Serverpod Backend | Express Bun Backend | Express Node Backend | C# .NET Backend | Go Mux Backend |
| --------------- | ---------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- |
| Benchmark Graph | ![Django Async Backend Benchmark Graph](/backends/python/django-async/tests/results/no_db_test/graph.png?v=1716754402) | ![Django Sync Backend Benchmark Graph](/backends/python/django-sync/tests/results/no_db_test/graph.png?v=1716754402) | ![Dart Serverpod Backend Benchmark Graph](/backends/dart/server-pod/tests/results/no_db_test/graph.png?v=1716754402) | ![Express Bun Backend Benchmark Graph](/backends/javascript/express-bun/tests/results/no_db_test/graph.png?v=1716754402) | ![Express Node Backend Benchmark Graph](/backends/javascript/express-node/tests/results/no_db_test/graph.png?v=1716754402) | ![C# .NET Backend Benchmark Graph](/backends/c_sharp/dot-net/tests/results/no_db_test/graph.png?v=1716754402) | ![Go Mux Backend Benchmark Graph](/backends/go/mux/tests/results/no_db_test/graph.png?v=1716754402) |
69 changes: 69 additions & 0 deletions README_template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# Backend Benchmark Repository

## Table of Contents
- [Introduction](#introduction)
- [Prerequisites](#prerequisites)
- [Completed Benchmarks](#completed-benchmarks)
- [Python](#python)
- [Dart](#dart)
- [JavaScript/TypeScript](#javascripttypescript)
- [C#](#c)
- [Go](#go)
- [Rust](#rust)
- [Testing Tool: Locust](#testing-tool-locust)
- [Benchmark Visualization](#benchmark-visualization)
- [Database Endpoints](#database-endpoints)
- [Static Endpoints](#static-endpoints)

## Introduction
This repository serves as a comprehensive resource for comparing backend technologies based on speed and load capabilities. Our benchmarks focus on write and read operations, as well as requests to static endpoints, reflecting real-world usage scenarios to assist developers and decision-makers in choosing the most suitable backend framework for their needs.

## Prerequisites
- Docker installed on the system to run the benchmarks.
- The `scripts/start_tests.sh` script is used to launch each test and create the graphs.

## Completed Benchmarks
Benchmarks are categorized into:
1. **Database Tests (`db_test`):** Involving database operations such as read and write requests.
2. **Static Endpoint Tests (`no_db_test`):** Involving requests to static endpoints without database interaction.

### Python
- **Django** (Sync and Async) - Connection Pooling with PgBouncer.

### Dart
- **Serverpod**

### JavaScript/TypeScript
- **Express** (Node and Bun)

### C#
- **.Net Core**

### Go

### Rust

## Testing Tool: Locust
- **Configuration:**
- Users: 10000
- Spawn Rate: 10 users/second
- Test Duration: 1000 seconds

## Benchmark Visualization
Visual comparisons for database endpoints and static endpoints are provided to showcase performance differences across technologies.

# Database Endpoints

## Comparison Graph with db endpoints
![Comparison Graph](comparison_graph_db_test.png?v={version})

## Detailed Graphs for each backend
{db_endpoint_graphs}

# Static Endpoints

## Comparison Graph with static endpoints
![Comparison Graph](comparison_graph_no_db_test.png?v={version})

## Detailed Graphs for each backend
{static_endpoint_graphs}
Binary file modified backends/dart/server-pod/.DS_Store
Binary file not shown.
2 changes: 1 addition & 1 deletion backends/dart/server-pod/benchmark_client/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
.dart_tool/
.packages

# Omit commiting pubspec.lock for library packages:
# Omit committing pubspec.lock for library packages:
# https://dart.dev/guides/libraries/private-files#pubspeclock
pubspec.lock

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,19 +25,6 @@ class EndpointExample extends _i1.EndpointRef {
'hello',
{'name': name},
);

_i2.Future<void> createNote(_i3.Note note) => caller.callServerEndpoint<void>(
'example',
'createNote',
{'note': note},
);

_i2.Future<List<_i3.Note>> getAllNotes() =>
caller.callServerEndpoint<List<_i3.Note>>(
'example',
'getAllNotes',
{},
);
}

/// {@category Endpoint}
Expand Down Expand Up @@ -80,13 +67,21 @@ class Client extends _i1.ServerpodClient {
_i1.AuthenticationKeyManager? authenticationKeyManager,
Duration? streamingConnectionTimeout,
Duration? connectionTimeout,
Function(
_i1.MethodCallContext,
Object,
StackTrace,
)? onFailedCall,
Function(_i1.MethodCallContext)? onSucceededCall,
}) : super(
host,
_i4.Protocol(),
securityContext: securityContext,
authenticationKeyManager: authenticationKeyManager,
streamingConnectionTimeout: streamingConnectionTimeout,
connectionTimeout: connectionTimeout,
onFailedCall: onFailedCall,
onSucceededCall: onSucceededCall,
) {
example = EndpointExample(this);
note = EndpointNote(this);
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
/* AUTOMATICALLY GENERATED CODE DO NOT MODIFY */
/* To generate run: "serverpod generate" */

// ignore_for_file: library_private_types_in_public_api
// ignore_for_file: public_member_api_docs
// ignore_for_file: implementation_imports
// ignore_for_file: use_super_parameters
// ignore_for_file: type_literal_in_constant_pattern

// ignore_for_file: no_leading_underscores_for_library_prefixes
import 'package:serverpod_client/serverpod_client.dart' as _i1;

abstract class Example implements _i1.SerializableModel {
Example._({
required this.name,
required this.data,
});

factory Example({
required String name,
required int data,
}) = _ExampleImpl;

factory Example.fromJson(Map<String, dynamic> jsonSerialization) {
return Example(
name: jsonSerialization['name'] as String,
data: jsonSerialization['data'] as int,
);
}

String name;

int data;

Example copyWith({
String? name,
int? data,
});
@override
Map<String, dynamic> toJson() {
return {
'name': name,
'data': data,
};
}

@override
String toString() {
return _i1.SerializationManager.encode(this);
}
}

class _ExampleImpl extends Example {
_ExampleImpl({
required String name,
required int data,
}) : super._(
name: name,
data: data,
);

@override
Example copyWith({
String? name,
int? data,
}) {
return Example(
name: name ?? this.name,
data: data ?? this.data,
);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
// ignore_for_file: no_leading_underscores_for_library_prefixes
import 'package:serverpod_client/serverpod_client.dart' as _i1;

abstract class Note extends _i1.SerializableEntity {
abstract class Note implements _i1.SerializableModel {
Note._({
this.id,
required this.title,
Expand All @@ -23,16 +23,11 @@ abstract class Note extends _i1.SerializableEntity {
required String content,
}) = _NoteImpl;

factory Note.fromJson(
Map<String, dynamic> jsonSerialization,
_i1.SerializationManager serializationManager,
) {
factory Note.fromJson(Map<String, dynamic> jsonSerialization) {
return Note(
id: serializationManager.deserialize<int?>(jsonSerialization['id']),
title:
serializationManager.deserialize<String>(jsonSerialization['title']),
content: serializationManager
.deserialize<String>(jsonSerialization['content']),
id: jsonSerialization['id'] as int?,
title: jsonSerialization['title'] as String,
content: jsonSerialization['content'] as String,
);
}

Expand All @@ -58,6 +53,11 @@ abstract class Note extends _i1.SerializableEntity {
'content': content,
};
}

@override
String toString() {
return _i1.SerializationManager.encode(this);
}
}

class _Undefined {}
Expand Down
Loading

0 comments on commit e817afa

Please sign in to comment.