Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Add benchamrks files for using as a demo #16

Merged
merged 1 commit into from
Jul 5, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 70 additions & 0 deletions benchmarks/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# Benchmarks

## Table of Contents

- [Benchmark Runs](#benchmark-runs)
- [performance.now()](#performancenow)
- [In An HTTP Server](#in-an-http-server)

## Benchmark Runs

### performance.now()

The below benchmarks were run with a set number of items in each `Map`; and
using `performance.now()` to check how long `.forEach()` method took to return a
record. The benchmarks application searched for the last item in each `Map`.
Below are the average times showing how long it took searches to complete and
what method was used.

Command used:

```
$ deno run -A benchmarks/app.ts [map|service] [number of seconds] [number of records to create]
```

#### Results

```
Performing search with 1,000 records(s) for 10s.
Searching took an avg of 0.00002s using Map.forEach().
Searching took an avg of 0.00001s using IndexService.search().

Performing search with 10,000 records(s) for 10s.
Searching took an avg of 0.00014s using Map.forEach().
Searching took an avg of 0.00001s using IndexService.search().

Performing search with 100,000 records(s) for 10s.
Searching took an avg of 0.00136s using Map.forEach().
Searching took an avg of 0.00001s using IndexService.search().

Performing search with 1,000,000 records(s) for 10s.
Searching took an avg of 0.01379s using Map.forEach().
Searching took an avg of 0.00001s using IndexService.search().
saragee3 marked this conversation as resolved.
Show resolved Hide resolved
```

### In An HTTP Server

The benchmarks below show how the index service performs in an HTTP server. The
dataset that was searched had 10,000,000 records. The search term used was
"Happy"; and the index service had to return records that included the word
"Happy". The index service returned 50,000 records per request.

Command used:

```
$ deno run --allow-net app_http.ts [number of records to create]
$ wrk -c 40 -d 10 http://localhost:8000/{searchTerm}
```

#### Results

```
Running 10s test @ http://localhost:8000/Happy
2 threads and 40 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 2.64ms 12.61ms 191.42ms 98.48%
Req/Sec 15.99k 1.85k 17.19k 94.95%
315254 requests in 10.01s, 18.34MB read
Requests/sec: 31503.92
Transfer/sec: 1.83MB
```
124 changes: 124 additions & 0 deletions benchmarks/app.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
import { Moogle } from "../mod.ts";

let args = Deno.args.slice();

const numRequests = 100000000000;
const seconds = Number(args[1]);
let records = Number(args[2]);

console.log(
`Performing search with ${
numberWithCommas(records)
} record(s) for ${seconds}s.`,
);

//
// deno run benchmarks_app.ts map 10 10000
//
if (args[0] === "map") {
map(records);
Deno.exit();
}

//
// deno run benchmarks_app.ts service 10 10000
//
if (args[0] === "service") {
service(records);
Deno.exit();
}

////////////////////////////////////////////////////////////////////////////////
// FILE MARKER - HELPERS ///////////////////////////////////////////////////////
////////////////////////////////////////////////////////////////////////////////

interface SearchResult {
id: number;
item: string;
search_input: string;
search_term: RegExp | string;
}

function benchmark(
process: () => void,
method: string,
): void {
const numbers: number[] = [];
const now = performance.now();
let i = 0;
while (i < numRequests) {
const pn = performance.now();
process();
const pt = performance.now();
numbers.push(pt - pn);
const then = performance.now();
if (((then + now) / 1000) >= seconds) {
i = 0;
break;
}
i++;
}

let total = 0;
for (let i = 0; i < numbers.length; i++) {
total += numbers[i];
}

let avg = total / numbers.length;

console.log(
`Searching took an avg of ${(avg / 1000).toFixed(5)}s using ${method}.`,
);
}

function map(records: number): void {
const m = new Map<string, SearchResult>();

for (let i = 0; i < records; i++) {
m.set(i.toString(), {
id: i,
item: i.toString() + "value",
search_term: i.toString(),
search_input: i.toString(),
});
}

m.set("last item", {
id: m.size + 1,
item: "test value",
search_term: "/test/g",
search_input: "test",
});

const results: SearchResult[] = [];
benchmark(() => {
m.forEach((item: SearchResult) => {
if (results.length > 0) {
return;
}
if ((item.search_term as string).includes("test")) {
results.push(item);
}
});
}, "Map.forEach()");
}

function service(records: number): void {
const s = new Moogle<string>();

for (let i = 0; i < records; i++) {
s.addItem([i.toString()], i.toString());
}

s.addItem(["last item"], "last item value");

benchmark(() => {
const result = s.search("last");
}, "IndexService.search()");
}

// https://stackoverflow.com/questions/2901102
// /how-to-print-a-number-with-commas-as-thousands-separators-in-javascript
function numberWithCommas(x: number): string {
return x.toString().replace(/\B(?=(\d{3})+(?!\d))/g, ",");
}
24 changes: 24 additions & 0 deletions benchmarks/app_http.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
import { Moogle } from "../mod.ts";
import { serve } from "../../deps.ts";
import { dataset } from "./dataset.ts";

const s = new Moogle<string>();

for (let i = 0; i < (Deno.args[0] || 1000); i++) {
dataset.data.children.forEach((child: any) => {
s.addItem([child.data.title], child.data);
});
i++;
}

console.log("Dataset has " + s.getLookupTable().size + " items.");

const deno = serve({ port: 8000 });

console.log("Server started at http://localhost:8000");

for await (const req of deno) {
const searchTerm = req.url.split("/")[1];
const results = s.search(searchTerm);
req.respond({ body: results.size + " record(s) found" });
}