Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Benchmarking #516

Open
hugusmaximus opened this issue Dec 29, 2023 · 8 comments
Open

Benchmarking #516

hugusmaximus opened this issue Dec 29, 2023 · 8 comments

Comments

@hugusmaximus
Copy link

hugusmaximus commented Dec 29, 2023 •
edited by mkroening

I'm trying to make a simple benchmark to a very simple code using "tiny_http" like this:

fn main(){

let server = Server::http("0.0.0.0:80");

	match server {
		Ok(_) => println!("Success"),
		Err(_) => println!("Error")
	}

	for request in server.expect("REASON").incoming_requests() {
	    let response = Response::from_string("Hello!");
		let _ = request.respond(response);
	}

}

I'm doing an extremely soft "benchmark" and getting those results:

hugo@anakin:~/hermit-rs-template$ ab -n 10 -c 3 -r "http://10.0.5.3/pruebastress"
This is ApacheBench, Version 2.3 <$Revision: 1879490 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 10.0.5.3 (be patient).....done


Server Software:        tiny-http
Server Hostname:        10.0.5.3
Server Port:            80

Document Path:          /pruebastress
Document Length:        0 bytes

Concurrency Level:      3
Time taken for tests:   0.286 seconds
Complete requests:      10
Failed requests:        20
   (Connect: 0, Receive: 10, Length: 0, Exceptions: 10)
Total transferred:      156 bytes
HTML transferred:       13 bytes
Requests per second:    34.97 [#/sec] (mean)
Time per request:       85.794 [ms] (mean)
Time per request:       28.598 [ms] (mean, across all concurrent requests)
Transfer rate:          0.53 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:    12   14   4.5     12      27
Waiting:        0    0   0.0      0       0
Total:         12   14   4.5     12      27

Percentage of the requests served within a certain time (ms)
  50%     12
  66%     12
  75%     13
  80%     14
  90%     27
  95%     27
  98%     27
  99%     27
 100%     27 (longest request)

Looks like concurrent connections are not handled also with no concurrency at all I get this:

hugo@anakin:~/hermit-rs-template$ ab -n 1000 -r "http://10.0.5.3/pruebastress"
This is ApacheBench, Version 2.3 <$Revision: 1879490 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 10.0.5.3 (be patient)
Completed 100 requests
Completed 200 requests
(...)
Finished 1000 requests

(...)
Document Path:          /pruebastress
Document Length:        13 bytes

Concurrency Level:      1
Time taken for tests:   213.315 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      156000 bytes
HTML transferred:       13000 bytes
Requests per second:    4.69 [#/sec] (mean)
Time per request:       213.315 [ms] (mean)
Time per request:       213.315 [ms] (mean, across all concurrent requests)
Transfer rate:          0.71 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       20   30   4.3     32      56
Processing:   146  183   6.2    185     208
Waiting:      132  166   5.9    167     189
Total:        170  213   5.9    213     256

Percentage of the requests served within a certain time (ms)
  50%    213
  66%    214
  75%    215
  80%    215
  90%    217
  95%    219
  98%    222
  99%    224
 100%    256 (longest request)

My console output (qemu-system-aarch64):

[LOADER][INFO] Loader: [0x40200000 - 0x4021f000]
[LOADER][INFO] Found ELF file with size 30194072
[LOADER][INFO] Parsing kernel from ELF at 0x48000000..0x49ccb998 (len = 0x1ccb998 B / 30194072 B)
[LOADER][INFO] Loading kernel to 0x40400000..0x40a2d228 (len = 0x62d228 B / 6476328 B)
[LOADER][INFO] TLS is at 0x408d90a8..0x408d9148 (len =  0xa0 B / 160 B)
[LOADER][INFO] Detect 1 CPU(s)
[LOADER][INFO] Detect UART at 0x9000000
[LOADER][INFO] Jumping to HermitCore Application Entry Point at 0x406b2794
[0][INFO] Welcome to Hermit 0.6.7
[0][INFO] Kernel starts at 40400000
[0][INFO] BSS starts at 0x40a2b7b0
[0][INFO] tls_info = Some(
    TlsInfo {
        start: 0x408d90a8,
        filesz: 0x20,
        memsz: 0xa0,
        align: 0x8,
    },
)
[0][INFO] RAM starts at physical address 40000000
[0][INFO] Physical address range: 16384GB
[0][INFO] Support of 4KB pages: true
[0][INFO] Support of 16KB pages: false
[0][INFO] Support of 64KB pages: true
[0][INFO] Total memory size: 500 MB
[0][INFO] Kernel region: [40400000 - 40c00000]
[0][INFO] A pure Rust application is running on top of Hermit!
[0][INFO] Heap: size 432 MB, start address 40c00000
[0][INFO] Heap is located at 0x40c00000..0x5bc00000 (0 Bytes unmapped)
[0][INFO] 
[0][INFO] ===================== PHYSICAL MEMORY FREE LIST ======================
[0][INFO] 0x0000005BCD4000 - 0x00000060000000
[0][INFO] ======================================================================
[0][INFO] 
[0][INFO] 
[0][INFO] ================== KERNEL VIRTUAL MEMORY FREE LIST ===================
[0][INFO] 0x0000005BC00000 - 0x00000100000000
[0][INFO] ======================================================================
[0][INFO] 
[0][INFO] Intialize generic interrupt controller
[0][INFO] Found GIC Distributor interface at 8000000 (size 0x10000)
[0][INFO] Found generic interrupt controller at 80a0000 (size 0xF60000)
[0][INFO] 
[0][INFO] ========================== CPU INFORMATION ===========================
[0][INFO] Processor compatiblity:  arm,cortex-a72
[0][INFO] Counter frequency:       62500000 Hz (from CNTFRQ_EL0)
[0][INFO] Run on hypervisor
[0][INFO] ======================================================================
[0][INFO] 
[0][INFO] Hermit booted on 2023-12-29 11:39:58.0 +00:00:00
[0][INFO] Mapping PCI Enhanced Configuration Space interface to virtual address 60000000 (size 0x10000000)
[0][INFO] Scanning PCI Busses 0 to 255
[0][INFO] Compiled with PCI support
[0][INFO] Compiled with ACPI support
[0][INFO] Compiled with FSGSBASE support
[0][INFO] Compiled with SMP support
[0][INFO] 
[0][INFO] ======================== PCI BUS INFORMATION =========================
[0][INFO] 00:00 Host bridge [0600]: Red Hat, Inc. QEMU PCIe Host bridge [1B36:0008]
[0][INFO] 00:01 Ethernet controller [0200]: Red Hat, Inc. Virtio network device [1AF4:1041], IRQ 4, BAR1 Memory32 { address: 0x0, size: 0x1000, prefetchable: false }, BAR4 Memory64 { address: 0x8000000000, size: 0x4000, prefetchable: true }
[0][INFO] ======================================================================
[0][INFO] 
[0][INFO] Hermit is running on common system!
[0][INFO] Found virtio network device with device id 0x1041
[0][WARN] Currently only mapping of 64 bit BAR's is supported!
[0][WARN] Currently only mapping of 64 bit BAR's is supported!
[0][INFO] Non Virtio PCI capability with id 11 found. And NOT used.
[0][ERROR] Found virtio capability whose BAR is not mapped or non existing. Capability of type 5 and id 0 for device 1041, can not be used!
[0][INFO] Feature set wanted by network driver are in conformance with specification.
[0][INFO] Feature set wanted by network driver are in conformance with specification.
[0][INFO] Driver found a subset of features for virtio device 1041. Features are: [VIRTIO_NET_F_CSUM, VIRTIO_NET_F_GUEST_CSUM, VIRTIO_NET_F_MAC, VIRTIO_NET_F_MRG_RXBUF, VIRTIO_NET_F_STATUS, VIRTIO_F_RING_INDIRECT_DESC, VIRTIO_F_VERSION_1]
[0][INFO] Features have been negotiated between virtio network device 1041 and driver.
[0][INFO] Created SplitVq: idx=0, size=256
[0][INFO] Created SplitVq: idx=1, size=256
[0][INFO] Network driver successfully initialized virtqueues.
[0][INFO] Device specific initialization for Virtio network device 1041 finished
[0][INFO] Network device with id 1041, has been initialized by driver!
[0][INFO] Virtio-net link is up after initialization.
[0][INFO] Virtio network driver initialized.
[0][INFO] Install virtio interrupt handler at line 4
[0][INFO] Trying to initialize network!
[0][INFO] MAC address 52-54-00-12-34-56
[0][INFO] Configure network interface with address 10.0.5.3/24
[0][INFO] Configure gateway with address 10.0.5.1
[0][INFO] MTU: 1514 bytes
[0][WARN] Unable to read entropy! Fallback to a naive implementation!
Success

I'm using Hermit 0.8.0. I'm missing something for sure as this low performance is not normal.... any hint?
@stlankes
Copy link
Contributor

I have to check. In the past, we had some performance issue. But I thought that we have solved it. Maybe it come back with an PR. Currently, our checks do not evaluate, if a PR creates some power losses.

@stlankes
Copy link
Contributor

stlankes commented Jan 4, 2024

On my system I got better performance. Naive question. Do you build the Hermit application with --release?

@hugusmaximus
Copy link
Author

hugusmaximus commented Jan 4, 2024 via email

@mkroening
Copy link
Member

Yeah, it's expected to be slow without --release. We should add a hint to the README in https://github.com/hermit-os/hermit-rs-template.

@hugusmaximus
Copy link
Author

Yes, with --release I got 40 req/sec. Not a rocket but it works for me. Thanks!

@mkroening
Copy link
Member

Network performance is something that we are actively looking into, so it should get better in the future. :)

@hugusmaximus
Copy link
Author

My apologies. I'm getting much, much more. This 40 req/sec is because an experimental KASLR I was implementing and also plus SSL offloading proxy before. Just tried your vanilla code with plain HTTP and I get 10.000 req/sec, which looks pretty good!!!

hugo@anakin:~/home/hugo$ wrk -c100 -d10000 -t 10 http://10.137.0.3:9975
Running 167m test @ http://10.137.0.3:9975
  10 threads and 100 connections
^C  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     8.97ms    2.27ms  25.83ms   82.25%
    Req/Sec     1.08k   323.60    10.69k    86.06%
  164378 requests in 15.32s, 38.22MB read
Requests/sec:  10731.98
Transfer/sec:      2.50MB

@mkroening
Copy link
Member

Ah, I am relieved. Thanks for clarifying!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@stlankes @mkroening @hugusmaximus