Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,22 +1,18 @@
---
title: Go Benchmarks with Sweet and Benchstat

draft: true
cascade:
draft: true
title: Benchmark Go performance with Sweet and Benchstat

minutes_to_complete: 60

who_is_this_for: This is an introductory topic for developers who are interested in measuring the performance of Go-based applications on Arm-based servers.
who_is_this_for: This introductory topic is for developers who want to measure and compare the performance of Go applications on Arm-based servers.

learning_objectives:
- Learn how to start up Arm64 and x64 instances of GCP VMs
- Install Go, benchmarks, benchstat, and sweet on the two VMs
- Use sweet and benchstat to compare the performance of Go applications on the two VMs
learning_objectives:
- Provision Arm64 and x86_64 VM instances on Google Cloud
- Install Go, Sweet, and Benchstat on each VM instance
- Run benchmarks and use Benchstat to compare Go application performance across architectures

prerequisites:
- A [Google Cloud account](https://console.cloud.google.com/). This learning path can be run on on-prem or on any cloud provider instance, but specifically documents the process for running on Google Axion.
- A local machine with [Google Cloud CLI](/install-guides/gcloud/) installed.
- A [Google Cloud account](https://console.cloud.google.com/). This Learning Path can be run on any cloud provider or on-premises, but it focuses on Google Cloud’s Axion Arm64-based instances.
- A local machine with [Google Cloud CLI](/install-guides/gcloud/) installed

author: Geremy Cohen

Expand Down
Original file line number Diff line number Diff line change
@@ -1,32 +1,31 @@
---
title: Launching a Intel Emerald Rapids Instance
title: Launch an Intel Emerald Rapids c4-standard-8 instance
weight: 30

### FIXED, DO NOT MODIFY
layout: learningpathall
---

## Section Overview
In this section, you will set up the second benchmarking system, an Intel Emerald Rapids `c4-standard-8` instance.
In this section, you'll set up the second benchmarking system: an Intel-based Emerald Rapids `c4-standard-8` instance on Google Cloud (referred to as **c4**).

## Creating the Instance
## Create the c4-standard-8 instance

To create the second system, follow the previous lesson's c4a install instructions, but make the following changes:
Follow the same steps from the previous section where you launched the c4a instance, but make the following changes for the Intel-based c4-standard-8:

1. **Name your instance:** For the `Name` field, enter "c4".
* In the **Name** field, enter "c4".
* In the **Machine types for common workloads** section, select the **c4** radio button.
![alt-text#center](images/launch_c4/3.png "Select the c4 radio button")

2. **Select machine series:** Scroll down to the Machine series section, and select the C4 radio button.
* In the **Machine configuration** section, open the dropdown select `c4-standard-8`.

![](images/launch_c4/3.png)
![alt-text#center](images/launch_c4/4.png "Open the dropdown and select `c4-standard-8`")

3. **View machine types:** Scroll down to the Machine type dropdown, and click it to show all available options.
* In the **Machine type** section, open the dropdown and select `c4-standard-8` under the **Standard** tab.

![](images/launch_c4/4.png)
![alt-text#center](images/launch_c4/5.png "Select `c4-standard-8`")

4. **Choose machine size:** Select "c4-standard-8" under the Standard tab.

![](images/launch_c4/5.png)

{{% notice Note %}} Don't forget to set the disk size for this c4 to 1000GB under the "OS and Storage" tab like you did for the c4a.{{% /notice %}}
{{% notice Note %}}
Be sure to set the disk size to **1000 GB** in the **OS and Storage** tab, just as you did for the `c4a` instance.
{{% /notice %}}

After the c4 instance starts up, you are ready to continue to the next section, where you'll install the benchmarking software.
Original file line number Diff line number Diff line change
@@ -1,67 +1,62 @@
---
title: Launching a Google Axion Instance
title: Launch an Arm-based c4a-standard-4 instance
weight: 20

### FIXED, DO NOT MODIFY
layout: learningpathall
---

## Overview
In this section, you'll learn how to spin up the first of two different VMs used for benchmarking Go tests, an Arm-based Google Axion c4a-standard-4 (c4a for short).
In this section, you'll launch the first of two VMs used for benchmarking Go applications: the Arm-based c4a-standard-4 instance on Google Cloud, (referred to as "c4a").

## Creating the c4a-standard-4 Instance
## Create the c4a-standard-4 instance

1. **Access Google Cloud Console:** Navigate to [https://console.cloud.google.com/welcome](https://console.cloud.google.com/welcome)
Go to the Google Cloud console: [https://console.cloud.google.com/welcome](https://console.cloud.google.com/welcome).

2. **Search for VM instances:** Click into the Search field.
In the search bar at the top, start typing `vm`, then select **VM instances** when it appears.

3. **Find VM Instances:** Start typing `vm` until the UI auto-completes `VM Instances`, then click it.
![alt-text#center](images/launch_c4a/3.png "Select VM instances")

![](images/launch_c4a/3.png)
On the **VM instances** page, click **Create instance**.

The VM Instances page appears.
![alt-text#center](images/launch_c4a/4.png "Select Create instance")

4. **Create a new instance:** Click `Create instance`
In the **Name** field, enter the name of the instance - here it should be `c4a`.

![](images/launch_c4a/4.png)
![alt-text#center](images/launch_c4a/5.png "Enter name of the instance")

The Machine configuration page appears.
Now select the machine series by scrolling down to the Machine series section, and selecting the **C4A** radio button.

5. **Name your instance:** Click the `Name` field, and enter "c4a" for the `Name`.
![alt-text#center](images/launch_c4a/7.png "Select C4A radio button")

![](images/launch_c4a/5.png)
To view machine types, scroll down to the **Machine type** dropdown, and select it to show all available options.

6. **Select machine series:** Scroll down to the Machine series section, and select the C4A radio button.
![alt-text#center](images/launch_c4a/8.png "Select Machine type dropdown")

![](images/launch_c4a/7.png)
Now choose machine size by selecting **c4a-standard-4** under the **Standard** tab.

7. **View machine types:** Scroll down to the Machine type dropdown, and click it to show all available options.
![alt-text#center](images/launch_c4a/9.png "Select machine size")

![](images/launch_c4a/8.png)
To configure storage, select the **OS and Storage** tab.

8. **Choose machine size:** Select "c4a-standard-4" under the Standard tab.
![alt-text#center](images/launch_c4a/10.png "Configure storage")

![](images/launch_c4a/9.png)
To modify storage settings, select **Change**.

9. **Configure storage:** Click the "OS and Storage" tab.
![alt-text#center](images/launch_c4a/11.png "Modify storage settings")

![](images/launch_c4a/10.png)
To set disk size, select the **Size (GB)** field and enter "1000" for the value.

10. **Modify storage settings:** Click "Change"
![alt-text#center](images/launch_c4a/16.png "Enter value in the Size (GB) field")

![](images/launch_c4a/11.png)
Now confirm storage settings by selecting **Select** to continue.

11. **Set disk size:** Double-click the "Size (GB)" field, then enter "1000" for the value.
![alt-text#center](images/launch_c4a/18.png "Confirm the selection of settings with the Select button")

![](images/launch_c4a/16.png)
To launch the instance, select **Create** to bring up the instance.

12. **Confirm storage settings:** Click "Select" to continue.
![alt-text#center](images/launch_c4a/19.png "Select the Create button to launch the instance")

![](images/launch_c4a/18.png)
After a few seconds, your c4a instance is up and running, and you are ready to continue to the next section.

13. **Launch the instance:** Click "Create" to bring up the instance.

![](images/launch_c4a/19.png)

After a few seconds, your c4a instance starts up, and you are ready to continue to the next section. In the next step, you will launch the second VM, an Intel-based Emerald Rapids c4-standard-8 (c4 for short), which will serve as the comparison system for our benchmarking tests.
In the next section, you'll launch the second VM, an Intel-based Emerald Rapids c4-standard-8 (referred to as "c4"), which serves as the comparison system for benchmarking.

Original file line number Diff line number Diff line change
@@ -1,38 +1,39 @@
---
title: Installing Go and Sweet
title: Install Go, Sweet, and Benchstat
weight: 40

### FIXED, DO NOT MODIFY
layout: learningpathall
---

In this section, you'll install Go, Sweet, and the Benchstat comparison tool on both VMs.
In this section, you'll install Go, Sweet, and Benchstat on both virtual machines:

## Installation Script

Sweet is a Go benchmarking tool that provides a standardized way to run performance tests across different systems. Benchstat is a companion tool that analyzes and compares benchmark results, helping you understand performance differences between systems. Together, these tools will allow you to accurately measure and compare Go performance on Arm and x86 architectures.
* Sweet is a Go benchmarking tool that provides a standardized way to run performance tests across systems.
* Benchstat is a companion tool that compares benchmark results to highlight meaningful performance differences.

Together, these tools help you evaluate Go performance on both Arm and x86 architectures.

{{% notice Note %}}
Subsequent steps in the learning path assume you are running this script (installing) from your home directory (`$HOME`), resulting in the creation of a `$HOME/benchmarks/sweet` final install path. If you decide to install elsewhere, you will need to adjust the path accordingly when prompted to run the benchmark logic later in the learning path.
Subsequent steps in this Learning Path assume you are running this script (installing) from your home directory (`$HOME`), resulting in the creation of a `$HOME/benchmarks/sweet` final install path. If you install to a different directory, update the paths in later steps to match your custom location.
{{% /notice %}}

## Installation script

Start by copying and pasting the script below on **both** of your GCP VMs. This script checks the architecture of your running VM, installs the required Go package on your VM. It then installs sweet, benchmarks, and the benchstat tools.
Start by copying and pasting the script below on both of your GCP VMs. This script automatically detects your system architecture, installs the appropriate Go version, and sets up Sweet, Benchstat, and the Go benchmark suite.

**You don't need to run it after pasting**, just paste it into your home directory and press enter to install all needed dependencies:
Paste the full block into your terminal. This creates and runs an installer script directly from your home directory:

```bash
#!/usr/bin/env bash

# Write the script to filesystem using a HEREDOC
# Write the install script to filesystem using a HEREDOC
cat <<'EOF' > install_go_and_sweet.sh

sudo apt-get -y update
sudo apt-get -y install git build-essential

# Detect architecture - this allows the same script to work on both
# our Arm (c4a) and x86 (c4) VMs without modification
# Arm (c4a) and x86 (c4) VMs without modification
ARCH=$(uname -m)
case "$ARCH" in
arm64|aarch64)
Expand Down Expand Up @@ -90,7 +91,9 @@ chmod 755 install_go_and_sweet.sh

```

The end of the output should look like:
## Expected output from sweet get

When `sweet get` completes successfully, you’ll see output similar to:

```output
Sweet v0.3.0: Go Benchmarking Suite
Expand All @@ -109,7 +112,7 @@ Usage: sweet get [flags]
```


## Verify Installation
## Verify installation

To test that everything is installed correctly, set the environment variables shown below on each VM:

Expand Down
Original file line number Diff line number Diff line change
@@ -1,34 +1,44 @@
---
title: Manually running benchmarks
title: Manually run benchmarks
weight: 51

### FIXED, DO NOT MODIFY
layout: learningpathall
---

In this section, you'll download the results of the benchmark you ran manually in the previous sections from each VM. You will use these results to understand how `sweet` and `benchstat` work together.
In this section, you'll download the benchmark results you ran manually in previous steps from each VM. You will use these results to understand how `sweet` and `benchstat` work together.

## Download Benchmark Results from each VM
Lets walk through the steps to manually download the sweet benchmark results from your initial run on each VM.
## Download benchmark results from each VM
Start by retrieving the results generated by Sweet from your earlier benchmark runs.


1. **Locate results:** Change directory to the `results/markdown` directory and list the files to see the `arm-benchmarks.result` file:
### Locate results

Change directory to the `results/markdown` directory and list the files to see the `arm-benchmarks.result` file:

```bash
cd results/markdown
ls -d $PWD/*
```

2. **Copy result path:** Copy the absolute pathname of `arm-benchmarks.result`.
### Copy result path

Copy the absolute pathname of `arm-benchmarks.result`. You'll need this to initiate the download.

### Download results

Select `DOWNLOAD FILE` in your GCP terminal interface. Paste the absolute pathname you copied into the dialog and confirm the download. This downloads the benchmark results to your local machine.

![alt-text#center](images/run_manually/6.png "Download the results")

3. **Download results:** Click `DOWNLOAD FILE`, and paste the **ABSOLUTE PATHNAME** you just copied for the filename, and then click `Download`. This will download the benchmark results to your local machine.
### Rename the file

![](images/run_manually/6.png)
After downloading the file to your local machine, rename it to `c4a.result` to distinguish it from the x86 results you'll download next. This naming convention helps you clearly identify which architecture each result came from. You'll know the download was successful if you see the file named `c4a.result` in your Downloads folder and receive a confirmation in your browser.

4. **Rename the file:** Once downloaded, on your local machine, rename this file to `c4a.result` so you can distinguish it from the x86 results you'll download later. This naming convention will help you clearly identify which results came from which architecture. You'll know the file downloaded successfully if you see the file in your Downloads directory with the name `c4a.result`, as well as the confirmation dialog in your browser:
![alt-text#center](images/run_manually/7.png "A successful download")

![](images/run_manually/7.png)
### Repeat for the second VM

5. **Repeat for c4 instance:** Repeat steps 2-8 with your `c4` (x86) instance. Do everything the same, except after downloading the c4's `arm-benchmarks.result` file, rename it to `c4.result`.
Repeat the same process with your c4 (x86) VM. Use the same results/markdown directory and download the `arm-benchmarks.result` file. This time, rename it to `c4.result` after downloading.

Now that you have the results from both VMs, in the next section, you'll learn how to use benchstat to analyze these results and understand the performance differences between the two architectures.
Now that you have the results from both VMs, in the next section, you'll learn how to use Benchstat to analyze these results and understand the performance differences between the two architectures.
Loading