Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add table gcp_compute_machine_type. Closes #219 #238

Merged
merged 7 commits into from
Jun 30, 2021
Merged

Add table gcp_compute_machine_type. Closes #219 #238

merged 7 commits into from
Jun 30, 2021

Conversation

rajeshbal65
Copy link
Contributor

@rajeshbal65 rajeshbal65 commented Jun 23, 2021 •
edited
Loading

Integration test logs

Logs 
No env file present for the current environment:  staging
 Falling back to .env config
No env file present for the current environment:  staging
customEnv TURBOT_TEST_EXPECTED_TIMEOUT undefined

SETUP: tests/gcp_compute_machine_type []

PRETEST: tests/gcp_compute_machine_type

TEST: tests/gcp_compute_machine_type
Running terraform

Warning: Deprecated Resource

The null_data_source was historically used to construct intermediate values to
re-use elsewhere in configuration, the same can now be achieved using locals


Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

id = 1.000048e+06
machine_type = "a2-highgpu-4g"
resource_aka = "gcp://compute.googleapis.com/projects/pikachu-aaa/machineTypes/a2-highgpu-4g"
zone = "us-central1-c"

Running SQL query: test-get-query.sql
[
  {
    "akas": [
      "gcp://compute.googleapis.com/projects/pikachu-aaa/machineTypes/a2-highgpu-4g"
    ],
    "name": "a2-highgpu-4g",
    "title": "a2-highgpu-4g"
  }
]
✔ PASSED

Running SQL query: test-list-query.sql
[
  {
    "akas": [
      "gcp://compute.googleapis.com/projects/pikachu-aaa/machineTypes/a2-highgpu-4g"
    ],
    "name": "a2-highgpu-4g",
    "title": "a2-highgpu-4g"
  }
]
✔ PASSED

Running SQL query: test-notfound-query.sql
null
✔ PASSED

Running SQL query: test-turbot-query.sql
[
  {
    "akas": [
      "gcp://compute.googleapis.com/projects/pikachu-aaa/machineTypes/a2-highgpu-4g"
    ],
    "title": "a2-highgpu-4g"
  }
]
✔ PASSED

POSTTEST: tests/gcp_compute_machine_type

TEARDOWN: tests/gcp_compute_machine_type

SUMMARY:

1/1 passed.

Example query results

Results 
### Compute machine type basic info

```sql
select
  name,
  id,
  description,
  guest_cpus,
  maximum_persistent_disks,
  maximum_persistent_disks_size_gb
from
  gcp_compute_machine_type;

+------------------+---------+------------------------------------------------------------------------+------------+--------------------------+----------------------------------+
| name             | id      | description                                                            | guest_cpus | maximum_persistent_disks | maximum_persistent_disks_size_gb |
+------------------+---------+------------------------------------------------------------------------+------------+--------------------------+----------------------------------+
| a2-highgpu-1g    | 1000012 | Accelerator Optimized: 1 NVIDIA Tesla A100 GPU, 12 vCPUs, 85GB RAM     | 12         | 128                      | 263168                           |
| e2-standard-32   | 335032  | Efficient Instance, 32 vCPUs, 128 GB RAM                               | 32         | 128                      | 263168                           |
| c2-standard-4    | 801004  | Compute Optimized: 4 vCPUs, 16 GB RAM                                  | 4          | 128                      | 263168                           |
| a2-highgpu-4g    | 1000048 | Accelerator Optimized: 4 NVIDIA Tesla A100 GPUs, 48 vCPUs, 340GB RAM   | 48         | 128                      | 263168                           |
| e2-highmem-2     | 336002  | Efficient Instance, 2 vCPUs, 16 GB RAM                                 | 2          | 128                      | 263168                           |
| c2-standard-16   | 801016  | Compute Optimized: 16 vCPUs, 64 GB RAM                                 | 16         | 128                      | 263168                           |
| c2-standard-60   | 801060  | Compute Optimized: 60 vCPUs, 240 GB RAM                                | 60         | 128                      | 263168                           |
| a2-highgpu-2g    | 1000024 | Accelerator Optimized: 2 NVIDIA Tesla A100 GPUs, 24 vCPUs, 170GB RAM   | 24         | 128                      | 263168                           |
| e2-standard-4    | 335004  | Efficient Instance, 4 vCPUs, 16 GB RAM                                 | 4          | 128                      | 263168                           |
| c2-standard-30   | 801030  | Compute Optimized: 30 vCPUs, 120 GB RAM                                | 30         | 128                      | 263168                           |
| e2-highcpu-32    | 337032  | Efficient Instance, 32 vCPUs, 32 GB RAM                                | 32         | 128                      | 263168                           |
| a2-highgpu-8g    | 1000096 | Accelerator Optimized: 8 NVIDIA Tesla A100 GPUs, 96 vCPUs, 680GB RAM   | 96         | 128                      | 263168                           |
| e2-highcpu-4     | 337004  | Efficient Instance, 4 vCPUs, 4 GB RAM                                  | 4          | 128                      | 263168                           |
| e2-highmem-4     | 336004  | Efficient Instance, 4 vCPUs, 32 GB RAM                                 | 4          | 128                      | 263168                           |
| e2-highcpu-16    | 337016  | Efficient Instance, 16 vCPUs, 16 GB RAM                                | 16         | 128                      | 263168                           |
| e2-highmem-8     | 336008  | Efficient Instance, 8 vCPUs, 64 GB RAM                                 | 8          | 128                      | 263168                           |
| a2-megagpu-16g   | 1001096 | Accelerator Optimized: 16 NVIDIA Tesla A100 GPUs, 96 vCPUs, 1360GB RAM | 96         | 128                      | 263168                           |
| n2-highmem-4     | 902004  | 4 vCPUs 32 GB RAM                                                      | 4          | 128                      | 263168                           |
| e2-medium        | 334004  | Efficient Instance, 2 vCPU (1/2 shared physical core) and 4 GB RAM     | 2          | 128                      | 263168                           |
| e2-standard-8    | 335008  | Efficient Instance, 8 vCPUs, 32 GB RAM                                 | 8          | 128                      | 263168                           |
| e2-micro         | 334002  | Efficient Instance, 2 vCPU (1/8 shared physical core) and 1 GB RAM     | 2          | 16                       | 3072                             |
| n2d-highcpu-80   | 910080  | 80 vCPUs 80 GB RAM                                                     | 80         | 128                      | 263168                           |
| n2d-highcpu-96   | 910096  | 96 vCPUs 96 GB RAM                                                     | 96         | 128                      | 263168                           |
| f1-micro         | 1000    | 1 vCPU (shared physical core) and 0.6 GB RAM                           | 1          | 16                       | 3072                             |
| e2-small         | 334003  | Efficient Instance, 2 vCPU (1/4 shared physical core) and 2 GB RAM     | 2          | 16                       | 3072                             |
| e2-highcpu-2     | 337002  | Efficient Instance, 2 vCPUs, 2 GB RAM                                  | 2          | 128                      | 263168                           |
| e2-highcpu-8     | 337008  | Efficient Instance, 8 vCPUs, 8 GB RAM                                  | 8          | 128                      | 263168                           |
| e2-highmem-16    | 336016  | Efficient Instance, 16 vCPUs, 128 GB RAM                               | 16         | 128                      | 263168                           |
| n2d-highmem-16   | 912016  | 16 vCPUs 128 GB RAM                                                    | 16         | 128                      | 263168                           |
| c2-standard-8    | 801008  | Compute Optimized: 8 vCPUs, 32 GB RAM                                  | 8          | 128                      | 263168                           |
| n2-highmem-48    | 902048  | 48 vCPUs 384 GB RAM                                                    | 48         | 128                      | 263168                           |
| e2-standard-16   | 335016  | Efficient Instance, 16 vCPUs, 64 GB RAM                                | 16         | 128                      | 263168                           |
| e2-standard-2    | 335002  | Efficient Instance, 2 vCPUs, 8 GB RAM                                  | 2          | 128                      | 263168                           |
| n2-standard-8    | 901008  | 8 vCPUs 32 GB RAM                                                      | 8          | 128                      | 263168                           |
| n2d-highmem-2    | 912002  | 2 vCPUs 16 GB RAM                                                      | 2          | 128                      | 263168                           |
| n1-standard-1    | 3001    | 1 vCPU, 3.75 GB RAM                                                    | 1          | 128                      | 263168                           |
| n1-standard-16   | 3016    | 16 vCPUs, 60 GB RAM                                                    | 16         | 128                      | 263168                           |
| n2-standard-80   | 901080  | 80 vCPUs 320 GB RAM                                                    | 80         | 128                      | 263168                           |

List machine types having or more CPUs

select
  name,
  id,
  description,
  guest_cpus
from
  gcp_compute_machine_type
where
  guest_cpus >= 64;

+------------------+---------+------------------------------------------------------------------------+------------+
| name             | id      | description                                                            | guest_cpus |
+------------------+---------+------------------------------------------------------------------------+------------+
| n1-highcpu-96    | 4096    | 96 vCPUs, 86 GB RAM                                                    | 96         |
| a2-megagpu-16g   | 1001096 | Accelerator Optimized: 16 NVIDIA Tesla A100 GPUs, 96 vCPUs, 1360GB RAM | 96         |
| a2-highgpu-8g    | 1000096 | Accelerator Optimized: 8 NVIDIA Tesla A100 GPUs, 96 vCPUs, 680GB RAM   | 96         |
| n2-standard-64   | 901064  | 64 vCPUs 256 GB RAM                                                    | 64         |
| m1-megamem-96    | 9196    | 96 vCPUs, 1.4 TB RAM                                                   | 96         |
| n1-highmem-64    | 5064    | 64 vCPUs, 416 GB RAM                                                   | 64         |
| m1-ultramem-160  | 11160   | 160 vCPUs, 3844 GB RAM                                                 | 160        |
| m1-ultramem-80   | 11080   | 80 vCPUs, 1922 GB RAM                                                  | 80         |
| n2d-highmem-80   | 912080  | 80 vCPUs 640 GB RAM                                                    | 80         |
| n2d-highmem-64   | 912064  | 64 vCPUs 512 GB RAM                                                    | 64         |
| n2-standard-80   | 901080  | 80 vCPUs 320 GB RAM                                                    | 80         |
| n2d-standard-224 | 911224  | 224 vCPUs 896 GB RAM                                                   | 224        |
| n1-ultramem-80   | 10080   | 80 vCPUs, 1922 GB RAM                                                  | 80         |
| n2d-highcpu-96   | 910096  | 96 vCPUs 96 GB RAM                                                     | 96         |
| n2d-highcpu-224  | 910224  | 224 vCPUs 224 GB RAM                                                   | 224        |
| n1-highmem-96    | 5096    | 96 vCPUs, 624 GB RAM                                                   | 96         |
| n2d-standard-96  | 911096  | 96 vCPUs 384 GB RAM                                                    | 96         |
| n2d-standard-80  | 911080  | 80 vCPUs 320 GB RAM                                                    | 80         |
| n2-highcpu-64    | 903064  | 64 vCPUs 64 GB RAM                                                     | 64         |
| n2d-standard-64  | 911064  | 64 vCPUs 256 GB RAM                                                    | 64         |
| n1-megamem-96    | 9096    | 96 vCPUs, 1.4 TB RAM                                                   | 96         |
| n2-highcpu-80    | 903080  | 80 vCPUs 80 GB RAM                                                     | 80         |
| n1-highcpu-64    | 4064    | 64 vCPUs, 57.6 GB RAM                                                  | 64         |
| n2d-highcpu-128  | 910128  | 128 vCPUs 128 GB RAM                                                   | 128        |
| n1-standard-64   | 3064    | 64 vCPUs, 240 GB RAM                                                   | 64         |
| n1-standard-96   | 3096    | 96 vCPUs, 360 GB RAM                                                   | 96         |
| n1-ultramem-160  | 10160   | 160 vCPUs, 3844 GB RAM                                                 | 160        |
| n2-highmem-64    | 902064  | 64 vCPUs 512 GB RAM                                                    | 64         |
| n2-highmem-80    | 902080  | 80 vCPUs 640 GB RAM                                                    | 80         |
| n2d-highmem-96   | 912096  | 96 vCPUs 768 GB RAM                                                    | 96         |
| n2d-highcpu-64   | 910064  | 64 vCPUs 64 GB RAM                                                     | 64         |
| n2d-standard-128 | 911128  | 128 vCPUs 512 GB RAM                                                   | 128        |
| n2d-highcpu-80   | 910080  | 80 vCPUs 80 GB RAM                                                     | 80         |
+------------------+---------+------------------------------------------------------------------------+------------+

List maching types having shared CPUs

select
  name,
  id,
  is_shared_cpu
from
  gcp_compute_machine_type
where
  is_shared_cpu;

+-----------+--------+---------------+
| name      | id     | is_shared_cpu |
+-----------+--------+---------------+
| f1-micro  | 1000   | true          |
| e2-medium | 334004 | true          |
| g1-small  | 2000   | true          |
| e2-micro  | 334002 | true          |
| e2-small  | 334003 | true          |
+-----------+--------+---------------+

List accelerator configurations assigned to this machine type

select
  name,
  id,
  a -> 'guestAcceleratorCount' as guest_accelerator_count,
  a ->> 'guestAcceleratorType' as guest_accelerator_type
from
  gcp_compute_machine_type,
  jsonb_array_elements(accelerators) as a;

+----------------+---------+-------------------------+------------------------+
| name           | id      | guest_accelerator_count | guest_accelerator_type |
+----------------+---------+-------------------------+------------------------+
| a2-highgpu-1g  | 1000012 | 1                       | nvidia-tesla-a100      |
| a2-highgpu-2g  | 1000024 | 2                       | nvidia-tesla-a100      |
| a2-highgpu-4g  | 1000048 | 4                       | nvidia-tesla-a100      |
| a2-highgpu-8g  | 1000096 | 8                       | nvidia-tesla-a100      |
| a2-megagpu-16g | 1001096 | 16                      | nvidia-tesla-a100      |
+----------------+---------+-------------------------+------------------------+
</details>

@rajeshbal65 rajeshbal65 self-assigned this Jun 23, 2021
Subhajit97
Subhajit97 reviewed Jun 23, 2021
View reviewed changes
gcp/plugin.go Outdated Show resolved Hide resolved
@Subhajit97 Subhajit97 linked an issue Jun 23, 2021 that may be closed by this pull request
Add table gcp_compute_instance_machine_type or it can be gcp_compute_machine_type #219
Closed
@Subhajit97 Subhajit97 changed the title Add table gcp_compute_machine_type. CLoses #219 Add table gcp_compute_machine_type. Closes #219 Jun 23, 2021
rajlearner17
rajlearner17 reviewed Jun 30, 2021
View reviewed changes
docs/tables/gcp_compute_machine_type.md
```


### List machine types with shared CPUs
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should it be CPU ? or virtual CPU (vCPU) ?

rajlearner17
rajlearner17 approved these changes Jun 30, 2021
View reviewed changes
Copy link

@rajlearner17 rajlearner17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@cbruno10 cbruno10 merged commit 3710fd3 into main Jun 30, 2021
@cbruno10 cbruno10 deleted the issue-219 branch June 30, 2021 16:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cbruno10 cbruno10 cbruno10 approved these changes

@rajlearner17 rajlearner17 rajlearner17 approved these changes

@Subhajit97 Subhajit97 Subhajit97 approved these changes

Assignees

@rajeshbal65 rajeshbal65

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add table gcp_compute_instance_machine_type or it can be gcp_compute_machine_type
4 participants
@rajeshbal65 @cbruno10 @rajlearner17 @Subhajit97