Skip to content

iacsecurity/tool-compare

Repository files navigation

MIT License Maintenance

tool-compare

In the world of infrastructure-as-code security there are several tools for users to choose from. The goal of this repository is to help compare the different options so that users can choose the tool that best fits their own needs.

What tools are there?

Checkov Cloudrail Kics Snyk Terrascan Tfsec
Vendor Bridgecrew Indeni Checkmarx Snyk Accurics Aqua Security
License OSS Freemium OSS Freemium OSS OSS
Written in Python Python Rego Unknown Rego Go
Custom Rule Support Yes Yes Yes No Yes Yes
CI/CD-specific Integrations CircleCI, GitLab, GitHub CircleCI, GitLab, GitHub GitHub None CircleCI, GitHub CircleCI, GitHub
Output Formats (for generic CI/CD support) Text, JSON, JUnit, SARIF Text, JSON, JUnit, SARIF, GitLab-SAST Text, JSON, SARIF, HTML Text, JSON, SARIF, HTML Text, JSON, JUnit Text, JSON, JUnit, SARIF
Coverage for live environment Not in OSS, use paid product Yes, integrated into scans No No Not in OSS, use paid product Yes via differnet product

(there are others, anyone can add to this list, sorted A-Z)

For a list of IaC languages supported and the coverage provided by each tool for different CSPs, scroll down to the test case tables.

How does this repo work?

This repository has a set of test-cases and a main script, called run_all_tools.sh which runs the above-listed tools against each of the test-cases. This allows any potential user to see what the tool can do, and how it compares, before even installing it.

Test case catch rate

The tables below list test cases included in this repository. For each case, it shows which tools are able to catch it specifically, and which don't. Most test cases originate from the cloud service provider's (CSP's) own recommendations and best practices, as well as the CIS benchmark for that specific CSP.

Summary

Last update: 2021-08-27

Checkov Indeni Cloudrail Kics Snyk Terrascan Tfsec
Tested Version 2.0.363 1.3.385 1.4.1 1.683.0 1.9.0 0.58.4
Terraform - AWS 69% 93% 94% 62% 73% 61%
Terraform - Azure 47% 35% 23% 30% 8% 18%
Terraform - Advanced Language Expressions 20% 100% 20% 0% 0% 0%
Total Catch Rate 59% 72% 65% 48% 47% 43%
test-cases/terraform/aws/best-practices
Test Case Checkov Indeni Cloudrail Kics Snyk Terrascan Tfsec
alb_drop_http_headers
cloudfront_not_using_waf
cloudtrail_enabled_on_multi_region
config_aggregator_all_regions
deploy_ec2_to_default_vpc
deploy_redshift_in_ec2_classic_mode
dynamodb_without_recovery_enabled
ec2_ebs_not_optimized
ecr_make_tags_immutable
ecr_use_image_scanning
ecs_cluster_container_insights
elasticache_automatic_backup
kms_uses_rotation
rds_retention_period_set
security_group_no_description_for_rules
security_group_no_description_for_securi..
security_group_no_unused
tag_all_items
using_public_amis
Sub-category Catch Rate 84% 84% 89% 63% 63% 79%
test-cases/terraform/aws/encryption/at-rest
Test Case Checkov Indeni Cloudrail Kics Snyk Terrascan Tfsec
athena_not_encrypted
cloudtrail_not_encrypted
cloudwatch_groups_not_encrypted
codbuild_using_aws_key
dax_cluster_not_encrypted
docdb_cluster_encrypted_at_rest_using_cm..
docdb_cluster_encrypted_without_kms_key
docdb_clusters_non_encrypted
dynamodb_not_encrypted
ecr_repo_not_encrypted
elasticache_replication_group_not_encryp..
elasticsearch_not_encrypted
kinesis_stream_not_encrypted
neptune_cluster_no_encryption
rds_cluster_encrypt_at_rest_disabled
redshift_not_encrypted
rest_api_cache_non_encrypted
s3_bucket_non_encrypted
s3_bucket_object_non_encrypted
sagemaker_not_encrypted
secretsmanager_secrets_encrypted_at_rest..
secretsmanager_secrets_encrypted_at_rest..
sns_topic_encrypted_at_rest_with_aws_man..
sqs_queue_not_encrypted
workgroups_non_encrypted
workspace_root_volume_not_encrypted_at_r..
workspace_user_volume_not_encrypted_at_r..
Sub-category Catch Rate 74% 100% 100% 81% 78% 89%
test-cases/terraform/aws/encryption/in-transit
Test Case Checkov Indeni Cloudrail Kics Snyk Terrascan Tfsec
alb_use_http
cloudfront_distribution_not_encrypted
cloudfront_protocol_version_is_low
ecs_task_definition_not_encrypted_in_tra..
elasticache_replication_group_not_encryp..
elasticsearch_encrypt_node_to_node_disab..
load_balancer_listener_http
vpc_has_only_dynamodb_vpce_gw_connection
Sub-category Catch Rate 75% 100% 88% 75% 88% 88%
test-cases/terraform/aws/iam/iam-entities
Test Case Checkov Indeni Cloudrail Kics Snyk Terrascan Tfsec
human_users_defined
iam_user_inline_policy_attach
iam_user_managed_policy_direct_attachmen..
passrole_and_lambda_permissions_cause_pr..
policy-too-broad
policy_missing_principal
public_and_private_ec2_same_role
role_assume_policy_principal_all
Sub-category Catch Rate 50% 100% 88% 38% 50% 0%
test-cases/terraform/aws/iam/resource-authentication
Test Case Checkov Indeni Cloudrail Kics Snyk Terrascan Tfsec
rds_without_authentication
rest_api_without_authorization
Sub-category Catch Rate 100% 50% 100% 100% 50% 0%
test-cases/terraform/aws/iam/resource-policies
Test Case Checkov Indeni Cloudrail Kics Snyk Terrascan Tfsec
cloudwatch_log_destination_insecure_poli..
ecr_not_secure_policy
efs_not_secure_policy
elasticsearch_domain_not_secure_policy
glacier_vault_not_secure_policy
glue_data_catalog_not_secure_policy
kms_key_not_secure_policy
lambda_not_secure_policy
rest_api_not_secure_policy
s3_bucket_acl_public_all_authenticated_u..
s3_bucket_acl_public_all_users_canned
s3_bucket_acl_public_all_users_canned_wi..
s3_bucket_policy_public_to_all_authentic..
secrets_manager_not_secure_policy
Sub-category Catch Rate 21% 100% 93% 21% 71% 21%
test-cases/terraform/aws/logging
Test Case Checkov Indeni Cloudrail Kics Snyk Terrascan Tfsec
api_gateway_no_xray
cloudfront_distribution_without_logging
cloudtrail_file_log_validation_disabled
cloudwatch_log_groups_no_retention
docdb_audit_logs_missing
ec2_without_monitoring
eks_logging_disabled
elasticsearch_domain_logging_disabled
elb_without_access_logs
globalaccelerator_accelerator_no_flow_lo..
lambda_without_explicit_log_group
lambda_without_xray
neptune_cluster_no_logging
rds_without_logging
redshift_without_logging
rest_api_no_access_logging
s3_access_logging_disabled
Sub-category Catch Rate 94% 82% 94% 71% 94% 59%
test-cases/terraform/aws/networking/vpc-endpoints
Test Case Checkov Indeni Cloudrail Kics Snyk Terrascan Tfsec
dynamodb-vpce-exist-without-routeassocia..
sqs-vpc-endpoint-without-dns-resolution
Sub-category Catch Rate 0% 100% 100% 0% 0% 0%
test-cases/terraform/azure/best-practices
Test Case Checkov Indeni Cloudrail Kics Snyk Terrascan Tfsec
defender_for_app_services_disabled
defender_for_container_registry_not_used
defender_for_keyvault_disabled
defender_for_kubernetes_not_used
defender_for_servers_not_used
defender_for_sql_servers_not_used
defender_for_storage_not_used
email_notifications_for_high_severity_al..
func_app_not_using_http2
func_app_not_using_latest_tls
functionapp_lin_java_isnot_latest
functionapp_python_isnot_latest
functionapp_win_java_isnot_latest
sql_vulnerability_assessment_not_enabled
sql_vulnerability_email_not_set
vm_unmanaged_disks
vmss_unmanaged_disks
vpn_gw_using_basic_sku
webapp_http2_not_enabled
webapp_lin_java_isnot_latest
webapp_php_isnot_latest
webapp_win_java_isnot_latest
Sub-category Catch Rate 59% 41% 32% 41% 0% 32%
test-cases/terraform/azure/encryption/at-rest
Test Case Checkov Indeni Cloudrail Kics Snyk Terrascan Tfsec
activitylog_storage_account_encryption_n..
sql_encryption_customer_key_not_set
storacc_encryption_not_enabled
Sub-category Catch Rate 33% 0% 0% 0% 0% 0%
test-cases/terraform/azure/encryption/in-transit
Test Case Checkov Indeni Cloudrail Kics Snyk Terrascan Tfsec
app_service_ftps_unused
app_service_use_most_recent_supported_tl..
func_app_ftps_not_required
mysql_not_forcing_ssl
postgresql_not_forcing_ssl
Sub-category Catch Rate 60% 80% 40% 60% 40% 40%
test-cases/terraform/azure/iam
Test Case Checkov Indeni Cloudrail Kics Snyk Terrascan Tfsec
app_service_authentication_missing
custom-role-owner-exists
func_app_authentication
func_app_client_cert_optional
functionapp_not_use_managedidentity
sql-server-ad-admin-not-set
storage_account_public_access_disabled
webapp_client_cert_not_enabled
webapp_not_use_managedidentity
Sub-category Catch Rate 67% 33% 11% 22% 0% 0%
test-cases/terraform/azure/logging
Test Case Checkov Indeni Cloudrail Kics Snyk Terrascan Tfsec
auto_prov_log_analytics_agent_disabled
batch_diagnostic_disabled
dl_analytics_diagnostic_not_enabled
dl_store_diagnostic_not_enabled
event_hub_diagnostic_not_enabled
iot_hub_diagnostic_not_enabled
logic_app_wf_diagnostic_not_enabled
postgresql_log_connections_not_enabled
postgresql_log_disconnections_not_enable..
postgresql_logcheckpoints_not_enabled
search_diagnostic_not_enabled
servicebus_namespace_not_enabled
sql-server-audit-retention-30
sql_server_audit_not_used
stream_analytics_diagnostic_not_enabled
vmss_win_diagnostic_log_disabled
Sub-category Catch Rate 25% 19% 25% 25% 19% 6%
test-cases/terraform/azure/networking
Test Case Checkov Indeni Cloudrail Kics Snyk Terrascan Tfsec
no_unused_nsg
public_access_sql_db
vm_public_rdp_lb_opened
vm_public_rdp_nat_opened
vmss_public_rdp_lb_opened
Sub-category Catch Rate 20% 40% 0% 0% 0% 20%
test-cases/terraform/hcl_language_complexity
Test Case Checkov Indeni Cloudrail Kics Snyk Terrascan Tfsec
using_count_and_ternary_expr
using_for_each
using_locals
using_module_multi
using_module_simple
Sub-category Catch Rate 20% 100% 20% 0% 0% 0%

Contributing

Anyone can contribute to this repository. The main areas of contribution are:

  • Adding an additional tool - simply add the tool to this readme and the run_all_tools.sh script. Then, execute that script and add all of its results as part of your PR. That's it, you're good to go.

  • Adding test-cases - you can add the test case in the correct spot in the tree under test-cases and run the run_all_tools.sh script against it. Make sure to include all of the tools' results as part of your PR.

NOTE: This repository has been initiated by @yi2020, CEO & Founder of Indeni, the company behind Indeni Cloudrail. While this was initiated by an employee of a vendor in the community, the intention is for this repository to be neutral and truly serve as a non-biased comparison tool of products offered. Contributions that help users make that choice, and are unbiased in nature, are very welcome. The aspiration is that over time all vendors will become equal contributors in this repository.

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published