EIP-1470: Smart Contract Weakness Classification (SWC)

[thec00n]

---

eip: 1470
title: Smart Contract Weakness Classification (SWC)
author: Gerhard Wagner (@thec00n)
discussions-to: #1469
status: Draft
type: Informational
created: 2018-09-18

Simple Summary

This EIP proposes a classification scheme for security weaknesses in Ethereum smart contracts.

Abstract

The SWC is a smart contract specific software weakness classification scheme for developers, tool vendors and security practitioners. The SWC is loosely aligned to the terminologies and structure used in the Common Weakness Enumeration - CWE scheme while overlaying a wide range of weakness variants that are specific to smart contracts.

The goals of the SWC scheme are as follows:

• Provide a straightforward way to classify weaknesses in smart contract systems.
• Provide a straightforward way to identify the weakness(es) that lead to a vulnerability in a smart contract system.
• Define a common language for describing weaknesses in smart contract systems' architecture, design and code.
• Train and increase the performance of smart contract security analysis tools.

Motivation

In the software security industry, it is a widely accepted practice to use a common terminology and to classify security related bugs and errors with a standardized scheme. While this has not stopped vulnerabilities from appearing in software, it has helped communities focusing on web applications, network protocols, IOT devices and various other fields to educate users and developers to understand the nature of security related issues in their software. It has also allowed the security community to quickly understand vulnerabilities that occur in production systems to perform root cause analysis or triage findings from various security analysis sources. In recent years various organizations and companies also published vulnerability data to find the most widespread security issues based on collected vulnerability data. Two examples that are widely used and referred to are the SANS TOP 25 Most Dangerous Software Errors and the OWASP TOP 10. None of those publications would have been possible without a common classification scheme.

At present no such weakness classification scheme exists for weaknesses specific to Ethereum Smart Contracts. Common language and awareness of security weaknesses is mostly derived from academic papers, best practice guides and published articles. Findings from audit reports and security tool analysis add to the wide range of terminologies that is used to describe the discovered weaknesses. It is often time consuming to understand the technical root cause and the risk associated to findings from different sources even for security experts.

Rationale

While recognizing the current gap, the SWC does not aim to reinvent the wheel in regards to classification of security weaknesses. It rather proposes to build on top of what has worked well in other parts of the software security community - specifically the Common Weakness Enumeration (CWE), a list of software vulnerability types that stands out in terms of adoption and breadth of coverage. While CWE does not describe any weaknesses specific to smart contracts, it does describe related weaknesses at higher abstraction layers. This EIP proposes to create smart contract specific variants while linking back to the larger spectrum of software errors and mistakes listed in the CWE that different platforms and technologies have in common.

Specification

Before discussing the SWC specification it is important to describe the terminology used:

• Weakness: A software error or mistake that in the right conditions can by itself or coupled with other weaknesses lead to a vulnerability.
• Vulnerability: A weakness or multiple weaknesses which directly or indirectly lead to an undesirable state in a smart contract system.
• Variant: A specific weakness that is described in a very low detail specific to Ethereum smart contracts. Each variant is assigned an unique SWC ID.
• Relationships: CWE has a wide range of Base and Class types that group weaknesses on higher abstraction layers. The CWE uses Relationships to link SWC smart contract weakness variants to existing Base or Class CWE types. Relationships are used to provide context on how SWCs are linked to the wider group of software security weaknesses and to be able to generate useful visualisations and insights through issue data sets. In its current revision it is proposed to link a SWC to its closest parent in the CWE.
• SWC ID: A numeric identifier linked to a variant (e.g. SWC-101).
• Test Case: A test case constitutes a micro-sample or real-world smart contract that demonstrates concrete instances of one or multiple SWC variants. Test cases serve as the basis for meaningful weakness classification and are useful to security analysis tool developers.

The SWC in its most basic form links a numeric identifier to a weakness variant. For example the identifier SWC-101 is linked to the Integer Overflow and Underflow variant. While a list with the weakness title and a unique id is useful by itself, it would also be ambiguous without further details. Therefore the SWC recommends to add a definition and test cases to any weakness variant.

SWC definition

A SWC definition is formated in markdown to allow good readability and tools to process them easily. It consists of the following attributes.

• Title: A name for the weakness that points to the technical root cause.
• Relationships: Links a CWE Base or Class type to its CWE variant. The Integer Overflow and Underflow variant for example is linked to CWE-682 - Incorrect Calculation.
• Description: Describes the nature and potential impact of the weakness on the contract system.
• Remediation: Describes ways on how to fix the weakness.
• References: Links to external references that contain relevant additional information on the weakness.

Test cases

Test cases include crafted as well as real-world samples of vulnerable smart contracts. A single test case consists of three components:

1. Source code of a smart contract sample; e.g. Solidity, Vyper, etc.
2. Compiled asset from an EVM compiler in machine readable format; e.g. JSON or ethPM.
3. Test result configuration that describes which and how many instances of a weakness variant can be found in a given sample. The YAML schema for the proposed test case configuration is listed below.

title: SWC config
type: object
required:
- description
- issues
properties:
  description:
    type: string
  issues:
    title: Issues
    type: array
    items:
      title: Issue
      type: object
      required:
      - id
      - count
      properties:
        id:
          type: string
        count:
          type: number
        locations:
          items:
            bytecode_offsets:
              type: object
            line_numbers:
              type: object

Implementation

The Smart Contract Weakness Classification registry located in this Github repository uses the SWC scheme proposed in this EIP. A Github Pages rendered version is also available here.

Copyright

Copyright and related rights waived via CC0.

[fubuloubu]

To reword the test case structure:

1. Compiled asset (JSON format) from an EVM compiler (solc, Vyper, etc.). Uses the solc output format, but eventually could use the ethPM packaging spec.
2. Source code (for reference only). Note: ethPM has an option to store source code and bytecode-sourcecode mappings.
3. Expected results (YAML format). Contains a test case description and a listing of bugs that are expectes to be found. If no list is provided, expected result is that no bug is found.

Do you agree with this rewording?

---

Also, the naming scheme should differentiate on compiler, so different languages can be tried.

[thec00n]

Thanks for your feedback @fubuloubu! I have reworded the paragraph based on your suggestions. I also realised that this should be worded more generically as we need to consider other languages and file formats in the future.

Test cases include crafted as well as real-world samples of vulnerable smart contracts. A single test case consists of three components:

1. Source code of a smart contract sample; e.g. Solidity, Vyper, etc.
2. Compiled asset from an EVM compiler in machine readable format; e.g. JSON or ethPM.
3. Test result configuration that describes which and how many instances of a weakness variant can be found in a given sample. The YAML schema for the proposed test result configuration is listed below.

[fubuloubu]

Let me also note that I think the primary result this EIP should seek to codify is in defining the "test result configuration" file and any supplemental files. ethPM packages are a great format for compiled assets; source code and execution paths may also make sense too.

Perhaps the configuration file should reference the relative location of these supplemental files?

The specification of a common weakness classification may be out of scope for an EIP. Indeed, there may be multiple such classifications covering different topics. We should seek to make them highly interoperable so as to enable the consensus and colloboration of different benchmarks and classifications.

[thec00n]

@fubuloubu It has become quickly apparent to me that with a growing set of test cases (or any security issue data set) it is very difficult to differentiate between weakness variants without a classification scheme as you create overlap and you get lost in the ambiguity of what different people call something. Also it’s helpful to discuss weakness classification based on micro-samples and real-world smart contracts where instances of a weakness occur together with description, mitigation strategies, references and how the weakness relates to the bigger picture of software security weaknesses -> CWE. So I think the test cases and the classification scheme go hand in hand and doing one them in isolation would be less meaningful.

[thec00n]

Update the EIP to reflect the changes proposed in SmartContractSecurity/SWC-registry#168.

[guylando]

Interesting idea, few comments:

1. Why to create a new standard for this instead of adding smart contracts weaknesses as new CWEs?
2. I think that several important properties which are missing for SWC are: how severe the weakness impact can be, how easy to fix is it, how easy to exploit is it.
3. I think for it to be useful, it should be very clear about where is the weakness in the test case and the SWC should be organized in a very strict and well defined manner. For example here: https://smartcontractsecurity.github.io/SWC-registry/docs/SWC-130 it is not clear where is the weakness in the test case contract other than pointing in the source to a 41 minutes video which has 1-2 minutes relevant for the actual weakness. Here is another example where it is left for the reader to discover where is the weakness in the test cases: https://smartcontractsecurity.github.io/SWC-registry/docs/SWC-105 and also this weakness seems to be a combination of other weaknesses so I am not sure it deserves a SWC of its own. Also this one: https://smartcontractsecurity.github.io/SWC-registry/docs/SWC-106 is also a consequence of other weaknesses and I dont think it should be a new SWC. If it has a new SWC for destroying a contract then it should have been named in a more general way since selfdestruct can be triggered also by a delegatecall and other ways. Also seems SWC 100 and SWC 108 should be united, even the link inside SWC 108 talks about the SWC 100 weakness too and since state variables auto-generate a getter by the same name and when called externally are basically functions (also in SWC 108 contract again it is not clearly stated where is the weakness, line number for example or some comment in the code). The list goes on and many of the test cases are contracts from CTFs which make it not so trivial to identify the weakness in them which is probably not the goal in the SWC.

[thec00n]

hey @guylando.

1.) If you mean why we do not add the weaknesses to https://cwe.mitre.org/, then the answer is because there is no edit or create a PR button. We wanted to have something that can receive contributions from anyone in the community and that does not require anyone to go through some kind of onboarding process to get started. That does not mean of course that someone could not take the content in the SWC-registry and put it on https://cwe.mitre.org/. It's intentionally compatible with CWE and under a permissive MIT license.

2.) There is a section for mitigation strategies Remediation. As for the risk related properties like severity and likelihood they should generally assigned when you find an instance of a particular weakness. You should use something like CVSS scoring to determine an objective risk value. Having said that there were discussions recently around having a "risk range" field that would indicate the CVSS categories that a particular weakness typically falls under.

3.)

I think for it to be useful, it should be very clear about where is the weakness in the test case and the SWC should be organized in a very strict and well defined manner. For example here: https://smartcontractsecurity.github.io/SWC-registry/docs/SWC-130 it is not clear where is the weakness in the test case contract other than pointing in the source to a 41 minutes video which has 1-2 minutes relevant for the actual weakness. Here is another example where it is left for the reader to discover where is the weakness in the test cases:

If you try guess_the_number.sol I think it becomes quickly clear what the issue is about.

https://smartcontractsecurity.github.io/SWC-registry/docs/SWC-105 and also this weakness seems to be a combination of other weaknesses so I am not sure it deserves a SWC of its own.

Generally I would agree that there should not be overlap between categories and this a strong consideration when merging new definitions. The reason why SWC-105 and SWC-106 have their own categories is because they represent very critical general negative properties that are always bad if you discover them in smart contracts. Therefore it seemed appropriate to assign them their own category.

I appreciate your feedback and I hope I could clarify your questions. Feel free to open up issues/PRs on the SWC-registry repo to discuss questions and improvements in more detail.

[guylando]

1,2,3. https://cwe.mitre.org/ has more fields such as those I wrote, see for example: https://cwe.mitre.org/data/definitions/682.html
a. "Likelihood Of Exploit"
b. "Common Consequences" + impacts.
regarding the examples:
c. "Demonstrative Examples" see the level of description, I believe it shouldn't be expected of everybody to waste their time trying "guess_the_number.sol" and instead providing better explanations for test cases will save a lot of people a lot of time.

If this is to be useful to everybody then the weaknesses should present not only the problem but also help prioritize dealing with it. Otherwise there will be hundreds of weaknesses without any information on how to prioritize and then you can't expect developers to go every time over EVERYTHING.

Those are suggestions in improving the standard proposed in the first comment here to make it more useful and the specific examples are to demonstrate the reasoning.

Would also suggest to add another field to the template which specifies the component affected to make it easier for developers to know which weaknesses are related to contract exploits and which are related to contract code presentation and which are related to dapps using the smart contract. For example the following weakness would not be considered an in-chain weakness which would make it easier to know when to deal with it (during contract code display/verification and not during contract development): https://smartcontractsecurity.github.io/SWC-registry/docs/SWC-130

[github-actions]

There has been no activity on this issue for two months. It will be closed in a week if no further activity occurs. If you would like to move this EIP forward, please respond to any outstanding feedback or add a comment indicating that you have addressed all required feedback and are ready for a review.

[thec00n]

This is an old EIP and not sure if people are using it but from what I understand it received the feedback it required. I think it can be merged.

[MicahZoltu]

This isn't an EIP, it is just an idea for an EIP. To be an EIP, someone needs to submit a pull request to this repository that adds a new file. Please see EIP-1 for details on this process. I'm going to close this for now, as ideas for EIPs should live at Ethereum Magicians, and this is incredibly old. If someone wants to turn this into an actual EIP I recommend opening a PR so it can start moving through the process to becoming an EIP.