New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Request: Add project to Coverity Scan or similar #1110

Open
JoeUX opened this Issue Oct 27, 2016 · 18 comments

Comments

Projects
None yet
4 participants
@JoeUX

JoeUX commented Oct 27, 2016

I like H2O a lot, but C makes me nervous given how easy it is to have exploitable memory bugs. Would you consider putting H2O on Coverity Scan so that at least some of those kinds of bugs can be detected more easily? It's free for open-source projects.

It would also be helpful to have a documented strategy for avoiding typical C security holes (buffer and memory bugs, mostly). Neverbleed is brilliant, and lower level code analysis tools like Coverity or the different sanitizers in clang/llvm would be a good combination with it.

@hbowden

This comment has been minimized.

Show comment
Hide comment
@hbowden

hbowden Jan 13, 2017

Collaborator

I fully agree with using Coverity, unfortunately only people with write access to this repo can setup Coverity. @kazuho Is there any chance you could setup Coverity? I could start going through bugs found and start fixing them but you or someone else with write access would have to do the original setup. If you need help using Coverity I can help.

Collaborator

hbowden commented Jan 13, 2017

I fully agree with using Coverity, unfortunately only people with write access to this repo can setup Coverity. @kazuho Is there any chance you could setup Coverity? I could start going through bugs found and start fixing them but you or someone else with write access would have to do the original setup. If you need help using Coverity I can help.

@kazuho

This comment has been minimized.

Show comment
Hide comment
@kazuho

kazuho Jan 16, 2017

Member

@hbowden If I gave you write permission, would you be willing to work on automating Coverity Scan of H2O using Travis CI? Reading https://scan.coverity.com/travis_ci my understanding is that it is possible. Thank you in advance.

PS. the reason I am referring to Travis integration is because we will need a repeatable build process for scanning.

Member

kazuho commented Jan 16, 2017

@hbowden If I gave you write permission, would you be willing to work on automating Coverity Scan of H2O using Travis CI? Reading https://scan.coverity.com/travis_ci my understanding is that it is possible. Thank you in advance.

PS. the reason I am referring to Travis integration is because we will need a repeatable build process for scanning.

@hbowden

This comment has been minimized.

Show comment
Hide comment
@hbowden

hbowden Jan 16, 2017

Collaborator

@kazuho Yea I could setup Travis CI to use Coverity. However Coverity only allows a limited amount of scans per week. Below is a snippet from their site showing the limits.

Up to 12 builds per week, with a maximum of 3 builds per day, for projects with fewer than 100K lines of code
Up to   8 builds per week, with a maximum of 2 builds per day, for projects with 100K to 500K lines of code
Up to   4 builds per week, with a maximum of 1 build per day, for projects with 500K to 1 million lines of code
Up to   2 builds per week, with a maximum of 1 build per day, for projects with more than 1 million lines of code

I could set up a separate branch and we could merge into it whenever we want to scan h2o with Coverity. This is actually what the Coverity docs suggest. Does that work for you?

Collaborator

hbowden commented Jan 16, 2017

@kazuho Yea I could setup Travis CI to use Coverity. However Coverity only allows a limited amount of scans per week. Below is a snippet from their site showing the limits.

Up to 12 builds per week, with a maximum of 3 builds per day, for projects with fewer than 100K lines of code
Up to   8 builds per week, with a maximum of 2 builds per day, for projects with 100K to 500K lines of code
Up to   4 builds per week, with a maximum of 1 build per day, for projects with 500K to 1 million lines of code
Up to   2 builds per week, with a maximum of 1 build per day, for projects with more than 1 million lines of code

I could set up a separate branch and we could merge into it whenever we want to scan h2o with Coverity. This is actually what the Coverity docs suggest. Does that work for you?

@kazuho

This comment has been minimized.

Show comment
Hide comment
@kazuho

kazuho Jan 17, 2017

Member

@hbowden

I could set up a separate branch and we could merge into it whenever we want to scan h2o with Coverity. This is actually what the Coverity docs suggest. Does that work for you?

Sounds great! Thank you very much. I've sent you an invitation that adds write privilege to the repo. Please feel free to create a dedicated branch for coverity scanning and make changes there.

Member

kazuho commented Jan 17, 2017

@hbowden

I could set up a separate branch and we could merge into it whenever we want to scan h2o with Coverity. This is actually what the Coverity docs suggest. Does that work for you?

Sounds great! Thank you very much. I've sent you an invitation that adds write privilege to the repo. Please feel free to create a dedicated branch for coverity scanning and make changes there.

@hbowden

This comment has been minimized.

Show comment
Hide comment
@hbowden

hbowden Jan 17, 2017

Collaborator

@kazuho

Thanks, I will add a pull request when I have it working.

Collaborator

hbowden commented Jan 17, 2017

@kazuho

Thanks, I will add a pull request when I have it working.

@hbowden

This comment has been minimized.

Show comment
Hide comment
@hbowden

hbowden Jan 17, 2017

Collaborator

@kazuho

So I got Coverity integrated with Travis CI on this branch: https://github.com/hbowden/h2o/tree/coverity_scan

I wasn't able to add the main h2o repo to travis ci because it says I don't have admin rights. You could just use the config below and sign in to Coverity using your github account and add h2o as a project. Make sure to set the issues as private, for some reason it defaults to public. Click submit build and then go to the Configure Travis CI tab. Half way down the page will be an example config, grab the key on the -secure line and replace that same line on the config I posted below. Then you should have Coverity integrated with Travis CI.

language: cpp

env:
  global:
   # The next declaration is the encrypted COVERITY_SCAN_TOKEN, created
   #   via the "travis encrypt" command using the project repo's public key
   - secure: "DW71w2slPUijF0OvlHvR3I6Hr1NoBTvGeQdaUTMfYLvNxZDCs9TlHv0ovcdiJnKLUPTBAqM9KHpPLdA36bflWAEZx1ZjFrtjFXnn1V1uQ9v+pP0w8DbbYjMBgncGKFDGJKmTSWXvtYaTz+OYWw2dQSu6dmPLUJx9zs+CYraelBAc7zCl9UidmGw4Cbd0RDY940+Dbkppp/pLKIOPKiXX4gxl34t1pwhBaLHs4smR+QoXzmqESvDSt30wQIugmUxg9GOKaWP75rbUD/PzawHIKKRI12fmLqfbR1f8di0EFeeofHLKfZJMqJ40UFul6yc0IECjBzRvaRcQPRVMeO6ioDURfNK86g/l3Aldtve/w53NolpruaMRok5MOafOd7kuskVS9OyUfBEp9BJxRXWBGcdPswhn/7epe2kHsVRBRjTp2ez7F+J8VRGYzVUkEGjMJJ02ILAkYk49bgcJz053Zrn0ejQD0eDTrIEF1p1JwP4GNtWyHlvSAynnJO6hbPPXM2H9DHqP9Bl8JFUeZiVumKhmqDW5mzfjNadUiJz4F8FMqgyajOj/4lAZkBm+5BvB3YDuxI/tV8q7vHBtsQaUL7uy+fBDKlV6lUtCc4qUoiU6XtLI/PMS7pfD+rG1JUqklZOZ4dQ5t405y/E4zMIHzuv+wceMId+P/Jyzqzc1RjA="

compiler:
  - gcc

addons:
  hosts:
    - 127.0.0.1.xip.io
    - alternate.127.0.0.1.xip.io
  coverity_scan:
    project:
      name: "h2o/h2o"
      description: "H2O - the optimized HTTP/1, HTTP/2 server https://h2o.examp1e.net"
    notification_email: kazuho@natadeco.co
    build_command_prepend: "mkdir build && cd build && cmake .."
    build_command: "cd build && make"
    branch_pattern: coverity_scan

before_install:
  # upgrade g++ and libstdc++ to build nghttp2
  - echo -n | openssl s_client -connect scan.coverity.com:443 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' | sudo tee -a /etc/ssl/certs/ca-
  - sudo add-apt-repository --yes ppa:ubuntu-toolchain-r/test
  - sudo apt-add-repository --yes ppa:smspillaz/cmake-2.8.12
  - sudo apt-get --yes update
  - sudo apt-get install --yes cmake cmake-data g++-4.8 libstdc++-4.8-dev php5-cgi wget
  - if [ "$CXX" = "g++" ]; then export CXX="g++-4.8"; fi
  - $CXX --version
  # for speed, pre-install deps installed in `before_script` section as ubuntu packages
  - sudo apt-get install -qq cpanminus libipc-signal-perl liblist-moreutils-perl libwww-perl libio-socket-ssl-perl zlib1g-dev

before_script:
  # install libuv >= 1.0.0 (optionally required for building / testing libh2o)
  - curl -L https://github.com/libuv/libuv/archive/v1.0.0.tar.gz | tar xzf -
  - (cd libuv-1.0.0 && ./autogen.sh && ./configure --prefix=/usr && make && sudo make install)
  # install perl modules (mandatory for testing)
  - misc/install-perl-module.pl Net::EmptyPort
  - misc/install-perl-module.pl Scope::Guard
  - misc/install-perl-module.pl Plack
  - misc/install-perl-module.pl FCGI
  - misc/install-perl-module.pl FCGI::ProcManager
  - misc/install-perl-module.pl Starlet
  - misc/install-perl-module.pl JSON
  - misc/install-perl-module.pl Path::Tiny
  - misc/install-perl-module.pl Test::Exception
  # install the `ab` command (a.k.a. ApacheBench; optionally required for running some of the tests)
  - sudo apt-get install -qq apache2-utils
  # install nghttp2 with `--enable-app` (optionally required for running HTTP/2 tests)
  - sudo apt-get install -qq libev-dev
  - curl -L https://github.com/tatsuhiro-t/nghttp2/releases/download/v1.4.0/nghttp2-1.4.0.tar.gz | tar xzf -
  - (cd nghttp2-1.4.0 && ./configure --prefix=/usr --disable-threads --enable-app && make && sudo make install)
  - curl -L https://curl.haxx.se/download/curl-7.50.0.tar.gz | tar xzf -
  - (cd curl-7.50.0 && ./configure --prefix=/usr --with-nghttp2 --disable-shared && make && sudo make install)

script:
  - cmake -DWITH_MRUBY=ON .
  - make all
  - make check
  - sudo make check-as-root
Collaborator

hbowden commented Jan 17, 2017

@kazuho

So I got Coverity integrated with Travis CI on this branch: https://github.com/hbowden/h2o/tree/coverity_scan

I wasn't able to add the main h2o repo to travis ci because it says I don't have admin rights. You could just use the config below and sign in to Coverity using your github account and add h2o as a project. Make sure to set the issues as private, for some reason it defaults to public. Click submit build and then go to the Configure Travis CI tab. Half way down the page will be an example config, grab the key on the -secure line and replace that same line on the config I posted below. Then you should have Coverity integrated with Travis CI.

language: cpp

env:
  global:
   # The next declaration is the encrypted COVERITY_SCAN_TOKEN, created
   #   via the "travis encrypt" command using the project repo's public key
   - secure: "DW71w2slPUijF0OvlHvR3I6Hr1NoBTvGeQdaUTMfYLvNxZDCs9TlHv0ovcdiJnKLUPTBAqM9KHpPLdA36bflWAEZx1ZjFrtjFXnn1V1uQ9v+pP0w8DbbYjMBgncGKFDGJKmTSWXvtYaTz+OYWw2dQSu6dmPLUJx9zs+CYraelBAc7zCl9UidmGw4Cbd0RDY940+Dbkppp/pLKIOPKiXX4gxl34t1pwhBaLHs4smR+QoXzmqESvDSt30wQIugmUxg9GOKaWP75rbUD/PzawHIKKRI12fmLqfbR1f8di0EFeeofHLKfZJMqJ40UFul6yc0IECjBzRvaRcQPRVMeO6ioDURfNK86g/l3Aldtve/w53NolpruaMRok5MOafOd7kuskVS9OyUfBEp9BJxRXWBGcdPswhn/7epe2kHsVRBRjTp2ez7F+J8VRGYzVUkEGjMJJ02ILAkYk49bgcJz053Zrn0ejQD0eDTrIEF1p1JwP4GNtWyHlvSAynnJO6hbPPXM2H9DHqP9Bl8JFUeZiVumKhmqDW5mzfjNadUiJz4F8FMqgyajOj/4lAZkBm+5BvB3YDuxI/tV8q7vHBtsQaUL7uy+fBDKlV6lUtCc4qUoiU6XtLI/PMS7pfD+rG1JUqklZOZ4dQ5t405y/E4zMIHzuv+wceMId+P/Jyzqzc1RjA="

compiler:
  - gcc

addons:
  hosts:
    - 127.0.0.1.xip.io
    - alternate.127.0.0.1.xip.io
  coverity_scan:
    project:
      name: "h2o/h2o"
      description: "H2O - the optimized HTTP/1, HTTP/2 server https://h2o.examp1e.net"
    notification_email: kazuho@natadeco.co
    build_command_prepend: "mkdir build && cd build && cmake .."
    build_command: "cd build && make"
    branch_pattern: coverity_scan

before_install:
  # upgrade g++ and libstdc++ to build nghttp2
  - echo -n | openssl s_client -connect scan.coverity.com:443 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' | sudo tee -a /etc/ssl/certs/ca-
  - sudo add-apt-repository --yes ppa:ubuntu-toolchain-r/test
  - sudo apt-add-repository --yes ppa:smspillaz/cmake-2.8.12
  - sudo apt-get --yes update
  - sudo apt-get install --yes cmake cmake-data g++-4.8 libstdc++-4.8-dev php5-cgi wget
  - if [ "$CXX" = "g++" ]; then export CXX="g++-4.8"; fi
  - $CXX --version
  # for speed, pre-install deps installed in `before_script` section as ubuntu packages
  - sudo apt-get install -qq cpanminus libipc-signal-perl liblist-moreutils-perl libwww-perl libio-socket-ssl-perl zlib1g-dev

before_script:
  # install libuv >= 1.0.0 (optionally required for building / testing libh2o)
  - curl -L https://github.com/libuv/libuv/archive/v1.0.0.tar.gz | tar xzf -
  - (cd libuv-1.0.0 && ./autogen.sh && ./configure --prefix=/usr && make && sudo make install)
  # install perl modules (mandatory for testing)
  - misc/install-perl-module.pl Net::EmptyPort
  - misc/install-perl-module.pl Scope::Guard
  - misc/install-perl-module.pl Plack
  - misc/install-perl-module.pl FCGI
  - misc/install-perl-module.pl FCGI::ProcManager
  - misc/install-perl-module.pl Starlet
  - misc/install-perl-module.pl JSON
  - misc/install-perl-module.pl Path::Tiny
  - misc/install-perl-module.pl Test::Exception
  # install the `ab` command (a.k.a. ApacheBench; optionally required for running some of the tests)
  - sudo apt-get install -qq apache2-utils
  # install nghttp2 with `--enable-app` (optionally required for running HTTP/2 tests)
  - sudo apt-get install -qq libev-dev
  - curl -L https://github.com/tatsuhiro-t/nghttp2/releases/download/v1.4.0/nghttp2-1.4.0.tar.gz | tar xzf -
  - (cd nghttp2-1.4.0 && ./configure --prefix=/usr --disable-threads --enable-app && make && sudo make install)
  - curl -L https://curl.haxx.se/download/curl-7.50.0.tar.gz | tar xzf -
  - (cd curl-7.50.0 && ./configure --prefix=/usr --with-nghttp2 --disable-shared && make && sudo make install)

script:
  - cmake -DWITH_MRUBY=ON .
  - make all
  - make check
  - sudo make check-as-root
@deweerdt

This comment has been minimized.

Show comment
Hide comment
@deweerdt

deweerdt Jan 18, 2017

Member

@hbowden could you add me to the project? I would like to help with the bug fixing, if possible. Thanks!

Member

deweerdt commented Jan 18, 2017

@hbowden could you add me to the project? I would like to help with the bug fixing, if possible. Thanks!

@hbowden

This comment has been minimized.

Show comment
Hide comment
@hbowden

hbowden Jan 18, 2017

Collaborator

@deweerdt I can send you an invitation to my repo so you can get access to the coverity results. A lot of the bugs appear to be in h2o dependencies.

Collaborator

hbowden commented Jan 18, 2017

@deweerdt I can send you an invitation to my repo so you can get access to the coverity results. A lot of the bugs appear to be in h2o dependencies.

@hbowden

This comment has been minimized.

Show comment
Hide comment
@hbowden

hbowden Jan 18, 2017

Collaborator

@deweerdt You should be able to see the results now, let me know if it doesn't work for you.

Collaborator

hbowden commented Jan 18, 2017

@deweerdt You should be able to see the results now, let me know if it doesn't work for you.

@deweerdt

This comment has been minimized.

Show comment
Hide comment
@deweerdt

deweerdt Jan 18, 2017

Member

@hbowden it works, thank you

Member

deweerdt commented Jan 18, 2017

@hbowden it works, thank you

@kazuho

This comment has been minimized.

Show comment
Hide comment
@kazuho

kazuho Jan 18, 2017

Member

@hbowden Thank you very much for all your help!

I've created a branch named coverity_scan on the master repository with your changes merged with the refs and secrets pointing out to the appropriate ones. So hopefully Coverity will start scanning the files once the build finishes.

I've also set project access rights to requires Maintainer approval, thank you for pointing that out.

I also sent an invitation that adds you as the owner of h2o/h2o project on scan.coverity.com; the fact that the project already existed (without giving you the appropriate rights there) might have been the cause of your being unable to access the settings.

Member

kazuho commented Jan 18, 2017

@hbowden Thank you very much for all your help!

I've created a branch named coverity_scan on the master repository with your changes merged with the refs and secrets pointing out to the appropriate ones. So hopefully Coverity will start scanning the files once the build finishes.

I've also set project access rights to requires Maintainer approval, thank you for pointing that out.

I also sent an invitation that adds you as the owner of h2o/h2o project on scan.coverity.com; the fact that the project already existed (without giving you the appropriate rights there) might have been the cause of your being unable to access the settings.

@hbowden

This comment has been minimized.

Show comment
Hide comment
@hbowden

hbowden Jan 18, 2017

Collaborator

@kazuho So it looks like Coverity got the build correctly, but it looks like the Travis CI script failed. So at least the Coverity integration is working but I'm not sure why the script failed. For some reason Coverity doesn't give you access to bugs for a day or two, if you want access before then I can give you access to them here: https://scan.coverity.com/projects/hbowden-h2o?tab=overview

Collaborator

hbowden commented Jan 18, 2017

@kazuho So it looks like Coverity got the build correctly, but it looks like the Travis CI script failed. So at least the Coverity integration is working but I'm not sure why the script failed. For some reason Coverity doesn't give you access to bugs for a day or two, if you want access before then I can give you access to them here: https://scan.coverity.com/projects/hbowden-h2o?tab=overview

@kazuho

This comment has been minimized.

Show comment
Hide comment
@kazuho

kazuho Jan 18, 2017

Member

@hbowden Thank you for looking into the issue.

I believe that is because I removed the script section of .travis.yml in 75b9e5d. I needed to do this since we were seeing a command not found error for build: "cd build && make" (please refer to https://s3.amazonaws.com/archive.travis-ci.org/jobs/192905916/log.txt).

In c62f81f, I have added a echo command into the script section to prevent the default build action from kicking in, so hopefully the travis error will go away.

Member

kazuho commented Jan 18, 2017

@hbowden Thank you for looking into the issue.

I believe that is because I removed the script section of .travis.yml in 75b9e5d. I needed to do this since we were seeing a command not found error for build: "cd build && make" (please refer to https://s3.amazonaws.com/archive.travis-ci.org/jobs/192905916/log.txt).

In c62f81f, I have added a echo command into the script section to prevent the default build action from kicking in, so hopefully the travis error will go away.

@deweerdt

This comment has been minimized.

Show comment
Hide comment
@deweerdt

deweerdt Jan 18, 2017

Member

I've done a first pass on the 'high' defects in h2o's code, and classified them. I believe that besides #1168 they're all false positives or intentional. @hbowden, do you know if there's a way to exclude the external dependencies from the coverity scan?

Member

deweerdt commented Jan 18, 2017

I've done a first pass on the 'high' defects in h2o's code, and classified them. I believe that besides #1168 they're all false positives or intentional. @hbowden, do you know if there's a way to exclude the external dependencies from the coverity scan?

@hbowden

This comment has been minimized.

Show comment
Hide comment
@hbowden

hbowden Jan 18, 2017

Collaborator

@deweerdt Yea I will have Coverity exclude bugs in the dependencies. Some of the 'medium' bugs are still kinda nasty and I found a few that weren't false positives. As for the false positives we can report them to Coverity and they can fix Coverity so it dosen't issue them anymore. I also think we should fix the bugs found in the dependencies after we fix all the bugs found in h2o it's self, they are still bugs that get compiled into h2o. I will exclude the dependencies bugs for now and lets re include them when we have the other bugs cleaned up.

Collaborator

hbowden commented Jan 18, 2017

@deweerdt Yea I will have Coverity exclude bugs in the dependencies. Some of the 'medium' bugs are still kinda nasty and I found a few that weren't false positives. As for the false positives we can report them to Coverity and they can fix Coverity so it dosen't issue them anymore. I also think we should fix the bugs found in the dependencies after we fix all the bugs found in h2o it's self, they are still bugs that get compiled into h2o. I will exclude the dependencies bugs for now and lets re include them when we have the other bugs cleaned up.

@deweerdt

This comment has been minimized.

Show comment
Hide comment
@deweerdt

deweerdt Jan 18, 2017

Member

Some of the 'medium' bugs are still kinda nasty and I found a few that weren't false positives.

Good to know, I'll do a pass on those today.

I will exclude the dependencies bugs for now and lets re include them when we have the other bugs cleaned up.

👍

Member

deweerdt commented Jan 18, 2017

Some of the 'medium' bugs are still kinda nasty and I found a few that weren't false positives.

Good to know, I'll do a pass on those today.

I will exclude the dependencies bugs for now and lets re include them when we have the other bugs cleaned up.

👍

@deweerdt

This comment has been minimized.

Show comment
Hide comment
@deweerdt

deweerdt Jan 18, 2017

Member

I've finished a pass on the remainder 'Medium' and 'Low' bugs in h2o's code. I've submitted #1172, this seemed to be the only semi-urgent fix. AFAICT all other bugs are relatively minor, I'll take a stab at them as time permits.

Member

deweerdt commented Jan 18, 2017

I've finished a pass on the remainder 'Medium' and 'Low' bugs in h2o's code. I've submitted #1172, this seemed to be the only semi-urgent fix. AFAICT all other bugs are relatively minor, I'll take a stab at them as time permits.

@kazuho

This comment has been minimized.

Show comment
Hide comment
@kazuho

kazuho Jan 20, 2017

Member

@hbowden @deweerdt Thank you for your efforts. https://scan.coverity.com/projects/h2o-h2o is now open and you are both given admin rights there.

And thank you to @deweerdt for triaging some of the issues reported there.

Regarding how to exclude the dependencies, we might consider installing openssl 1.0.2 separately (so that the bundled libressl would not get built under Coverity), and also disable mruby for the coverity_scan branch.

Please let me know if you find any issue that should be considered as an vulnerability. I haven't seen such one yet, but I might be missing something.

Member

kazuho commented Jan 20, 2017

@hbowden @deweerdt Thank you for your efforts. https://scan.coverity.com/projects/h2o-h2o is now open and you are both given admin rights there.

And thank you to @deweerdt for triaging some of the issues reported there.

Regarding how to exclude the dependencies, we might consider installing openssl 1.0.2 separately (so that the bundled libressl would not get built under Coverity), and also disable mruby for the coverity_scan branch.

Please let me know if you find any issue that should be considered as an vulnerability. I haven't seen such one yet, but I might be missing something.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment