New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Third Party Cryptography Audit #8

Closed
paragonie-scott opened this Issue Dec 29, 2016 · 11 comments

Comments

Projects
4 participants
@paragonie-scott
Member

paragonie-scott commented Dec 29, 2016

After I'm confident that this library is...

  • Secure, even against sophisticated attackers
  • Correctly implemented
  • Adequately pedantic about types, etc.
  • Perfectly API compatible with the relevant libsodium features we're bringing over

...then the next step will be to obtain an independent third party security assessment.

@paragonie-scott

This comment has been minimized.

Show comment
Hide comment
@paragonie-scott

paragonie-scott Jan 17, 2017

Member

So far the plan is as follows:

  1. Get quotes/estimates from a few companies qualified to perform this sort of audit.
  2. Figure out a way to pay for it.
  3. ????
  4. VERIFIABLE SECURITY!

I'll update this thread later when I have even rough estimates on hand.

Member

paragonie-scott commented Jan 17, 2017

So far the plan is as follows:

  1. Get quotes/estimates from a few companies qualified to perform this sort of audit.
  2. Figure out a way to pay for it.
  3. ????
  4. VERIFIABLE SECURITY!

I'll update this thread later when I have even rough estimates on hand.

@paragonie-scott

This comment has been minimized.

Show comment
Hide comment
@paragonie-scott

paragonie-scott Jan 18, 2017

Member

After hearing back from four different companies on this, the total cost is going to be in the $30,000 to $50,000 range.

One quoted a bit lower, but they don't specialize in cryptography and might not know what they're getting into.

Two didn't provide a time estimate, but rather a day-rate (which is, I guess, pretty standard), so their contribution to the total cost estimate is based on the time estimate provided by the other two.

None of the numbers given are totally unexpected.

Member

paragonie-scott commented Jan 18, 2017

After hearing back from four different companies on this, the total cost is going to be in the $30,000 to $50,000 range.

One quoted a bit lower, but they don't specialize in cryptography and might not know what they're getting into.

Two didn't provide a time estimate, but rather a day-rate (which is, I guess, pretty standard), so their contribution to the total cost estimate is based on the time estimate provided by the other two.

None of the numbers given are totally unexpected.

@paragonie-scott

This comment has been minimized.

Show comment
Hide comment
@paragonie-scott

paragonie-scott Jan 19, 2017

Member

Proposed Auditing Scope

These are the sort of questions I'm hoping can be answered.

  • Does our PHP code contain any addressable bad practices?
  • Does the PHP interpreter introduce any exploitable side-channels (e.g. integer multiplication)?
    • PHP 5.6 on 32-bit Linux is safe?
    • PHP 5.6 on 64-bit Linux is safe?
    • PHP 7.0 on 32-bit Linux is safe?
    • PHP 7.0 on 64-bit Linux is safe?
    • PHP 7.1 on 32-bit Linux is safe?
    • PHP 7.1 on 64-bit Linux is safe?
    • PHP 5.6 on 32-bit Windows is safe?
    • PHP 5.6 on 64-bit Windows is safe?
    • PHP 7.0 on 32-bit Windows is safe?
    • PHP 7.0 on 64-bit Windows is safe?
    • PHP 7.1 on 32-bit Windows is safe?
    • PHP 7.1 on 64-bit Windows is safe?
    • Although PHP < 5.6 is worth considering overall since they're tentatively supported, the first run should focus on supported versions of PHP. Argument: If you're running PHP < 5.6 in 2017, you don't care about security, so we shouldn't burden the auditors to analyze these version too.
  • Are there any inputs for which our implementations produce incorrect results?
  • Are there any concerns about int/float conversion? (Timing leaks, etc.)
  • Does PHP's use of signed ints everywhere pose any security risk?
Member

paragonie-scott commented Jan 19, 2017

Proposed Auditing Scope

These are the sort of questions I'm hoping can be answered.

  • Does our PHP code contain any addressable bad practices?
  • Does the PHP interpreter introduce any exploitable side-channels (e.g. integer multiplication)?
    • PHP 5.6 on 32-bit Linux is safe?
    • PHP 5.6 on 64-bit Linux is safe?
    • PHP 7.0 on 32-bit Linux is safe?
    • PHP 7.0 on 64-bit Linux is safe?
    • PHP 7.1 on 32-bit Linux is safe?
    • PHP 7.1 on 64-bit Linux is safe?
    • PHP 5.6 on 32-bit Windows is safe?
    • PHP 5.6 on 64-bit Windows is safe?
    • PHP 7.0 on 32-bit Windows is safe?
    • PHP 7.0 on 64-bit Windows is safe?
    • PHP 7.1 on 32-bit Windows is safe?
    • PHP 7.1 on 64-bit Windows is safe?
    • Although PHP < 5.6 is worth considering overall since they're tentatively supported, the first run should focus on supported versions of PHP. Argument: If you're running PHP < 5.6 in 2017, you don't care about security, so we shouldn't burden the auditors to analyze these version too.
  • Are there any inputs for which our implementations produce incorrect results?
  • Are there any concerns about int/float conversion? (Timing leaks, etc.)
  • Does PHP's use of signed ints everywhere pose any security risk?
@AshleyPinner

This comment has been minimized.

Show comment
Hide comment
@AshleyPinner

AshleyPinner Jan 23, 2017

Argument: If you're running PHP < 5.6 in 2017, you don't care about security

Counter-argument: OS vendors are backporting security fixes to versions < 5.6, so they should, IMO, be given due care for checking this library in.

AshleyPinner commented Jan 23, 2017

Argument: If you're running PHP < 5.6 in 2017, you don't care about security

Counter-argument: OS vendors are backporting security fixes to versions < 5.6, so they should, IMO, be given due care for checking this library in.

@paragonie-scott

This comment has been minimized.

Show comment
Hide comment
@paragonie-scott

paragonie-scott Jan 23, 2017

Member

Counter-counter-argument: Those OS vendors aren't funding the audit.

If this statement becomes false, then the scope can be widened.

Member

paragonie-scott commented Jan 23, 2017

Counter-counter-argument: Those OS vendors aren't funding the audit.

If this statement becomes false, then the scope can be widened.

@AshleyPinner

This comment has been minimized.

Show comment
Hide comment
@AshleyPinner

AshleyPinner Jan 23, 2017

Counter-clockwise-argument: Reach out to Ubuntu/RHEL and see if they're willing to chip in?

AshleyPinner commented Jan 23, 2017

Counter-clockwise-argument: Reach out to Ubuntu/RHEL and see if they're willing to chip in?

@paragonie-scott

This comment has been minimized.

Show comment
Hide comment
@paragonie-scott

paragonie-scott Jan 23, 2017

Member

If you know any contacts at either organization, can you link them to this Github issue? I'd rather these discussions happen out in the open as much as possible.

Also, that will probably double the time needed, and therefore the cost of the audit.

Member

paragonie-scott commented Jan 23, 2017

If you know any contacts at either organization, can you link them to this Github issue? I'd rather these discussions happen out in the open as much as possible.

Also, that will probably double the time needed, and therefore the cost of the audit.

@defuse

This comment has been minimized.

Show comment
Hide comment
@defuse

defuse Jan 23, 2017

The scope looks good to me.

The only thing I would add is: "Are there kinds of automated tests or tools we should be running against the codebase to help uncover bugs in future versions?" They usually build some sort of custom tooling/tests to help with the audit, so I would see if it's possible to get some of the code they happen to write (even if it's shitty and needs to be fixed up), or at least get their opinion on what kinds of additional automated tests would be most effective.

defuse commented Jan 23, 2017

The scope looks good to me.

The only thing I would add is: "Are there kinds of automated tests or tools we should be running against the codebase to help uncover bugs in future versions?" They usually build some sort of custom tooling/tests to help with the audit, so I would see if it's possible to get some of the code they happen to write (even if it's shitty and needs to be fixed up), or at least get their opinion on what kinds of additional automated tests would be most effective.

@paragonie-scott

This comment has been minimized.

Show comment
Hide comment
@paragonie-scott

paragonie-scott Feb 19, 2017

Member

The purpose of sodium_compat was largely motivated by a desire to make WordPress more secure against its own infrastructure, which was in turn motivated by ethics. If you could prevent 27.5% of the Internet from getting erased or conscripted into a botnet just because one server got compromised, almost anyone would say, "I should do that."

However, given that WordPress will not be considering signing their updates any time soon, I don't see any point in asking the PHP community to reach into their own wallets to cover the cost of something that will currently not help the situation.

If someone else wants to pursue collecting the funds and hiring an auditor for this library: Feel free.

I'm still not tagging v1.0.0 unless this is audited by a team of crypto experts. If that never happens, then neither will v1.0.0.

Member

paragonie-scott commented Feb 19, 2017

The purpose of sodium_compat was largely motivated by a desire to make WordPress more secure against its own infrastructure, which was in turn motivated by ethics. If you could prevent 27.5% of the Internet from getting erased or conscripted into a botnet just because one server got compromised, almost anyone would say, "I should do that."

However, given that WordPress will not be considering signing their updates any time soon, I don't see any point in asking the PHP community to reach into their own wallets to cover the cost of something that will currently not help the situation.

If someone else wants to pursue collecting the funds and hiring an auditor for this library: Feel free.

I'm still not tagging v1.0.0 unless this is audited by a team of crypto experts. If that never happens, then neither will v1.0.0.

@mcordingley

This comment has been minimized.

Show comment
Hide comment
@mcordingley

mcordingley Apr 7, 2017

I've started a GoFundMe campaign to make this happen. I know that I and the projects that I work on could benefit from this more widely entering the PHP ecosystem, and I'm sure that's true of other developers and projects.

For those interested, (hey, this is a crypto thread) I've signed the GoFundMe story with this key: D821FCE9D9919CB77F4AF7922C3DBF3A600B432F It's available on the pgp.mit.edu keyserver. I'm not yet signed by anyone, but hope to be soon.

mcordingley commented Apr 7, 2017

I've started a GoFundMe campaign to make this happen. I know that I and the projects that I work on could benefit from this more widely entering the PHP ecosystem, and I'm sure that's true of other developers and projects.

For those interested, (hey, this is a crypto thread) I've signed the GoFundMe story with this key: D821FCE9D9919CB77F4AF7922C3DBF3A600B432F It's available on the pgp.mit.edu keyserver. I'm not yet signed by anyone, but hope to be soon.

@paragonie-scott

This comment has been minimized.

Show comment
Hide comment
@paragonie-scott

paragonie-scott Jun 9, 2017

Member

Due to various reasons (unsuccessful GoFundMe, the Joomla investment falling apart despite @mbabker's best effort at pushing through on their budget), I'm going to proceed with a non-audited version 1.0.0 release. I am, however, going to make sure our unit tests are testing the unhappy paths before we get there.

Member

paragonie-scott commented Jun 9, 2017

Due to various reasons (unsuccessful GoFundMe, the Joomla investment falling apart despite @mbabker's best effort at pushing through on their budget), I'm going to proceed with a non-audited version 1.0.0 release. I am, however, going to make sure our unit tests are testing the unhappy paths before we get there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment