Need to enhance the OpenSSL::SSL::Context loads Certificate/PrivateKey from Memory #7897

ghost · 2019-06-18T21:52:28Z

Summary

I saw the Crystal API documentation, it can only be loaded from the file path.

OpenSSL::SSL::Context
- def ca_certificates=(file_path : String)
- def private_key=(file_path : String)

In Golang it can be loaded using X509KeyPair? (probably?),
But I don't seem to see similar features in Crystal.

In Golang

func (srv *embeddedServer) ListenAndServeTLS(addr string) error {

   config := &tls.Config{
      MinVersion: tls.VersionTLS10,
   }
   if srv.TLSConfig != nil {
      *config = *srv.TLSConfig
   }
   if config.NextProtos == nil {
      config.NextProtos = []string{"http/1.1"}
   }

   var err error
   config.Certificates = make([]tls.Certificate, 1)
   config.Certificates[0], err = tls.X509KeyPair([]byte(srv.webserverCertificate), []byte(srv.webserverKey))
   if err != nil {
      return err
   }

   conn, err := net.Listen("tcp", addr)
   if err != nil {
      return err
   }

   tlsListener := tls.NewListener(conn, config)
   return srv.Serve(tlsListener)
}

References

Golang - Modifying net/http server.go, using decrypted TLS certificates from memory
Golang - Practical embedding in GoLang
Golang - TLS server with in-memory self-signed certificate

The text was updated successfully, but these errors were encountered:

ghost · 2019-06-20T17:08:37Z

Please, if possible, please consider giving priority to this feature,
I need to use this feature, I need to use it to make MITM proxy server,
otherwise, I can only try to use Rust to rewrite this server... :(

ysbaddaden · 2019-06-20T17:43:46Z

A pull request will likely be welcomed :)

ghost · 2019-06-20T18:04:57Z

@ysbaddaden Okay.

ghost · 2019-06-23T19:35:10Z

Update

@ysbaddaden Hi, I have implemented this feature, But I think some methods need to be renamed.
Because in lib_crypto.cr, it also use the same format: (e.g. x509_free, evp_pkey_free).

i.e.
- fun BIO_new -> fun bio_new = BIO_new
- fun BIO_write -> fun bio_write = BIO_write
- fun BIO_free -> fun bio_free = BIO_free

And some methods of ssl/context.cr should also be renamed.

i.e.
- ca_certificates= -> ca_certificates_path=
- private_key= -> private_key_path=

Or better method name.

Enhance

src/openssl/lib_crypto.cr

lib LibCrypto
  alias PasswordCallback = (UInt8*, Int, Int, Void*) -> Int
  alias EVP_PKEY = Void*

  fun pem_read_bio_x509 = PEM_read_bio_X509(bio : Bio*, x509 : X509*, cb : PasswordCallback, user_data : Void*) : X509
  fun bio_s_mem = BIO_s_mem : BioMethod*
  fun bio_new = BIO_new(type : BioMethod*) : Bio*
  fun bio_write = BIO_write(bio : Bio*, data : UInt8*, len : Int) : Int
  fun bio_free = BIO_free(Bio*) : Int

  fun pem_read_bio_privatekey = PEM_read_bio_PrivateKey(bio : Bio*, pk : EVP_PKEY*, cb : PasswordCallback, user_data : Void*) : EVP_PKEY
  fun evp_pkey_free = EVP_PKEY_free(pkey : EVP_PKEY)
end

src/openssl/lib_ssl.cr

lib LibSSL
  fun ssl_ctx_use_certificate = SSL_CTX_use_certificate(ctx : SSLContext, x509 : LibCrypto::X509) : Int
  fun ssl_ctx_use_privatekey = SSL_CTX_use_PrivateKey(ctx : SSLContext, pkey : LibCrypto::EVP_PKEY) : Int
end

src/ssl/context.cr

abstract class OpenSSL::SSL::Context

  # Set the CA certificate by string, in PEM format, used to
  # validate the peers certificate.
  def ca_certificate_text=(certificate : String)
    bio = LibCrypto.bio_new LibCrypto.bio_s_mem
    LibCrypto.bio_write bio, certificate, certificate.size
    x509 = LibCrypto.pem_read_bio_x509 bio, nil, nil, nil
    ret = LibSSL.ssl_ctx_use_certificate @handle, x509
    LibCrypto.bio_free bio ensure LibCrypto.x509_free x509
    raise OpenSSL::Error.new("SSL_CTX_use_certificate") unless ret == 1_i32
  end

  # Set the private key by string, The key must in PEM format.
  def private_key_text=(private_key : String)
    bio = LibCrypto.bio_new LibCrypto.bio_s_mem
    LibCrypto.bio_write bio, private_key, private_key.size
    pkey = LibCrypto.pem_read_bio_privatekey bio, nil, nil, nil
    ret = LibSSL.ssl_ctx_use_privatekey @handle, pkey
    LibCrypto.bio_free bio ensure LibCrypto.evp_pkey_free pkey
    raise OpenSSL::Error.new("SSL_CTX_use_PrivateKey") unless ret == 1_i32
  end
end

Usage

require "openssl"
context = OpenSSL::SSL::Context::Server.new
context.ca_certificate_text = File.read "/Users/User/Certificate/server.pem"
context.private_key_text = File.read "/Users/User/Certificate/localhost.key"

References

Finally

Sorry, I will only issue, not going to pull.

ghost · 2019-06-23T19:48:55Z

@ysbaddaden @asterite I have implemented this feature, If anyone is interested in this, please merge, thanks.

ghost · 2019-06-24T21:33:03Z

Update

Other classes will use similar features, so I redesigned them.
Some Reference: datanoise/openssl.cr

Better implementation

crystal/src/openssl/lib_crypto.cr

lib LibCrypto
  alias PasswordCallback = (UInt8*, Int, Int, Void*) -> Int
  alias EVP_PKEY = Void*
  alias RSA = Void*

  NID_rsaEncryption = 6_i32
  BIO_CTRL_RESET    = 1_i32

  fun bio_s_mem = BIO_s_mem : BioMethod*
  fun bio_new = BIO_new(type : BioMethod*) : Bio*
  fun bio_read = BIO_read(bio : Bio*, data : UInt8*, len : Int) : Int
  fun bio_write = BIO_write(bio : Bio*, data : UInt8*, len : Int) : Int
  fun bio_free = BIO_free(Bio*) : Int
  fun bio_free_all = BIO_free_all(bio : Bio*) : Int
  fun bio_ctrl = BIO_ctrl(bio : Bio*, cmd : Int, larg : Int, parg : Void*) : Int

  fun evp_pkey_assign = EVP_PKEY_assign(pkey : EVP_PKEY, type : Int, key : Void*) : Int
  fun evp_pkey_new = EVP_PKEY_new : EVP_PKEY
  fun evp_pkey_free = EVP_PKEY_free(pkey : EVP_PKEY)
  fun evp_pkey_get1_rsa = EVP_PKEY_get1_RSA(pk : EVP_PKEY) : RSA

  fun rsa_generate_key = RSA_generate_key(bits : Int, e : UInt, cb : (Int32, Int32, Void*) ->, ud : Void*) : RSA
  fun rsa_free = RSA_free(rsa : RSA) : Int
  fun rsapublickey_dup = RSAPublicKey_dup(rsa : RSA) : RSA
  fun rsaprivateKey_dup = RSAPrivateKey_dup(rsa : RSA) : RSA

  fun pem_read_bio_privatekey = PEM_read_bio_PrivateKey(bio : Bio*, pk : EVP_PKEY, cb : PasswordCallback, user_data : Void*) : EVP_PKEY
  fun pem_read_bio_x509 = PEM_read_bio_X509(bio : Bio*, x509 : X509*, cb : PasswordCallback, user_data : Void*) : X509
  fun pem_write_bio_rsaprivatekey = PEM_write_bio_RSAPrivateKey(bio : Bio*, rsa : RSA, enc : EVP_CIPHER, kstr : UInt8*, klen : Int, cb : PasswordCallback, user_data : Void*) : Int
  fun pem_write_bio_rsa_pubkey = PEM_write_bio_RSA_PUBKEY(bio : Bio*, rsa : RSA) : Int

  fun x509_get_pubkey = X509_get_pubkey(x509 : X509) : EVP_PKEY
end

crystal/src/openssl/lib_ssl.cr

lib LibSSL
  fun ssl_ctx_use_certificate = SSL_CTX_use_certificate(ctx : SSLContext, x509 : LibCrypto::X509) : Int
  fun ssl_ctx_use_privatekey = SSL_CTX_use_PrivateKey(ctx : SSLContext, pkey : LibCrypto::EVP_PKEY) : Int
end

NewFile - crystal/src/openssl/bio/mem_bio.cr

class OpenSSL::MemBIO < IO
  class Error < OpenSSL::Error
  end

  def initialize(@bio : LibCrypto::Bio*)
    raise Error.new "Invalid handle" unless @bio
  end

  def initialize
    initialize LibCrypto.bio_new(LibCrypto.bio_s_mem)
  end

  def read(data : Slice(UInt8))
    LibCrypto.bio_read self, data, data.size
  end

  def write(data : String)
    write data.to_slice
  end

  def write(data : Slice(UInt8))
    LibCrypto.bio_write self, data, data.size
  end

  def reset
    LibCrypto.bio_ctrl self, LibCrypto::BIO_CTRL_RESET, 0_i64, nil
  end

  def finalize
    LibCrypto.bio_free_all self
  end

  def to_io(io : IO)
    IO.copy self, io
  end

  def to_s
    io = IO::Memory.new
    to_io io
    io.to_s ensure io.close
  end

  def to_unsafe
    @bio
  end
end

NewFile - crystal/src/openssl/pkey/pkey.cr

module OpenSSL
  class PKey
    class Error < OpenSSL::Error
    end

    enum KeyType
      PrivateKey
      PublicKey
    end

    def initialize(@pkey : LibCrypto::EVP_PKEY, @keyType = KeyType::PublicKey)
      raise Error.new "Invalid EVP_PKEY" unless @pkey
    end

    def initialize(keyType : KeyType)
      initialize LibCrypto.evp_pkey_new, keyType
    end

    def self.from_text(private_key : String, password = nil)
      bio = MemBIO.new
      bio.write private_key
      pkey = LibCrypto.pem_read_bio_privatekey bio, nil, nil, nil
      bio.finalize
      new pkey, KeyType::PrivateKey
    end

    def private_key?
      KeyType::PrivateKey == keyType
    end

    def public_key?
      KeyType::PublicKey == keyType
    end

    def to_rsa
      OpenSSL::PKey::RSA.new LibCrypto.evp_pkey_get1_rsa(self), @keyType
    end

    def to_unsafe
      @pkey
    end

    def finalize
      LibCrypto.evp_pkey_free @pkey
    end
  end
end

NewFile - crystal/src/openssl/pkey/rsa.cr

module OpenSSL
  class PKey::RSA
    alias KeyType = OpenSSL::PKey::KeyType

    class Error < PKey::Error
    end

    def initialize(@rsa : LibCrypto::RSA, @keyType = KeyType::PublicKey)
      raise Error.new "Invalid EVP_PKEY" unless @rsa
    end

    def self.new(size : Int32)
      generate size
    end

    def self.generate(size : Int32, exponent = 65537_u32)
      new LibCrypto.rsa_generate_key size, exponent, nil, nil
    end

    def to_io(io : IO, keyType : KeyType)
      bio = MemBIO.new
      case keyType
      when KeyType::PrivateKey
        LibCrypto.pem_write_bio_rsaprivatekey bio, self, nil, nil, 0, nil, nil
      when KeyType::PublicKey
        LibCrypto.pem_write_bio_rsa_pubkey bio, self
      end ensure bio.to_io io
    end

    def to_s(keyType : KeyType)
      io = IO::Memory.new
      to_io io, keyType
      io.to_s ensure io.close
    end

    def to_s
      to_s @keyType
    end

    def to_unsafe
      @rsa
    end

    def finalize
      LibCrypto.rsa_free self
    end

    def self.free(rsa : LibCrypto::RSA)
      LibCrypto.rsa_free rsa
    end

    def private_key
      private_rsa = LibCrypto.rsaprivateKey_dup self
      raise Error.new "Could not get PrivateKey from RSA" unless private_rsa
      private_rsa
    end

    def public_key
      public_rsa = LibCrypto.rsapublickey_dup self
      raise Error.new "Could not get PublicKey from RSA" unless public_rsa
      public_rsa
    end
  end
end

crystal/src/openssl/x509/certificate.cr

module OpenSSL::X509
  class Certificate
    def self.from_text(certificate : String)
      bio = MemBIO.new
      bio.write certificate
      x509 = LibCrypto.pem_read_bio_x509 bio, nil, nil, nil
      bio.finalize
      new x509
    end

    def public_key
      OpenSSL::PKey.new LibCrypto.x509_get_pubkey(self),
        OpenSSL::PKey::KeyType::PublicKey
    end
  end
end

crystal/src/openssl/ssl/context.cr

abstract class OpenSSL::SSL::Context
  # Set the CA certificate by string, in PEM format, used to
  # validate the peers certificate.
  def ca_certificate_text=(certificate : String)
    x509 = OpenSSL::X509::Certificate.from_text certificate
    ret = LibSSL.ssl_ctx_use_certificate @handle, x509.to_unsafe
    x509.finalize
    raise OpenSSL::Error.new("SSL_CTX_use_certificate") unless ret == 1_i32
  end

  # Set the private key by string, The key must in PEM format.
  def private_key_text=(private_key : String)
    pkey = OpenSSL::PKey.from_text private_key
    ret = LibSSL.ssl_ctx_use_privatekey @handle, pkey.to_unsafe
    pkey.finalize
    raise OpenSSL::Error.new("SSL_CTX_use_PrivateKey") unless ret == 1_i32
  end
end

Usage

# 1 - Load Certificate from String | Extract the RSA public key data in the X509 certificate
x509 = OpenSSL::X509::Certificate.from_text File.read "/Users/User/Certificate/server.pem"
puts [x509.subject.to_a] # => [[{"CN", "CustomName"}]]
rsa_key = x509.public_key.to_rsa
puts [rsa_key.to_s] ensure rsa_key.finalize # => RSA PublicKey...

# 2 - Load PrivateKey from String | Extract the RSA private key data in the PKey certificate
pkey = OpenSSL::PKey.from_text File.read "/Users/User/Certificate/localhost.key"
rsa_key = pkey.to_rsa
puts [rsa_key.to_s] ensure rsa_key.finalize # => RSA PrivateKey...
x509.finalize ensure pkey.finalize

# 3 - Load string from `OpenSSL::SSL::Context::Server` (Certificate/PrivateKey)
context = OpenSSL::SSL::Context::Server.new
context.ca_certificate_text = File.read "/Users/User/Certificate/server.pem"
context.private_key_text = File.read "/Users/User/Certificate/localhost.key"

# 4 - Test `OpenSSL::MemBIO` IO read and write features.
bio = OpenSSL::MemBIO.new
bio.write File.read "/Users/User/Certificate/server.pem"
bio_memory = IO::Memory.new
bio.to_io bio_memory
puts [bio_memory] # => [#<IO::Memory:0x108acaae0 ...]
bio_memory.close ensure bio.finalize

# 5 - Test `OpenSSL::PKey::RSA` KeyPair generation
rsa_key = OpenSSL::PKey::RSA.new 4096
private_key = OpenSSL::PKey::RSA.new rsa_key.private_key, OpenSSL::PKey::KeyType::PrivateKey
public_key = OpenSSL::PKey::RSA.new rsa_key.public_key, OpenSSL::PKey::KeyType::PublicKey
puts str = {private_key: private_key.to_s, public_key: public_key.to_s}
private_key.finalize ensure public_key.finalize ensure rsa_key.finalize

Last Modified

9:11 PM | Tuesday, June 25, 2019 (CDT) | Time in Davison County, SD

jhass · 2019-06-25T11:23:19Z

If anyone wants to take this over and go through a proper pull request cycle, taking care of any codereview feedback, please do :)

rileythomp · 2019-11-15T21:34:05Z

Does this still need to be made into a pr?

ghost changed the title ~~HTTP::Server loads Certificate/PrivateKey from Memory~~ OpenSSL::SSL::Context loads Certificate/PrivateKey from Memory Jun 18, 2019

ghost changed the title ~~OpenSSL::SSL::Context loads Certificate/PrivateKey from Memory~~ Need to enhance the OpenSSL::SSL::Context loads Certificate/PrivateKey from Memory Jun 20, 2019

ghost closed this as completed Jun 23, 2019

asterite reopened this Jun 23, 2019

jhass added good first issue This is an issue suited for newcomers to become aquianted with working on the codebase. kind:feature topic:stdlib labels Jun 25, 2019

636f7374 mentioned this issue Nov 16, 2019

Some TCPSocket connections will cause HTTP::Server accept (freeze | blocking | hangs | waiting)? #8376

Closed

straight-shoota added help wanted This issue is generally accepted and needs someone to pick it up and removed good first issue This is an issue suited for newcomers to become aquianted with working on the codebase. labels Nov 26, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Need to enhance the OpenSSL::SSL::Context loads Certificate/PrivateKey from Memory #7897

Need to enhance the OpenSSL::SSL::Context loads Certificate/PrivateKey from Memory #7897

ghost commented Jun 18, 2019 •

edited by ghost

Loading

ghost commented Jun 20, 2019 •

edited by ghost

Loading

ysbaddaden commented Jun 20, 2019

ghost commented Jun 20, 2019

ghost commented Jun 23, 2019 •

edited by ghost

Loading

ghost commented Jun 23, 2019 •

edited by ghost

Loading

ghost commented Jun 24, 2019 •

edited by ghost

Loading

jhass commented Jun 25, 2019

rileythomp commented Nov 15, 2019

Need to enhance the OpenSSL::SSL::Context loads Certificate/PrivateKey from Memory #7897

Need to enhance the OpenSSL::SSL::Context loads Certificate/PrivateKey from Memory #7897

Comments

ghost commented Jun 18, 2019 • edited by ghost Loading

Summary

In Golang

References

ghost commented Jun 20, 2019 • edited by ghost Loading

ysbaddaden commented Jun 20, 2019

ghost commented Jun 20, 2019

ghost commented Jun 23, 2019 • edited by ghost Loading

Update

Enhance

Usage

References

Finally

ghost commented Jun 23, 2019 • edited by ghost Loading

ghost commented Jun 24, 2019 • edited by ghost Loading

Update

Better implementation

Usage

Last Modified

jhass commented Jun 25, 2019

rileythomp commented Nov 15, 2019

ghost commented Jun 18, 2019 •

edited by ghost

Loading

ghost commented Jun 20, 2019 •

edited by ghost

Loading

ghost commented Jun 23, 2019 •

edited by ghost

Loading

ghost commented Jun 23, 2019 •

edited by ghost

Loading

ghost commented Jun 24, 2019 •

edited by ghost

Loading