This project provides cryptography helper slots for Magic, allowing you to use both symmetric and asymmetric cryptography operations in your Hyperlambda applications. The symmetric parts of the project is using AES internally, and the asymmetric parts is using RSA. In addition to a bunch of "low level slots", the project also contains combination slots, combining RSA and AES, allowing you to both encrypt and sign a message, using a single signal invocation.
This slot create a bunch of random characters, or bytes for you. The slot can optionally take a [min] and [max] argument, which defines the min/max length of the random bytes/characters returned. If not supplied, the default values for these arguments are respectively 10 and 20. This slot is useful for creating random secrets, and similar types of random strings, where you need cryptographically secured random strings. An example of generating a cryptographically secure random string of text, between 50 and 100 characters in lenght, can be found below.
crypto.random
min:50
max:100
Notice, the [crypto.random] slot will only return characters from a-z, A-Z and 0-9. Which makes
it easily traversed using any string library. However, you can provide a [raw] argument, and set its
value to boolean true, at which point the slot will return the raw bytes as a byte[]. This has a much
larger amount of entropy than simply using alphanumeric characters for the same size - Which is
important as you start creating keys for AES encryption, etc.
The [crypto.hash] slot can be used to generate hash values. When you invoke it, you can choose
between having the hash returned as raw a byte[], as a "fingerprint" or as its bit encoded hex
version. Below is an example of all three of these formats.
.data:Some data to hash
crypto.hash:x:@.data
format:fingerprint
crypto.hash:x:@.data
format:text
crypto.hash:x:@.data
format:raw
To override the hashing algorithm used when creating hash values, apply an [algorithm] argument, and set its value to the hashing algorithm you want to use. You can choose between the following hashing algorithms as you consume the above slot.
- SHA256
- SHA384
- SHA512
Neither SHA1 nor MD5 are supported, since they're both considered "insecure" hashing algorithms.
This library also supports several cryptographic services, but first a bit of cryptography theory. Public key cryptography, or what's often referred to as "asymmetric cryptography" is based upon a key pair. One of your keys are intended for being publicly shared, and is often referred to as "your public key". This key can do two important things.
- Your public key can encrypt data such that only its private counterpart key can decrypt the data
- Your public key can verify that a message originated from a party that has access to its private counterpart
Hence, keeping your private key as private, is of outmost importance, otherwise 3rd parties might read messages others send to you, and also impersonate you in front of others. In addition, securely delivering your public key to the other party, is of equal importance, to make sure they're using the correct public key in their communication with you. If you can keep your private key private, and securely deliver your public key to others, you have a 100% secure channel to use for communication, preventing malicious individuals from both reading what others send to you, and also tampering with the content you send to others, before the other party receives it. Hence, cryptography is about two main subjects.
- Encrypting messages sent to you
- Allowing you to provide guarantees that a message originated from you
Depending upon your paranoia level, you might just send your public key in an email, which is considered insecure - Or you might need to physically meet the person whom you want to communicate with, and give him a USB stick with your public key, which is considered full paranoia level. The latter might be important if you fear what's often referred to as a "man in the middle attack", where some malicious adversary, takes your public key, and gives a bogus and fake public key to the other party. This results in that the man in the middle can intercept your communication, decrypt it, and re-encrypt it with your public key, before he or she sends it to you - In addition to that he can use a similar mechanism to impersonate your signatures, allowing the other party to falsely believe some message originated from you, when it did indeed originate from a malicious "man in the middle".
There are several different ways to create a key pair, just have the above in mind as you start using cryptography in your Hyperlambda applications. Most of the cryptography functions in this library is using Bouncy Castle, which is a thoroughly tested library for doing cryptography. Bouncy Castle is owned by a charitable organisation in Australia, so they don't need to obey by American laws, reducing American intelligence services ability to coerce them to build backdoors and similar constructs into their code. Bouncy Castle is also Open Source, allowing others to scrutinise their code for such backdoors. However, with cryptography, there are no guarantees, only a "general feeling and concent" amongst developers that it's secure.
The asymmetric parts of this project is built upon RSA for its public and private key pairs.
To create an RSA keypair that you can use for other cryptographic services later, you can use something as follows.
crypto.rsa.create-key
strength:2048
seed:some random jibberish text
Both the [strength] and [seed] is optional above. Strength will default to 2048, which might be too weak for serious cryptography, but increasing your strength too much, might result in that the above function spends several seconds, possibly minutes to return if you set it too high - In addition to that your key pair becomes very large. The [seed] is optional, and even if you don't provide a seed argument, the default seed should still be strong enough to avoid predictions.
A good strength for an RSA key, is considered to be 4096, which developers around the world feels are secure enough to avoid brute force "guessing" of your private key. According to what we know about cryptography, all other concerns set aside, a 4096 bit strength key pair, should be impossible to break. If you're just playing around with cryptography to learn, 1024 is probably more than enough.
Notice, if you want the key back as raw bytes, you can supply a [raw] argument, and set its value to boolean
true, at which point the returned key(s) will only be DER encoded, and returned as a raw byte[]. This might be
useful, if you for instance need to persist the key to disc, as a binary file, etc. All the RSA slots can return
their results as byte[] values, if you provide a [raw] argument to them, and set its value to true.
If you don't provide a raw argument, the returned value will be base64 encoded.
This slot will also return the fingerprint of your public key, which is useful to keep around somewhere, since it's used in other cryptographic operations to identify keys used in operation, etc. The public key's fingerprint is usually used to identify a specific key somehow.
You can use a previously created private RSA key to cryptographically sign some data or message, intended to be passed over an insecure context, allowing the caller to use your public key to verify the message was in fact created by the owner of the private key. To sign some arbitrary content using your private key, and also verify the message was correctly signed with a specific key, you can use something as follows.
// The message can also by byte arrays.
.message:Some message you wish to sign
crypto.rsa.create-key
// Notice, using PRIVATE key
crypto.rsa.sign:x:@.message
private-key:x:@crypto.rsa.create-key/*/private
// Uncommenting these lines, will make the verify process throw an exception
// set-value:x:@.message
// .:Some message you wish to sign - XXXX
// Notice, using PUBLIC key
crypto.rsa.verify:x:@.message
signature:x:@crypto.rsa.sign
public-key:x:@crypto.rsa.create-key/*/public
If somebody tampers with the content between the signing process and the verify process, an exception will be thrown during the verify stage. Something you can verify yourself by uncommenting the above [set-value] invocation. Throwing an exception is a conscious choice, due to the potential security breaches an error in your code might have, creating a false positive if you erronously invert an [if] statement. Even though this is technically "using exceptions for control flow", it has been a conscious design choice as the library was created, to avoid false positives during the verification process of a signature.
Notice, if you want the signature back as raw bytes, you can supply a [raw] argument, and set its value to boolean
true, at which point the returned signature will be returned as a raw byte[]. This might be
useful, if you for instance need to persist the signature to disc, as a binary file, etc.
If you don't provide a raw argument, the returned value will be a base64 encoded byte array.
To encrypt a message, you can use something as follows.
.message:Some message you want to encrypt
crypto.rsa.create-key
crypto.rsa.encrypt:x:@.message
public-key:x:@crypto.rsa.create-key/*/public
crypto.rsa.decrypt:x:@crypto.rsa.encrypt
private-key:x:@crypto.rsa.create-key/*/private
Notice how the encryption above is using the public key, and the decryption is using the private key. The encrypt slot
will internally base64 encode the encrypted data for simplicity reasons, allowing you to immediately inspect it as text,
since encryption will result in a byte array, which is often inconvenient to handle.
You can override this by passing in a [raw] argument, and set its value to true, at which point a byte[] will be
returned.
You can also supply [raw] as you invoke [crypto.rsa.decrypt] if you know the content in the message is
not a string, but rather an array of byte[]. Base64 encoding a byte array normally makes it larger in size,
and also require CPU resources in both ends of the communication, implying it is sometimes important to have the
raw byte array, instead of its base64 encoded equivalent.
RSA is asymmetric cryptography, implying a different key is used for decrypting the data, than that which was used to encrypt the data. This project also supports symmetric cryptography, more specifically the AES encryption algorithm. This algorithm requires the same key to decrypt some content that was used to encrypt the data, and the key must either be 128, 192 or 256 bits long. Below is an example.
crypto.aes.encrypt:Howdy, this is cool
password:Howdy World this is a passphrase that guarantees 256 bits strength
crypto.aes.decrypt:x:-
password:Howdy World this is a passphrase that guarantees 256 bits strength
The length of the key argument you provide, becomes the bit strength of the encryption, ranging from
128 through 192 to 256 bits. However, even though the key you normally use for encrypting and decrypting
when using AES is supposed to be a byte[], this project will automatically convert any passphrase
specified from a string to a SHA256 hash value. This allows you to use any passphrase you wish,
while avoiding reducing entropy, making it harder to crack the encrypted message.
Even though AES has low bit strength, it's still considered one of the strongest forms of cryptography
that exists, assuming you use it correct. For the record, this library does not use the built in
AES library from .Net, which has several security issues, due to the way it handles padding, among other
things - Neither does this library simply convert strings to byte[] arrays using Encoding.UTF.GetBytes,
which significantly reduces entropy, and makes your message easily cracked by a malicious agent with
some resources. Instead Magic uses Bouncy Castle, which does not have these security holes, in addition
to that it creates a SHA256 hash of passphrases used, if you provide a string, keeping as much entropy
as possible. However, if you want to decrypt it using other libraries, you'll have to inform the other
party of that the passphrase supplied is actually supposed to be hashed using SHA256 before supplied
as the key during decryption.
The library also supports using raw byte[] values as its [password] value, allowing you
to generate a random array of bytes, either 16, 24 or 32 bytes in size, and use this as your
passphrase directly - At which point the byte array will be used as is, and not hashed in any ways
before encryption/decryption occurs.
Due to AES' blistering speed and strength, it is often wise to combine asymmetric cryptography with symmetric cryptography, which can be used by generating a random symmetric key/passphrase, then encrypt this passphrase using asymmetric cryptography, such as for instance RSA, for then to use the passphrase to encrypt the actual main data the caller wants to transmit. This has several advantages, such as reducing the size of the data sent, while still providing the benefits from asymmetric cryptography, such as securely sharing the public key, etc. Of course, sharing a symmetric key without major hassle, and/or making adversaries also get a hold of it, is practically very difficult for obvious reasons, unless you can asymmetrically encrypt the symmetric key.
If you only need a random array of 32 bytes, to use as your passphrase, in combination with for instance RSA asymmetric cryptography - You can use the [crypto.random] slot as follows.
crypto.random
min:32
max:32
raw:true
AES and RSA are only really useful when combined. Hence, this project contains the following convenience slots.
- [crypto.encrypt] - Encrypts some message using AES + RSA, and signs the message in the process
- [crypto.decrypt] - Decrypts some message using AES + RSA, optionally verifying a signature in the process
- [crypto.sign] - Cryptographically signs a message using RSA and creates a package containing both signature, signing key's fingerprint, and the content that was signed
- [crypto.verify] - The opposite of the above, that verifies the integrity of a package created with [crypto.sign]
- [crypto.get-key] - Returns the fingerprint of the RSA key that was used to encrypt some message using [crypto.encrypt] or sign some message using [crypto.sign]
The [crypto.encrypt] slot requires some message/content, a signing key, an encryption key, and your signing key's fingerprint. This slot will first cryptographically sign your message using the private key. Then it will use the public key supplied to encrypt the message. Below is an example.
// Recipient's key.
crypto.rsa.create-key
strength:512
// Sender's key.
crypto.rsa.create-key
strength:512
// Encrypting some message.
crypto.encrypt:Some super secret message
encryption-key:x:././*/crypto.rsa.create-key/[0,1]/*/public
signing-key:x:././*/crypto.rsa.create-key/[1,2]/*/private
signing-key-fingerprint:x:././*/crypto.rsa.create-key/[1,2]/*/fingerprint
// Decrypting the above encrypted message.
crypto.decrypt:x:-
decryption-key:x:././*/crypto.rsa.create-key/[0,1]/*/private
verify-key:x:././*/crypto.rsa.create-key/[1,2]/*/public
Notice - We're using only 512 bit strength in the above example. Make sure you (at least) use 2048, preferably 4096 in real world usage. The [crypto.encrypt] slot can also optionally handle a [seed] argument, which will seed the CS RNG that's used to generate a symmetric AES encryption key.
To understand what occurs in the above Hyperlambda example, let's walk through it step by step, starting from the [crypto.encrypt] invocation.
- The message "Some super secret message" is first cryptographically signed using the [signing-key]
- The signed message is then encrypted using a CSRNG generated AES key
- The AES key from the above is then encrypted using the [encryption-key], that's assumed to be the recipient's public key
- The signing key's fingerprint is stored inside of the encrypted content, such that when the message is decrypted, the other party can verify that the signature originated from some trusted party
- The encryption key's fingerprint is stored as bytes, prepended before the encrypted message, which allows the other party to retrieve the correct decryption key, according to what encryption key the caller encrypted the message with. To retrieve a cryptography operation key fingerprint, you can use [crypto.get-key]
Hence, the only thing that is in plain sight in the above encrypted message, is the fingerprint of the public key that was used to encrypt the message. Only after the message is decrypted, the signature for the message can be retrieved, together with the fingerprint of the key that was used to sign the message. Hence, what would normally be a more complete process, is that after the receiver decrypts the message, he should also verify that the signature originates from some trusted party. This can be done by simply omitting the [verify-key] argument as you invoke [crypto.decrypt], and then invoke [crypto.get-key] on the result of the decryption process, for then to use the result of [crypto.get-key] to lookup the public key used to verify the signature of the package.
// Recipient's key.
crypto.rsa.create-key
strength:512
// Sender's key.
crypto.rsa.create-key
strength:512
// Encrypting some message.
crypto.encrypt:Some super secret message
encryption-key:x:././*/crypto.rsa.create-key/[0,1]/*/public
signing-key:x:././*/crypto.rsa.create-key/[1,2]/*/private
signing-key-fingerprint:x:././*/crypto.rsa.create-key/[1,2]/*/fingerprint
// Decrypting the above encrypted message.
crypto.decrypt:x:-
decryption-key:x:././*/crypto.rsa.create-key/[0,1]/*/private
// Uncomment this line to retrieve signing key's fingerprint
// That you can use to lookup the public key needed to verify
// the signature
// crypto.get-key:x:-
// Verifying signature of encrypted message.
crypto.verify:x:@crypto.decrypt
public-key:x:././*/crypto.rsa.create-key/[1,2]/*/public
Only after the message is verified, the actual content of the message is possible to read, as the value of the [crypto.verify] slot - Unless you pass in a [verify-key] during the invocation to [crypto.decrypt], at which point that key will be used to verify the signature of the message, after the package has been decrypted.
If the above invocation to [crypto.verify] does not throw an exception, we know for a fact that the message was cryptographically signed with the private key that matches its [public-key] argument. Normally the fingerprint of the sender's key is asssociated with some sort of "authorisation object" to elevate the rights of the user, only after having verified the message originated from a trusted party.
Hence, from the caller's perspective it's one invocation to encrypt and sign a message. From the receiver's perspective it's normally two steps to both decrypt and verify the integrity of a message, unless you know who the message originated from.
Notice - We're using only 512 bit strength in the above example. Make sure you (at least) use 2048, preferably 4096 in real world usage.
The encrypted package has the following format.
- Signing key's fingerprint in SHA256
byte[]format, 32 bytes long - The length of the signature as
int, 4 bytes long - The actual signature of the message
- The content of the message in UTF encoded
byte[]format
Afterwards the result from the above steps is encrypted using AES, with a random generated session key that is 32 bytes long. And another package is created, which is the final package, intended for being sent to the recipient. The final encryption package has a structure as follows.
- Encryption key's fingerprint in SHA256
byte[]format, 32 bytes long - The length of the encrypted session key as
int, 4 bytes long - The encrypted session key, encrypted using the recipient's public RSA key
- The AES encrypted content from the above signing step
Hence, only when both of the above lists are done, you have a final encryption package to send to some recipient.
The other party can retrieve the encryption key used for encrypting the package, using for instance the [crypto.get-key] slot on the package. Then the receiver can use his private RSA key to decrypt the AES key, and use the decrypted AES key to decrypt the rest of the package - Which will result in getting the fingerprint of the RSA key used to sign the package, then the signature, and only then the content of the message. However, all of these steps are done automatically if you use the [crypto.decrypt] slot, except the signature verification process, unless you provide a [verify-key] argument to the decryption process.
The AES key is generated using Bouncy Castle's SecureRandom implementation, resulting in a 256 bit
cryptography key. This key again is encrypted using whatever bit strength you selected as you created
your RSA key pair. Hence, the message as a whole, is not stronger than whatever key strength you use
as you supply a [strength] argument to the [crypto.rsa.create-key].
The above format results in that the only "meta information" an adversary can possibly pick up, is the fingerprint of the public RSA key used to encrypt the AES key, in addition to also the bit strength of this RSA key, since the bit strength of an RSA key will result in differences in the length of the encrypted AES key. An adversary will not have access to who encrypted/transmitted the package, he will not know who, if any signed the package - Or any other parts of the message - Assuming he is not able to somehow crack the AES encryption, and/or somehow retrieve the private RSA key the AES package's encryption key was encrypted with.
This project provides the following slots.
- [crypto.hash] - Creates a hash of the specified string value/expression's value, using the specified [algorithm], that defaults to SHA256
- [crypto.password.hash] - Creates a cryptographically secure hash from the specified password, expected to be found in its value node. Uses blowfish, or more specifically BCrypt internally, to create the hash with individual salts.
- [crypto.password.verify] - Verifies that a [hash] argument matches towards the password specified in its value. The [hash] is expected to be in the format created by BCrypt, implying the hash was created with e.g. [crypto.password.hash].
- [crypto.random] - Creates a cryptographically secured random string for you, with the characters [a-zA-Z0-9].
- [crypto.rsa.create-key] - Creates an RSA keypair for you, allowing you to pass in [strength], and/or [seed] to override the default strength being 2048, and apply a custom seed to the random number generator. The private/public keypair will be returned to caller as [public] and [private] after invocation, which is the DER encoded keys, encoded by default as base64.
- [crypto.rsa.sign] - Cryptographically signs a message (provided as value) with the given private [private-key], and returns the signature for your content as value. The signature content will be returned as the base64 encoded raw bytes being your signature.
- [crypto.rsa.verify] - Verifies a previously created RSA signature towards its message (provided as value), with the specified public [public-key], optionally allowing the caller to provide a hashing [algorithm], defaulting to SHA256. The slot will throw an exception if the signature is not matching the message passed in for security reasons.
- [crypto.rsa.encrypt] - Encrypts the specified message (provided as value) using the specified public [public-key], and returns the encrypted message as a base64 encoded encrypted message by default.
- [crypto.rsa.decrypt] - Decrypts the specified message (provided as value) using the specified private [private-key], and returns the decrypted message as its original value.
- [crypto.aes.encrypt] - Encrypts a piece of data using the AES encryption algorithm
- [crypto.aes.decrypt] - Decrypts a piece of data previously encrypted using AES encryption
- [crypto.encrypt] - Convenience slot combining AES and RSA encryption to encrypt some message
- [crypto.decrypt] - The opposite of the above
- [crypto.sign] - Signs a package, and returns the combination of the signature and package to caller
- [crypto.verify] - Verifies a signature created using the [crypto.sign] slot
- [crypto.get-key] - Returns the public key that was used to encrypt a message using the above slot. Result is returned in "fingerprint format".
