MIPS RSA pk encryption produces invalid result #1722
Description
May be related to #1093 and seen impacting big and little endian MIPS
Description
- Type: Bug
- Priority: Major | Minor
Mitigated by following suggestion to remove embedded assembly inbn_mul.hbased on Getting MBEDTLS_ERR_RSA_PRIVATE_FAILED error #755 (comment)
Bug
OS
Mbed OS linux MIPS and MIPSEL
mbed TLS build:
Version: 2.6.0 and still present in 2.9.0
OS version: reproduced in qemu-mips and Openwrt 15.05 (Broadcom BCM5300)
Configuration: attached config.h
config.h.txt
Compiler and options (if you used a pre-built binary, please indicate how you obtained it):
https://bitbucket.org/GregorR/musl-cross
Peer device TLS stack and version
N/A failure can be reproduced in sample code, although this was identified by encrypting data with an RSA public key on MIPS and attempting to decrypt in Ruby 2.5.1 on x86_64.
RSA key was generated with OpenSSL 1.0.2j
openssl req -new -newkey rsa:2048 -nodes -x509 -subj '/CN=localhost' -days 365 -keyout server.key -out server.crt && cat server.key server.crt > server.pem
openssl rsa -in server.pem -pubout > server.pub
Expected behavior
Sample code should report success on decrypt of public key encrypted data.
qemu-mips ./test
loaded public key
ALL IS WELL IN LOCAL DECRYPT
Actual behavior
Decryption fails with error code
qemu-mips ./test
loaded public key
FAILED TO DECRYPT mbedtls_pk_decrypt RETURNED -16640
Steps to reproduce
Sample failing code on MIPS, this code reports success on ARM & x86_64
#include <stdlib.h>
#include <mbedtls/aes.h>
#include <mbedtls/entropy.h>
#include <mbedtls/ctr_drbg.h>
#include <mbedtls/pk.h>
int main (argv)
{
size_t olen = 0;
size_t data_len = 16;
unsigned char data[16] = "ABCDEFGHIJKLMNO";
mbedtls_pk_context pk;
mbedtls_ctr_drbg_context ctr_drbg;
mbedtls_entropy_context entropy;
mbedtls_entropy_init(&entropy);
mbedtls_ctr_drbg_init(&ctr_drbg);
mbedtls_pk_init(&pk);
if (!(mbedtls_pk_parse_public_keyfile( &pk, "server.pub" ))) {
printf("loaded public key\n");
if (!(mbedtls_ctr_drbg_seed(&ctr_drbg, mbedtls_entropy_func, &entropy, NULL, 0))) {
mbedtls_pk_context priv_k;
mbedtls_pk_init(&priv_k);
unsigned char buf[MBEDTLS_MPI_MAX_SIZE] = {'\0'};
if (!(mbedtls_pk_encrypt(&pk, data, data_len, buf, &olen, sizeof(buf), mbedtls_ctr_drbg_random, &ctr_drbg))) {
// now try to decrypt
if (!(mbedtls_pk_parse_keyfile( &priv_k, "server.key", "" ))) {
size_t dec_len = 0;
unsigned char result[MBEDTLS_MPI_MAX_SIZE] = {'\0'};
int ret = 0;
if(!(ret = mbedtls_pk_decrypt( &priv_k, buf, olen, result, &dec_len, sizeof(result),
mbedtls_ctr_drbg_random, &ctr_drbg)))
{
printf("ALL IS WELL IN LOCAL DECRYPT\n");
}
else
{
printf("FAILED TO DECRYPT mbedtls_pk_decrypt RETURNED %d\n", ret);
}
}
else
{
printf("FAILED TO PARSE THE PRIVATE KEY\n");
}
} else {
printf("FAILED TO ENCRYPT DATA\n");
}
}
} else
{
printf("FAILED TO PARSE THE PUBLIC KEY\n");
}
mbedtls_ctr_drbg_free(&ctr_drbg);
mbedtls_entropy_free(&entropy);
return 0;
}
compiled with sample mips-linux-gnu-gcc -static main.c -o test -I"../mbedtls/include/" -L"../lib" -lmbedtls -lmbedcrypto using https://hub.docker.com/r/asmimproved/qemu-mips/