qat_engine_init error after reload nginx

[guobinchn]

Hi:
I run nginx-1.10.3 with qat_engine(0.5.22) and openssl 1.1.0b on centos 6.5.
When it reload (nginx -s reload)nginx after change nginx.conf, the error.log write a lot of error ,
but when I restart nginx services, the nginx is ok,
please help me , thks.

errorlog:
[WARNING][e_qat.c:993:qat_engine_init()] icp_sal_userStart failed
[WARNING][e_qat.c:256:get_next_inst()] Failure in qat_engine_init function
[WARNING][qat_rsa.c:433:qat_rsa_decrypt()] Failed to get an instance
[WARNING][qat_rsa.c:1047:qat_rsa_priv_dec()] Failure in qat_rsa_decrypt
2017/06/29 11:01:35 [crit] 11927#11927: *239209 SSL_do_handshake() failed (SSL: error:8007E094:lib(128):qat_engine_init:icp sal userstart fail error:8008E044:lib(128):qat_rsa_decrypt:internal error error:80090044:lib(128):qat_rsa_priv_dec:internal error error:1419F093:SSL routines:tls_process_cke_rsa:decryption failed) while SSL handshaking, client: 120.80.57.18, server: 0.0.0.0:443
[error] icp_sal_userStartMultiProcess() - : icp_adf_userProcessToStart failed

[stevelinsell]

Hi @guobinchn,

When you perform an nginx -s reload it does the following:

• Starts a new set of processes to handle the nginx connections.
• All new incoming connections will use these new processes.
• Existing connections will continue to use the old processes and only once all the connections on that process have finished will the process be terminated.

This means that while the new processes have been created and the old processes have not been terminated it is possible to have up to twice as many processes running as expected.
The errors you are seeing above are caused as the new nginx processes come up because the old ones are still running and icp_sal_userStart will fail because you have exceeded the maximum number of processes that can utilize QAT.
As the Intel Quickassist Driver conf file declares the maximum amount of QAT enabled processes you can run with it is important to declare enough processes in the NumProcesses field to allow for both the new and old nginx processes. So for instance if you are running 16 nginx server processes during normal operation then you should set NumProcesses to 32.

Hope this information is helpful,

Steve.

[guobinchn]

what is the maximum NumProcesses of value?

How to understand the NumProcesses value of 1?
thanks.

[stevelinsell]

Hi @guobinchn

It depends how many userspace crypto/compression instances you declare in your config file. I'll refer you to the Intel Quickassist Driver documentation below for full details:

If you are using an Intel Communication Chipset 8925 - 8955 Series Device then please see the following document:
https://01.org/sites/default/files/page/330751-006_clc_pg.pdf
Section 5.3.3.1 Maximum Number of Process Calculations.

If you are using an Intel Communication Chipset 8900 - 8920 Series Device then please see the following document:
https://01.org/sites/default/files/page/comms_8900_8920_sw_pg_rev_006.pdf
Section 5.3.3.1 Maximum Number of Process Calculations.

Kind Regards,

Steve.

[guobinchn]

thanks, I will be try.