Exception has occurred: ValueError not enough values to unpack (expected 1, got 0)

[ilpadrinohack]

Hi:

I am trying to use pyhanko to perform a Pades-LTV signature using the spanish eID card. First I was trying with the CLI:

pyhanko sign addsig pkcs11 --lib /usr/lib64/pkcs11/opensc-pkcs11.so --token-label 'DNI electrónico (PIN1)' --cert-label CertFirmaDigital lorem_field.pdf lorem-f.pdf and get the following error, which I suppose is related to not being able to get the right certificate from the card:

Traceback (most recent call last):
File "/usr/local/lib64/python3.8/site-packages/pyhanko/cli.py", line 63, in pyhanko_exception_manager
yield
File "/usr/local/lib64/python3.8/site-packages/pyhanko/cli.py", line 829, in addsig_pkcs11
generic_sign(
File "/usr/local/lib64/python3.8/site-packages/pyhanko/cli.py", line 689, in generic_sign
result = signers.PdfSigner(
File "/usr/local/lib64/python3.8/site-packages/pyhanko/sign/signers.py", line 2277, in sign_pdf
test_signature_cms = signer.sign(
File "/usr/local/lib64/python3.8/site-packages/pyhanko/sign/signers.py", line 710, in sign
signed_attrs = self.signed_attrs(
File "/usr/local/lib64/python3.8/site-packages/pyhanko/sign/signers.py", line 533, in signed_attrs
general.as_signing_certificate(self.signing_cert)
File "/usr/local/lib64/python3.8/site-packages/pyhanko/sign/pkcs11.py", line 159, in signing_cert
self._load_objects()
File "/usr/local/lib64/python3.8/site-packages/pyhanko/sign/pkcs11.py", line 268, in _load_objects
kh, = list(q)
ValueError: not enough values to unpack (expected 1, got 0)
Error: Generic processing error.

Then, I tried to do it using a python code which it is as follows:

`import os
 from io import BytesIO

 import pytest
 import logging
 from freezegun import freeze_time

 from pyhanko.pdf_utils.incremental_writer import IncrementalPdfFileWriter
 from pyhanko.pdf_utils.reader import PdfFileReader
from pyhanko.sign import signers, pkcs11

#from pyhanko_tests.test_signing import val_trusted, SIMPLE_ECC_V_CONTEXT

def read_all(fname):
     with open(fname, 'rb') as f:
         return f.read()



MINIMAL_PATH = '../lorem-f.pdf'

MINIMAL = read_all(MINIMAL_PATH)
logger = logging.getLogger(__name__)

SKIP_PKCS11 = False



def _simple_sess():
   lib = "/usr/lib64/libpkcs11-fnmtdnie.so"
   session=pkcs11.open_pkcs11_session(
   lib, user_pin='p4rp4rp4arpxp2', token_label='DNI electrónico (PIN1)'
    )

   return session


default_other_certs = ('CertCAIntermediaDGP',)



def test_simple_sign(bulk_fetch, pss):

   w = IncrementalPdfFileWriter(BytesIO(MINIMAL))
   meta = signers.PdfSignatureMetadata(field_name='Sigg2')
   with _simple_sess() as sess:
        signer = pkcs11.PKCS11Signer(
              sess, 'CertFirmaDigital', other_certs_to_pull=default_other_certs,
              bulk_fetch=bulk_fetch, prefer_pss=pss
    )
    print(signer)
    
    out = signers.sign_pdf(w, meta, signer,None,None,False,None,False,None)

r = PdfFileReader(out)
emb = r.embedded_signatures[0]
assert emb.field_name == 'Sigg1'
val_trusted(emb)




test_simple_sign(True,False)`

And get the same error in "out=signers.sign_pdf (....)" whic is "Exception has occurred: ValueError
not enough values to unpack (expected 1, got 0)"

But using:

`for cert in session.get_objects({ Attribute.LABEL: "CertFirmaDigital",
 Attribute.CLASS: ObjectClass.CERTIFICATE,}):   
    der_bytes = cert[Attribute.VALUE]

 # Load a certificate object from the DER-encoded value
  cert = x509.Certificate.load(der_bytes)
  print(cert)`

I am getting the signature certificate.

Any help will be appreciated
Thanks in advance

[MatthiasValvekens]

From the stack trace, it appears that pyHanko isn't able to find the private key handle (although the error message is definitely less than enlightening — I'll do something about that).

Probably, the PKCS#11 label of the private key is not the same as that of the certificate object. Can you try to list the labels of the keys on the card using a tool like pkcs11-tool (or using python-pkcs11, whichever you prefer)? Once you know what the label of the key is, you should pass it to the PKCS11Signer constructor as the key_label parameter.

Let me know if that helps. Incidentally, you might also want to pass in "CertFirmaDigital" as the cert_label parameter, by the way.

PS: The ECDSA signing commands for PKCS#11 are almost completely untested, since SoftHSMv2 doesn't (didn't?) support ECDSA properly when I wrote the tests for it. So far, there's no indication that points towards that being the issue here, though.

PPS: By the way, validating against the validation contexts from the test suite won't help you, I'm afraid. Those have been set up to only accept the trust roots from my testing CAs. You'll want to set up a validation context with at least one trust root that covers your ID cert. If there's such a trust root in the system trust store, initialising your validation context without any arguments might already be sufficient.

[ilpadrinohack]

Thanks for your reply:

List of label of the keys using the pkcs11-tool are:
Using slot 0 with a present token (0x10000)
Certificate Object; type = X.509 cert
label: CertAutenticacion
subject: DN: C=ES/serialNumber=3400xxxxx SN=LOPEZ, GN=FERNANDO, CN=LOPEZ MOZOS, FERNANDO (AUTENTICACI\xC3\x93N)
ID: 413032303934423xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Certificate Object; type = X.509 cert
label: CertFirmaDigital
subject: DN: C=ES/serialNumber=3400xxxxx SN=LOPEZ, GN=FERNANDO, CN=LOPEZ MOZOS, FERNANDO (FIRMA)
ID: 463032303934xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Certificate Object; type = X.509 cert
label: CertCAIntermediaDGP
subject: DN: C=ES, O=DIRECCION GENERAL DE LA POLICIA, OU=DNIE, CN=AC DNIE 005
ID: 53303230393442xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Public Key Object; RSA 2048 bits
label: KpuAutenticacion
ID: 41303230393442xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Usage: verify
Access: local
Public Key Object; RSA 2048 bits
label: KpuFirmaDigital
ID: 463032303934xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Usage: verify
Access: local
Data object 2097159
label: 'DG1'
application: ''
app_id:
flags: modifiable
Data object 2097160
label: 'DG11'
application: ''
app_id:
flags: modifiable
Data object 2097161
label: 'DG13'
application: ''
app_id:
flags: modifiable
Data object 2097162
label: 'DG2'
application: ''
app_id:
flags: modifiable
Data object 2097163
label: 'DG7'
application: ''
app_id:
flags: modifiable
Data object 2097164
label: 'DG3'
application: ''
app_id:
flags: modifiable
Data object 2097165
label: 'DG14'
application: ''
app_id:
flags: modifiable
Data object 2097166
label: 'EFCOM'
application: ''
app_id:
flags: modifiable
Data object 2097167
label: 'EFSOD'
application: ''
app_id:
flags: modifiable

As you can see, there is public labelled "KpuFirmaDigital". I have tried to pass it as the key_label parameter in the PKCS11Signer with no luck. Same error. I have also used your improvement in the code to get an error message on a missing key handle, but have the same error on "signers.sign.pdf(....)" on the "test_wrong_key_label(buk_fetch)" function.
But if I use another wrong cert name (invented one), give "Exception has occurred: PKCS11Error
Could not find (unique) cert with label ...), which means that it is succesfully accessing to the certs stored in the card.

This also happens when the CLI is used where you do not specify the name of the cert to use, so is it possible that this is not related with the key label management?

Thanks again

[MatthiasValvekens]

Just to confirm: I'm not seeing any private keys in the pkcs11-tool listing. Possibly those are only accessible after logging in to the PKCS#11 token (but it depends). Did you run pkcs11-tool --module /path/to/p11-library.so -O -l?

Anyway, the certificates and public key objects won't help you when signing things, so you'll need to find a way to talk to the private keys :/
Are you able to sign things using other PKCS#11-based tools?

[MatthiasValvekens]

For comparison: this is what I see on my Belgian eID. Note in particular that there are two objects of type "Private Key" (labelled "Authentication" and "Signature", respectively).

Certificate Object; type = X.509 cert
  label:      Authentication
  subject:    <censored>
  ID:         0200000000000000
Certificate Object; type = X.509 cert
  label:      Signature
  subject:    <censored>
  ID:         0300000000000000
Certificate Object; type = X.509 cert
  label:      CA
  subject:    DN: C=BE, CN=Citizen CA/serialNumber=<censored>
  ID:         0400000000000000
Certificate Object; type = X.509 cert
  label:      Root
  subject:    DN: C=BE, CN=Belgium Root CA4
  ID:         0600000000000000
Private Key Object; RSA 
  label:      Authentication
  ID:         0200000000000000
  Usage:      sign
  Access:     sensitive
Public Key Object; RSA 2048 bits
  label:      Authentication
  ID:         0200000000000000
  Usage:      verify
  Access:     none
Private Key Object; RSA 
  label:      Signature
  ID:         0300000000000000
  Usage:      sign
  Access:     sensitive
Public Key Object; RSA 2048 bits
  label:      Signature
  ID:         0300000000000000
  Usage:      verify
  Access:     none

[ilpadrinohack]

Hi Mathias, you were right:

After logging in my eID card, I got the label of the private key which was "KprivFirmaDigital". So it seems there is not error if I pass this as key_value parameter to the PKCS11Signer constructor. Now I am struggling how to save this in a pdf after the sign to check if this has worked.

Anyway, thanks for your help and congratulation for your code.
I owe you a beer/paella if you come to Valencia.

Fernando

[MatthiasValvekens]

Haha, good to hear that that part got solved ;)

As for saving the output to a file: there you have multiple options.

Assuming that you're calling sign_pdf more or less as in this test, you should be able to do something like this:

with open('output-file.pdf', 'wb') as outf:
     # preparation goes here
     # ...
     signers.sign_pdf(w, meta, signer=signer, output=outf)

PyHanko should take care of the buffer management behind the scenes. If the input buffer is writable as well, you can also pass in_place=True instead. See here for more info.

For what it's worth, the PKCS#11 signer is also available directly from the CLI (with pyhanko sign addsig pkcs11), and there's a command line flag (--key-label) to specify the key label you need. Depending on what you want to do, that might be easier.

I'll keep this issue open for now; let me know if that solves your use case.

[ilpadrinohack]

Well, that works like a charm and now I am able to get a valid pades-b signature. Thank you very much.
My last question (I promise) is that I want to make a visible signature in a box. I am able to print the box with the timestamp, but neither the subject nor the issuer. This is the code I am using:

 `def _simple_sess():
  lib = "/usr/lib64/pkcs11/opensc-pkcs11.so"
  session=pkcs11.open_pkcs11_session(
                        lib, user_pin='p4rp4rp4arpxp2', token_label='DNI electrónico (PIN1)'
                         )

   return session


   default_other_certs = ('CertCAIntermediaDGP',)



   def test_simple_sign(bulk_fetch, pss):
        with _simple_sess() as sess:
             for certi in sess.get_objects({ Attribute.LABEL: "CertFirmaDigital",
                    Attribute.CLASS: ObjectClass.CERTIFICATE,}):   
                    der_bytes = certi[Attribute.VALUE]
                    certi = x509.Certificate.load(der_bytes)
             w = IncrementalPdfFileWriter(BytesIO(MINIMAL))
   
             signerp = pkcs11.PKCS11Signer(
             sess, 'CertFirmaDigital',None,key_label='KprivFirmaDigital', other_certs_to_pull=default_other_certs,
                   bulk_fetch=bulk_fetch, prefer_pss=pss
                           )
             vc = ValidationContext(trust_roots=[signerp.signing_cert])
            #tsl_client = timestamps.HTTPTimeStamper('http://timestamp.digicert.com')
             tsl_client=datetime.now()
             identificador= fields.SigCertConstraints(subjects=[certi])
             sv = fields.SigSeedValueSpec(
                   reasons=[ ],
             cert=fields.SigCertConstraints( subjects=[certi],  flags=fields.SigCertConstraintFlags.SUBJECT  ),
             digest_methods=['sha256', 'sha384', 'sha512'],
             flags=fields.SigSeedValFlags.REASONS | fields.SigSeedValFlags.DIGEST_METHOD)
     
             sp = fields.SigFieldSpec('Signature', seed_value_dict=sv, box=(200, 50, 400, 100),)
             #image = PdfImage(image=settings.STAMP_IMAGE)
             style = TextStampStyle(background=None, border_width=0)
    
   
             with open('output-file.pdf', 'wb') as outf:
        
                       out = signers.PdfSigner(signature_meta=signers.PdfSignatureMetadata(
                                        field_name='Signature',
                                        reason='Firma',
                                        location='',
                                        use_pades_lta=True,
                                        subfilter=fields.SigSeedSubFilter.PADES,
                                        embed_validation_info=True,
                                        validation_context=vc
                                        ),
                        signer=signerp,
                        timestamper=None,
                        stamp_style=style,
                        new_field_spec=sp
                        ).sign_pdf(w,output=outf)


              test_simple_sign(True,False)`

Obviously, I am not managing / understanding the "SigSeedValueSpec" in a proper way. Last code I made to get the subject, issuer and serial number was:

  `def obtcer():

    pk11objects = session2.findObjects([(PK11.CKA_CLASS,PK11.CKO_CERTIFICATE), 
   (PK11.CKA_LABEL,"CertFirmaDigital")])
       
        
    all_attributes = [
            PK11.CKA_SUBJECT,
            PK11.CKA_VALUE,
            #PK11.CKA_ISSUER,
            #PK11.CKA_CERTIFICATE_CATEGORY,
            #PK11.CKA_END_DATE,
            PK11.CKA_LABEL,
            
            PK11.CKA_ID,
    ]
        
    for pk11object in pk11objects:
            try:
                attributes = session2.getAttributeValue(pk11object, all_attributes)
                #print(attributes)
            except PK11.PyKCS11Error as e:
                continue

            attrDict = dict(list(zip(all_attributes, attributes)))
            
            #cert = bytes(attrDict[PK11.CKA_VALUE])

            certs = [OpenSSL.crypto.load_certificate(OpenSSL.crypto.FILETYPE_ASN1, bytes(attrDict[PK11.CKA_VALUE]))]
            for certi in certs:
                x509_cert = XX509()
                x509_cert.parse(OpenSSL.crypto.dump_certificate(OpenSSL.crypto.FILETYPE_PEM, certi).decode('utf-8'))
                subject = certi.get_subject()
                nombre = dict(subject.get_components())[b'CN'].decode()
                dni = dict(subject.get_components())[b'serialNumber'].decode()
                return nombre,dni  
                `

Thanks Matthias

[MatthiasValvekens]

A couple things:

• Don't bother with seed values if you're creating signature fields and then immediately signing them yourself. Seed values are a somewhat obscure feature of the standard intended for form authors to pass instructions to (later) signers. Support for those is fairly sparse "in the wild" anyhow.
• The visual appearance of the signature can be customised, but it's admittedly not that well-documented. I don't have the bandwidth to put together a full example for the documentation right now, but I can point you in the right direction (see below).

To put in a custom appearance, you'll have to work directly with the (slightly more detailed) API on which signers.sign_pdf() is built, i.e. use the sign_pdf method of PdfSigner directly: https://pyhanko.readthedocs.io/en/latest/api-docs/pyhanko.sign.signers.html#pyhanko.sign.signers.PdfSigner.sign_pdf. Note the following facts:

• PdfSigner takes a stamp_style parameter. You can override that one, either to a parametrised style (like the default one), or just put in the text that you want to show directly. For reference, this is how the default appearance is initialised:

  pyHanko/pyhanko/sign/signers.py

  Line 1823 in f43c3c0

  DEFAULT_SIGNING_STAMP_STYLE = TextStampStyle(

• If you decide to go with a parametrised style, you'll have to add string interpolation parameters for the subject & issuer DN, since pyHanko doesn't include those by default. When calling sign_pdf on your PdfSigner, you can then pass in those extra parameters as a dictionary via the appearance_text_params keyword argument.

Granted, it's a bit complicated, but the APIs are there. :) I've made a note to add a proper example to the documentation when I find the time, but I can't promise when that'll happen.

[ilpadrinohack]

Thank you for your help Matthias. All working now. And once more, congratulations for your code.

Best regards
Fernando

[MatthiasValvekens]

Great! And thank you for making good use of it :)

I'll go ahead and close the issue now, if you don't mind.