gsub : invalid byte sequence in US-ASCII

[ec-ecss]

When I try to follow example of implementation, I got an issue on the hashing part. Here is the error

Traceback (most recent call last):
        7: from test.rb:289:in `<main>'
        6: from /var/lib/gems/2.7.0/gems/zatca-1.1.0/lib/zatca/ubl/invoice.rb:188:in `to_base64'
        5: from /var/lib/gems/2.7.0/gems/zatca-1.1.0/lib/zatca/ubl/invoice.rb:214:in `generate_xml'
        4: from /var/lib/gems/2.7.0/gems/zatca-1.1.0/lib/zatca/ubl/base_component.rb:135:in `generate_xml'
        3: from /var/lib/gems/2.7.0/gems/zatca-1.1.0/lib/zatca/ubl/builder.rb:24:in `build'
        2: from /var/lib/gems/2.7.0/gems/zatca-1.1.0/lib/zatca/ubl/builder.rb:37:in `apply_hacks_to_invoice'
        1: from /var/lib/gems/2.7.0/gems/zatca-1.1.0/lib/zatca/ubl/builder.rb:45:in `apply_qualifying_properties_hacks'
/var/lib/gems/2.7.0/gems/zatca-1.1.0/lib/zatca/ubl/builder.rb:45:in `gsub': invalid byte sequence in US-ASCII (ArgumentError)

Here is my code

#encoding: utf-8

require "zatca"

vat_id = "310000000000003"

# Four digits, each digit acting as a bool. The order is as follows: Standard Invoice, Simplified, future use, future use
invoice_type = "1100"

options = {
  common_name: "The common name to be used in the certificate",
  organization_identifier: vat_id,
  organization_name: "The name of your organization",
  organization_unit: "A subunit in your organization",
  country: "SA",
  invoice_type: invoice_type,
  address: "Riyadh 1234 Street",
  business_category: "Your business category",

  # The solution provider name
  egs_solution_name: "mrsoolsdk",

  # The model of the unit the stamp is being generated for
  egs_model: "1",

  # If you have multiple devices each should have a unique serial number
  egs_serial_number: "1-Zatca|2-Hububl|3-fa4e99b8-c9c5-4c37-957a-3926352c34e6"
}

generator = ZATCA::Signing::CSR.new(csr_options: options, mode: :sandbox)


csr = generator.generate

# ZATCA's API expects us to encode the PEM to Base64
csr_base64 = Base64.strict_encode64(csr)

client = ZATCA::Client.new(username: "", password: "", environment: :sandbox)

otp = "123345"
response = client.issue_csid(csr: csr_base64, otp: otp)

request_id = response["requestID"]
binary_security_token = response["binarySecurityToken"]
secret = response["secret"]
#puts response ;

#binarySecurityToken as user
#secret as passwort

client = ZATCA::Client.new(username: binary_security_token, password: secret, environment: :sandbox)



invoice_id = "SME00010"
invoice_uuid = "8e6000cf-1a98-4174-b3e7-b5d5954bc10d"
note = "ABC"
note_language_id = "ar"
issue_date = "2022-08-17"
issue_time = "17:41:08"

invoice_subtype = ZATCA::UBL::InvoiceSubtypeBuilder.build(
  simplified: true,
  third_party: false,
  nominal: false,
  exports: false,
  summary: false,
  self_billed: false
) # => "0200000"

payment_means_code = ZATCA::UBL::Invoice::PAYMENT_MEANS[:bank_card] # => "48"
invoice_type = ZATCA::UBL::Invoice::TYPES[:invoice] # => "388"

invoice_counter_value = "10"
previous_invoice_hash = "NWZlY2ViNjZmZmM4NmYzOGQ5NTI3ODZjNmQ2OTZjNzljMmRiYzIzOWRkNGU5MWI0NjcyOWQ3M2EyN2ZiNTdlOQ=="
currency_code = "SAR"
vat_registration_number = "311111111101113"

#===============================================================================
# 2. Setup the Invoice's Nested Properties
#===============================================================================

# Create the supplier party (the party issuing the invoice, e.g. the seller)
accounting_supplier_party = ZATCA::UBL::CommonAggregateComponents::Party.new(
  party_identification: ZATCA::UBL::CommonAggregateComponents::PartyIdentification.new(
    id: "324223432432432"
  ),
  postal_address: ZATCA::UBL::CommonAggregateComponents::PostalAddress.new(
    street_name: "الامير سلطان",
    additional_street_name: nil,
    building_number: "3242",
    plot_identification: "4323",
    city_subdivision_name: "32423423",
    city_name: "الرياض | Riyadh",
    postal_zone: "32432",
    country_subentity: nil,
    country_identification_code: "SA"
  ),

  party_tax_scheme: ZATCA::UBL::CommonAggregateComponents::PartyTaxScheme.new(
    company_id: vat_registration_number
  ),

  party_legal_entity: ZATCA::UBL::CommonAggregateComponents::PartyLegalEntity.new(
    registration_name: "Acme Widgets LTD"
  )
)

# Create the customer party (the party receiving the invoice, e.g. the buyer)
accounting_customer_party = ZATCA::UBL::CommonAggregateComponents::Party.new(
  # party_identification: ZATCA::UBL::CommonAggregateComponents::PartyIdentification.new(
  #   id: "2345",
  #   scheme_id: "NAT"
  # ),
  party_identification: nil,
  postal_address: ZATCA::UBL::CommonAggregateComponents::PostalAddress.new(
    street_name: nil,
    additional_street_name: nil,
    building_number: nil,
    plot_identification: nil,
    city_subdivision_name: "32423423",
    city_name: nil,
    postal_zone: nil,
    country_subentity: nil,
    country_identification_code: "SA"
  ),

  party_tax_scheme: ZATCA::UBL::CommonAggregateComponents::PartyTaxScheme.new,
  party_legal_entity: nil
)

# Create the (optional) delivery object (detailing the delivery date and time)
# delivery = ZATCA::UBL::CommonAggregateComponents::Delivery.new(
#   actual_delivery_date: "2022-03-13",
#   latest_delivery_date: "2022-03-15"
# )

delivery = nil

# Create the allowance charges (e.g. discounts) for the invoice
allowance_charges = [
  ZATCA::UBL::CommonAggregateComponents::AllowanceCharge.new(
    charge_indicator: false,
    amount: "0.00",
    allowance_charge_reason: "discount",
    currency_id: "SAR",
    tax_categories: [
      # Yes, ZATCA's official valid sample duplicates these, not sure why
      ZATCA::UBL::CommonAggregateComponents::TaxCategory.new(
        tax_percent: "15"
      ),
      ZATCA::UBL::CommonAggregateComponents::TaxCategory.new(
        tax_percent: "15"
      )
    ]
  )
]

# Create the tax totals for the invoice
# ZATCA's official valid sample has two of these, not sure why
tax_totals = [
  ZATCA::UBL::CommonAggregateComponents::TaxTotal.new(
    tax_amount: "30.15"
  ),
  ZATCA::UBL::CommonAggregateComponents::TaxTotal.new(
    tax_amount: "30.15",
    tax_subtotal_amount: "30.15",
    taxable_amount: "201.00",
    tax_category: ZATCA::UBL::CommonAggregateComponents::TaxCategory.new(
      tax_percent: "15.00"
    )
  )
]

# Create the legal monetary total for the invoice
legal_monetary_total = ZATCA::UBL::CommonAggregateComponents::LegalMonetaryTotal.new(
  line_extension_amount: "201.00",
  tax_exclusive_amount: "201.00",
  tax_inclusive_amount: "231.15",
  allowance_total_amount: "0.00",
  prepaid_amount: "0.00",
  payable_amount: "231.15"
)

# Create the invoice lines for the invoice (the list of items that were sold)
invoice_lines = [
  # Book
  ZATCA::UBL::CommonAggregateComponents::InvoiceLine.new(
    invoiced_quantity: "33.000000",
    invoiced_quantity_unit_code: "PCE",
    line_extension_amount: "99.00",
    tax_total: ZATCA::UBL::CommonAggregateComponents::TaxTotal.new(
      tax_amount: "14.85",
      rounding_amount: "113.85"
    ),
    item: ZATCA::UBL::CommonAggregateComponents::Item.new(
      name: "كتاب"
    ),
    price: ZATCA::UBL::CommonAggregateComponents::Price.new(
      price_amount: "3.00",
      allowance_charge: ZATCA::UBL::CommonAggregateComponents::AllowanceCharge.new(
        charge_indicator: false,
        allowance_charge_reason: "discount",
        amount: "0.00",
        add_tax_category: false,

        # ZATCA's samples can sometimes have a nested tax scheme with an ID
        # and sometimes they omit it. Setting this boolean controls whether it is
        # present or not
        add_id: false
      )
    )
  ),

  # Pen
  ZATCA::UBL::CommonAggregateComponents::InvoiceLine.new(
    invoiced_quantity: "3.000000",
    invoiced_quantity_unit_code: "PCE",
    line_extension_amount: "102.00",
    tax_total: ZATCA::UBL::CommonAggregateComponents::TaxTotal.new(
      tax_amount: "15.30",
      rounding_amount: "117.30"
    ),
    item: ZATCA::UBL::CommonAggregateComponents::Item.new(
      name: "قلم"
    ),
    price: ZATCA::UBL::CommonAggregateComponents::Price.new(
      price_amount: "34.00",
      allowance_charge: ZATCA::UBL::CommonAggregateComponents::AllowanceCharge.new(
        charge_indicator: false,
        allowance_charge_reason: "discount",
        amount: "0.00",
        add_tax_category: false,
        add_id: false
      )
    )
  )
]

#===============================================================================
# 3. Construct the Invoice
#===============================================================================
# Construct the invoice using all of the above
invoice = ZATCA::UBL::Invoice.new(
  add_ids_to_allowance_charges: false,
  id: invoice_id,
  uuid: invoice_uuid,
  issue_date: issue_date,
  issue_time: issue_time,
  subtype: invoice_subtype,
  type: invoice_type,
  invoice_counter_value: invoice_counter_value,
  previous_invoice_hash: previous_invoice_hash,
  accounting_supplier_party: accounting_supplier_party,
  accounting_customer_party: accounting_customer_party,
  delivery: delivery,
  payment_means_code: payment_means_code,
  allowance_charges: allowance_charges,
  tax_totals: tax_totals,
  legal_monetary_total: legal_monetary_total,
  invoice_lines: invoice_lines,
  currency_code: currency_code,
  note: note,
  note_language_id: note_language_id
)

private_key_path = "path/to_your/private.key"

# Must be in pem format with header blocks
certificate_path = "path/to_your/certificate.pem"

# You can omit this if you don't need to customize it,
# the default value is the same as you see here.
signing_time = Time.now.utc.strftime("#{invoice.issue_date}T#{invoice.issue_time}")

# Sign the invoice (this adds a couple of new elements to the invoice)

# openssl ecparam -name secp256k1 -genkey -noout -out ec-secp256k1-priv-key.pem
invoice.sign(
  private_key_path: "/var/www/html/public/ec-secp256k1-priv-key.pem",
  certificate_path: "/var/www/html/public/.private/app.pem",
  signing_time: signing_time,
  decode_private_key_from_base64: false # Optional
)

# 2. Send the invoice to ZATCA
# Assuming you have already constructed a ZATCA::UBL::Invoice
response = client.compliance_check(
  invoice: invoice.to_base64,
  invoice_hash: invoice.generate_hash,
  uuid: invoice.uuid
)

puts response

[obahareth]

Hey @ec-ecss,

Could you share the value of invoice.qualifying_properties after signing?

[ec-ecss]

I fixed my problem on remove all arabic characacters and keep only alphabetics chars

[obahareth]

@ec-ecss Could you share what part you had those characters in? It could be either something we need to change on our end, or it could be something unsupported (e.g. some fields in certificates don't allow UTF-8).

[ec-ecss]

Consider I have remove every arabic occurence in the example in my first post, to keep only alphabetic ones.
street name, city name,
Item.new (2 occurences)
If one is present the error occurs "invalid byte sequence in US-ASCII"
Maybee it's a OS unicode library problem ? (I use debian bullseye)