asn1tools.codecs.EncodeError: enumeration value

[URockMan]

I'm facing below error when im trying to pass to enumerated value.
`
ASN.1 specification:

-- A simple protocol taken from Wikipedia.

Foo DEFINITIONS ::= BEGIN

Question ::= SEQUENCE {
    id        Tp4table,
    question  IA5String
}

Answer ::= SEQUENCE {
    id        INTEGER,
    answer    BOOLEAN
}

    Tp4table ::= ENUMERATED
    {
    labelA  (0),
    labelB  (1)
    }

END`

Input : question = {'id': 1, 'question': 'What is this problem'}

Error :asn1tools.codecs.EncodeError: enumeration value '1' not found in ['labelA', 'labelB']

[eerimoq]

Well, the error message is quite clear, one of the strings "labelA" and "labelB" are expected, not a numerical value.

Though, it might be handy if the package allows numerical values as well.

[URockMan]

Thanks for your reply. When we are testing in below site it was expecting numerical values so i was trying with numerical values.

Link: http://asn1-playground.oss.com/

[alexroat]

IS it possible to extract the Enumeration Mapping from a compiled .asn file for a specific enumerated type in order to automatically translate the numerical value to the string ones? I've tried to dig into the stub classes but i was unable to extract the mapping. I think it should be stored somewhere in a property of the enumerated type but I don't know where.
Could you provide an example?

[eerimoq]

There is no public API to get this information, sorry. I don't recommend anyone to use internal stuff that may be removed at any point without notice.

[alexroat]

Inspecting the classes i've found that the mapping is in the root_index_to_name and root_name_to_index attributes of the type.

It should be possible add a index2index mapping acting on these maps also after the compilation (that is without modifying the asn1tools sourcecode) throught an "fixer function"

print etsi.types["RoadType"].type.root_name_to_index

#out: {'urban-NoStructuralSeparationToOppositeLanes': 0, 'urban-WithStructuralSeparationToOppositeLanes': 1, 'nonUrban-WithStructuralSeparationToOppositeLanes': 3, 'nonUrban-NoStructuralSeparationToOppositeLanes': 2}


def add_numeric_enumeration_mapping(asn):
    for k,v in asn.types.items():
        if (v.type.type_name=="ENUMERATED"):
            mm=v.type.root_name_to_index
            for i in mm.values():
                mm[i]=i
    return asn

add_numeric_enumeration_mapping(etsi)




print etsi.types["RoadType"].type.root_name_to_index

#out: {0: 0, 1: 1, 2: 2, 3: 3, 'nonUrban-NoStructuralSeparationToOppositeLanes': 2, 'nonUrban-WithStructuralSeparationToOppositeLanes': 3, 'urban-NoStructuralSeparationToOppositeLanes': 0, 'urban-WithStructuralSeparationToOppositeLanes': 1}

This should prevent the error in the topic of this thread make the encoder working with namestring or indexes.

[alexroat]

There is no public API to get this information, sorry. I don't recommend anyone to use internal stuff that may be removed at any point without notice.

I agree, however i need this feature beacause my source is sending all the enumerated values in a numeric fashion. My solution should prevent me to traverse the object tree and re-encode all the enumerated values.

[alexroat]

mmm... there is another type check:

......Expected data of type str, but got 1

Anyway, I've found a way to get the mapping, I'm close to a solution :)

[eerimoq]

We could possibly add an optional parameter to compile_files(), like numeric_enums=False, to force numeric enumeration values instead of strings, both when encoding and decoding.

[alexroat]

We could possibly add an option to compile_files(), like numeric_enums=False, to force numeric enumeration values both when encoding and decoding.

this is a great idea!
Of course also for the decoding so the decoded object is in the same form of the source.

I'm trying to apply a sort of real-time patching of the encode method in order to obtain that ... the problem is that the Enumerated.encode is called after the String.encode one ...
It seems, if I'm correctly understand, that the argument is first parsed as type_checker string and the it tryes to convert it to a enumerated .... the problem is that the string is not allowing integer as input so it raises a parsing error.

[eerimoq]

@alexroat Try asn1tools version 0.135.0, available on PyPI, and compile as compile_files(..., numeric_enums=True).

[alexroat]

Wonderful!
It works :)
Thanks a lot

#### INPUT ####
{
  "denm": {
    "alacarte": {}, 
    "location": {
      "traces": [
        [
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": -14364, 
              "deltaLongitude": 4223
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2414, 
              "deltaLongitude": -721
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2416, 
              "deltaLongitude": -716
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2416, 
              "deltaLongitude": -716
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2418, 
              "deltaLongitude": -710
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2420, 
              "deltaLongitude": -701
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2420, 
              "deltaLongitude": -701
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2420, 
              "deltaLongitude": -701
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2420, 
              "deltaLongitude": -701
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2420, 
              "deltaLongitude": -701
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2420, 
              "deltaLongitude": -701
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2420, 
              "deltaLongitude": -701
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2420, 
              "deltaLongitude": -701
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2420, 
              "deltaLongitude": -701
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2420, 
              "deltaLongitude": -701
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2421, 
              "deltaLongitude": -699
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2421, 
              "deltaLongitude": -699
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2421, 
              "deltaLongitude": -699
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2421, 
              "deltaLongitude": -699
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2421, 
              "deltaLongitude": -699
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2421, 
              "deltaLongitude": -699
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2421, 
              "deltaLongitude": -699
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2421, 
              "deltaLongitude": -699
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2421, 
              "deltaLongitude": -699
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2421, 
              "deltaLongitude": -699
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2421, 
              "deltaLongitude": -699
            }
          }
        ]
      ]
    }, 
    "management": {
      "DestArea": {
        "Latitude": 461064172, 
        "Longitude": 110892373
      }, 
      "__v": 0, 
      "_id": "5bd9d9da9863ffbc8f4fb2e2", 
      "actionID": {
        "originatingStationID": 123456, 
        "sequenceNumber": 98765
      }, 
      "detectionTime": 468499984714, 
      "enabled": true, 
      "eventPosition": {
        "altitude": {
          "altitudeConfidence": 15, 
          "altitudeValue": 800001
        }, 
        "latitude": 461064172, 
        "longitude": 110892373, 
        "positionConfidenceEllipse": {
          "semiMajorConfidence": 4095, 
          "semiMajorOrientation": 3601, 
          "semiMinorConfidence": 4095
        }
      }, 
      "geometry": [
        11.089237299999999, 
        46.106417199999996
      ], 
      "location": {
        "traces": [
          [
            {
              "pathPosition": {
                "deltaAltitude": 12800, 
                "deltaLatitude": -14364, 
                "deltaLongitude": 4223
              }
            }, 
            {
              "pathPosition": {
                "deltaAltitude": 12800, 
                "deltaLatitude": 2414, 
                "deltaLongitude": -721
              }
            }, 
            {
              "pathPosition": {
                "deltaAltitude": 12800, 
                "deltaLatitude": 2416, 
                "deltaLongitude": -716
              }
            }, 
            {
              "pathPosition": {
                "deltaAltitude": 12800, 
                "deltaLatitude": 2416, 
                "deltaLongitude": -716
              }
            }, 
            {
              "pathPosition": {
                "deltaAltitude": 12800, 
                "deltaLatitude": 2418, 
                "deltaLongitude": -710
              }
            }, 
            {
              "pathPosition": {
                "deltaAltitude": 12800, 
                "deltaLatitude": 2420, 
                "deltaLongitude": -701
              }
            }, 
            {
              "pathPosition": {
                "deltaAltitude": 12800, 
                "deltaLatitude": 2420, 
                "deltaLongitude": -701
              }
            }, 
            {
              "pathPosition": {
                "deltaAltitude": 12800, 
                "deltaLatitude": 2420, 
                "deltaLongitude": -701
              }
            }, 
            {
              "pathPosition": {
                "deltaAltitude": 12800, 
                "deltaLatitude": 2420, 
                "deltaLongitude": -701
              }
            }, 
            {
              "pathPosition": {
                "deltaAltitude": 12800, 
                "deltaLatitude": 2420, 
                "deltaLongitude": -701
              }
            }, 
            {
              "pathPosition": {
                "deltaAltitude": 12800, 
                "deltaLatitude": 2420, 
                "deltaLongitude": -701
              }
            }, 
            {
              "pathPosition": {
                "deltaAltitude": 12800, 
                "deltaLatitude": 2420, 
                "deltaLongitude": -701
              }
            }, 
            {
              "pathPosition": {
                "deltaAltitude": 12800, 
                "deltaLatitude": 2420, 
                "deltaLongitude": -701
              }
            }, 
            {
              "pathPosition": {
                "deltaAltitude": 12800, 
                "deltaLatitude": 2420, 
                "deltaLongitude": -701
              }
            }, 
            {
              "pathPosition": {
                "deltaAltitude": 12800, 
                "deltaLatitude": 2420, 
                "deltaLongitude": -701
              }
            }, 
            {
              "pathPosition": {
                "deltaAltitude": 12800, 
                "deltaLatitude": 2421, 
                "deltaLongitude": -699
              }
            }, 
            {
              "pathPosition": {
                "deltaAltitude": 12800, 
                "deltaLatitude": 2421, 
                "deltaLongitude": -699
              }
            }, 
            {
              "pathPosition": {
                "deltaAltitude": 12800, 
                "deltaLatitude": 2421, 
                "deltaLongitude": -699
              }
            }, 
            {
              "pathPosition": {
                "deltaAltitude": 12800, 
                "deltaLatitude": 2421, 
                "deltaLongitude": -699
              }
            }, 
            {
              "pathPosition": {
                "deltaAltitude": 12800, 
                "deltaLatitude": 2421, 
                "deltaLongitude": -699
              }
            }, 
            {
              "pathPosition": {
                "deltaAltitude": 12800, 
                "deltaLatitude": 2421, 
                "deltaLongitude": -699
              }
            }, 
            {
              "pathPosition": {
                "deltaAltitude": 12800, 
                "deltaLatitude": 2421, 
                "deltaLongitude": -699
              }
            }, 
            {
              "pathPosition": {
                "deltaAltitude": 12800, 
                "deltaLatitude": 2421, 
                "deltaLongitude": -699
              }
            }, 
            {
              "pathPosition": {
                "deltaAltitude": 12800, 
                "deltaLatitude": 2421, 
                "deltaLongitude": -699
              }
            }, 
            {
              "pathPosition": {
                "deltaAltitude": 12800, 
                "deltaLatitude": 2421, 
                "deltaLongitude": -699
              }
            }, 
            {
              "pathPosition": {
                "deltaAltitude": 12800, 
                "deltaLatitude": 2421, 
                "deltaLongitude": -699
              }
            }
          ]
        ]
      }, 
      "location_option": 1, 
      "referenceTime": 0, 
      "relevanceDistance": 4, 
      "relevanceDistance_option": true, 
      "relevanceTrafficDirection": 1, 
      "relevanceTrafficDirection_option": true, 
      "situation": {
        "eventType": {
          "causeCode": 19, 
          "subCauseCode": 1
        }, 
        "informationQuality": 4
      }, 
      "situation_option": true, 
      "stationType": 0, 
      "tccId": "274897_5e26a193-27bb-4901-9c0a-95230d34e961_1", 
      "validityDuration": 600
    }, 
    "situation": {
      "eventType": {
        "causeCode": 19, 
        "subCauseCode": 1
      }, 
      "informationQuality": 4
    }
  }, 
  "header": {
    "messageID": 0, 
    "protocolVersion": 0, 
    "stationID": 123456
  }
}
#### ASN1 UPER  HEX OUTPUT ####
00000001e240e70000f120c0e68da298ce2940000000000512030ec71e5e755ffffffe11dbba1f880960004130100d1c7e3841fb1ce2096d7f4bb1ce2096f7f4cf1ce2096f7f4cf1ce209717f4e71ce209737f50b1ce209737f50b1ce209737f50b1ce209737f50b1ce209737f50b1ce209737f50b1ce209737f50b1ce209737f50b1ce209737f50b1ce209737f50b1ce209747f5131ce209747f5131ce209747f5131ce209747f5131ce209747f5131ce209747f5131ce209747f5131ce209747f5131ce209747f5131ce209747f5131ce209747f5131ce00
#### DECODED ####
{
  "denm": {
    "alacarte": {}, 
    "location": {
      "traces": [
        [
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": -14364, 
              "deltaLongitude": 4223
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2414, 
              "deltaLongitude": -721
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2416, 
              "deltaLongitude": -716
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2416, 
              "deltaLongitude": -716
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2418, 
              "deltaLongitude": -710
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2420, 
              "deltaLongitude": -701
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2420, 
              "deltaLongitude": -701
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2420, 
              "deltaLongitude": -701
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2420, 
              "deltaLongitude": -701
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2420, 
              "deltaLongitude": -701
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2420, 
              "deltaLongitude": -701
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2420, 
              "deltaLongitude": -701
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2420, 
              "deltaLongitude": -701
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2420, 
              "deltaLongitude": -701
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2420, 
              "deltaLongitude": -701
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2421, 
              "deltaLongitude": -699
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2421, 
              "deltaLongitude": -699
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2421, 
              "deltaLongitude": -699
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2421, 
              "deltaLongitude": -699
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2421, 
              "deltaLongitude": -699
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2421, 
              "deltaLongitude": -699
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2421, 
              "deltaLongitude": -699
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2421, 
              "deltaLongitude": -699
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2421, 
              "deltaLongitude": -699
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2421, 
              "deltaLongitude": -699
            }
          }, 
          {
            "pathPosition": {
              "deltaAltitude": 12800, 
              "deltaLatitude": 2421, 
              "deltaLongitude": -699
            }
          }
        ]
      ]
    }, 
    "management": {
      "actionID": {
        "originatingStationID": 123457, 
        "sequenceNumber": 33229
      }, 
      "detectionTime": 468499984714, 
      "eventPosition": {
        "altitude": {
          "altitudeConfidence": 15, 
          "altitudeValue": 800001
        }, 
        "latitude": 461064172, 
        "longitude": 110892373, 
        "positionConfidenceEllipse": {
          "semiMajorConfidence": 4095, 
          "semiMajorOrientation": 3601, 
          "semiMinorConfidence": 4095
        }
      }, 
      "referenceTime": 0, 
      "relevanceDistance": 4, 
      "relevanceTrafficDirection": 1, 
      "stationType": 0, 
      "validityDuration": 600
    }, 
    "situation": {
      "eventType": {
        "causeCode": 19, 
        "subCauseCode": 1
      }, 
      "informationQuality": 4
    }
  }, 
  "header": {
    "messageID": 0, 
    "protocolVersion": 0, 
    "stationID": 123456
  }
}

[alexroat]

PS: just a feature request:
Could it possible to add a "hook" function that is called to preformat the "data" before the encode.
This could be convenient in case there is a not compatible data in order to convert to the input expected by the asn1tools. For example: BIT STRING encoding is expecting to receive a tuple(bytes, int), however could be possible to feed the encoding process passing a string like "0010010101", having a hook function it could be possible to convert a string like this into the required tuple(bytes, int) without having to recursively patch the input.

Of course: we can open a new thread for this.

[eerimoq]

I don't quite understand the benefit of a "hook" function. Isn't it just easier to convert the data before calling encode and after decode returns?

[alexroat]

Yes, but let me explain.
Lets suppose that I use the same ASN type in different part of the input structure hiearchy, for example at a level 0, 7, 4, inside a choice and inside a sequence, all with different names.
Let suppose that my source is sending BitString in the form of "0101010010" instead tuple(bytes,n).
Instead of patching the input structure it should be nice to have an "hook" function that is called by the encoder right before to feed the encode function of BitString type in order to convert the string form into the tuple form.
This should be more maintaneble and clean when writing an encoder.

I've obtained a result like that using a run-time method overriding

Look at this
(ps: here you can see also a patch which convert {glc:{....}} into the form tuple(glc,{....}) for the choice and add a parsing support from string to the integer)

def bitstring2tublebytelen(s):
    n=len(s)
    a2=[ s[_:_+8] for _ in xrange(0,len(s),8)]
    return bytes(struct.pack("B"*len(a2),*map(lambda x: int(x,2),a2))),n



def patchTcChoice():
    oencode=asn1tools.codecs.type_checker.Choice.encode
    def nencode(self, data):
        if isinstance(data,dict):
            data=[(k,v) for k,v in data.items()][0]
        return oencode(self,data)
    asn1tools.codecs.type_checker.Choice.encode=nencode

def patchPerChoice():
    oencode=asn1tools.codecs.per.Choice.encode
    def nencode(self, data, encoder):
        if isinstance(data,dict):
            data=[(k,v) for k,v in data.items()][0]
        return oencode(self, data, encoder)
    asn1tools.codecs.per.Choice.encode=nencode


def patchUperInteger():
    oencode=asn1tools.codecs.uper.Integer.encode
    def nencode(self, data, encoder):
        return oencode(self, int(data), encoder)
    asn1tools.codecs.uper.Integer.encode=nencode


def patchPerBitString():
    oencode=asn1tools.codecs.per.BitString.encode
    def nencode(self, data, encoder):
        if isinstance(data,(str,unicode)):
            data=bitstring2tublebytelen(data)
        return oencode(self, data, encoder)
    asn1tools.codecs.per.BitString.encode=nencode



def patchTcBitString():
    oencode=asn1tools.codecs.type_checker.BitString.encode
    def nencode(self, data):
        if isinstance(data,(str,unicode)):
            data=bitstring2tublebytelen(data)
        return oencode(self, data)
    asn1tools.codecs.type_checker.BitString.encode=nencode


patchTcChoice()
patchUperInteger()
patchPerChoice()
patchPerBitString()
patchTcBitString()

[eerimoq]

I think you'll have to continue patching asn1tools as you do today, sorry.

[alexroat]

I think you'll have to continue patching asn1tools as you do today, sorry.

Yes. Finally I've found a way that is clean and effective enough :) It works great making asn1tools tollerant about the input syntax. Thanks for the support Erik, asn1tools is a great piece of software.