Cannot decode byte '\xe0'

[dongsin-kim]

Hi, @zoranbosnjak and @dsalantic

I would like to express my gratitude for this wonderful library-! Based on these instructions, I am trying to convert .ast file to .json file.

When I use a command, I am facing an error as follows:
>> aspecs validate -f tp1_20191221_220000_Cat20.ast --ast --warnings
aspecs: Cannot decode byte '\xe0': Data.Text.Internal.Encoding.decodeUtf8: Invalid UTF-8 stream

Should I modify an internal source code of the aspecs ...?

[zoranbosnjak]

It looks like the problem is unicode related. I can not yet tell if the source code need any modification. I will try to reproduce the problem first.

Please attach:

• The input ast file tp1_20191221_220000_Cat20.ast
• what linux distro and version your are working on?
• Output of the locale command

Do you get the same problem if you try with one of the original .ast files from the project?

Have you built the aspecs tool yourself or you are using the pre-built binary from the project's web page?

[dongsin-kim]

Hi @zoranbosnjak, thank you for your swift reply. I am so sorry for bothering you. Please find each answer as follows:

---

• The input ast file tp1_20191221_220000_Cat20.ast

Oh, it is definitely a good idea to reproduce the issue. Well... this is a multilateration data of flights in South Korea. Unfortunately, I have signed up for a confidentiality agreement for this .ast file. So I do not.. currently have permission to upload it on Github.. :(

---

• what linux distro and version your are working on?

>> cat /etc/os-release

NAME="Ubuntu"
VERSION="18.04.5 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.5 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic

>> uname -r
4.15.0-132-generic

---

• Output of the locale command

>> locale

LANG=en_US.UTF-8
LANGUAGE=
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

---

• Do you get the same problem if you try with one of the original .ast files from the project?

No, everything is perfect with the original .ast file from the project. I downloaded asterix-specs-master.zip, browsed to a directory (asterix-specs-master/specs/cat020), and tested with cat-1.9.ast file.

>> aspecs validate -f cat-1.9.ast --ast --warnings
"ok"
>> aspecs checksum -f cat-1.9.ast --ast
d6681e5ac1639cf2f2c88af77e13af18a77bc33f
>> aspecs convert -f cat-1.9.ast --ast --json > out.json
(I could get a out.json which is working perfect)

---

• Have you built the aspecs tool yourself or you are using the pre-built binary from the project's web page?

I used a pre-built binary (aspecs-static.exe) from the project's web page. :)

---

I could find that both files have the same extension (.ast). However, the difference is that tp1_20191221_220000_Cat20.ast is not a human readable file.. (it seems that it is a hex data.) Well.. please let me ask this issue to an organization in South Korea.

[zoranbosnjak]

@dongsin-kim
The .ast and .json files in the context of this project are the asterix category description files. The aspecs tool can convert/validate the descriptions. You would need it only in case if you want to define a new asterix category or edition.

It looks like you have the actual data file. That is: data that came from the sensors. It maybe has the .ast file extension, but it's not the same kind of file. For this kind of processing, you need an asterix decoder. See for example projects like:

• https://github.com/zoranbosnjak/deseo
• https://github.com/CroatiaControlLtd/asterix

Both projects above are generic to process any asterix category, including category 020 (multicateration), as long as the category description is available. And this (the category descriptions) is what this project provides.