Import from TeslaFi fails if vehicle name contains international characters

[miksen]

Describe the bug

Attempted to import my TeslaFI export. It failed with:

teslamate_1  | 06:39:51.377 [info] Importing 11 file(s) ...
teslamate_1  | 06:39:51.383 [error] GenStateMachine TeslaMate.Import terminating
teslamate_1  | ** (FunctionClauseError) no function clause matching in CSV.Decoding.Preprocessing.Lines.starts_sequence?/5
teslamate_1  |     (csv 2.3.1) CSV.Decoding.Preprocessing.Lines.starts_sequence?(<<248, 100, 101, 32, 79, 114, 109, 34, 44, 78, 111, 110, 101, 44, 78, 111, 110, 101, 44, 44, 53, 89, 74, 51, 69, 55, 69, 66, 57, 75, 70, 51, 56, 50, 52, 54, 56, 44, 78, 111, 110, 101, 44, 54, 55, 57, 56, 50, 50, 49, ...>>, "R", true, 44, <<82, 248, 100, 101, 32, 79, 114, 109, 34, 44, 78, 111, 110, 101, 44, 78, 111, 110, 101, 44, 44, 53, 89, 74, 51, 69, 55, 69, 66, 57, 75, 70, 51, 56, 50, 52, 54, 56, 44, 78, 111, 110, 101, 44, 54, 55, 57, 56, 50, 50, ...>>)
teslamate_1  |     (csv 2.3.1) lib/csv/decoding/preprocessing/lines.ex:85: CSV.Decoding.Preprocessing.Lines.start_sequence/3
teslamate_1  |     (elixir 1.10.3) lib/stream.ex:902: Stream.do_transform_user/6
teslamate_1  |     (elixir 1.10.3) lib/stream.ex:1609: Enumerable.Stream.do_each/4
teslamate_1  |     (elixir 1.10.3) lib/stream.ex:880: Stream.do_transform/5
teslamate_1  |     (elixir 1.10.3) lib/stream.ex:1609: Enumerable.Stream.do_each/4
teslamate_1  |     (elixir 1.10.3) lib/stream.ex:880: Stream.do_transform/5
teslamate_1  |     (elixir 1.10.3) lib/stream.ex:1609: Enumerable.Stream.do_each/4

Expected behavior

Import should succeed.

How to reproduce it (as minimally and precisely as possible):

Use a TeslaFi export from a vehicle containing an international character. For example, mine is named Røde Orm, which contains the Danish ø character. In TeslaFi exports this shows up like:

153325,"2020-01-01 00:00:53",1,,1789577095,"R�de Orm",,<invalid>,,5YJ3E7EB9KFxxxxxx,...

After search-replacing R�de Orm with Rode Orm my import completed successfully.

Environment

• TeslaMate version: 1.19.4
• Type of installation: Docker
• OS TeslaMate is installed on: macOS
• User OS & Browser: macOS & Chrome
• Others:

[BabyYeggie]

You should delete/obfuscate your VIN for privacy purposes. The numbers after 5YJ3E7EB9KF3 are unnecessary.

[adriankumpf]

I'm unable to reproduce the error. Could you provide a sample file (you can obfuscate the vin, coordinates etc)?

[techgaun]

Looks like this may no longer be an issue with the switch from CSV package to NimbleCSV based implementation.

[adriankumpf]

I couldn't even reproduce it with the previous CSV based implementation. But it may well be that this is no longer an issue with either implementation.