Repository navigation
Sources And Encodings
Source.Source says where one file's text comes from:
| Variant | Reads |
|---|---|
File(path) |
a file, located in directory when relative |
Text(label, text) |
text the program already holds |
Data(label, bytes) |
bytes decoded in the options' encoding |
withSources(&[...]) reads them in order instead of discovering files; each is one layer. The label is what a problem names.
Bytes are decoded as Encoding.Utf8 unless withEncoding says Ascii, Latin1, Utf16Le, Utf16Be, Utf32Le, or Utf32Be — import PuduLangEnvironment.Domain.Encoding for the names. A UTF-8, UTF-16, or UTF-32 byte-order mark always wins and is dropped, and a text source that starts with one loses it too. Bytes that do not decode are refused with Undecodable rather than read with substitute characters, because a secret with a substituted character is a wrong secret:
module Ex09Encodings
import Std.Bytes as Bytes
import Std.Io as Io
import Std.Map as Map
import Std.Result as Result
import PuduLangEnvironment as Environment
import PuduLangEnvironment.Domain.Encoding as Encoding
import PuduLangEnvironment.Loader as Loader
import PuduLangEnvironment.Options as Options
import PuduLangEnvironment.Source as Source
fn attempt(label: Str, hex: Str, encoding: Encoding.Encoding) -> () {
let data = Result.unwrapOr(Bytes.decodeHex(hex), Bytes.empty())
let options = Options.defaults().withSources(&[Source.Data(label, data)]).withEncoding(encoding).withStrict()
let line = match Loader.readOver(&options, &Map.empty()) {
case Ok(values) => label + ": " + show(values.get("CITY"))
case Err(problem) => label + ": " + Environment.describe(&problem)
}
let _shown = Io.writeLine(line)
}
export fn main() -> Int {
attempt("utf-8", "434954593d4dc3a16c616761", Encoding.Utf8)
attempt("latin-1 bytes", "434954593d4de16c616761", Encoding.Latin1)
attempt("utf-16 with mark", "fffe43004900540059003d004d00e1006c00610067006100", Encoding.Utf8)
attempt("utf-32 with mark", "fffe0000430000004900000054000000590000003d0000004d000000e10000006c000000610000006700000061000000", Encoding.Utf8)
attempt("ascii", "434954593d4d616c616761", Encoding.Ascii)
attempt("latin-1 read as ascii", "434954593d4de16c616761", Encoding.Ascii)
attempt("latin-1 read as utf-8", "434954593d4de16c616761", Encoding.Utf8)
0
}
Output:
utf-8: Some("Málaga")
latin-1 bytes: Some("Málaga")
utf-16 with mark: Some("Málaga")
utf-32 with mark: Some("Málaga")
ascii: Some("Malaga")
latin-1 read as ascii: The environment source 'latin-1 read as ascii' is not valid ASCII.
latin-1 read as utf-8: The environment source 'latin-1 read as utf-8' is not valid UTF-8.