postal.parser.parse_address() returns a list instead of a dict

[antimirov]

Just began using libpostal and decided to try its python binding.

Whats I immediately noticed is that comparing to the command-line 'src/address_parser' the corresponding python function return a list. Compare:

src/address_parser:

1010 EASY ST OTTAWA K1A 0B1

Result:

{
"house_number": "1010",
"road": "easy st",
"city": "ottawa",
"postcode": "k1a 0b1"
}

python postal:

parse('1010 EASY ST OTTAWA K1A 0B1')
[(u'1010', u'house_number'), (u'easy st', u'road'), (u'ottawa', u'city'), (u'k1a 0b1', u'postcode')]

I could not find a way to output the data in a more organized, JSON way. What's the reason it's done like that? Is it always guaranteed to have a value as a first element, and a key as a second?

Please advise.

[jdemaris]

Easiest route from my experience is:

{k: v for (v, k) in parse_address('330 11th Ave Apt 1')}

[antimirov]

Yes, that's what I did. What I first did not see was that the order of the output tuple was always the same. The issue is solved, thank you!

[albarrentine]

Hi @antimirov, yes, the dictionary comprehension should do the trick.

As a quick background on why it's formatted this way, the underlying sequence model takes an array of tokens like this:

['1010', 'easy', 'st', 'ottawa', 'k1a', '0b1']

and predicts an equal-length array of labels, one per token, like this:

['house_number', 'road', 'road', 'city', 'postcode', 'postcode']

For convenience, we roll those up into phrases wherever there's a contiguous sequence of the same label. However, a dictionary would make an implicit assumption that there's only one of each address component.

Most of the time, and for most components, this is true. The output roughly corresponds to fields that one might store in a database or search index. It's usually a parser mistake when there are duplicate components, but in, say, India, the UK, Ireland, etc. it's possible to see something like "{venue/company name} {road} {house name}" in which case there might be two different legitimate components tagged house. While a dictionary is more convenient to work with, we try not to lose any information from the response, so it was either a dictionary of lists or a list of tuples. Decided on the latter as it used less memory and can be converted easily to a dict as needed.

Note: the address parser client from the C library prints something that looks like JSON, but it's not necessarily valid, as there can be duplicate keys. It's just easier to visualize the output as pseudo-JSON on the command line.

[adriangb]

Would it be possible to return the tuples in the opposite order? If the keys come first you can do dict(parse_address('330 11th Ave Apt 1')) which is both more succinct and performant.

On a related topic, it might be nice to think of this as a multi-dict. This is pretty common in the web world, e.g. for headers.
It's essentially a data structure with both references to the dictionary and list of tuples. When accessed as a dictionary duplicate keys are either dropped or merged. When iterated over duplicates are preserved.