encrichment tables removes support for GeoIP-ASN and other databases (Regression)

[splitice]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Problem

Previously with the GeoIP transform GeoIP ASN (and other) databases were compatible. While the documentation mentions this limitation suboptimal for everyone already using the GeoIP transform.

Could this perhaps be prioritised for 0.27.0 so that we may rejoin latest?

Configuration

Old:

[transforms.http_access_log_geoip]
  type = "geoip"
  inputs = [ "http_access_log" ]
  database = "/opt/GeoLite2-ASN.mmdb"
  source = "client_ip"
  target = "geoip"

Version

0.25.0 - 0.26.0

Debug Output

N/A

Example Data

N/A

Additional Context

N/A

References

No response

[jszwedko]

Hi @splitice !

ASN databases should work, are you having any issues using them? I tested with this config:

[enrichment_tables.geoip_table]
type = "geoip"
path = "/Users/jesse.szwedko/Downloads/GeoLite2-ASN_20221220/GeoLite2-ASN.mmdb"

[sources.source0]
type = "demo_logs"
format = "shuffle"
lines = [
  """{"ip_address": "8.8.8.8"}"""
]
decoding.codec = "json"

[transforms.transform0]
inputs = ["source0"]
runtime = "ast"
timezone = "local"
type = "remap"
source = """
.geoip = get_enrichment_table_record!("geoip_table", { "ip": .ip_address })
"""

[sinks.sink0]
inputs = ["transform0"]
type = "console"
encoding.codec = "json"

And got:

{"geoip":{"autonomous_system_number":15169,"autonomous_system_organization":"GOOGLE","isp":null,"organization":null},"ip_address":"8.8.8.8","source_type":"demo_logs","timestamp":"2022-12-22T22:57:50.246233Z"}

You mentioned the docs said they weren't supported. Do you have a link to that? https://vector.dev/docs/reference/configuration/global-options/#enrichment_tables.geoip mentions ASN as one of the supported databases.

[splitice]

I could have sworn that there was a page of the documentation that only mentioned the City database. However it appears I must have been blind.

Thank you for all your informaion. I will confirm soon once I've given the ASN database a shot with the enrichment VRL functions.

[splitice]

I can confirm everything is working with the ASN database.

I found the page of the documentation that threw me:

Path to the [MaxMind GeoIP2](https://dev.maxmind.com/geoip/geoip2/downloadable) or [GeoLite2 binary city database](https://dev.maxmind.com/geoip/geoip2/geolite2/#Download_Access) file (GeoLite2-City.mmdb). Other databases, such as the country database, are not supported.

enrichment_tables.geoip.path

[jszwedko]

I can confirm everything is working with the ASN database.

I found the page of the documentation that threw me:

Path to the [MaxMind GeoIP2](https://dev.maxmind.com/geoip/geoip2/downloadable) or [GeoLite2 binary city database](https://dev.maxmind.com/geoip/geoip2/geolite2/#Download_Access) file (GeoLite2-City.mmdb). Other databases, such as the country database, are not supported.

enrichment_tables.geoip.path

Aha, thanks for the pointer. I'll see about updating that. Glad to hear it is working!