spatial and other semantic encoding...

[EvaMaeRey]

Just want to collect some stuff here for discussion, as work on ggsql's spatial capabilites are underway as well as ggregions, and it might be good to compare notes, spatial expertise, pitfalls, solutions, excitement 🎉!

I think a central motivating idea is, 'when I say "North Carolina", I should 'get' (see) North Carolina'... ❓🤷‍♀️ aka 'dish out some simple chloropleths.'?

From @teunbrand on 'geometry scale' posit-dev/ggsql#453:

... a scale task to translate names to shapes. It'd be a scale with fixed breaks (country names, country codes, state names, state abbreviations), a palette of geographical shapes , which outputs a geometry column. The detail level would just be a property of the palette. I don't ggplot2 easily allows this approach, as scales only map data after position scales already (should) have taken stock of all the bounding boxes and such. But in ggsql we're not hindered by backwards compatibilities :)

It also felt like @mdsumner's (hello!👋🙂) comment about ggXspatial might be related... ?

I long thought gg was perfect for construction of spatial data ie group_by, arrange is how you create lines (or animal tracks). It doesn't really work for polygons because no one stores them that way but they do decompose polygons that way for validity and geometric ops..Anyhoo no coherent or long thoughts here but I'd appreciate any explorations 🙏this project is surprising and not in many ways. There's a marriage of gg and spatial that got buried under the sf era , and I thought euclid would uncover that. Spatial primitives at least fit very neatly here

So, ggregions does introduces 'region' and 'geom_region', I think with the same motivations in play - but with the ggplot2 constraints.

tribble(~state, ~info,
        "Texas", TRUE,
        "NC",    FALSE) |>
  ggplot() + 
  aes(region = state,
      fill = info) + 
  geom_region(ref_data = us_states_ref) # reference data from which to get geometries, based on other columns of unique identifiers, state names, state abbreviations. coords is set baseed on crs from this ref data.

It sounds like the 'fixed breaks' in ggsql will help with the task of providing context automatically, like it would draw all the other states boundaries for example - sounds really nice! 🤩 (is that right?) ggregions approach to this has been to provide stamp_region - so another layer.

tribble(~state, ~info,
        "Texas", TRUE,
        "NC",    FALSE) |>
  ggplot() + 
  aes(region = state,
         fill = info) + 
  stamp_region(ref_data = us_states_ref) +
  geom_region(ref_data = us_states_ref)

Also, instead of defining the ref data locally, you can set options with set_regions() so that the ref data is automatically used across ggregion layers.

set_regions(us_states_ref)

tribble(~state, ~info,
        "Texas", TRUE,
        "NC",    FALSE) |>
  ggplot() + 
  aes(region = state,
         fill = info) + 
  stamp_region() +
  geom_region()

[teunbrand]

The stamp_regions() seems similar in some way to annotation_borders("state"), what do you reckon the user facing differences are?

It sounds like the 'fixed breaks' in ggsql will help with the task of providing context automatically

No that isn't the aim. The idea (translated to R) is that you have a mapping geometry = c("Uganda", "India", ...) that then will be match()'ed to scale/palette specific breaks (universe of country names) and the column will then become a proper geometry column with polygons and whatnot. So if this were ggplot2, it'd be scale_geometry_countries() or scale_geometry_us_states() and the like. I think there is some discussion to be had on what to match (country names, codes, whatnot?)

W.r.t annotation/context; the idea I had for ggsql is that context would work like bounding-box-inert pieces of geometry that fill in the edges around the true user-data-geometry. So you plot North Carolina, and you only see pieces of neighbouring states that happen to overlap the bounding box of the plot, not all of the states. Another idea is that one might have annotation_oceans(), annotation_coastlines() or annotion_landmass() or other derived geometry that give contexts.

[EvaMaeRey]

Ah. Then intent is what I first understood (translation), but then 'fixed' made me thing of limit setting like this in ggplot2...

ggplot() +
aes("texas") +
geom_bar() +
scale_x_discrete(limits = c("texas", "north carolina"))

leaving a space for 'north carolina'. Scale thinking feels a little slippery to me - would like to be more fluent! Would you be able to do this kind of limit setting in ggsql spatial, such that faint north carolina appears with analogous scale specification?

I think the main thing about stamp_region() is that it's anticipated to be used with geom_region() - and then using the set_regions() function, you'd also not need to specify that they are the same. Users are anticipated to bring their own data a data.frame input rather than predefined borders that ggplot2 provides.

[teunbrand]

What is that data you're using for the US there? Is Alaska, Hawai, Puerto Rico(?) moved there in the data or is it some projection that carves out these places? And is Alaska size-proportional?

me thing of limit setting like this in ggplot2...

I don't think that is too far off as limits propagate to breaks. I think the 'providing context automatically' threw me off what you meant before.

[EvaMaeRey]

Alaska is not proportional, and likely neither is Puerto Rico or Hawaii. I remember this as called out as 'bad' in 'fundamentals of data visualization'. Somehow, though, I find myself unbothered by a miniaturized and moved Alaska. 😬

The us_states_ref in the above workflow is created from usmapdata::us_map().

us_states_ref <- usmapdata::us_map() |>
  select( 
    state_name = full, # first variable will also be 'id' var, keep and drop
    state_abbr = abbr, 
    state_fips = fips,
    geometry = geom,  # one column 'geometry' is required
         )  

I suspect usmapdata::us_map moves the non-continental regions instead of doing projection carve-outs, though not certain of this. geom_region is like geom_sf in that it also bundles up coord_sf alongside layer specification and will project according to the CRS of the data. Do you have a US states w projection carve-outs go-to? I guess that'd be better for combining w/ cities for example, where you'd have true, unmoved coords.

I don't think that is too far off as limits propagate to breaks.

That sounds great!! 🚀

[EvaMaeRey]

Having checked out annotate_borders and seeing usage with us.cities, I just added geom_locale and friends. I'd wanted to do for bit but was I guess waiting to come across some easy example data. geom_locale and friends are essentially the same as geom_regions, but refers to different referenced positional aesthetic. Since GeomSf supports both, and the reference data contains point sf Geometry, feels easy to translate and to use. 🤞 I think geom_way() (streets, rivers) and others might be good to add, as well as geom_metaregion (state borders, when region references counties for example, or country borders when region references sub-country areas like states).

nc_cities_ref <- maps::us.cities |> 
  filter(country.etc == "NC") |> 
  mutate(name = str_remove(name, " NC")) |> 
  sf::st_as_sf(coords = c("long", "lat"), agr = "constant") |> 
  select(name, geometry)

sf::st_crs(nc_cities_ref) <-  "NAD27"

# Step 1 Declare reference data
set_regions(regions = nc_ref, 
            locales = nc_cities_ref)

nc_cities_name_pop |> # data with only city name and pop value
  ggplot() + 
  stamp_region() +
  aes(locale = city_name) + 
  geom_locale(shape = 21, color = "steelblue") + 
  geom_locale_label(size = 2,
                    data = . %>% filter(pop > 200000)) + 
  aes(size = pop) + 
  scale_size(range = c(5,10))

[teunbrand]

I've made a barebones POC of what a geometry scale could look like in ggplot2: tidyverse/ggplot2#6896

[teunbrand]

I'd anticipate people working with rather clean reference data if they are mapping with any regularity.

The spatial people might have more elegant ways to deal with this, but my novice approach would be to just pull in natural earth data. NE datasets have north of 100 columns, so I'd recon that 'clean' in the sense of 'it is obvious which column to use as ID' doesn't neatly apply there.

[EvaMaeRey]

sf can probably be bended to give a bbox,crs, number of parts/verts etc but it's not really the right tool for this sophisticated project that requires reasoning about entities

Hi @mdsumner! I think this is an important and interesting perspective. I wonder if some of my experiences with region data is relevant - like I've worked with some sports fields and human anatomy data that has regions but the curators haven't conceived of it as 'sf'. But ggregions assumes it will be sf, so I do the conversion so I can use the semantic, name-the-place--draw-the-place, API. It does feels a bit circular - there's more structure (crs, bounding boxes) than is really required by the data. Is that an illustration of this critique, or am i misunderstanding?

https://github.com/EvaMaeRey/ggregions

[EvaMaeRey]

The spatial people might have more elegant ways to deal with this, but my novice approach would be to just pull in natural earth data. NE datasets have north of 100 columns, so I'd recon that 'clean' in the sense of 'it is obvious which column to use as ID' doesn't neatly apply there.

NE datasets certainly do have a lot of cols, yes! So, what I'm proposing (using the scale_geometry proposal) is being able to do the following...

world_ids_geometries <- rnaturalearth::ne_countries() |> select(iso_a3, iso_a2)  #  this will keep geometry column, so no need to type that 

# 1:1 mapping using three-letter country codes
df_countries_iso_a3 <- data.frame(id = c("AUS", "NZL", "PNG", "IDN"))

df_countries_iso_a3 |>
  ggplot() +
  aes(geometry = id, fill = id) +
  geom_sf() +
  scale_geometry_discrete(world_ids_geometries)

or alternatively...

# 1:1 mapping using three-letter country codes
iso_a3 <- data.frame(id = c("AUS", "NZL", "PNG", "IDN"))

df_countries_iso_a3 |>
  ggplot() +
  aes(geometry = id, fill = id) +
  geom_sf() +
  scale_geometry_discrete( rnaturalearth::ne_countries() |> select(iso_a3))

'recon'

😅

In both NE and nc.shp there are id columns, geometries, and features (e.g. BIR74, pop_est). It feels like features should be dropped (and like, I just wonder why they are there in the first place - I think ease of demonstration?)

sf::st_read(system.file("shape/nc.shp", package="sf")) |> 
   tibble()

If data 'cleaned' by dropping features, then id columns could be checked for uniqueness, requiring the second argument ("iso_a3") if uniqueness isn't met?

[EvaMaeRey]

Wanted to highlight a new brings-its-own-data semantic spatial (but anatomocal) encoding project to this thread: https://github.com/Sanofi-Public/ggcolonviz/ from @yonicd 🙂👀

A very specific regional focus ('part' is the aesthetic name, contrasting with region/geometery). And polygons are the regional Geom instead of Sf. Followed a ggregions (bring your own data) X teethr demonstration.

In discussion with @mdsumner, I think getting away from SF (and instead using polygons?) has been part of the discussion... I've thought about that, but no implementation w/ ggregions here-to-fore, point 1: https://evamaerey.github.io/mytidytuesday/2026-04-03-r-medicine-ggregions/asa-cowy-ggregions.html#104

The non-choropleth fills also seem like they might be of interest for this crowd and I believe @jespermaag brought this idea up (not tackled by ggregions) when I experiemented w/ ggregions X gganatogram.

First example:

Heatmap:

[EvaMaeRey]

ggregionsXteethr::dental_arcade_mapping api, for comparison...

https://github.com/EvaMaeRey/ggregions#dental-example-teethr

[mdsumner]

I'd look at wk as sf compatible but less rigid on "being geospatial". Some interesting stuff here I haven't really thought about what's being said enough That heat map is reminiscent of SOAP film smoothing, in mgcv I think, but another route is mesh triangulation and barycentric interpolation (which can happen on the fly visually with GPU)

[EvaMaeRey]

That heat map is reminiscent of SOAP film smoothing, in mgcv I think, but another route is mesh triangulation and barycentric interpolation (which can happen on the fly visually with GPU)

So much here... 🫠🙏🙃 Thanks for the link below!

[mdsumner]

I'm working on a better support for polygon triangulation in a few packages with (heavy)llm support: https://mdsumner.github.io/trianglewins/

[EvaMaeRey]

https://github.com/tidyverse/ggplot2/blob/poc_geometry_scale/R/scale-geometry.R

ScaleDiscreteGeometry <- ggproto(
"ScaleGeometry", ScaleDiscreteIdentity,
transform = function(self, x) {
self$trans(x)
}
)

@teunbrand I finally had a look at the scale_geometry work! 🤩🙌 :-)

I just have a really quick question.. I'm stuck on ScaleDiscreteGeometry and ScaleGeometry ... I just really expected the names to be identical. It is challenging what I think I know about everything! 🙃 Is this intentional?

[teunbrand]

Ah well, you have just exposed my sloppiness in putting together a POC PR 😁
The class name should be the same as the object: ScaleDiscreteGeometry I think.

[EvaMaeRey]

I was just curious, and glad that the convention holds for scales. 😮‍💨🙏

[EvaMaeRey]

I was actually looking at it as a potential guide to a scale_fill_semantic() utility which would do translation between categories and pick appropriate colors from a larger palette... so a scales approach to #151 but I think it doesn't quite 'map' to this other space 🙃🗺️ But liked taking a look. I like the needle/haystack functions.

[EvaMaeRey]

Hearing about @willgearty's scale_fill_geopattern 🙌🚀🤩🙏🙏, I wondered if this is in neighboring space. I guess all scales for me look a little bit alike, because I haven't done a ton of scale specification (I'm not color confident, which seems like a prerequisite for a lot of extension scale work).

Anyway, it seems like there's a lookup table that scale_fill_geopattern uses which helps translate from numeric id (maybe name too) to pattern, and scale_geometry() translates from id to to boundaries (or other geometry like coordinates, paths)... Could we say that scale_fill_geopattern is a semantic encoder? (Can we say that of scale_geometry?)

And the more non-deterministic translations discussed in #151 translators from category -> color, assisted by LLMs (maybe MALL) would have a slightly different configuration, where you'd have a vector of items (menu) from which to select from instead of a lookup table.

I guess I think a scale_translate_deteministic(aesthetics = "fill, lookup_table = id_geopattern_df, etc, etc )... would deliver the appropriate translation (as scale_fill_geopattern already does).

And scale_translate_stochastic(aesthetics = "color" , to_menu = c("olivedrab", "bisque", "saddlebrown", "plum3"), translation_method = mall::llm_vec_classify, etc, etc )... would 🤞 translate aes(color = c("frog", "horse")) to rendered "olivedrab" and "saddlebrown"

Maybe these scales sit between scale_identity and scale_discrete, cause that their identity is informative, and the 'vanilla' discrete scale ignores pretty much strips away identity?