Release 11

Release 11 has a lot more changes than I originally expected it to. One of my constantly repeated notes across my docs is "Don't try to run a global-scale game", and I've chosen to ignore my own advice and see what it takes to do that anyways with a new 'core' game. It has taken quite a bit of time since the last release, but it's also revealed a lot of optimizations and better approaches that should benefit games of any scale.

I continue to think that my simplest bootstrap process is my best, and have found some fragility in the previously-preferred geomdata/tagsdata file loading process as I make changes and expand on what PraxisMapper could or should do and increase the options available. The next release may well focus on cleaning up that and PbfReader to make it simpler/easier to set up larger instances.

The biggest core change is that I now use TagParser on each Place when loading it to the database the first time, and save all matched styles to the PlaceData table. Pre-tagging styles to Places this way allows the database to match on a given style without loading every geometry and passing that to the C# server. This is the biggest optimization that allows for maptile drawing and searching for specific entries to be dramatically faster.

Core/Server:

• New GameTool: PointPicker lets you ask for a number of individual Cell10s (or 11s, if cellSize is set to 11 instead of 10) inside a given area or polygon. By default, attempts to ensure that a third of the points are in a walkable location (trail or tertiary road assumed to have a sidewalk) EX: can ask for 20 Cell10s inside a Cell8. Potentially useful for quickly determining random spawn points for items or enemies.
• A new PMD file format has been set up to replace the geomdata/tagsdata intermediate files. The new format is a single file, self-contained, significantly smaller, and should be very easy to drop into an existing database to add or update contents. It supports resuming after stopping and limiting results to bounds, handles processing modes on read (full data should always be written). It does not currently support splitting a PBF into 1 file per style entry the way that PBFReader does.
• GeometryTracker is now 2 classes. CellTracker handles storing and casting an arbitrary list of PlusCode cells to Geometry for drawing purposes, and is dramatically faster for this purpose when only tracking cells to be drawn. GeometryTracker specifically handles arbitrary geometry objects and no longer has the AddCell/RemoveCell calls on it because CellTracker is the correct way to handle that.
• A new StyleSet was added. AdminBoundsFilled is split into each possible value for admin bounds and draws them in a static-random color (the color is random based on the places name, but its the same color every time for that particular name)
• AdminBounds, the original style that draws the dashed lines to indicate boundaries, has been adjusted to look a little better.
• AdminBounds and AdminBoundsFilled should pick the primary, complete geography for a country more often, instead of every piece or component of a country's border or administration.
• suggestedMini style has been repurposed: It is now the smallest set of reasonable playable locations for a location-based game (park, university, natureReserve, cemetery, historical, artsCulture). This set can handle a global server in under 25GB of storage space, including indexes and PreTag info.
• suggestedMini style has also had the M de-capitalized now when actually used.
• ArtsCulture now has all of the relevant entries from the Tourism category added to it and is the appropriate gameplay style.
• Tourism is now exclusively lodging, and is no longer a gameplay style. It has also been removed from SuggestedGameplay and SuggestedMini styles.
• MapTiles no longer matches admin bounds with a transparent entry in the mapTiles style set. This should speed up drawing maptiles after data is pretagged, but does mean that loading admin bounds into a database requires a second pass across the source PBF file specifically for them.
• teamColor style was expanded to 8 values (red, green, blue, white, orange, teal, yellow, pink) and now uses color names instead of number IDs
• teamColor style has an outline entry added, which can be used to draw a black border with no fill color for use when you need to draw a thing that you want to appear.
• teamColor outlines are now fixed-size at 5px
• StylePaints can now draw a random color based on an entry's name tag. This is used in AdminBoundsFilled.
• Water polygons processed from shapefiles will now always be drawn under other elements in the mapTiles style.
• SkiaSharp will now draw icons from an internal cache instead of reading the bitmap file every draw.
• Added Place.RandomPoint() to pick a random point inside a GeoArea or Polygon[using the envelope of Polygon, so points may not be perfectly in bounds].
• Added GeometrySupport.GeometryFromWKB(byte[]) to read Well-Known Binary formatted geometry objects.
• Added Places.GetPlacesByData. Use this to query places on a PlaceData key/value pair. This is intended for use with the new feature in Larry to tag-match all places and save the result.
• Added Places.GetPlacesByTags. As above, but searches for existing OSM tags. Doesn't need you to run Larry with the -preTag command, but won't replicate results of any style that uses more than 1 key/value pair, which is most of them.
• Added GeometrySupport.CalculateDrawSizeHint(Geometry, ICollection). This lets you determine the draw size hint for a standalone Geometry object if you know how you want to paint it. Used by the below function.
• Added MapTileSupport.GetPaintOpsForGeometry. This is intended for when you have a Geometry object that isn't a Place (EX: CellTracker.ToGeometry()) and know what style you want to paint it. You can add the results to the list from GetPaintOpsForPlaces() and draw the two together.
• Added ImageStats.FitToImageSize(GeoArea). This leaves the set image size in pixels alone, and changes the area to fit the aspect ratio of the existing pixel size values while fitting the entire requested GeoArea inside those bounds.
• MapTileController can now draw Places from a single matching entry in a StyleSet. In the URL, add the single style you want to draw between the styleSet and zoom entries for Slippy tiles, or after the StyleSet for PlusCode tiles. This is not available for drawing Area data.
• • EX: MapTile/Slippy/adminBoundsFilled/city/10/272/390.png
• Can now call TagParser.PickName() instead of TagParser.GetName() in a plugin to avoid getting errors about references that aren't actually needed.
• allowList moved to config instead of code. You can now add paths to the comma-separated list in appsettings.config to change which paths work without logging in. Useful if you change if MapTiles are or are not allowed by anonymous requests, and for adjusting plugin call access.
• Places.GetPlaces now allows you to search on a tag key or value, and a data key or value. This still uses the spatial index and requires an Area to search. EX: You want to figure out which country covers a Cell10, you could set tagKey = "admin_level" and tagValue = "2".
• GetPlaces() has a new optional flag: skipType. If passed, it will specifically skip items pre-tagged with that styleSet when loading from the database. AdminBounds are usually the largest and most complicated entries available when those are present in the database. This flag is generally a huge increase performance when set and admin bounds are not needed. (EX: the United States relation is 2.2MB by itself, and if the boundary isn't being drawn is a lot of data transferred from the db that isn't needed).
• GetPlaces now sorts by DrawSizeHint instead of area/length. This should be much faster and provide equivalent results, since drawing was re-ordered by DrawSizeHint anyways and most other calls that depend on order only needed First() or Last().
• MapTiles.DrawAreaAtSize() now has a skipType parameter that's passed to GetPlaces. In quick testing, this cuts a 2,000ms call down to 90ms when admin bounds are present on the server and "adminBounds" is passed in.
• GenericData's calls can be cached automatically now. Requires setting "enableServerCaching" to true in appsettings.json to use.
• MapTileSupport (and therefore MapTileController) can now cache map tile data automatically now if "enableServerCaching" in appsettings.json is true. Images will be pulled from or added to the cache when GetExisting[Slippy]Tile is called.
• MapTileSupport.SaveMapTile and SaveSlippyMapTile can now be passed an expiration date. This is the absolute date the tile expires, not a difference in time like the expiration values for the GenericData calls.
• bannedReason column added to the authentication data table, so admins can track/remember why a ban was set on a user.
• PBFReader usage is being simplified. The default plan will be to read .osm.pbf files and write them directly to the database. This will handle creating and updating a database from source files in a single command. Entries will be run through the PreTag command to allow searching Places by style db-side. The previous suggested plan of processing data to geomdata/tagsdata files and then bulk loading those is being deprecated. The new way is slower because EntityFramework actions must be single-threaded, and a database read per group occurs, but this way can be done live on a running database.
• GenericData.DeserializeAnonymousType(string, T) added. A helper to decode JSON into an anonymous type. Intended for use when a call is the only place a particular object type is used and you don't want to make a new class for it. Probably also works on non-anonymous classes too, but thats's not likely necessary.
• The server now allows you to directly do some GDPR requests as a user. Go to /Server/Gdpr to get to the management screen. You can view a copy of all data on your account, and force the server to immediately delete all of it. Banned users are still able to access the GDPR page and perform these tasks, since those rights aren't overridden by the game or any TOS involved.
• DrawSizeHint math has been corrected, and no longer uses GameTileScale to decide what the hint should be.
• Places now serialize to JSON correctly.
• Tweaked default timer value on deleting expired database/auth values to 5 minutes instead of 30. Can now be set explicitly in appsettings.json
• ReprocessFile option removed. This logic will now only happen on read, not writes.
• Updated to .NET 8, and updated all dependencies to match.

Plugins:

• Slippy plugin can now load it's valid choices from the database, which means other plugins can add Slippy entries. A plugin can create a GlobalData entry with a key starting with "SlippyOverlay-" or "SlippyBase-" to indicate how the tiles should be handled, the name that should appear in Slippy after the -, and the base URL for maptiles (no zoom/x/y or other parameters) as the key for that entry.
• • EX: GenericData.SetGlobalData("SlippyBase-OSMLike", "MapTile/Slippy/mapTiles");
• • Defaults have been added for mapTiles (OSMLike), suggestedGameplay, adminBounds (all), outlines, teamColor, and each style inside adminBoundsFilled has its own overlay.
• Slippy now uses the internal cache setup for the list of tiles it can draw.
• Slippy now property sorts base layers from overlays. Overlays have a transparent background, base layers have an opaque background. Plugins will be expected to declare the correct type for layers they add to Slippy
• Slippy now has a box to let you jump to a specific PlusCode. Type in your PlusCode or partial code, and click Jump. It should jump you to that point. It will also show you the current PlusCode of the map's center.
• RoutineBreaker demo has been updated to work on a more specific set of entries that allows for a global-scale demo server. It will also work on a local server with more details, and will draw better examples if so.

Larry:

• Larry can now create sub-map files from an existing server. -extractSubMap:placeId:placeTypeId will look up a place by its OSM id and type, then create PMD files for all places intersecting the given place. These will
• Larry can re-tag all Places with their resulting match for all stylesets using the "-retag" command, in the event that styles are added or edited after initial database loading. Each Place in the database will get matched on most of the default styles, and will save the style's name and matched style to the PlaceData table. This can allow for faster lookups on things that are hard to query efficiently otherwise, like Countries or style entries that have a lot of NOT logic in their rules. Use Places.GetPlacesByData to search these results.
• Loading commands are getting simplifed. -makeServerDb remains the primary first-time command. There will be 1 command to import additional data later: loadData, and it will handle PBF and PMD files at the same time, and most options will not be needed. When using -makeServerDb, indexes will be dropped and created as appropriate for maximum performance.
• Added the -autoCreateSlippyMapTiles command, to make the Slippy tiles for zoom levels 2-18 ahead of time the way that -autoCreateMapTiles makes all the Cell8 game tiles ahead of time. Will take much, much, longer and use much more storage space, since there will be 16 levels of zoom saved.
• Output folder is now the same as the PBF input folder.
• To optionally adjust some values in the database for better behavior, a -fudgeIt command was added to Larry. This will alter entries in the database to work better for most gameplay at the cost of some proportionally small tradeoffs. Currently, the only change this command does it to treat the Continental United States as the actual country entry (because otherwise attempting to generate images based on the USA results in the entire planet being drawn)
• Various bugfixes across the app.