surplus

surplus is a Python script to convert Google Maps Plus Codes to iOS Shortcuts-like shareable text.

installation
usage
- command-line usage
- example api usage
developer's guide
contributor's guide
api reference
- details on the fingerprinted user agent
licence

$ surplus 9R3J+R9 Singapore
surplus version 2.2.0
Thomson Plaza
301 Upper Thomson Road
Sin Ming, Bishan
574408
Central, Singapore

installation

Important

python 3.11 or later is required due to a bug in earlier versions. (python/cpython#88089)

for most, you can install surplus built from the latest stable release:

pipx install https://github.com/markjoshwel/surplus/releases/latest/download/surplus-latest-py3-none-any.whl

or directly from the repository using pip:

pipx install git+https://github.com/markjoshwel/surplus.git@main

Termux users: consider surplus on wheels, a sister project that allows you to run surplus regularly throughout the day and send it to someone on a messaging platform.

surplus is also a public domain dedicated single python file, so feel free to grab that and embed it into your own program as you see fit.

see licence for licensing information.

usage

command-line usage

usage: surplus [-h] [-d] [-v] [-c {pluscode,localcode,latlong,sharetext}]
               [-u USER_AGENT] [--show-user-agent] [-t]
               [query ...]

Google Maps Plus Code to iOS Shortcuts-like shareable text

positional arguments:
  query                 full-length Plus Code (6PH58QMF+FX), shortened
                        Plus Code/'local code' (8QMF+FX Singapore),
                        latlong (1.3336875, 103.7749375), string query
                        (e.g., 'Wisma Atria'), or '-' to read from stdin

options:
  -h, --help            show this help message and exit
  -d, --debug           prints lat, long and reverser response dict to
                        stderr
  -v, --version         prints version information to stderr and exits
  -c {pluscode,localcode,latlong,sharetext}, --convert-to {pluscode,localcode,latlong,sharetext}
                        converts query a specific output type, defaults
                        to 'sharetext'
  -u USER_AGENT, --user-agent USER_AGENT
                        user agent string to use for geocoding service,
                        defaults to fingerprinted user agent string
  --show-user-agent     prints fingerprinted user agent string and exits
  -t, --using-termux-location
                        treats input as a termux-location output json
                        string, and parses it accordingly

example api usage

here are a few examples to get you quickly started using surplus in your own program:

let surplus do the heavy lifting

>>> from surplus import surplus, Behaviour
>>> result = surplus("Ngee Ann Polytechnic, Singapore", Behaviour())
>>> result.get()
'Ngee Ann Polytechnic\n535 Clementi Road\nBukit Timah\n599489\nNorthwest, Singapore'

handle queries separately

>>> import surplus
>>> behaviour = surplus.Behaviour("6PH59R48+WP")
>>> query = surplus.parse_query(behaviour)
>>> result = surplus.surplus(query.get(), behaviour)
>>> result.get()
'MacRitchie Nature Trail\nCentral Water Catchment\n574325\nCentral, Singapore'

start from a Query object

>>> import surplus
>>> localcode = surplus.LocalCodeQuery(code="8R3M+F8", locality="Singapore")
>>> geocoder = surplus.SurplusDefaultGeocoding().geocoder
>>> pluscode_str = localcode.to_full_plus_code(geocoder=geocoder).get()
>>> pluscode = surplus.PlusCodeQuery(pluscode_str)
>>> result = surplus.surplus(pluscode, surplus.Behaviour())
>>> result.get()
'Wisma Atria\n435 Orchard Road\n238877\nCentral, Singapore'

notes:

you can change what surplus does when passing in a custom Behaviour object
most surplus functions return a Result object. while you can call .get() to obtain the proper return value, this is dangerous and might raise an exception

see the api reference for more information.

api reference

constants
exception classes
types
class Behaviour
class SurplusDefaultGeocoding
class ConversionResultTypeEnum
class Result
class Latlong
- Latlong.__str__()
class PlusCodeQuery
- PlusCodeQuery.to_lat_long_coord()
- PlusCodeQuery.__str__()
class LocalCodeQuery
class LatlongQuery
- LatlongQuery.to_lat_long_coord()
- LatlongQuery.__str__()
class StringQuery
- StringQuery.to_lat_long_coord()
- StringQuery.__str__()
def surplus()
def parse_query()
def generate_fingerprinted_user_agent
- details on the fingerprinted user agent

constants

VERSION: tuple[int, int, int]

a tuple of integers representing the version of surplus, in the format [major, minor, patch]
VERSION_SUFFIX: typing.Final[str]
BUILD_BRANCH: typing.Final[str]
BUILD_COMMIT: typing.Final[str]
BUILD_DATETIME: typing.Final[datetime]

string and a datetime.datetime object containing version and build information, set by releaser.py
CONNECTION_MAX_RETRIES: int = 9
CONNECTION_WAIT_SECONDS: int = 10

defines if and how many times to retry a connection, alongside how many seconds to wait in between tries, for Nominatim

[!NOTE]
this constant only affects the default surplus Nominatim geocoding functions. custom functions do not read from this, unless deliberately programmed to do so
SHAREABLE_TEXT_LINE_0_KEYS: dict[str, tuple[str, ...]]
SHAREABLE_TEXT_LINE_1_KEYS: dict[str, tuple[str, ...]]
SHAREABLE_TEXT_LINE_2_KEYS: dict[str, tuple[str, ...]]
SHAREABLE_TEXT_LINE_3_KEYS: dict[str, tuple[str, ...]]
SHAREABLE_TEXT_LINE_4_KEYS: dict[str, tuple[str, ...]]
SHAREABLE_TEXT_LINE_5_KEYS: dict[str, tuple[str, ...]]
SHAREABLE_TEXT_LINE_6_KEYS: dict[str, tuple[str, ...]]

a dictionary of iso3166-2 country-portion string keys with a tuple of Nominatim keys used in shareable text line 0-6 as their values
```
{
  "default": (...),
  "SG": (...,),
  ...
}
```
SHAREABLE_TEXT_LINE_SETTINGS: dict[str, dict[int, tuple[str, bool]]]

a dictionary of iso3166-2 country-portion string keys with a dictionary as their values

the dictionary values are dictionaries with integers as keys, and a tuple of two strings

the first string is the separator string to use, and the second string is a boolean flag that if True will check the line for seen names
```
{
    "default": {
        0: (", ", False),
        ...
        6: (", ", False),
    },
    "IT": {
        0: (", ", False),
        ...
        6: (", ", False),
    },
    ...
}
```
SHAREABLE_TEXT_NAMES: dict[str, tuple[str, ...]]

a dictionary of iso3166-2 country-portion string keys with a tuple of strings as their values a tuple of strings containing Nominatim keys used in shareable text line 0-2 and special keys in line 3

used for seen name checks
SHAREABLE_TEXT_LOCALITY: dict[str, tuple[str, ...]]

a dictionary of iso3166-2 country-portion string keys with a tuple of strings as their values

used when generating the locality portions of shortened Plus Codes/local codes
```
{
  "default": (...),
  "SG": (...,),
  ...
}
```
SHAREABLE_TEXT_DEFAULT: typing.Final[str]
constant for what is the "default" key in the SHAREABLE* constants
EMPTY_LATLONG: typing.Final[Latlong]
a constant for an empty latlong coordinate, with latitude and longitude set to 0.0

exception classes

class SurplusError(Exception)
base skeleton exception for handling and typing surplus exception classes
class NoSuitableLocationError(SurplusError)
class IncompletePlusCodeError(SurplusError)
class PlusCodeNotFoundError(SurplusError)
class LatlongParseError(SurplusError)
class EmptyQueryError(SurplusError)
class UnavailableFeatureError(SurplusError)

types

`Query`

Query: typing.TypeAlias = PlusCodeQuery | LocalCodeQuery | LatlongQuery | StringQuery

type alias representing either a PlusCodeQuery, LocalCodeQuery, LatlongQuery or StringQuery

`ResultType`

ResultType = TypeVar("ResultType")

generic type used by Result

`SurplusGeocoderProtocol`

typing_extensions.Protocol class for documentation and static type checking of surplus geocoder functions

signature and conforming function signature

class SurplusGeocoderProtocol(Protocol):
    def __call__(self, place: str) -> Latlong:
        ...

functions that conform to this protocol should have the following signature:

def example(place: str) -> Latlong: ...

information on conforming functions

function takes in a location name as a string, and returns a Latlong.

function MUST supply a bounding_box attribute to the to-be-returned Latlong. the bounding box is used when surplus shortens Plus Codes.

function can and should be at minimum functools.lru_cache()-wrapped if the geocoding service asks for caching

exceptions are handled by the caller

`SurplusReverserProtocol`

typing_extensions.Protocol class for documentation and static type checking of surplus reverser functions

signature and conforming function signature

class SurplusReverserProtocol(Protocol):
    def __call__(self, latlong: Latlong, level: int = 18) -> dict[str, Any]:
        ...

functions that conform to this protocol should have the following signature:

def example(latlong: Latlong, level: int = 18) -> dict[str, Any]: ...

information on conforming functions

function takes in a Latlong object and return a dictionary with SHAREABLE_TEXT_LINE_*_KEYS keys at the dictionaries' top-level.
keys are used to access address information.

function should also take in an int representing the level of detail for the returned address, 0-18 (country-level to building), inclusive. should default to 18.

keys for latitude, longitude and an iso3166-2 (or closest equivalent) should also be included at the dictionaries top level as the keys latitude, longitude and ISO3166-2 (non-case sensitive, or at least something starting with ISO3166) respectively.
```
{
    'ISO3166-2-lvl6': 'SG-03',
    'amenity': 'Ngee Ann Polytechnic',
    ...
    'country': 'Singapore',
    'latitude': 1.33318835,
    'longitude': 103.77461234638255,
    'postcode': '599489',
    'raw': {...},
}
```
function can and should be at minimum functools.lru_cache()-wrapped if the geocoding service asks for caching

exceptions are handled by the caller

`class Behaviour`

typing.NamedTuple representing how surplus operations should behave

attributes

query: str | list[str] = ""
original user-passed query string or a list of strings from splitting user-passed query string by spaces
geocoder: SurplusGeocoderProtocol = default_geocoding.geocoder
name string to location function, see SurplusGeocoderProtocol for more information
reverser: SurplusReverserProtocol = default_geocoding.reverser
Latlong object to address information dictionary function, see SurplusReverserProtocol for more information
stderr: typing.TextIO = sys.stderr
TextIO-like object representing a writeable file. defaults to sys.stderr.
stdout: typing.TextIO = sys.stdout
TextIO-like object representing a writeable file. defaults to sys.stdout.
debug: bool = False
whether to print debug information to stderr
version_header: bool = False
whether to print version information and exit
convert_to_type: ConversionResultTypeEnum = ConversionResultTypeEnum.SHAREABLE_TEXT
what type to convert the query to
using_termux_location: bool = False
treats query as a termux-location output json string, and parses it accordingly
show_user_agent: bool = False
whether to print the user agent string to stderr

`class SurplusDefaultGeocoding`

Important

this has replaced the now deprecated default geocoding functions, default_geocoder() and default_reverser(), in surplus 2.1 and later.

see SurplusGeocoderProtocol and SurplusReverserProtocol for more information how to implement a compliant custom geocoder functions.

dataclasses.dataclass providing the default geocoding functionality for surplus, via OpenStreetMap Nominatim

attributes

user_agent: str = default_fingerprint
pass in a custom user agent here, else it will be the default fingerprinted user agent

example usage

from surplus import surplus, Behaviour, SurplusDefaultGeocoding

geocoding = SurplusDefaultGeocoding("custom user agent")
geocoding.update_geocoding_functions()  # not necessary but recommended

behaviour = Behaviour(
    ...,
    geocoder=geocoding.geocoder,
    reverser=geocoding.reverser
)

result = surplus("query", behaviour=behaviour)

...

methods

def update_geocoding_functions(self) -> None: ...
def geocoder(self, place: str) -> Latlong: ...
def reverser(self, latlong: Latlong, level: int = 18) -> dict[str, Any]: ...

`SurplusDefaultGeocoding.update_geocoding_functions()`

re-initialise the geocoding functions with the current user agent, also generate a new user agent if not set properly

it is recommended to call this before using surplus as by default the geocoding functions are uninitialised

signature

def update_geocoding_functions(self) -> None: ...

`SurplusDefaultGeocoding.geocoder()`

Warning

this function is primarily given to be passed into a Behaviour object, and is not meant to be called directly.

default geocoder for surplus

see SurplusGeocoderProtocol for more information on surplus geocoder functions

`SurplusDefaultGeocoding.reverser()`

Warning

this function is primarily given to be passed into a Behaviour object, and is not meant to be called directly.

default reverser for surplus

see SurplusReverserProtocol for more information on surplus reverser functions

`class ConversionResultTypeEnum`

enum.Enum representing what the result type of conversion should be

values

PLUS_CODE: str = "pluscode"
LOCAL_CODE: str = "localcode"
LATLONG: str = "latlong"
SHAREABLE_TEXT: str = "sharetext"

`class Result`

typing.NamedTuple representing the result for safe value retrieval

attributes

value: ResultType
value to return or fallback value if erroneous
error: BaseException | None = None
exception if any

example usage

# do something
def some_operation(path) -> Result[str]:
    try:
        file = open(path)
        contents = file.read()

    except Exception as exc:
        # must pass a default value
        return Result[str]("", error=exc)

    else:
        return Result[str](contents)

# call function and handle result
result = some_operation("some_file.txt")

if not result:  # check if the result is erroneous
    # .cry() raises the exception
    # (or returns it as a string error message using string=True)
    result.cry()
    ...

else:
    # .get() raises exception or returns value,
    # but since we checked for errors this is safe
    print(result.get())

methods

def __bool__(self) -> bool: ...
def cry(self, string: bool = False) -> str: ...
def get(self) -> ResultType: ...

`Result.bool()`

method that returns True if self.error is not None

signature
```
def __bool__(self) -> bool: ...
```
returns bool

`Result.cry()`

method that raises self.error if is an instance of BaseException, returns self.error if is an instance of str, or returns an empty string if self.error is None

signature

def cry(self, string: bool = False) -> str: ...

arguments
- string: bool = False
  if self.error is an Exception, returns it as a string error message
returns str

`Result.get()`

method that returns self.value if Result is non-erroneous else raises error

signature
```
def get(self) -> ResultType: ...
```
returns self.value

`class Latlong`

typing.NamedTuple representing a latitude-longitude coordinate pair

attributes

latitude: float
longitude: float
bounding_box: tuple[float, float, float, float] | None = None
a four-tuple representing a bounding box, (lat1, lat2, lon1, lon2) or None.

the user does not need to enter this. the attribute is only used when shortening plus codes, and would be supplied by the geocoding service during shortening.

methods

def __str__(self) -> str: ...

`Latlong.str()`

method that returns a comma-and-space-separated string of self.latitude and self.longitude

signature
```
def __str__(self) -> str: ...
```
returns str

`class PlusCodeQuery`

typing.NamedTuple representing a full-length Plus Code (e.g., 6PH58QMF+FX)

attributes

code: str

methods

def to_lat_long_coord(self, ...) -> Result[Latlong]: ...
def __str__(self) -> str: ...

`PlusCodeQuery.to_lat_long_coord()`

signature

def to_lat_long_coord(self, geocoder: SurplusGeocoderProtocol) -> Result[Latlong]:
    ...

arguments
- geocoder: SurplusGeocoderProtocol
  name string to location function, see SurplusGeocoderProtocol for more information
returns Result[Latlong]

`PlusCodeQuery.str()`

method that returns string representation of query

signature
```
def __str__(self) -> str: ...
```
returns str

`class LocalCodeQuery`

typing.NamedTuple representing a shortened Plus Code with locality, referred to by surplus as a "local code"

attributes

code: str
Plus Code portion of local code, e.g., "8QMF+FX"
locality: str
remaining string of local code, e.g., "Singapore"

methods

def to_full_plus_code(self, ...) -> Result[str]: ...
def to_lat_long_coord(self, ...) -> Result[Latlong]: ...
def __str__(self) -> str: ...

`LocalCodeQuery.to_full_plus_code()`

exclusive method that returns a full-length Plus Code as a string

signature

def to_full_plus_code(self, geocoder: SurplusGeocoderProtocol) -> Result[str]:
    ...

arguments
- geocoder: SurplusGeocoderProtocol
  name string to location function, see SurplusGeocoderProtocol for more information
returns Result[str]

`LocalCodeQuery.to_lat_long_coord()`

method that returns a latitude-longitude coordinate pair

signature

def to_lat_long_coord(self, geocoder: SurplusGeocoderProtocol) -> Result[Latlong]:
    ...

arguments
- geocoder: SurplusGeocoderProtocol
  name string to location function, see SurplusGeocoderProtocol for more information
returns Result[Latlong]

`LocalCodeQuery.str()`

method that returns string representation of query

signature
```
def __str__(self) -> str: ...
```
returns str

`class LatlongQuery`

typing.NamedTuple representing a latitude-longitude coordinate pair

attributes

latlong: Latlong

methods

def to_lat_long_coord(self, ...) -> Result[Latlong]: ...
def __str__(self) -> str: ...

`LatlongQuery.to_lat_long_coord()`

method that returns a latitude-longitude coordinate pair

signature

def to_lat_long_coord(self, geocoder: SurplusGeocoderProtocol) -> Result[Latlong]:
    ...

arguments
- geocoder: SurplusGeocoderProtocol
  name string to location function, see SurplusGeocoderProtocol for more information
returns Result[Latlong]

`LatlongQuery.str()`

method that returns string representation of query

signature
```
def __str__(self) -> str: ...
```
returns str

`class StringQuery`

typing.NamedTuple representing a pure string query

attributes

query: str

methods

def to_lat_long_coord(self, ...) -> Result[Latlong]: ...
def __str__(self) -> str: ...

`StringQuery.to_lat_long_coord()`

method that returns a latitude-longitude coordinate pair

signature

def to_lat_long_coord(self, geocoder: SurplusGeocoderProtocol) -> Result[Latlong]:
    ...

arguments
- geocoder: SurplusGeocoderProtocol
  name string to location function, see SurplusGeocoderProtocol for more information
returns Result[Latlong]

`StringQuery.str()`

method that returns string representation of query

signature
```
def __str__(self) -> str: ...
```
returns str

`def surplus()`

query to shareable text conversion function

signature

def surplus(query: Query | str, behaviour: Behaviour) -> Result[str]: ..

arguments
- query: str | Query
  query object to convert or string to attempt to query for then convert
- behaviour: Behaviour
  surplus behaviour namedtuple
returns Result[str]

`def parse_query()`

function that parses a query string into a query object

signature

def parse_query(behaviour: Behaviour) -> Result[Query]: ...

arguments
- behaviour: Behaviour
  surplus behaviour namedtuple
returns Result[Query]

`def generate_fingerprinted_user_agent()`

function that attempts to return a unique user agent string.

signature

def generate_fingerprinted_user_agent() -> Result[str]:

returns Result[str]

this result will always have a valid value as erroneous results will have a resulting value of 'surplus/<version> (generic-user)'

valid results will have a value of 'surplus/<version> (<fingerprin hasht>)', where the fingerprint hash is a 12 character hexadecimal string

details on the fingerprinted user agent

why do this in the first place?
if too many people use surplus at the same time, Nominatim will start to think it's just one person being greedy. so to prevent this, surplus will try to generate a unique user agent string for each user through fingerprinting.

at the time of writing, the pre-hashed fingerprint string is as follows:

unique_info: str = f"{version}-{system_info}-{hostname}-{mac_address}"

it contains the following, in order, alongside an example:

version - the surplus version alongside a suffix, if any
```
2.2.0-local
```
system_info - generic machine and operating system information
```
Linux-6.5.0-locietta-WSL2-xanmod1-x86_64-with-glibc2.35
```
hostname - your computer's hostname
```
mark
```
mac_address - your computer's mac address
```
A9:36:3C:98:79:33
```

after hashing, this string becomes a 12 character hexadecimal string, as shown below:

surplus/2024.0.0 (1fdbfa0b0cfb)
                  ^^^^^^^^^^^^
                  this is the hashed result of unique_info

if at any time the retrieval of any of these four elements fail, surplus will just give up and default to 'surplus/<version> (generic-user)'.

you can see the fingerprinted user agent string by running the following command:

$ surplus --show-user-agent
...

if any of this seems weird to you, that's fine. pass in a custom user agent flag to surplus with -u or --user-agent to override the default user agent, or override the default user agent in your own code by passing in a custom user agent string to Behaviour.

$ surplus --user_agent "a-shiny-custom-and-unique-user-agent" 77Q4+7X Austin, Texas, USA
...

>>> from surplus import surplus, Behaviour
>>> surplus(..., Behaviour(user_agent="a-shiny-custom-and-unique-user-agent"))
...

licence

surplus is free and unencumbered software released into the public domain. for more information, please refer to the UNLICENCE, https://unlicense.org, or the python module docstring.

however, direct dependencies of surplus are licensed under different, but still permissive and open-source licences.

geopy: Python Geocoding Toolbox

MIT Licence
- geographiclib: The geodesic routines from GeographicLib
  
  MIT Licence
pluscodes: Compute Plus Codes (Open Location Codes)

Apache 2.0

Name		Name	Last commit message	Last commit date
Latest commit History 161 Commits
.github		.github
surplus		surplus
.gitignore		.gitignore
CHANGELOG.md		CHANGELOG.md
CONTRIBUTING.md		CONTRIBUTING.md
DEVELOPING.md		DEVELOPING.md
README.md		README.md
UNLICENCE		UNLICENCE
devbox.json		devbox.json
devbox.lock		devbox.lock
pyproject.toml		pyproject.toml
releaser.py		releaser.py
requirements.txt		requirements.txt

License

markjoshwel/surplus

Folders and files

Latest commit

History

Repository files navigation

surplus

installation

usage

command-line usage

example api usage

api reference

constants

exception classes

types

Query

ResultType

SurplusGeocoderProtocol

SurplusReverserProtocol

class Behaviour

class SurplusDefaultGeocoding

SurplusDefaultGeocoding.update_geocoding_functions()

SurplusDefaultGeocoding.geocoder()

SurplusDefaultGeocoding.reverser()

class ConversionResultTypeEnum

class Result

Result.__bool__()

Result.cry()

Result.get()

class Latlong

Latlong.__str__()

class PlusCodeQuery

PlusCodeQuery.to_lat_long_coord()

PlusCodeQuery.__str__()

class LocalCodeQuery

LocalCodeQuery.to_full_plus_code()

LocalCodeQuery.to_lat_long_coord()

LocalCodeQuery.__str__()

class LatlongQuery

LatlongQuery.to_lat_long_coord()

LatlongQuery.__str__()

class StringQuery

StringQuery.to_lat_long_coord()

StringQuery.__str__()

def surplus()

def parse_query()

def generate_fingerprinted_user_agent()

details on the fingerprinted user agent

licence

About

Topics

Resources

License

Code of conduct

Stars

Watchers

Forks

Sponsor this project

Languages

`Query`

`ResultType`

`SurplusGeocoderProtocol`

`SurplusReverserProtocol`

`class Behaviour`

`class SurplusDefaultGeocoding`

`SurplusDefaultGeocoding.update_geocoding_functions()`

`SurplusDefaultGeocoding.geocoder()`

`SurplusDefaultGeocoding.reverser()`

`class ConversionResultTypeEnum`

`class Result`

`Result.bool()`

`Result.cry()`

`Result.get()`

`class Latlong`

`Latlong.str()`

`class PlusCodeQuery`

`PlusCodeQuery.to_lat_long_coord()`

`PlusCodeQuery.str()`

`class LocalCodeQuery`

`LocalCodeQuery.to_full_plus_code()`

`LocalCodeQuery.to_lat_long_coord()`

`LocalCodeQuery.str()`

`class LatlongQuery`

`LatlongQuery.to_lat_long_coord()`

`LatlongQuery.str()`

`class StringQuery`

`StringQuery.to_lat_long_coord()`

`StringQuery.str()`

`def surplus()`

`def parse_query()`

`def generate_fingerprinted_user_agent()`