Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
f692d46
feat(overture): add conflation engine, category map, and PostGIS sche…
Medformatik Jun 20, 2026
a23e7c3
feat(overture): add Berlin ingest pipeline and conflation eval harness
Medformatik Jun 20, 2026
dbeb371
feat(overture): add poi-overture provider and orchestrator search fusion
Medformatik Jun 20, 2026
2d1f1af
fix(overture): align poi-overture SQL query to actual PostGIS schema
Medformatik Jun 20, 2026
fe26871
feat(overture): add knowledge-overture enrichment provider
Medformatik Jun 20, 2026
c69f673
feat(overture): add embeddings residual and conflation link-building job
Medformatik Jun 20, 2026
a5e28d0
fix(overture): embed only in-window candidates and ensure the model i…
Medformatik Jun 20, 2026
cec7b1f
feat(overture): add pull/ingest/conflate routes, CLI, and monthly cha…
Medformatik Jun 20, 2026
bfbe752
fix(overture): track changelog counts, stream conflate progress, fix …
Medformatik Jun 20, 2026
c2d2cba
feat(overture): consult precomputed conflation link first in POI fusion
Medformatik Jun 20, 2026
57b1597
feat(overture): register overture place resolver and enabled-aware di…
Medformatik Jun 20, 2026
f710f4f
feat(overture): register overture place resolver and enabled-aware di…
Medformatik Jun 20, 2026
057581e
fix(overture): exclude permanently-closed venues from link-building a…
Medformatik Jun 20, 2026
bd00f02
fix(data-manager): resolve @openmapx/core via pure subpath exports + …
Medformatik Jun 20, 2026
4f1b323
fix(overture): ingest the real Overture GeoParquet geometry column
Medformatik Jun 20, 2026
49f675b
fix(overture): emit geometry as hex-WKB and use explicit INSERT colum…
Medformatik Jun 20, 2026
a579b79
fix(overture): correct the OSM POI extractor for real runs
Medformatik Jun 20, 2026
73a07fd
fix(overture): emit OSM object ids in the extractor geojson export
Medformatik Jun 20, 2026
61ea8bc
perf(overture): H3-block the eval candidate generation
Medformatik Jun 20, 2026
e04bbc5
feat(overture): expand category map via overture2osm bridge + fix bac…
Medformatik Jun 20, 2026
955d1c2
fix(overture): correct Berlin Geofabrik path + scrub DB password from…
Medformatik Jun 20, 2026
ac79a15
refactor(overture): derive region URL + bbox from Geofabrik, drop har…
Medformatik Jun 20, 2026
80396b4
fix(overture): address xhigh code-review findings
Medformatik Jun 21, 2026
20117e2
feat(overture): normalize + token-aware name matching in conflation c…
Medformatik Jun 21, 2026
eb0127d
fix(overture): lower conflation confidence floor 0.7 to 0.5 (calibrated)
Medformatik Jun 21, 2026
e6c4ae5
fix(overture): keep NULL operating_status places (was excluding 99.8%)
Medformatik Jun 21, 2026
cbbaf72
fix(overture): require a name signal in the always-merge band
Medformatik Jun 21, 2026
50ed79c
fix(overture): clear poi_conflation_link before rebuilding it
Medformatik Jun 21, 2026
a019197
feat(overture): address + wikidata corroboration in conflation matching
Medformatik Jun 21, 2026
9055c30
fix(overture): fold compound German street suffixes in address keys
Medformatik Jun 21, 2026
0698058
feat(overture): single-tenant address trust to cut multi-tenant mis-l…
Medformatik Jun 21, 2026
79d0e8c
fix(overture): require name/category corroboration at every shared ad…
Medformatik Jun 21, 2026
a3260ea
feat(overture): trust same-address category-equality only for singula…
Medformatik Jun 21, 2026
d6b0069
fix(overture): drop museums from singular-per-address categories
Medformatik Jun 21, 2026
137bde4
feat(overture): add phone/website corroboration to POI conflation
Medformatik Jun 21, 2026
1c33edb
fix(overture): harden phone/website conflation per code review
Medformatik Jun 21, 2026
8f9afdf
chore(overture): clear deferred review minors
Medformatik Jun 21, 2026
31eeb2e
fix(overture): credit data sources via name/url + license/licenseUrl
Medformatik Jun 21, 2026
782d447
fix(overture): resolve CodeQL findings + add changeset
Medformatik Jun 21, 2026
43e5a3e
fix(overture): route Geofabrik fetch through the host-validated URL o…
Medformatik Jun 21, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/overture-poi-conflation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@openmapx/core": minor
---

Add the OSM↔Overture POI conflation toolkit used by the Overture Places integration. Exposes the bipartite `conflate` matcher (`ConflationPoint`/`ConflationThresholds`/`ConflationResult`/`DEFAULT_CONFLATION_THRESHOLDS`), query-time `fusePoiResults`, the Overture category mappings (`overtureCategoryToOpenMapX`, `openMapXCategoryToOverture`, `OVERTURE_COMMERCIAL_CATEGORIES`, `openmapxCategoryToOvertureLeaves`), and the name-independent matching utilities in `geo-server` (`nameSimilarity`, `normalizeName`, `normalizeStreet`, `osmAddressKey`, `overtureAddressKey`, `normalizePhone`, `parsePhones`, `websiteDomain`). Matching corroborates on address keys, wikidata, and phone/website signals beyond fuzzy name similarity.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ docs/superpowers/
docs/plans/
docs/legal/
plans/
.superpowers/

# CLI generated outputs
infra/docker/docker-compose.generated.yml
Expand Down
20 changes: 20 additions & 0 deletions apps/api/src/integration-host.ts
Original file line number Diff line number Diff line change
Expand Up @@ -895,6 +895,26 @@ export function isIntegrationScheme(scheme: string): boolean {
return integrations.has(scheme);
}

/**
* True when `scheme` matches an installed integration that is **not**
* config-disabled (`config.enabled !== false`). The places route uses this
* to decide whether a missing resolver should 404 (the integration owns the
* scheme and is enabled, so the missing resolver is a boot failure) or fall
* through to coord-fallback (the integration is config-disabled, so a stale
* deep-link should degrade gracefully rather than hard-404).
*
* Keyed on `config.enabled`, not the runtime `enabled` flag: both a
* config-disabled integration and an enabled-but-`setup()`-threw integration
* have `enabled === false` at runtime, but only `config.enabled === false`
* uniquely marks config-disabled. Using the runtime flag would let a broken
* enabled integration fall through to coord-fallback, re-opening the
* tag-substitution leak the 404 trap closes.
*/
export function isEnabledIntegrationScheme(scheme: string): boolean {
const i = integrations.get(scheme);
return i !== undefined && i.config?.enabled !== false;
}

export function getIntegrationsByDomain(domain: string): LoadedIntegration[] {
return Array.from(integrations.values()).filter(
(i) => i.enabled && i.manifest.domains.includes(domain),
Expand Down
57 changes: 49 additions & 8 deletions apps/api/src/routes/__tests__/places.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -48,9 +48,11 @@ vi.mock("@integrations/reviews/orchestrator", () => ({
}));

const mockIsIntegrationScheme = vi.fn().mockReturnValue(false);
const mockIsEnabledIntegrationScheme = vi.fn().mockReturnValue(false);
vi.mock("../../integration-host.js", () => ({
getAllIntegrations: vi.fn().mockReturnValue([]),
isIntegrationScheme: (scheme: string) => mockIsIntegrationScheme(scheme),
isEnabledIntegrationScheme: (scheme: string) => mockIsEnabledIntegrationScheme(scheme),
}));

// Mock DB RIS service
Expand Down Expand Up @@ -348,14 +350,14 @@ describe("GET /places/:id", () => {
expect(mockLookupByCoords).toHaveBeenCalled();
});

it("returns 404 for an integration scheme whose resolver didn't register", async () => {
it("returns 404 for an enabled integration scheme whose resolver didn't register", async () => {
// Mirrors the failure mode that produced the original leak: a data-
// source integration (here scooter-sharing) is installed — its manifest
// is on disk so `isIntegrationScheme` returns true — but its `setup()`
// threw at boot, so no resolver was registered. Without this gate the
// route would fall through to lookupByCoords and substitute the
// nearest OSM POI's tags onto the scooter.
mockIsIntegrationScheme.mockImplementation((scheme) => scheme === "scooter-sharing");
// source integration (here scooter-sharing) is installed and enabled —
// `isEnabledIntegrationScheme` returns true — but its `setup()` threw at
// boot, so no resolver was registered. Without this gate the route would
// fall through to lookupByCoords and substitute the nearest OSM POI's
// tags onto the scooter.
mockIsEnabledIntegrationScheme.mockImplementation((scheme) => scheme === "scooter-sharing");

const res = await app.inject({
method: "GET",
Expand All @@ -370,7 +372,46 @@ describe("GET /places/:id", () => {
expect(mockLookupByNameAndCoords).not.toHaveBeenCalled();
expect(mockLookupByCoords).not.toHaveBeenCalled();

mockIsIntegrationScheme.mockReturnValue(false);
mockIsEnabledIntegrationScheme.mockReturnValue(false);
});

it("coord-falls-back for a config-disabled integration scheme with lat/lng/name", async () => {
// A config-disabled integration's scheme should NOT 404 — the request
// should reach the coord-fallback so a shared link degrades gracefully.
// `isEnabledIntegrationScheme` returns false (config-disabled); no resolver is registered.
mockIsEnabledIntegrationScheme.mockReturnValue(false);
mockLookupByNameAndCoords.mockResolvedValue(MOCK_PLACE);
mockGetPlaceKnowledge.mockResolvedValue({ externalIds: {} });
mockBuildReviewLinks.mockReturnValue([]);

const res = await app.inject({
method: "GET",
url: `/places/${encodeURIComponent("overture:some-gers-id")}?${qs({
lat: "52.52",
lng: "13.37",
name: "Some Place",
})}`,
});

expect(res.statusCode).toBe(200);
expect(mockLookupByNameAndCoords).toHaveBeenCalled();
});

it("returns 404 for an enabled integration scheme with no resolver (no coords supplied)", async () => {
// When no lat/lng are provided AND the scheme belongs to an enabled
// integration whose resolver never registered, the route must 404.
mockIsEnabledIntegrationScheme.mockImplementation((scheme) => scheme === "scooter-sharing");

const res = await app.inject({
method: "GET",
url: `/places/${encodeURIComponent("scooter-sharing:dott-456")}`,
});

expect(res.statusCode).toBe(404);
expect(mockLookupByNameAndCoords).not.toHaveBeenCalled();
expect(mockLookupByCoords).not.toHaveBeenCalled();

mockIsEnabledIntegrationScheme.mockReturnValue(false);
});

it("allows coord-fallback for a non-integration freeform scheme (stylePoi)", async () => {
Expand Down
4 changes: 2 additions & 2 deletions apps/api/src/routes/places.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ import {
type PlaceResolverContext,
} from "@openmapx/place-ids";
import type { FastifyPluginAsync } from "fastify";
import { getAllIntegrations, isIntegrationScheme } from "../integration-host.js";
import { getAllIntegrations, isEnabledIntegrationScheme } from "../integration-host.js";
import { getPlaceKnowledge } from "../services/knowledge/index";
import { buildReviewLinks } from "../services/review-links";
import { TTL, withCache } from "../utils/cache.js";
Expand Down Expand Up @@ -235,7 +235,7 @@ export const placesRoute: FastifyPluginAsync = async (fastify) => {
// never got to register its resolver. The manifest registry
// tells us which is which: any scheme matching an installed
// integration id is strict; everything else is freeform.
if (isIntegrationScheme(parsedId.scheme)) {
if (isEnabledIntegrationScheme(parsedId.scheme)) {
fastify.log.warn(
{ scheme: parsedId.scheme, rawId },
"places: integration scheme has no resolver; refusing coord-fallback",
Expand Down
61 changes: 59 additions & 2 deletions apps/api/src/services/knowledge/__tests__/index.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -61,14 +61,36 @@ function makePlace(osmTags?: Record<string, string>) {
}

describe("getPlaceKnowledge", () => {
it("returns {} immediately when place has no osmTags", async () => {
it("returns {} immediately when place has neither osmTags nor coordinates", async () => {
const { getPlaceKnowledge } = await import("../index.js");
const result = await getPlaceKnowledge(makePlace(undefined));
const placeWithoutBoth = {
id: "overture:gers-abc",
primaryScheme: "overture",
ids: { overture: "gers-abc" },
name: "Test",
address: "",
coordinates: undefined,
osmTags: undefined,
} as unknown as Parameters<typeof getPlaceKnowledge>[0];
const result = await getPlaceKnowledge(placeWithoutBoth);
expect(result).toEqual({});
expect(wikidataLookup).not.toHaveBeenCalled();
expect(wikipediaLookup).not.toHaveBeenCalled();
});

it("calls sources with empty osmTags when coordinates are present but osmTags absent (Overture brandless)", async () => {
wikidataLookup.mockResolvedValueOnce(null);
wikipediaLookup.mockResolvedValueOnce(null);
const { getPlaceKnowledge } = await import("../index.js");
const result = await getPlaceKnowledge(makePlace(undefined));
expect(result).toEqual({});
expect(wikidataLookup).toHaveBeenCalledWith(
{},
undefined,
expect.objectContaining({ coordinates: [13.4, 52.5] }),
);
});

it("scalar fields: first non-null description wins", async () => {
wikidataLookup.mockResolvedValueOnce({ description: "Wikidata desc" });
wikipediaLookup.mockResolvedValueOnce({ description: "Wikipedia desc" });
Expand Down Expand Up @@ -176,4 +198,39 @@ describe("getPlaceKnowledge", () => {
expect(wikidataLookup).not.toHaveBeenCalled();
expect(result.description).toBe("Wikipedia desc");
});

it("optionality: enrichment output is unchanged when knowledge-overture is not registered", async () => {
wikidataLookup.mockReset();
wikipediaLookup.mockReset();
wikidataLookup.mockResolvedValueOnce({ description: "Wikidata desc" });
wikipediaLookup.mockResolvedValueOnce({ description: "Wikipedia desc" });

const { getPlaceKnowledge } = await import("../index.js");
const result = await getPlaceKnowledge(makePlace({ wikidata: "Q42" }));

expect(result.description).toBe("Wikidata desc");
expect(result.brand).toBeUndefined();
expect(result.names).toBeUndefined();
expect(result.structuredOpeningHours).toBeUndefined();
});

it("merges brand, names, structuredOpeningHours from first non-null source", async () => {
wikidataLookup.mockReset();
wikipediaLookup.mockReset();
wikidataLookup.mockResolvedValueOnce({
brand: { name: "Starbucks", wikidata: "Q37158" },
names: { de: "Starbucks" },
structuredOpeningHours: "Mo-Fr 07:00-21:00",
});
wikipediaLookup.mockResolvedValueOnce({
brand: { name: "Other Brand" },
});

const { getPlaceKnowledge } = await import("../index.js");
const result = await getPlaceKnowledge(makePlace({ wikidata: "Q42" }));

expect(result.brand).toEqual({ name: "Starbucks", wikidata: "Q37158" });
expect(result.names).toEqual({ de: "Starbucks" });
expect(result.structuredOpeningHours).toBe("Mo-Fr 07:00-21:00");
});
});
13 changes: 11 additions & 2 deletions apps/api/src/services/knowledge/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -26,15 +26,16 @@ function getKnowledgeSources(disallowedIntegrations: Set<string>): KnowledgeProv
* Never throws — failures are silently dropped.
*/
export async function getPlaceKnowledge(place: Place, lang?: string): Promise<KnowledgeResult> {
if (!place.osmTags) return {};
if (!place.osmTags && !place.coordinates) return {};

const sources = getKnowledgeSources(await getGatedIntegrationIds());

const settled = await Promise.allSettled(
sources.map((source) =>
source.lookup(place.osmTags as Record<string, string>, lang, {
source.lookup((place.osmTags ?? {}) as Record<string, string>, lang, {
coordinates: place.coordinates,
name: place.name,
ids: place.ids,
}),
),
);
Expand All @@ -52,6 +53,9 @@ export async function getPlaceKnowledge(place: Place, lang?: string): Promise<Kn
facts,
externalIds,
airport,
brand,
names,
structuredOpeningHours,
} = result.value;

if (description && !merged.description) merged.description = description;
Expand All @@ -66,6 +70,11 @@ export async function getPlaceKnowledge(place: Place, lang?: string): Promise<Kn
merged.externalIds = { ...externalIds, ...(merged.externalIds ?? {}) };
}
if (airport && !merged.airport) merged.airport = airport;
if (brand && !merged.brand) merged.brand = brand;
if (names && !merged.names) merged.names = names;
if (structuredOpeningHours && !merged.structuredOpeningHours) {
merged.structuredOpeningHours = structuredOpeningHours;
}
}

return merged;
Expand Down
Loading