Skip to content

Releases: kpubdata-lab/kpubdata

v0.9.0

Choose a tag to compare

@github-actions github-actions released this 06 Oct 02:43
c5ecddf

Added

  • DatasetRef.to_dict() (#784): a dataset reference serialises to a JSON-safe dict with a stable set of keys — identity, status (#783), query_support, license, request_parameters, application and verified_at. dataclasses.asdict(ref) raised TypeError on raw_metadata, so a consumer had to read raw_metadata, which carries no stability promise. Every key is always present; None means nothing is declared. The key list is in docs/compatibility.md §3. kpubdata datasets list --format json and datasets show --format json print this dict: list entries gain the new keys next to id, name, provider and operations, and show keeps capabilities and raw_metadata_keys beside them.
  • kpubdata.SENSITIVE_PARAM_KEYS (#782): the names kpubdata masks as credentials — servicekey, service_key, api_key, apikey, token, authorization, secret, password, key, oc, accesstoken, consumer_key, consumer_secret — are importable from kpubdata and kpubdata.transport. They lived only in the private kpubdata.transport._sensitive, so Builder kept its own list, and that list lacks key, oc, consumer_key and consumer_secret. Names are casefolded and matched exactly; a release only adds names. Value-based masking stays private.
  • DatasetRef.status (#783): a dataset reference says how far the dataset has been verified, as a DatasetStatus (now exported from kpubdata). A dataset awaiting an application, one checked against fixtures only and one verified against the live API used to look the same at run time; the levels lived only in SUPPORTED_DATA.md, which is not in the wheel. scripts/gen_dataset_status.py writes src/kpubdata/dataset_status.json from that document's level column (through SUPPORTED_DATA_LEVELS, with a spec's status: override applied), and tests/unit/test_dataset_ref_status.py fails when the two differ. Of the 155 datasets the table and the runtime share: 94 application_required, 36 fixture_verified, 24 live_verified, 1 unstable (datago.ocean_buoy, by its spec's override); datago.generic is not in the table and reports None. The status is the recorded one, not a live check — that is Client.probe.
  • Evidence bound to the spec, the commit and the run (#522, #713): a recorded fixture's meta.json now carries spec_sha256, a digest of the spec file's pipeline-relevant content (kpubdata.core.spec.spec_file_digest, which normalises away the last_verified line the recorder itself rewrites), and record_commit/run_ref from the CI environment (GITHUB_SHA/GITHUB_RUN_ID) when present. make verify gains a spec-binding step: a fixture whose recorded digest no longer matches its spec is void — the spec changed after recording — and fails with re-record guidance. Fixtures recorded before the digest existed are reported as legacy evidence rather than failed — since #717, only those on a frozen baseline.
  • Public probe API (#694): Client.probe(dataset_id) -> ProbeResult | None and Client.probe_all(*, provider=None) -> list[ProbeResult] classify reachability with the client's own keys, and ProbeResult and PROBE_STATUSES (the ADR 0005 ProbeStatus vocabulary) are exported from kpubdata. Probing keeps its fast-fail transport (PROBE_TIMEOUT_SECONDS, PROBE_RETRIES, no cache), and a run shares one transport that is closed at the end — previously every dataset opened its own and none was closed. kpubdata probe now goes through these methods; its output and report are unchanged, and it now honors the global --provider-key option, which it silently ignored before.
  • Explicit-keys-only mode (#694): Client(..., env_keys=False) (backed by KPubDataConfig.env_fallback) uses only provider_keys and never reads a credential from the environment — not KPUBDATA_<P>_API_KEY, <P>_API_KEY or KPUBDATA_SGIS_CONSUMER_SECRET. A service probing with a user's key no longer has a missing entry silently filled with the operator's key. With no key the probe makes no call and reports auth_unknown. The default (env_keys=True) is unchanged.
  • R3 review gate (#722, kpubdata-builder#905): a pull request labelled review:R3 now needs an approval from someone other than its author before it can merge. scripts/r3_review.py, wrapped by the .github/actions/r3-review composite action, is the one implementation all four repositories call; the R3 review workflow runs it on every pull request (passing when it is not R3) and again on label changes, pushes and review submissions or dismissals, with pull-requests: read only. Each reviewer's latest approving or change-requesting review decides; approvals on an older head count, as with branch protection's stale-approval dismissal off; the author, bots and accounts without write access do not count. POLICY 14.1 and AGENTS.md describe it. R3 review becomes a required status check next to CI gate; the approval count in branch protection stays at 0.
  • Probe evidence fields (#625, #710): ProbeResult carries http_status (200 on success — the executor raises on anything else — and the raised error's status otherwise), result_code (read off the raised error; a successful query does not parse the envelope), latency_ms, schema_hash (a fingerprint of the returned field names — data changes daily, field names are the contract) and classification (the DriftClassification this outcome feeds the dataset status machine), each None where it could not be observed — never guessed. The report envelope records who ran the probe (runner; the nightly workflow passes github-actions).
  • Nightly drift detection (#625, #711): scripts/drift_detect.py runs the dataset status machine over two consecutive probe reports — a differing schema_hash emits SCHEMA_CHANGED itself (it is not a probe outcome), transient failures move nothing before the third consecutive night, restores restore previous_status and file nothing, RETIRED moves nothing but files. The live-probe workflow grows a drift job that chains the previous night's artifacts, keeps streak state in a drift-state artifact, and files one [drift] issue per transition that needs a person — skipping datasets that already have an open one, with the status_history.json entry riding in the issue body. The exit status is always 0: an upstream failure is a drift event, not a CI failure.
  • Production-grade gate (#463, #711): scripts/check_production_grade.py evaluates the machine-checkable criteria of docs/production_grade.yaml — source and listing gate every spec; a recorded fixture (any *.meta.json: the recorder names fixtures after the example, tour_kor_area records seoul_restaurants.meta.json) and an example script gate the verified tier only (an in-progress row gates nothing — ocean_buoy's missing fixture is #627, not a surprise); fields_declared is reported, not gated. Its unit test is the CI gate: every spec dataset in the repository must pass.
  • Example recency windows (#734, #750): a spec can declare params[].max_age_days, and make verify fails an example outside the window before the live call does — naming the cause (expired, or a future issue that does not answer yet — the #731 morning trap) and the refresh route. Parameters without a window emit nothing. The relative-time representation the issue considered first was rejected: replay matches on recorded literals, and issue-slot granularity would leak provider specifics into the generic schema.
  • Recorder run reference (#523, #729, #739): the evidence-authorship gate grows a second layer — a changed meta.json by a recorder name must also carry record_commit and run_ref, and run_ref must resolve to a run of this repository's Build Dataset workflow whose head SHA is exactly that commit (gh run view; any lookup failure fails closed). A local tool committing under the recorder's name — the #728 path — no longer passes. ci.yml gains actions: read and the job token for the lookup.
  • Unconfirmed terms are not redistributable (#732): a spec whose licence says redistribution: allowed must carry the attribution text that proves the terms were confirmed from the provider page (#525); terms nobody checked say redistribution: unknown — the two claims #728 wrongly shipped (bus_arrival, social_enterprise) were fixed to unknown first (#740, #744), because Builder's publish gate (kpubdata-builder#892) reads exactly this flag. make verify enforces the rule with a shrink-only baseline, scripts/unconfirmed_terms_baseline.txt, frozen at the 16 legacy violators — mirroring the legacy evidence ratchet (#717): a listed spec passes while its terms stay unconfirmed, an unlisted violation fails verify, and a stale entry (terms confirmed, or the claim withdrawn) must be removed from the list.
  • PR title blocks the merge (#741): PR title joined CI gate and R3 review as a required status check in all four product repositories, so a failing title check now closes the merge button instead of only showing a red X. The check re-runs on title edits (titles.yml), which ci.yml cannot see, so it is registered in branch protection rather than added to the aggregate gate's needs. scripts/check_required_checks.py verifies the required list against the workflows (the studio#416 lesson), and POLICY 14.2 records the rule.
  • Unjustified plain http is refused (#738): a spec whose endpoint.base_url says http:// must carry endpoint.insecure_http_reason — the service key rides the query string, and over plain http anyone on the network path reads it. make verify gains a transport-security step enforced with a shrink-only baseline, scripts/insecure_http_baseline.txt, frozen at the 23 legacy http specs — all of them on apis.data.go.kr, which answers https with identical envelopes (the reproducible record — commands and raw envelopes for every service path behind the baseline ...
Read more

v0.8.0

Choose a tag to compare

@github-actions github-actions released this 30 Sep 10:10
22de31a

Security

  • RecordBatch.meta["provenance"]["url"] no longer leaks the API key. It was masked with str.replace of the plain key, which missed a percent-encoded data.go.kr key (+, /, =) and a path-segment key; it is now masked by parameter name (including the spec's own auth parameter) and by every encoded form of the key, and omitted if a key form survives (#612).
  • Dependencies with known vulnerabilities are upgraded in uv.lock (kpubdata-builder#691). pip-audit over every extra reported 87 findings (48 distinct advisories) in 14 packages; none remain. urllib3 2.6.3 → 2.8.0, idna 3.11 → 3.20, cryptography 46.0.6 → 50.0.1, pyjwt 2.12.1 → 2.15.1, pillow 12.2.0 → 12.3.0, mcp 1.26.0 → 2.2.0 (with starlette 1.7.0, python-multipart 0.0.32, click 8.5.0; anyio 4.14.2 on Python < 3.12 and 4.15.1 on ≥ 3.12, typing-extensions 4.15.0 / 4.16.0 likewise; adds httpx2, httpcore2, httpx2-jsfetch, truststore, opentelemetry-api and mcp-types; drops httpx-sse, python-dotenv and pydantic-settings), pytest 9.1.1, mkdocs-material 9.7.7, pymdown-extensions 12.1 — all inside the declared ranges, so only the lock moves. The gitleaks secret scan over the full history now runs in CI and gates merges through CI gate; a Security workflow runs pip-audit over the locked dependencies (for Python 3.10 and 3.12, so both sides of the < 3.12 markers are audited) and CodeQL on every pull request and weekly — gating those two is left to #631.

Added

  • Spec fields can declare semantic_kind (code, measure, date, period, text, flag), title and format (ADR 0006). Dataset.schema() returns them as FieldDescriptor.semantic_kind, FieldDescriptor.title and FieldConstraints.format. A kind that contradicts the storage type, a numeric transform on a code, a unit on anything but a measure, or an unknown kind fails the spec load and validate_spec.py; fields without a kind are not checked. The 16 code columns from #613 declare semantic_kind: code (#651).

  • Spec license gains redistribution (allowed / non_commercial / forbidden / unknown; absent means unknown), attribution (the exact text to display), quota and pii_columns (#525, #605).

  • DatasetRef.license carries a spec's licence terms — redistribution, attribution, quota, PII columns — exactly as declared, and LicenseSpec is exported from kpubdata (#609). A dataset that declares no licence has None, which means unknown rather than unrestricted; quota is the provider's own wording and is not parsed. Catalogue-only datasets have None.

  • Dataset.schema() returns a SchemaDescriptor for spec datasets, built from the spec's fields in declaration order (name, type, description; unit, source_name, transform in raw), and spec datasets with fields declare Operation.SCHEMA. A spec with no fields still returns None. title/format stay unset until the column-metadata contract (#644) decides them (#643).

  • ADR 0006: the column-metadata contract separates storage type (type), meaning (semantic_kind: code, measure, date, period, text, flag) and display (reusing FieldDescriptor.title and FieldConstraints.format). Implementation is #651 (#644).

  • kpubdata.core.status — one canonical dataset status vocabulary
    (DatasetStatus) with mappings from the spec, probe, SUPPORTED_DATA.md and
    production-grade vocabularies; spec._STATUSES and _probe.PROBE_STATUSES are
    now derived from it. ADR 0005 records the decision, and
    tests/unit/test_status_vocabulary.py fails when a design document uses a name
    the code does not define (#619).

  • kpubdata.core.status.transition() — the pure dataset status state machine from docs/DATASET_STATUS.md. A table-driven test runs every row of that document's transition table against the function. unstable now also breaks immediately on a structural change, and application_required recovers on HEALTHY (#625).

  • ValidationReport.to_dict() returns a JSON-serialisable form of the report (#615).

  • Provenance now reports cached=True for responses served from the response cache, and fetched_at keeps the original fetch time instead of the time of the cache read. The transport marks a cache hit in httpx.Response.extensions, and ResponseCache.get_entry() returns the stored created_at (#616).

Changed

  • BREAKING: code columns keep their leading zeros and come back as str (#613). Declared string now: apt_trade bonbun/bubun/roadNmBonbun/roadNmBubun/roadNmSeq, apt_rent roadnmbonbun/roadnmbubun, hospital_info clCd/postNo, metro_fare arvlStnCd/dptreStnCd, tour_kor_* zipcode, village_fcst fcstTime. As a safety net, a zero-led value ("06102") no longer casts to a number, so a code column declared numeric stays text and is reported uncastable. Replay verification (make verify) now also fails when normalization drops a leading zero. See kpubdata-builder#702.

  • KPubData is documented as a standalone Python SDK; KPubData Builder and KPubData Studio are listed as related projects, not as stages after it (#666). ADR 0007 records the independence rules — dependencies run Studio → Builder → KPubData only, and the public API is what kpubdata.__all__ and API_SPEC.md name (#667) — and scripts/check_independence.py fails CI if KPubData imports or depends on Builder or Studio (#668).

  • All 59 localdata catalogue datasets are marked retired and LocaldataAdapter.query_records emits a DeprecationWarning (#527, #603). The retirement is disputed by the recorded evidence — see #618.

  • CODEOWNERS covers the credential host allowlist, the scripts run by contents: write release jobs, pyproject.toml and uv.lock; a test checks the required paths and that every pattern still names an existing path (#629).

  • FieldIssue.kind and ValidationReport.issues_of() take the IssueKind literal ("uncastable", "missing", "undeclared"); issues_of() raises ValueError on an unknown kind instead of silently returning nothing (#615).

Fixed

  • scripts/release_notes.py promote keeps a CRLF CHANGELOG's line endings instead of rewriting every line as LF (#628).

  • kpubdata scaffold generates English docstrings, so the files it writes pass check_english_comments.py; a test runs the gate on the generated files (#626).

  • A provider-reported total of 0 is kept as RecordBatch.total_count == 0 instead of becoming None, so "no results" and "count unknown" are distinguishable; a reported 0 explicitly ends paging (#642).

  • Client(...).dataset(...).list_all() now reaches the spec path with column casting decided across all pages. CompositeProviderAdapter had no query_records_all, so every spec dataset reached through Client still cast per page and a column could be int on one page and str on the next — the split 0.7.0 recorded as fixed (#611).

  • docs/status.md no longer depends on the calendar: "recent" is measured from the newest last_verified, so --check stops failing every pull request from 2026-12-09. It also counts all six SUPPORTED_DATA levels instead of three (#620).

  • docs/DATASET_STATUS.md and docs/LIVE_PROBE.md contradicted each other and the
    code: the failure threshold (3 against "2+"), code 32's classification, the
    PARAM_CHANGED spelling, a nonexistent dataset key, a nonexistent module and the
    neis/fds key sharing claim (#619).

  • scripts/release_notes.py promote for a final version folds that version's pre-release sections (aN, bN, rcN) into one section, instead of failing with "nothing to release" after a pre-release. A prerelease command and the release-notes action's prerelease output tell the release job to mark a pre-release (#622).

  • The spec-dataset list_all() path honours page_size, max_size and max_pages (#614). page_size drives pagination instead of going out as a raw page_size parameter; a page_size above the spec's max_size no longer ends the walk after the first page; exceeding max_pages raises InvalidRequestError like the legacy path instead of truncating silently; and each batch keeps its own raw, meta["provenance"], next_page and validation, with the whole-result report in meta["validation_total"]. The path still buffers every page before the first batch, because casting is decided across all pages (#481); this is now documented.

  • The field validation report no longer miscounts (#615): a 0-row page is reported clean instead of every declared field being missing; numeric null markers ("", "-") count as nulls rather than non-null values; undeclared columns are detected across all records, not only the first.

Full changelog: v0.7.0...v0.8.0

v0.7.0

Choose a tag to compare

@github-actions github-actions released this 28 Sep 13:49
18b6ed0

kpubdata 0.7.0 is mostly a security and correctness release. If you use 0.6.x, upgrade: several paths could leak your API key or send it somewhere it should not go.

pip install -U "kpubdata==0.7.0"

kpubdata-builder 0.4.0 is pinned to kpubdata<0.7 and still installs 0.6.x. The Builder release that moves to 0.7 will be 0.4.1.

Security

  • Your API key could reach logs, tracebacks and error trackers. When the key travelled as a query parameter (params=, used by datago, localdata, semas, sgis and every spec dataset), the exception chain kept the full URL with ?serviceKey=… (#486). bok carried its key in the URL path and law in a parameter named OC; neither was masked (#475). sgis tokens and consumer secrets were not masked either (#484).
  • lofin turned TLS certificate verification off. Every lofin request, including the key, would accept any certificate (#488). Verification is now on; only the cipher level is relaxed, which is all the server needs.
  • A spec could send your provider key to any host. The executor now refuses to attach a credential when endpoint.base_url is not a host listed for that provider (#532, #519).
  • Error responses and tokens were cached for 24 hours. Korean public APIs report quota and key errors in an HTTP 200 body, so a momentary quota breach was served from cache for a day (#490).

Behaviour changes — check before upgrading

  • Numeric columns cast differently. A declared numeric column is cast only if every value in it casts (#468). Thousands separators are understood: "1,200" becomes 1200, and datago.apt_trade.dealAmount is now an integer, not a string (#574). list_all() applies the rule across all pages at once, so page 1 and page 2 can no longer disagree on a column's type (#575).
  • An invalid license field in a spec now fails the load instead of being dropped silently (#476).
  • 4xx responses are no longer retried (#490). A Retry-After longer than TransportConfig.max_retry_delay (default 60 s) now raises a retryable RateLimitError instead of blocking (#477).
  • A credential sent to an unlisted host raises (#532) — only relevant if you wrote a spec that points somewhere unusual.
  • krx rejects raw operation names it does not have (#493). Previously any name returned the same listing.

Added

  • RecordBatch.validation — a typed report of which fields could not be cast, which were missing, and which were undeclared, with counts and sample values (#576, #582).
  • RecordBatch.meta["provenance"] — fetch time, SHA-256 of the raw response, content type, cache hit, and the masked URL and parameters (#583).
  • kpubdata probe — calls each dataset once with your key and sorts it into reachable, needs 활용신청 (403), needs parameters (400) or retired (#504).
  • Spec request parameters (type, required, description, example, enum) are exposed on DatasetRef metadata (#469, #376).
  • Spec datasets: 18 → 23, including the ocean buoy observation spec (#446, marked unstable until checked against the live API), and a license field in the spec schema (#443).

Fixed

  • data.go.kr gateway rejections are reported as what they are, on both the adapter and the spec path, instead of "malformed response envelope" (#478, #485).
  • The documented key names localdata / KPUBDATA_LOCALDATA_API_KEY and the semas equivalents now work; the shared datago key is still accepted (#492).
  • pip install kpubdata without pandas no longer breaks datasets.list() (#487).
  • HTTP errors carry status_code, and a 429 that exhausts retries raises RateLimitError (#484).
  • The response cache is written atomically, and keeps Content-Type across a hit (#484, #496).
  • localdata: resultCode "03" (no data) returns an empty result, empty wrappers no longer become a phantom row, and a trailing slash in base_url no longer produces // (#482, #483).
  • datago.g2b_catalog sends its required inqryDiv automatically (#421).

For contributors

Code comments and docstrings are now in English, with a CI gate against new Korean comments (#517). The project policy (docs/governance/POLICY.md), language policy (ADR 0003) and versioning policy (ADR 0004) were written down, and releases now run through a gated release pull request (#586, #588).

Full changelog: v0.6.0...v0.7.0

v0.6.0

Choose a tag to compare

@yeongseon yeongseon released this 09 Sep 11:13
f235858

kpubdata 0.6.0 removes 141 datasets whose upstream service no longer exists, adds 36 datasets and four providers, and closes several ways an API key could leak. The removal is breaking: check the list below before upgrading. The catalogue goes from 261 datasets to 156.

pip install "kpubdata==0.6.0"

Security

  • API keys are masked in transport logs and exception messages, and when a URL had to be masked the original httpx exception is no longer chained, so the key does not surface through tracebacks either (#260, #331). A key that seoul carries in the URL path is masked too (#354).
  • The response cache no longer shares entries between credentials. Sensitive parameters and Authorization headers are now part of the cache key as a SHA-256 fingerprint (#263).
  • datago.generic refuses hosts other than data.go.kr instead of logging a warning and sending your key anyway. Add hosts with KPUBDATA_DATAGO_EXTRA_HOSTS (#261).
  • HTTP response bodies are capped at 50 MiB by default (TransportConfig.max_response_bytes) (#271).

Behaviour changes — check before upgrading

  • BREAKING: 141 retired datasets removed — 136 from localdata (195 → 59) and 5 from datago: building_area, building_floor, building_recap_title, building_title and metro_path. A body-level re-probe on 2026-09-09 found no OpenAPI service behind them, so they are gone from the catalogue (#411, #412).
  • Canonical query parameters are validated before the adapter is called. page, page_size, cursor, start_date, end_date, fields and sort are checked by name and type, and a bad value raises InvalidRequestError rather than TypeError/ValueError or being passed through as a filter. For example, dataset.list(page="1") and an empty cursor now raise (#264, #328).
  • list_all(max_pages=...) and kpubdata fetch --all -p max_pages=N reject anything but a positive integer with InvalidRequestError (#320), and an invalid cache TTL environment value is rejected (#316).
  • Registering a provider whose catalogue makes a false capability claim now raises CapabilityContractError (#231, #258).
  • 15 existing datasets (12 datago, 3 localdata) are now served from declarative YAML specs instead of adapter code. Dataset ids are unchanged: a spec replaces the catalogue entry of the same name.
  • KPubDataConfig.from_env() and Client.from_env() take typed keyword parameters instead of **kwargs (#276).
  • kpubdata now depends on PyYAML, and the xml and mcp extras accept newer major versions (xmltodict<2, mcp<3).

Added

  • New providers: neis (school meals and school information, #164, #218), fds (MFDS food traceability, #165), korean (Standard Korean Dictionary search, #222) and kipris (patent family search, #223). sgis is now registered in the default Client(); in 0.3.0–0.5.0 it had to be added with Client.register_provider() (#332).
  • datago: weather — asos_daily, asos_hourly (#217), mid-term forecasts mid_fcst, mid_land_fcst, mid_sea_fcst, mid_ta (#251) and ultra_srt_fcst; air quality — airkorea_station_realtime, airkorea_forecast (#224) and air_station; nine MFDS DUR drug-safety datasets (dur_*); and agri_price (#248), bond_price and sports_facility (#163, #166), road_traffic (#87), subway_passengers (#93, #259) and culture_facility.
  • seoul: bike_realtime, bike_station_master, park_info (#246), park_usage (#227) and citydata (#225).
  • bok: money_supply (legacy M2 series, 1986-01 to 2004-09) (#245).
  • Declarative dataset specs: a dataset can be defined in YAML under src/kpubdata/specs/, validated against a JSON Schema; 18 specs ship in this release (#378).
  • kpubdata scaffold provider <name> generates the skeleton of a new provider adapter (#61, #257).
  • with_retry_async and an injectable sleep function for retry backoff (#270).

Fixed

  • Concurrent first use of a provider no longer raises ProviderNotRegisteredError, and a failing provider factory can be retried (#262).
  • bok treats every RESULT payload as an error regardless of its code; seoul detects top-level API errors before parsing the envelope; lofin URL-encodes filter values.
  • Catalogue search reuses its index across unchanged listings (#279).

For contributors

A record/replay verification pipeline (make record, make verify, KPUBDATA_MODE=replay), bulk catalogue-to-spec migration scripts, daily live smoke tests with drift reporting (#382), an MkDocs site, and the cross-repository compatibility matrix and release policy (#233, #256).

Full changelog: v0.5.0...v0.6.0

v0.5.0

Choose a tag to compare

@github-actions github-actions released this 27 Apr 15:35

kpubdata 0.5.0 adds two providers (law and krx), 10 datasets on existing providers, and richer dataset metadata with relevance-ranked search. The catalogue grows from 245 datasets to 261.

pip install "kpubdata==0.5.0"

Behaviour changes — check before upgrading

  • datasets.search() is now fuzzy and ranked. It used to be a substring match delegated to each adapter. It now scores name, description, tags and id (NFC-normalised, so Korean text matches consistently), returns results sorted by relevance, and drops anything below threshold (default 0.5). Expect more, and differently ordered, results (#63, #185).

Added

  • law provider (법제처, law.go.kr): law.law_search, law.law_detail and law.ordin_search (#180, #182).
  • krx provider (Korea Exchange): krx.kospi_index, krx.investor_flow and krx.market_valuation. It needs no API key and uses pykrx, which you install with pip install "kpubdata[krx]" (#199, #200, #204, #205).
  • Adapters declare requires_api_key, and Client.iter_authenticated_providers() lists only the providers that need a key (#204).
  • bok: bok.usd_krw (KRW/USD daily rate) and bok.bond_yield_3y (3-year treasury yield) from ECOS (#201, #202).
  • kosis: kosis.industrial_production, and dataset-level default query parameters that your own filters override (#203).
  • datago: four building-register datasets (building_area, building_floor, building_recap_title, building_title) (#181), and g2b_contract, g2b_catalog and social_enterprise (#193, #195). social_enterprise is served from api.odcloud.kr; the datago adapter now handles that host's page/perPage paging and flat data[] responses.
  • DatasetRef gains description, tags and source_url, filled in across the provider catalogues (#55, #183).
  • FieldConstraints describes a schema field's max_length, min_value, max_value, pattern, allowed_values and format, exposed as FieldDescriptor.constraints (#57, #184).
  • Cookbook guides (getting started, discovery, raw access) and example scripts (#59, #186).

For contributors

Built-in providers are listed in providers/manifest.py (#178), and semas, sgis and law gained live integration tests (#80, #187).

Full changelog: v0.4.0...v0.5.0

v0.4.0

Choose a tag to compare

@yeongseon yeongseon released this 23 Apr 15:11
f300ca5

kpubdata 0.4.0 adds the semas provider and makes every data.go.kr-based provider use a single API key. If you set KPUBDATA_LOCALDATA_API_KEY or KPUBDATA_LOFIN_API_KEY, read the behaviour change below before upgrading.

pip install "kpubdata==0.4.0"

Behaviour changes — check before upgrading

  • BREAKING: localdata and lofin now read the datago key. KPUBDATA_LOCALDATA_API_KEY and KPUBDATA_LOFIN_API_KEY are no longer read, and neither is provider_keys={"localdata": ...} or {"lofin": ...}. Set KPUBDATA_DATAGO_API_KEY (or provider_keys={"datago": ...}) once for all data.go.kr-based providers (#176). The new semas provider uses the same key. (0.7.0 accepts the localdata key name again, #492; lofin still reads only the datago key.)

Added

  • semas provider (소상공인시장진흥공단 commercial district and store data) with 17 datasets: 4 commercial-zone, 10 store and 3 industry-category lookups (#170).

For contributors

Contract tests use the shared datago key (#177). The roadmap was realigned around core, plugin and provider work, and ARCHITECTURE.md states that the MCP adapter lives outside this repository.

Full changelog: v0.3.1...v0.4.0

v0.3.1

Choose a tag to compare

@yeongseon yeongseon released this 23 Apr 03:51
f84b3e6

A patch release for 0.3.0 with two fixes.

pip install "kpubdata==0.3.1"

Fixed

  • kpubdata.__version__ reported 0.2.3 in 0.3.0. It is now read from the installed package metadata, so it always matches the release (#127, #158).
  • A 403 from data.go.kr now raises AuthError with a hint: the API usually has not been activated (활용신청) for your key. Previously it surfaced as a generic transport error. The quickstart and datago docs explain the activation step (#128, #159).

For contributors

The metro integration tests pass their required parameters (#140, #160).

Full changelog: v0.3.0...v0.3.1

v0.3.0

Choose a tag to compare

@yeongseon yeongseon released this 22 Apr 15:29
b82e7ba

kpubdata 0.3.0 adds three providers (localdata, sgis and seoul), an escape hatch for data.go.kr APIs that are not in the catalogue, a command-line interface and an opt-in response cache. The catalogue grows from 21 datasets to 228.

pip install "kpubdata==0.3.0"

Added

  • localdata provider (지방행정인허가, local business permits) with 195 permit datasets (#136, #138, #155). It reads its own key, KPUBDATA_LOCALDATA_API_KEY (0.4.0 moved it to the shared datago key).
  • sgis provider for administrative boundary GeoJSON: sgis.boundary.sido, sgis.boundary.sigungu and sgis.boundary.emd. It handles SGIS's access-token flow (consumer_key + consumer_secret), caches the token in memory and refreshes it on an auth failure (#150, #151). Note: sgis was not registered in the default Client() until 0.6.0 (#332), so in this release you register it yourself with client.register_provider(...).
  • seoul provider for Seoul Open Data Plaza: seoul.subway_realtime_arrival and seoul.bike_rent_month (#149).
  • datago.generic, a raw escape hatch for data.go.kr APIs that are not in the catalogue. It returns the decoded response as-is, with no normalisation, pagination or schema, and list() on it raises InvalidRequestError (#133).
  • Six more datago datasets: four TourAPI 4 datasets (tour_kor_area, tour_kor_festival, tour_kor_keyword, tour_kor_location) plus metro_fare and metro_path (#134).
  • kpubdata command-line interface: datasets list, datasets show, fetch (with --all and --output) and raw (#145).
  • Opt-in disk response cache: Client(cache=True, cache_ttl_seconds=...). It is off by default, and the TTL defaults to 24 hours (#144).
  • Structured logging across modules, including debug logs on adapter and transport failure paths (#131, #142).

Fixed

  • datago.bus_arrival moved to the v2 endpoint and understands its msgHeader/msgBody envelope (#146, #147).

For contributors

Live integration tests no longer break when their hard-coded dates expire, and SUPPORTED_DATA.md marks more datago datasets as live-verified (#141, #143, #148).

Full changelog: v0.2.3...v0.3.0

v0.2.3

Choose a tag to compare

@github-actions github-actions released this 19 Apr 09:34

kpubdata 0.2.3 makes the MOLIT real-estate transaction datasets and the lofin provider actually work, and adds seven more real-estate datasets. If you use datago.apt_trade or any lofin dataset, upgrade.

pip install -U "kpubdata==0.2.3"

Added

  • Seven real-estate transaction datasets on datago: apt_rent, offi_trade, offi_rent, rh_trade, rh_rent, sh_trade and sh_rent — apartment rentals, and sales and rentals for officetels, row houses and detached houses (#123). These were checked against recorded fixtures, not yet against the live API.

Fixed

  • datago.apt_trade always failed with ProviderResponseError: OK. The real-estate APIs report success as resultCode "000", which was treated as an error. Any all-zero result code now counts as success (#130).
  • lofin datasets failed with an SSL handshake error when used through Client. Since 0.2.1, the adapter's TLS settings for the LOFIN server were skipped whenever Client supplied the shared transport; the adapter now declares them through TransportRequirements (#122, #124). Note: those settings also turned certificate verification off for LOFIN requests. That was fixed in 0.7.0 (#488).

Full changelog: v0.2.2...v0.2.3

v0.2.2

Choose a tag to compare

@github-actions github-actions released this 17 Apr 12:32

kpubdata 0.2.2 is a small follow-up to 0.2.1's paging change: list_all() now understands cursor-based paging as well as page numbers.

pip install -U "kpubdata==0.2.2"

Added

  • Dataset.list_all() follows next_cursor when a batch returns one, and falls back to next_page otherwise (#121). No built-in provider returns a cursor yet; this matters if you write your own adapter.

Known issue

  • lofin datasets still fail with an SSL handshake error when used through Client (since 0.2.1). Fixed in 0.2.3 (#122, #124).

For contributors

The paging contract for adapters, including the best-effort len(items) == page_size heuristic datago uses when a response has no total count, is documented in ARCHITECTURE.md and PROVIDER_ADAPTER_CONTRACT.md (#121).

Full changelog: v0.2.1...v0.2.2