Repository navigation
Releases: kpubdata-lab/kpubdata
Release list
v0.9.0
Added
DatasetRef.to_dict()(#784): a dataset reference serialises to a JSON-safe dict with a stable set of keys — identity,status(#783),query_support,license,request_parameters,applicationandverified_at.dataclasses.asdict(ref)raisedTypeErroronraw_metadata, so a consumer had to readraw_metadata, which carries no stability promise. Every key is always present;Nonemeans nothing is declared. The key list is indocs/compatibility.md§3.kpubdata datasets list --format jsonanddatasets show --format jsonprint this dict:listentries gain the new keys next toid,name,providerandoperations, andshowkeepscapabilitiesandraw_metadata_keysbeside them.kpubdata.SENSITIVE_PARAM_KEYS(#782): the names kpubdata masks as credentials —servicekey,service_key,api_key,apikey,token,authorization,secret,password,key,oc,accesstoken,consumer_key,consumer_secret— are importable fromkpubdataandkpubdata.transport. They lived only in the privatekpubdata.transport._sensitive, so Builder kept its own list, and that list lackskey,oc,consumer_keyandconsumer_secret. Names are casefolded and matched exactly; a release only adds names. Value-based masking stays private.DatasetRef.status(#783): a dataset reference says how far the dataset has been verified, as aDatasetStatus(now exported fromkpubdata). A dataset awaiting an application, one checked against fixtures only and one verified against the live API used to look the same at run time; the levels lived only inSUPPORTED_DATA.md, which is not in the wheel.scripts/gen_dataset_status.pywritessrc/kpubdata/dataset_status.jsonfrom that document's level column (throughSUPPORTED_DATA_LEVELS, with a spec'sstatus:override applied), andtests/unit/test_dataset_ref_status.pyfails when the two differ. Of the 155 datasets the table and the runtime share: 94application_required, 36fixture_verified, 24live_verified, 1unstable(datago.ocean_buoy, by its spec's override);datago.genericis not in the table and reportsNone. The status is the recorded one, not a live check — that isClient.probe.- Evidence bound to the spec, the commit and the run (#522, #713): a recorded fixture's
meta.jsonnow carriesspec_sha256, a digest of the spec file's pipeline-relevant content (kpubdata.core.spec.spec_file_digest, which normalises away thelast_verifiedline the recorder itself rewrites), andrecord_commit/run_reffrom the CI environment (GITHUB_SHA/GITHUB_RUN_ID) when present.make verifygains a spec-binding step: a fixture whose recorded digest no longer matches its spec is void — the spec changed after recording — and fails with re-record guidance. Fixtures recorded before the digest existed are reported as legacy evidence rather than failed — since #717, only those on a frozen baseline. - Public probe API (#694):
Client.probe(dataset_id) -> ProbeResult | NoneandClient.probe_all(*, provider=None) -> list[ProbeResult]classify reachability with the client's own keys, andProbeResultandPROBE_STATUSES(the ADR 0005ProbeStatusvocabulary) are exported fromkpubdata. Probing keeps its fast-fail transport (PROBE_TIMEOUT_SECONDS,PROBE_RETRIES, no cache), and a run shares one transport that is closed at the end — previously every dataset opened its own and none was closed.kpubdata probenow goes through these methods; its output and report are unchanged, and it now honors the global--provider-keyoption, which it silently ignored before. - Explicit-keys-only mode (#694):
Client(..., env_keys=False)(backed byKPubDataConfig.env_fallback) uses onlyprovider_keysand never reads a credential from the environment — notKPUBDATA_<P>_API_KEY,<P>_API_KEYorKPUBDATA_SGIS_CONSUMER_SECRET. A service probing with a user's key no longer has a missing entry silently filled with the operator's key. With no key the probe makes no call and reportsauth_unknown. The default (env_keys=True) is unchanged. - R3 review gate (#722, kpubdata-builder#905): a pull request labelled
review:R3now needs an approval from someone other than its author before it can merge.scripts/r3_review.py, wrapped by the.github/actions/r3-reviewcomposite action, is the one implementation all four repositories call; theR3 reviewworkflow runs it on every pull request (passing when it is not R3) and again on label changes, pushes and review submissions or dismissals, withpull-requests: readonly. Each reviewer's latest approving or change-requesting review decides; approvals on an older head count, as with branch protection's stale-approval dismissal off; the author, bots and accounts without write access do not count. POLICY 14.1 and AGENTS.md describe it.R3 reviewbecomes a required status check next toCI gate; the approval count in branch protection stays at 0. - Probe evidence fields (#625, #710):
ProbeResultcarrieshttp_status(200 on success — the executor raises on anything else — and the raised error's status otherwise),result_code(read off the raised error; a successful query does not parse the envelope),latency_ms,schema_hash(a fingerprint of the returned field names — data changes daily, field names are the contract) andclassification(theDriftClassificationthis outcome feeds the dataset status machine), eachNonewhere it could not be observed — never guessed. The report envelope records who ran the probe (runner; the nightly workflow passesgithub-actions). - Nightly drift detection (#625, #711):
scripts/drift_detect.pyruns the dataset status machine over two consecutive probe reports — a differingschema_hashemitsSCHEMA_CHANGEDitself (it is not a probe outcome), transient failures move nothing before the third consecutive night, restores restoreprevious_statusand file nothing,RETIREDmoves nothing but files. The live-probe workflow grows a drift job that chains the previous night's artifacts, keeps streak state in adrift-stateartifact, and files one[drift]issue per transition that needs a person — skipping datasets that already have an open one, with thestatus_history.jsonentry riding in the issue body. The exit status is always 0: an upstream failure is a drift event, not a CI failure. - Production-grade gate (#463, #711):
scripts/check_production_grade.pyevaluates the machine-checkable criteria ofdocs/production_grade.yaml— source and listing gate every spec; a recorded fixture (any*.meta.json: the recorder names fixtures after the example,tour_kor_arearecordsseoul_restaurants.meta.json) and an example script gate the verified tier only (an in-progress row gates nothing — ocean_buoy's missing fixture is #627, not a surprise);fields_declaredis reported, not gated. Its unit test is the CI gate: every spec dataset in the repository must pass. - Example recency windows (#734, #750): a spec can declare
params[].max_age_days, andmake verifyfails an example outside the window before the live call does — naming the cause (expired, or a future issue that does not answer yet — the #731 morning trap) and the refresh route. Parameters without a window emit nothing. The relative-time representation the issue considered first was rejected: replay matches on recorded literals, and issue-slot granularity would leak provider specifics into the generic schema. - Recorder run reference (#523, #729, #739): the evidence-authorship gate grows a second layer — a changed
meta.jsonby a recorder name must also carryrecord_commitandrun_ref, andrun_refmust resolve to a run of this repository's Build Dataset workflow whose head SHA is exactly that commit (gh run view; any lookup failure fails closed). A local tool committing under the recorder's name — the #728 path — no longer passes.ci.ymlgainsactions: readand the job token for the lookup. - Unconfirmed terms are not redistributable (#732): a spec whose licence says
redistribution: allowedmust carry theattributiontext that proves the terms were confirmed from the provider page (#525); terms nobody checked sayredistribution: unknown— the two claims #728 wrongly shipped (bus_arrival, social_enterprise) were fixed to unknown first (#740, #744), because Builder's publish gate (kpubdata-builder#892) reads exactly this flag.make verifyenforces the rule with a shrink-only baseline,scripts/unconfirmed_terms_baseline.txt, frozen at the 16 legacy violators — mirroring the legacy evidence ratchet (#717): a listed spec passes while its terms stay unconfirmed, an unlisted violation fails verify, and a stale entry (terms confirmed, or the claim withdrawn) must be removed from the list. - PR title blocks the merge (#741):
PR titlejoinedCI gateandR3 reviewas a required status check in all four product repositories, so a failing title check now closes the merge button instead of only showing a red X. The check re-runs on title edits (titles.yml), whichci.ymlcannot see, so it is registered in branch protection rather than added to the aggregate gate's needs.scripts/check_required_checks.pyverifies the required list against the workflows (the studio#416 lesson), and POLICY 14.2 records the rule. - Unjustified plain http is refused (#738): a spec whose
endpoint.base_urlsayshttp://must carryendpoint.insecure_http_reason— the service key rides the query string, and over plain http anyone on the network path reads it.make verifygains a transport-security step enforced with a shrink-only baseline,scripts/insecure_http_baseline.txt, frozen at the 23 legacy http specs — all of them onapis.data.go.kr, which answers https with identical envelopes (the reproducible record — commands and raw envelopes for every service path behind the baseline ...
v0.8.0
Security
RecordBatch.meta["provenance"]["url"]no longer leaks the API key. It was masked withstr.replaceof the plain key, which missed a percent-encoded data.go.kr key (+,/,=) and a path-segment key; it is now masked by parameter name (including the spec's own auth parameter) and by every encoded form of the key, and omitted if a key form survives (#612).- Dependencies with known vulnerabilities are upgraded in
uv.lock(kpubdata-builder#691). pip-audit over every extra reported 87 findings (48 distinct advisories) in 14 packages; none remain. urllib3 2.6.3 → 2.8.0, idna 3.11 → 3.20, cryptography 46.0.6 → 50.0.1, pyjwt 2.12.1 → 2.15.1, pillow 12.2.0 → 12.3.0, mcp 1.26.0 → 2.2.0 (with starlette 1.7.0, python-multipart 0.0.32, click 8.5.0; anyio 4.14.2 on Python < 3.12 and 4.15.1 on ≥ 3.12, typing-extensions 4.15.0 / 4.16.0 likewise; adds httpx2, httpcore2, httpx2-jsfetch, truststore, opentelemetry-api and mcp-types; drops httpx-sse, python-dotenv and pydantic-settings), pytest 9.1.1, mkdocs-material 9.7.7, pymdown-extensions 12.1 — all inside the declared ranges, so only the lock moves. The gitleaks secret scan over the full history now runs in CI and gates merges throughCI gate; aSecurityworkflow runs pip-audit over the locked dependencies (for Python 3.10 and 3.12, so both sides of the< 3.12markers are audited) and CodeQL on every pull request and weekly — gating those two is left to #631.
Added
-
Spec fields can declare
semantic_kind(code,measure,date,period,text,flag),titleandformat(ADR 0006).Dataset.schema()returns them asFieldDescriptor.semantic_kind,FieldDescriptor.titleandFieldConstraints.format. A kind that contradicts the storage type, a numeric transform on acode, auniton anything but ameasure, or an unknown kind fails the spec load andvalidate_spec.py; fields without a kind are not checked. The 16 code columns from #613 declaresemantic_kind: code(#651). -
Spec
licensegainsredistribution(allowed/non_commercial/forbidden/unknown; absent means unknown),attribution(the exact text to display),quotaandpii_columns(#525, #605). -
DatasetRef.licensecarries a spec's licence terms — redistribution, attribution,quota, PII columns — exactly as declared, andLicenseSpecis exported fromkpubdata(#609). A dataset that declares no licence hasNone, which means unknown rather than unrestricted;quotais the provider's own wording and is not parsed. Catalogue-only datasets haveNone. -
Dataset.schema()returns aSchemaDescriptorfor spec datasets, built from the spec'sfieldsin declaration order (name, type, description;unit,source_name,transforminraw), and spec datasets with fields declareOperation.SCHEMA. A spec with no fields still returnsNone.title/formatstay unset until the column-metadata contract (#644) decides them (#643). -
ADR 0006: the column-metadata contract separates storage type (
type), meaning (semantic_kind: code, measure, date, period, text, flag) and display (reusingFieldDescriptor.titleandFieldConstraints.format). Implementation is #651 (#644). -
kpubdata.core.status— one canonical dataset status vocabulary
(DatasetStatus) with mappings from the spec, probe,SUPPORTED_DATA.mdand
production-grade vocabularies;spec._STATUSESand_probe.PROBE_STATUSESare
now derived from it. ADR 0005 records the decision, and
tests/unit/test_status_vocabulary.pyfails when a design document uses a name
the code does not define (#619). -
kpubdata.core.status.transition()— the pure dataset status state machine fromdocs/DATASET_STATUS.md. A table-driven test runs every row of that document's transition table against the function.unstablenow also breaks immediately on a structural change, andapplication_requiredrecovers onHEALTHY(#625). -
ValidationReport.to_dict()returns a JSON-serialisable form of the report (#615). -
Provenance now reports
cached=Truefor responses served from the response cache, andfetched_atkeeps the original fetch time instead of the time of the cache read. The transport marks a cache hit inhttpx.Response.extensions, andResponseCache.get_entry()returns the storedcreated_at(#616).
Changed
-
BREAKING: code columns keep their leading zeros and come back as
str(#613). Declaredstringnow:apt_tradebonbun/bubun/roadNmBonbun/roadNmBubun/roadNmSeq,apt_rentroadnmbonbun/roadnmbubun,hospital_infoclCd/postNo,metro_farearvlStnCd/dptreStnCd,tour_kor_*zipcode,village_fcstfcstTime. As a safety net, a zero-led value ("06102") no longer casts to a number, so a code column declared numeric stays text and is reporteduncastable. Replay verification (make verify) now also fails when normalization drops a leading zero. See kpubdata-builder#702. -
KPubData is documented as a standalone Python SDK; KPubData Builder and KPubData Studio are listed as related projects, not as stages after it (#666). ADR 0007 records the independence rules — dependencies run Studio → Builder → KPubData only, and the public API is what
kpubdata.__all__andAPI_SPEC.mdname (#667) — andscripts/check_independence.pyfails CI if KPubData imports or depends on Builder or Studio (#668). -
All 59
localdatacatalogue datasets are marked retired andLocaldataAdapter.query_recordsemits aDeprecationWarning(#527, #603). The retirement is disputed by the recorded evidence — see #618. -
CODEOWNERS covers the credential host allowlist, the scripts run by
contents: writerelease jobs,pyproject.tomlanduv.lock; a test checks the required paths and that every pattern still names an existing path (#629). -
FieldIssue.kindandValidationReport.issues_of()take theIssueKindliteral ("uncastable","missing","undeclared");issues_of()raisesValueErroron an unknown kind instead of silently returning nothing (#615).
Fixed
-
scripts/release_notes.py promotekeeps a CRLF CHANGELOG's line endings instead of rewriting every line as LF (#628). -
kpubdata scaffoldgenerates English docstrings, so the files it writes passcheck_english_comments.py; a test runs the gate on the generated files (#626). -
A provider-reported total of
0is kept asRecordBatch.total_count == 0instead of becomingNone, so "no results" and "count unknown" are distinguishable; a reported 0 explicitly ends paging (#642). -
Client(...).dataset(...).list_all()now reaches the spec path with column casting decided across all pages.CompositeProviderAdapterhad noquery_records_all, so every spec dataset reached throughClientstill cast per page and a column could beinton one page andstron the next — the split 0.7.0 recorded as fixed (#611). -
docs/status.mdno longer depends on the calendar: "recent" is measured from the newestlast_verified, so--checkstops failing every pull request from 2026-12-09. It also counts all six SUPPORTED_DATA levels instead of three (#620). -
docs/DATASET_STATUS.mdanddocs/LIVE_PROBE.mdcontradicted each other and the
code: the failure threshold (3 against "2+"), code 32's classification, the
PARAM_CHANGEDspelling, a nonexistent dataset key, a nonexistent module and the
neis/fds key sharing claim (#619). -
scripts/release_notes.py promotefor a final version folds that version's pre-release sections (aN,bN,rcN) into one section, instead of failing with "nothing to release" after a pre-release. Aprereleasecommand and the release-notes action'sprereleaseoutput tell the release job to mark a pre-release (#622). -
The spec-dataset
list_all()path honourspage_size,max_sizeandmax_pages(#614).page_sizedrives pagination instead of going out as a rawpage_sizeparameter; apage_sizeabove the spec'smax_sizeno longer ends the walk after the first page; exceedingmax_pagesraisesInvalidRequestErrorlike the legacy path instead of truncating silently; and each batch keeps its ownraw,meta["provenance"],next_pageandvalidation, with the whole-result report inmeta["validation_total"]. The path still buffers every page before the first batch, because casting is decided across all pages (#481); this is now documented. -
The field validation report no longer miscounts (#615): a 0-row page is reported clean instead of every declared field being
missing; numeric null markers ("","-") count as nulls rather than non-null values; undeclared columns are detected across all records, not only the first.
Full changelog: v0.7.0...v0.8.0
v0.7.0
kpubdata 0.7.0 is mostly a security and correctness release. If you use 0.6.x, upgrade: several paths could leak your API key or send it somewhere it should not go.
pip install -U "kpubdata==0.7.0"kpubdata-builder 0.4.0 is pinned to kpubdata<0.7 and still installs 0.6.x. The Builder release that moves to 0.7 will be 0.4.1.
Security
- Your API key could reach logs, tracebacks and error trackers. When the key travelled as a query parameter (
params=, used by datago, localdata, semas, sgis and every spec dataset), the exception chain kept the full URL with?serviceKey=…(#486). bok carried its key in the URL path and law in a parameter namedOC; neither was masked (#475). sgis tokens and consumer secrets were not masked either (#484). - lofin turned TLS certificate verification off. Every lofin request, including the key, would accept any certificate (#488). Verification is now on; only the cipher level is relaxed, which is all the server needs.
- A spec could send your provider key to any host. The executor now refuses to attach a credential when
endpoint.base_urlis not a host listed for that provider (#532, #519). - Error responses and tokens were cached for 24 hours. Korean public APIs report quota and key errors in an HTTP 200 body, so a momentary quota breach was served from cache for a day (#490).
Behaviour changes — check before upgrading
- Numeric columns cast differently. A declared numeric column is cast only if every value in it casts (#468). Thousands separators are understood:
"1,200"becomes1200, anddatago.apt_trade.dealAmountis now an integer, not a string (#574).list_all()applies the rule across all pages at once, so page 1 and page 2 can no longer disagree on a column's type (#575). - An invalid
licensefield in a spec now fails the load instead of being dropped silently (#476). - 4xx responses are no longer retried (#490). A
Retry-Afterlonger thanTransportConfig.max_retry_delay(default 60 s) now raises a retryableRateLimitErrorinstead of blocking (#477). - A credential sent to an unlisted host raises (#532) — only relevant if you wrote a spec that points somewhere unusual.
- krx rejects raw operation names it does not have (#493). Previously any name returned the same listing.
Added
RecordBatch.validation— a typed report of which fields could not be cast, which were missing, and which were undeclared, with counts and sample values (#576, #582).RecordBatch.meta["provenance"]— fetch time, SHA-256 of the raw response, content type, cache hit, and the masked URL and parameters (#583).kpubdata probe— calls each dataset once with your key and sorts it into reachable, needs 활용신청 (403), needs parameters (400) or retired (#504).- Spec request parameters (type, required, description, example, enum) are exposed on
DatasetRefmetadata (#469, #376). - Spec datasets: 18 → 23, including the ocean buoy observation spec (#446, marked unstable until checked against the live API), and a
licensefield in the spec schema (#443).
Fixed
- data.go.kr gateway rejections are reported as what they are, on both the adapter and the spec path, instead of "malformed response envelope" (#478, #485).
- The documented key names
localdata/KPUBDATA_LOCALDATA_API_KEYand the semas equivalents now work; the shared datago key is still accepted (#492). pip install kpubdatawithout pandas no longer breaksdatasets.list()(#487).- HTTP errors carry
status_code, and a 429 that exhausts retries raisesRateLimitError(#484). - The response cache is written atomically, and keeps
Content-Typeacross a hit (#484, #496). - localdata:
resultCode "03"(no data) returns an empty result, empty wrappers no longer become a phantom row, and a trailing slash inbase_urlno longer produces//(#482, #483). datago.g2b_catalogsends its requiredinqryDivautomatically (#421).
For contributors
Code comments and docstrings are now in English, with a CI gate against new Korean comments (#517). The project policy (docs/governance/POLICY.md), language policy (ADR 0003) and versioning policy (ADR 0004) were written down, and releases now run through a gated release pull request (#586, #588).
Full changelog: v0.6.0...v0.7.0
v0.6.0
kpubdata 0.6.0 removes 141 datasets whose upstream service no longer exists, adds 36 datasets and four providers, and closes several ways an API key could leak. The removal is breaking: check the list below before upgrading. The catalogue goes from 261 datasets to 156.
pip install "kpubdata==0.6.0"Security
- API keys are masked in transport logs and exception messages, and when a URL had to be masked the original httpx exception is no longer chained, so the key does not surface through tracebacks either (#260, #331). A key that
seoulcarries in the URL path is masked too (#354). - The response cache no longer shares entries between credentials. Sensitive parameters and
Authorizationheaders are now part of the cache key as a SHA-256 fingerprint (#263). datago.genericrefuses hosts other than data.go.kr instead of logging a warning and sending your key anyway. Add hosts withKPUBDATA_DATAGO_EXTRA_HOSTS(#261).- HTTP response bodies are capped at 50 MiB by default (
TransportConfig.max_response_bytes) (#271).
Behaviour changes — check before upgrading
- BREAKING: 141 retired datasets removed — 136 from
localdata(195 → 59) and 5 fromdatago:building_area,building_floor,building_recap_title,building_titleandmetro_path. A body-level re-probe on 2026-09-09 found no OpenAPI service behind them, so they are gone from the catalogue (#411, #412). - Canonical query parameters are validated before the adapter is called.
page,page_size,cursor,start_date,end_date,fieldsandsortare checked by name and type, and a bad value raisesInvalidRequestErrorrather thanTypeError/ValueErroror being passed through as a filter. For example,dataset.list(page="1")and an emptycursornow raise (#264, #328). list_all(max_pages=...)andkpubdata fetch --all -p max_pages=Nreject anything but a positive integer withInvalidRequestError(#320), and an invalid cache TTL environment value is rejected (#316).- Registering a provider whose catalogue makes a false capability claim now raises
CapabilityContractError(#231, #258). - 15 existing datasets (12 datago, 3 localdata) are now served from declarative YAML specs instead of adapter code. Dataset ids are unchanged: a spec replaces the catalogue entry of the same name.
KPubDataConfig.from_env()andClient.from_env()take typed keyword parameters instead of**kwargs(#276).- kpubdata now depends on PyYAML, and the
xmlandmcpextras accept newer major versions (xmltodict<2,mcp<3).
Added
- New providers:
neis(school meals and school information, #164, #218),fds(MFDS food traceability, #165),korean(Standard Korean Dictionary search, #222) andkipris(patent family search, #223).sgisis now registered in the defaultClient(); in 0.3.0–0.5.0 it had to be added withClient.register_provider()(#332). - datago: weather —
asos_daily,asos_hourly(#217), mid-term forecastsmid_fcst,mid_land_fcst,mid_sea_fcst,mid_ta(#251) andultra_srt_fcst; air quality —airkorea_station_realtime,airkorea_forecast(#224) andair_station; nine MFDS DUR drug-safety datasets (dur_*); andagri_price(#248),bond_priceandsports_facility(#163, #166),road_traffic(#87),subway_passengers(#93, #259) andculture_facility. - seoul:
bike_realtime,bike_station_master,park_info(#246),park_usage(#227) andcitydata(#225). - bok:
money_supply(legacy M2 series, 1986-01 to 2004-09) (#245). - Declarative dataset specs: a dataset can be defined in YAML under
src/kpubdata/specs/, validated against a JSON Schema; 18 specs ship in this release (#378). kpubdata scaffold provider <name>generates the skeleton of a new provider adapter (#61, #257).with_retry_asyncand an injectable sleep function for retry backoff (#270).
Fixed
- Concurrent first use of a provider no longer raises
ProviderNotRegisteredError, and a failing provider factory can be retried (#262). - bok treats every
RESULTpayload as an error regardless of its code; seoul detects top-level API errors before parsing the envelope; lofin URL-encodes filter values. - Catalogue search reuses its index across unchanged listings (#279).
For contributors
A record/replay verification pipeline (make record, make verify, KPUBDATA_MODE=replay), bulk catalogue-to-spec migration scripts, daily live smoke tests with drift reporting (#382), an MkDocs site, and the cross-repository compatibility matrix and release policy (#233, #256).
Full changelog: v0.5.0...v0.6.0
v0.5.0
kpubdata 0.5.0 adds two providers (law and krx), 10 datasets on existing providers, and richer dataset metadata with relevance-ranked search. The catalogue grows from 245 datasets to 261.
pip install "kpubdata==0.5.0"Behaviour changes — check before upgrading
datasets.search()is now fuzzy and ranked. It used to be a substring match delegated to each adapter. It now scores name, description, tags and id (NFC-normalised, so Korean text matches consistently), returns results sorted by relevance, and drops anything belowthreshold(default 0.5). Expect more, and differently ordered, results (#63, #185).
Added
lawprovider (법제처, law.go.kr):law.law_search,law.law_detailandlaw.ordin_search(#180, #182).krxprovider (Korea Exchange):krx.kospi_index,krx.investor_flowandkrx.market_valuation. It needs no API key and usespykrx, which you install withpip install "kpubdata[krx]"(#199, #200, #204, #205).- Adapters declare
requires_api_key, andClient.iter_authenticated_providers()lists only the providers that need a key (#204). - bok:
bok.usd_krw(KRW/USD daily rate) andbok.bond_yield_3y(3-year treasury yield) from ECOS (#201, #202). - kosis:
kosis.industrial_production, and dataset-level default query parameters that your own filters override (#203). - datago: four building-register datasets (
building_area,building_floor,building_recap_title,building_title) (#181), andg2b_contract,g2b_catalogandsocial_enterprise(#193, #195).social_enterpriseis served fromapi.odcloud.kr; the datago adapter now handles that host'spage/perPagepaging and flatdata[]responses. DatasetRefgainsdescription,tagsandsource_url, filled in across the provider catalogues (#55, #183).FieldConstraintsdescribes a schema field'smax_length,min_value,max_value,pattern,allowed_valuesandformat, exposed asFieldDescriptor.constraints(#57, #184).- Cookbook guides (getting started, discovery, raw access) and example scripts (#59, #186).
For contributors
Built-in providers are listed in providers/manifest.py (#178), and semas, sgis and law gained live integration tests (#80, #187).
Full changelog: v0.4.0...v0.5.0
v0.4.0
kpubdata 0.4.0 adds the semas provider and makes every data.go.kr-based provider use a single API key. If you set KPUBDATA_LOCALDATA_API_KEY or KPUBDATA_LOFIN_API_KEY, read the behaviour change below before upgrading.
pip install "kpubdata==0.4.0"Behaviour changes — check before upgrading
- BREAKING: localdata and lofin now read the
datagokey.KPUBDATA_LOCALDATA_API_KEYandKPUBDATA_LOFIN_API_KEYare no longer read, and neither isprovider_keys={"localdata": ...}or{"lofin": ...}. SetKPUBDATA_DATAGO_API_KEY(orprovider_keys={"datago": ...}) once for all data.go.kr-based providers (#176). The newsemasprovider uses the same key. (0.7.0 accepts the localdata key name again, #492; lofin still reads only the datago key.)
Added
semasprovider (소상공인시장진흥공단 commercial district and store data) with 17 datasets: 4 commercial-zone, 10 store and 3 industry-category lookups (#170).
For contributors
Contract tests use the shared datago key (#177). The roadmap was realigned around core, plugin and provider work, and ARCHITECTURE.md states that the MCP adapter lives outside this repository.
Full changelog: v0.3.1...v0.4.0
v0.3.1
A patch release for 0.3.0 with two fixes.
pip install "kpubdata==0.3.1"Fixed
kpubdata.__version__reported0.2.3in 0.3.0. It is now read from the installed package metadata, so it always matches the release (#127, #158).- A 403 from data.go.kr now raises
AuthErrorwith a hint: the API usually has not been activated (활용신청) for your key. Previously it surfaced as a generic transport error. The quickstart and datago docs explain the activation step (#128, #159).
For contributors
The metro integration tests pass their required parameters (#140, #160).
Full changelog: v0.3.0...v0.3.1
v0.3.0
kpubdata 0.3.0 adds three providers (localdata, sgis and seoul), an escape hatch for data.go.kr APIs that are not in the catalogue, a command-line interface and an opt-in response cache. The catalogue grows from 21 datasets to 228.
pip install "kpubdata==0.3.0"Added
localdataprovider (지방행정인허가, local business permits) with 195 permit datasets (#136, #138, #155). It reads its own key,KPUBDATA_LOCALDATA_API_KEY(0.4.0 moved it to the shared datago key).sgisprovider for administrative boundary GeoJSON:sgis.boundary.sido,sgis.boundary.sigunguandsgis.boundary.emd. It handles SGIS's access-token flow (consumer_key+consumer_secret), caches the token in memory and refreshes it on an auth failure (#150, #151). Note: sgis was not registered in the defaultClient()until 0.6.0 (#332), so in this release you register it yourself withclient.register_provider(...).seoulprovider for Seoul Open Data Plaza:seoul.subway_realtime_arrivalandseoul.bike_rent_month(#149).datago.generic, a raw escape hatch for data.go.kr APIs that are not in the catalogue. It returns the decoded response as-is, with no normalisation, pagination or schema, andlist()on it raisesInvalidRequestError(#133).- Six more datago datasets: four TourAPI 4 datasets (
tour_kor_area,tour_kor_festival,tour_kor_keyword,tour_kor_location) plusmetro_fareandmetro_path(#134). kpubdatacommand-line interface:datasets list,datasets show,fetch(with--alland--output) andraw(#145).- Opt-in disk response cache:
Client(cache=True, cache_ttl_seconds=...). It is off by default, and the TTL defaults to 24 hours (#144). - Structured logging across modules, including debug logs on adapter and transport failure paths (#131, #142).
Fixed
datago.bus_arrivalmoved to the v2 endpoint and understands itsmsgHeader/msgBodyenvelope (#146, #147).
For contributors
Live integration tests no longer break when their hard-coded dates expire, and SUPPORTED_DATA.md marks more datago datasets as live-verified (#141, #143, #148).
Full changelog: v0.2.3...v0.3.0
v0.2.3
kpubdata 0.2.3 makes the MOLIT real-estate transaction datasets and the lofin provider actually work, and adds seven more real-estate datasets. If you use datago.apt_trade or any lofin dataset, upgrade.
pip install -U "kpubdata==0.2.3"Added
- Seven real-estate transaction datasets on
datago:apt_rent,offi_trade,offi_rent,rh_trade,rh_rent,sh_tradeandsh_rent— apartment rentals, and sales and rentals for officetels, row houses and detached houses (#123). These were checked against recorded fixtures, not yet against the live API.
Fixed
datago.apt_tradealways failed withProviderResponseError: OK. The real-estate APIs report success asresultCode "000", which was treated as an error. Any all-zero result code now counts as success (#130).lofindatasets failed with an SSL handshake error when used throughClient. Since 0.2.1, the adapter's TLS settings for the LOFIN server were skipped wheneverClientsupplied the shared transport; the adapter now declares them throughTransportRequirements(#122, #124). Note: those settings also turned certificate verification off for LOFIN requests. That was fixed in 0.7.0 (#488).
Full changelog: v0.2.2...v0.2.3
v0.2.2
kpubdata 0.2.2 is a small follow-up to 0.2.1's paging change: list_all() now understands cursor-based paging as well as page numbers.
pip install -U "kpubdata==0.2.2"Added
Dataset.list_all()followsnext_cursorwhen a batch returns one, and falls back tonext_pageotherwise (#121). No built-in provider returns a cursor yet; this matters if you write your own adapter.
Known issue
lofindatasets still fail with an SSL handshake error when used throughClient(since 0.2.1). Fixed in 0.2.3 (#122, #124).
For contributors
The paging contract for adapters, including the best-effort len(items) == page_size heuristic datago uses when a response has no total count, is documented in ARCHITECTURE.md and PROVIDER_ADAPTER_CONTRACT.md (#121).
Full changelog: v0.2.1...v0.2.2