fix: handle missing and null fields in API responses - #270
Merged
Conversation
Five crashes that share one root pattern: the code assumed a field was present and non-null, and raised an unhelpful low-level exception when the API said otherwise. `filter_by_date()` called `.get()` on `library_status` without checking it is a dict, so an item with `purchase_date: null` and a missing or null `library_status` raised `AttributeError` as soon as a date bound was set. The `else` branch already handles "cannot determine date added" by logging and keeping the item; it just could not be reached. This shape is not hypothetical: podcast episodes fetched through the catalog endpoint carry neither field (#267). `_reuse_voucher()` tested for key presence and then converted the value, so an explicit `null` passed the guard and reached `strptime` as `None`. Test the value instead of the key. `get_license()` did the same for `license_denial_reasons`, turning a denied license into `TypeError: 'NoneType' object is not iterable` instead of `LicenseDenied`. A null `content_metadata` or `content_url` likewise raised `AttributeError` instead of reaching the intended `NoDownloadUrl`. `is_published()` had no explicit return for a missing `publication_datetime` and fell through to `None`. Callers read that as "not published" and passed the same missing value into `ItemNotPublished`, which then failed in the parser. A missing publication date means unknown, not unpublished, so skipping the item on absent evidence silently drops it — return `True` and let the download attempt itself decide. An `AudioPart` whose parent has no publication date now falls back to its own instead of being assumed published, so a known future date is still honoured. `ItemNotPublished` additionally no longer raises while building its message: without a usable date it reports the ASIN without a countdown. Closes #268
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #268.
Problem
Five crashes sharing one root pattern: the code assumes a field is present and non-null in an API response, and raises an unhelpful low-level exception when it is not.
filter_by_date()purchase_date: null+ missing/nulllibrary_status, with any date boundAttributeError: 'NoneType' object has no attribute 'get'_reuse_voucher()refresh_date: nullTypeError: strptime() argument 1 must be str, not Noneget_license()license_denial_reasons: nullon a denied licenseTypeError: 'NoneType' object is not iterableinstead ofLicenseDeniedget_license()content_metadata: nullorcontent_url: nullAttributeErrorinstead ofNoDownloadUrlis_published()publication_datetimeNone, callers pass it toItemNotPublished, which fails in the parserThe first three rows all come from testing key presence where the value needed testing. Case 1 is reachable in practice: podcast episodes fetched through the catalog endpoint carry neither
purchase_datenorlibrary_status(#267).Behaviour change
A missing
publication_datetimenow means unknown, not unpublished. Skipping an item on absent evidence silently drops content, sois_published()returnsTrueand lets the download attempt itself decide. AnAudioPartwhose parent carries no publication date falls back to its own date, so a known future date is still honoured — only when neither is available is the item assumed published.ItemNotPublishedno longer raises while building its message; without a usable date it reports the ASIN without a countdown.Verification
library_statusmissing / null /date_addednull — all three keep the item; thedate_addedbranch still filters correctlyrefresh_datenull and absent are skipped; expired →VoucherNeedRefresh, valid → accepted, expiredExpires→DownloadUrlExpiredall unchangedlicense_denial_reasonsnull →LicenseDenied;content_metadata/content_urlnull →NoDownloadUrl; a denied license with real reasons is unchangedis_published()across the parent/child matrix — parent date stays authoritative, fallback only when the parent has none, both absent →True; non-AudioPartunchangedItemNotPublishedwithNone,"","garbage"and an overflowing offset → clean message; a real date still yields the countdownruff check src plugin_cmds: +1G004for the newlogger.debugf-string, matching the six already in that fileaudible library list --resolve-podcasts --start-date 2008-01-01against an account with 168 titles returns 914 items, identical tomasterin the same window, exit 0, no stderrReview notes
A review pass caught a regression in the first version of the
is_published()change: returningTrueon a null parent date ignored a valid future date on the child. That is fixed and covered by the parent/child matrix above. The same pass found the twoget_license()cases, which are the same defect class as the three in #268 and are therefore included here rather than deferred to another issue.Two related items deliberately left out, both pre-existing:
self.publication_datetimetoItemNotPublishedeven whenis_published()evaluated the parent's date, so a future parent with a past child can produce a misleading countdown.library_status["date_added"]incmd_goodreads-transform.py,product_images.items()incmd_image-urls.py). Not built-in command paths.