{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":324879897,"defaultBranch":"main","name":"hdx-python-scraper","ownerLogin":"OCHA-DAP","currentUserCanPush":false,"isFork":false,"isEmpty":false,"createdAt":"2020-12-28T01:03:28.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/5087346?v=4","public":true,"private":false,"isOrgOwned":true},"refInfo":{"name":"","listCacheKey":"v0:1706842178.0","currentOid":""},"activityList":{"items":[{"before":"80054ca1ca84867ccbdf50cbd5eaf73fb5dba506","after":"4957b61a878ed311507f77166e84e37a9dba5e88","ref":"refs/heads/main","pushedAt":"2024-02-02T02:48:01.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"mcarans","name":"Mike","path":"/mcarans","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/3799212?s=80&v=4"},"commit":{"message":"HDXDS-481 Allow override of countries when adding one or more configurable scrapers (#33)\n\n* Per scraper configuration of countries to process\r\n\r\n* Add delete_scraper to Runner\r\n\r\n* Add test for delete_scraper","shortMessageHtmlLink":"HDXDS-481 Allow override of countries when adding one or more configu…"}},{"before":"ee2ed2c47497b3e48dfd21729e05611f3b7b5bf4","after":"3894027ef7f56eb8160a90aaa54b1347529a18e1","ref":"refs/heads/countries_per_scraper","pushedAt":"2024-02-02T01:06:57.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"mcarans","name":"Mike","path":"/mcarans","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/3799212?s=80&v=4"},"commit":{"message":"Add test for delete_scraper","shortMessageHtmlLink":"Add test for delete_scraper"}},{"before":"864fca1a67e6c038d3dcf3069999d5aab51cc792","after":"ee2ed2c47497b3e48dfd21729e05611f3b7b5bf4","ref":"refs/heads/countries_per_scraper","pushedAt":"2024-02-01T02:33:47.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"mcarans","name":"Mike","path":"/mcarans","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/3799212?s=80&v=4"},"commit":{"message":"Add delete_scraper to Runner","shortMessageHtmlLink":"Add delete_scraper to Runner"}},{"before":null,"after":"864fca1a67e6c038d3dcf3069999d5aab51cc792","ref":"refs/heads/countries_per_scraper","pushedAt":"2024-02-01T01:12:21.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"mcarans","name":"Mike","path":"/mcarans","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/3799212?s=80&v=4"},"commit":{"message":"Per scraper configuration of countries to process","shortMessageHtmlLink":"Per scraper configuration of countries to process"}},{"before":"c0fc042f21f894a3a84df04d48dce8c8465ffc47","after":"80054ca1ca84867ccbdf50cbd5eaf73fb5dba506","ref":"refs/heads/main","pushedAt":"2024-01-31T03:05:19.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"mcarans","name":"Mike","path":"/mcarans","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/3799212?s=80&v=4"},"commit":{"message":"Use 23:59:59.999999 for end time period (#32)\n\nWhen using same dataset multiple times in HAPI, update tiem period to have max and min of all uses","shortMessageHtmlLink":"Use 23:59:59.999999 for end time period (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2109190666\" data-permission-text=\"Title is private\" data-url=\"https://github.com/OCHA-DAP/hdx-python-scraper/issues/32\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/OCHA-DAP/hdx-python-scraper/pull/32/hovercard\" href=\"https://github.com/OCHA-DAP/hdx-python-scraper/pull/32\">#32</a>)"}},{"before":null,"after":"060570c6787aacda32899515839cf3c0ce9ef797","ref":"refs/heads/max_time_period","pushedAt":"2024-01-31T02:53:06.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"mcarans","name":"Mike","path":"/mcarans","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/3799212?s=80&v=4"},"commit":{"message":"Use 23:59:59.999999 for end time period\nWhen using same dataset multiple times in HAPI, update tiem period to have max and min of all uses","shortMessageHtmlLink":"Use 23:59:59.999999 for end time period"}},{"before":"5ef4dee8e9768fddba7fad8a9d710262cf510f62","after":"c0fc042f21f894a3a84df04d48dce8c8465ffc47","ref":"refs/heads/main","pushedAt":"2024-01-30T02:57:52.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"mcarans","name":"Mike","path":"/mcarans","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/3799212?s=80&v=4"},"commit":{"message":"Update requirements","shortMessageHtmlLink":"Update requirements"}},{"before":"220dc2d769d2a8bee81a693e6f33b167e55d42e0","after":"5ef4dee8e9768fddba7fad8a9d710262cf510f62","ref":"refs/heads/main","pushedAt":"2024-01-30T02:52:41.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"mcarans","name":"Mike","path":"/mcarans","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/3799212?s=80&v=4"},"commit":{"message":"HDXDS-449 HDXDS-470 Get HAPI time period from source_date (#31)\n\nGet HAPI time period from source_date\r\nEnsure that we read resource even if url is provided so we can populate hapi resource metadata\r\nRemove Black, add ruff-format and remove deprecated ruff option","shortMessageHtmlLink":"HDXDS-449 HDXDS-470 Get HAPI time period from source_date (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2106845797\" data-permission-text=\"Title is private\" data-url=\"https://github.com/OCHA-DAP/hdx-python-scraper/issues/31\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/OCHA-DAP/hdx-python-scraper/pull/31/hovercard\" href=\"https://github.com/OCHA-DAP/hdx-python-scraper/pull/31\">#31</a>)"}},{"before":"99a37c79256cde3fdeb606b1926a458d48dcb591","after":"073dd03a7834d40e8ddb49375ffa216538bfbf55","ref":"refs/heads/hapi_time_period","pushedAt":"2024-01-29T23:59:39.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"mcarans","name":"Mike","path":"/mcarans","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/3799212?s=80&v=4"},"commit":{"message":"Get HAPI time period from source_date\nEnsure that we read resource even if url is provided so we can populate hapi resource metadata\nRemove Black, add ruff-format and remove deprecated ruff option","shortMessageHtmlLink":"Get HAPI time period from source_date"}},{"before":"31dee1a79eda2bd626f3053bbe7282c56125c412","after":"99a37c79256cde3fdeb606b1926a458d48dcb591","ref":"refs/heads/hapi_time_period","pushedAt":"2024-01-29T22:54:25.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"mcarans","name":"Mike","path":"/mcarans","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/3799212?s=80&v=4"},"commit":{"message":"Get HAPI time period from source_date\nEnsure that we read resource even if url is provided so we can populate hapi resource metadata\nRemove Black, add ruff-format and remove deprecated ruff option","shortMessageHtmlLink":"Get HAPI time period from source_date"}},{"before":"5b071fa7c32f11219a177bf191107066dc981bd3","after":"31dee1a79eda2bd626f3053bbe7282c56125c412","ref":"refs/heads/hapi_time_period","pushedAt":"2024-01-29T01:30:33.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"mcarans","name":"Mike","path":"/mcarans","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/3799212?s=80&v=4"},"commit":{"message":"Get HAPI time period from source_date\nRemove Black, add ruff-format and remove deprecated ruff option","shortMessageHtmlLink":"Get HAPI time period from source_date"}},{"before":"0c6e541e2f31d0710414f98837072af481469e1c","after":"5b071fa7c32f11219a177bf191107066dc981bd3","ref":"refs/heads/hapi_time_period","pushedAt":"2024-01-29T01:23:58.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"mcarans","name":"Mike","path":"/mcarans","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/3799212?s=80&v=4"},"commit":{"message":"Get HAPI time period from source_date\nRemove Black","shortMessageHtmlLink":"Get HAPI time period from source_date"}},{"before":null,"after":"0c6e541e2f31d0710414f98837072af481469e1c","ref":"refs/heads/hapi_time_period","pushedAt":"2024-01-29T00:50:32.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"mcarans","name":"Mike","path":"/mcarans","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/3799212?s=80&v=4"},"commit":{"message":"Get HAPI time period from source_date","shortMessageHtmlLink":"Get HAPI time period from source_date"}},{"before":"302bce211e364a4e773097f215174309a7bd31a1","after":"220dc2d769d2a8bee81a693e6f33b167e55d42e0","ref":"refs/heads/main","pushedAt":"2024-01-03T21:33:56.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"mcarans","name":"Mike","path":"/mcarans","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/3799212?s=80&v=4"},"commit":{"message":"Add to HAPI results rather than overwriting (#30)\n\nCo-authored-by: Mike <rans@un.org>","shortMessageHtmlLink":"Add to HAPI results rather than overwriting (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2063206961\" data-permission-text=\"Title is private\" data-url=\"https://github.com/OCHA-DAP/hdx-python-scraper/issues/30\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/OCHA-DAP/hdx-python-scraper/pull/30/hovercard\" href=\"https://github.com/OCHA-DAP/hdx-python-scraper/pull/30\">#30</a>)"}},{"before":null,"after":"29b312970c60684754299f8857fe2c2db79e6e76","ref":"refs/heads/hapi_results_values_changes","pushedAt":"2024-01-03T03:36:39.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"mcarans","name":"Mike","path":"/mcarans","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/3799212?s=80&v=4"},"commit":{"message":"Add to HAPI results rather than overwriting","shortMessageHtmlLink":"Add to HAPI results rather than overwriting"}},{"before":"d5a4545d5191cdd1a88b9132d85fb3a74b8b8115","after":"302bce211e364a4e773097f215174309a7bd31a1","ref":"refs/heads/main","pushedAt":"2024-01-02T03:07:58.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"mcarans","name":"Mike","path":"/mcarans","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/3799212?s=80&v=4"},"commit":{"message":"HDXDS-423 File prefix changes (#29)\n\nIf filename is found or put in datasetinfo, then ignore file_prefix\r\n\r\nreference_period -> time_period\r\nfile_type -> format","shortMessageHtmlLink":"HDXDS-423 File prefix changes (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2061820061\" data-permission-text=\"Title is private\" data-url=\"https://github.com/OCHA-DAP/hdx-python-scraper/issues/29\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/OCHA-DAP/hdx-python-scraper/pull/29/hovercard\" href=\"https://github.com/OCHA-DAP/hdx-python-scraper/pull/29\">#29</a>)"}},{"before":"78926a6f3f72588e64d1479c6be0d57e86e5cce7","after":"f6bc0df6e1d7c902fd418a0bd6bf6037de8b1a93","ref":"refs/heads/file_prefix_changes","pushedAt":"2024-01-02T02:45:43.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"mcarans","name":"Mike","path":"/mcarans","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/3799212?s=80&v=4"},"commit":{"message":"Increase coverage of new code","shortMessageHtmlLink":"Increase coverage of new code"}},{"before":"c6f154a4060e6b4f6cd1385327255bdaad824073","after":"78926a6f3f72588e64d1479c6be0d57e86e5cce7","ref":"refs/heads/file_prefix_changes","pushedAt":"2024-01-01T22:31:09.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"mcarans","name":"Mike","path":"/mcarans","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/3799212?s=80&v=4"},"commit":{"message":"Small improvement","shortMessageHtmlLink":"Small improvement"}},{"before":"e4f492d2059c29ddbccf958cca972ef04251d28f","after":"c6f154a4060e6b4f6cd1385327255bdaad824073","ref":"refs/heads/file_prefix_changes","pushedAt":"2024-01-01T20:31:01.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"mcarans","name":"Mike","path":"/mcarans","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/3799212?s=80&v=4"},"commit":{"message":"Fix test\nfile_type -> format","shortMessageHtmlLink":"Fix test"}},{"before":null,"after":"e4f492d2059c29ddbccf958cca972ef04251d28f","ref":"refs/heads/file_prefix_changes","pushedAt":"2023-12-29T04:40:31.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"mcarans","name":"Mike","path":"/mcarans","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/3799212?s=80&v=4"},"commit":{"message":"If filename is found or put in datasetinfo, then ignore file_prefix","shortMessageHtmlLink":"If filename is found or put in datasetinfo, then ignore file_prefix"}},{"before":"db00ef4d94bf0154fc037fb67c3d37895788e579","after":"d5a4545d5191cdd1a88b9132d85fb3a74b8b8115","ref":"refs/heads/main","pushedAt":"2023-12-18T04:30:36.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"mcarans","name":"Mike","path":"/mcarans","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/3799212?s=80&v=4"},"commit":{"message":"Rename reference period to time period (except HAPI for now)","shortMessageHtmlLink":"Rename reference period to time period (except HAPI for now)"}},{"before":"fc6b8ebf114d8e68aedd2ae55409e52d573abe5a","after":"db00ef4d94bf0154fc037fb67c3d37895788e579","ref":"refs/heads/main","pushedAt":"2023-12-14T00:49:07.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"mcarans","name":"Mike","path":"/mcarans","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/3799212?s=80&v=4"},"commit":{"message":"Add precommit exclusions","shortMessageHtmlLink":"Add precommit exclusions"}},{"before":"b92ccb2db3079d48c3e87b5faad7d6c1d0cacb99","after":"fc6b8ebf114d8e68aedd2ae55409e52d573abe5a","ref":"refs/heads/main","pushedAt":"2023-12-13T01:46:14.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"mcarans","name":"Mike","path":"/mcarans","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/3799212?s=80&v=4"},"commit":{"message":"Remove use of Reader member variable prefix and explicitly pass file_prefix instead","shortMessageHtmlLink":"Remove use of Reader member variable prefix and explicitly pass file_…"}},{"before":"dc5d3d506c40941f3f8b6276330b71f9a9e0af69","after":"b92ccb2db3079d48c3e87b5faad7d6c1d0cacb99","ref":"refs/heads/main","pushedAt":"2023-12-13T01:37:54.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"mcarans","name":"Mike","path":"/mcarans","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/3799212?s=80&v=4"},"commit":{"message":"Remove use of Reader member variable prefix and explicitly pass file_prefix instead","shortMessageHtmlLink":"Remove use of Reader member variable prefix and explicitly pass file_…"}},{"before":"9312b2324e71af2e9e7dd6bd928cd5d8f2595ee6","after":"dc5d3d506c40941f3f8b6276330b71f9a9e0af69","ref":"refs/heads/main","pushedAt":"2023-12-13T01:27:11.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"mcarans","name":"Mike","path":"/mcarans","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/3799212?s=80&v=4"},"commit":{"message":"Add is_hxl output to HAPI resource metadata (#28)\n\nAdd is_hxl to HAPI resource metadata\r\nUse resource name and format where available rather than generating filename from URL (breaks names of files used for tests)\r\nRemove \"identifier\" and rely on file_prefix in kwargs to be consistent\r\nSet filename once available\r\nMove fixtures to fixtures/input\r\nFix and test for: when use_hxl is True, filename is created in the constructor if the ConfigurableScraper, but then when data is read later, the filename is not used\r\nRemove use of Reader member variable prefix and explicitly pass file_prefix instead","shortMessageHtmlLink":"Add is_hxl output to HAPI resource metadata (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2034780332\" data-permission-text=\"Title is private\" data-url=\"https://github.com/OCHA-DAP/hdx-python-scraper/issues/28\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/OCHA-DAP/hdx-python-scraper/pull/28/hovercard\" href=\"https://github.com/OCHA-DAP/hdx-python-scraper/pull/28\">#28</a>)"}},{"before":"da33a81fa9923a9b67620bd34e308f244aa75419","after":"c19b96f6709dc930c51969fb3b72c2a7a12feaa3","ref":"refs/heads/is_hxl_resource","pushedAt":"2023-12-12T05:15:04.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"mcarans","name":"Mike","path":"/mcarans","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/3799212?s=80&v=4"},"commit":{"message":"Remove use of Reader member variable prefix and explicitly pass file_prefix instead","shortMessageHtmlLink":"Remove use of Reader member variable prefix and explicitly pass file_…"}},{"before":"1fafbbe57a42d498624f5195f83aaaebf1fe7c01","after":"da33a81fa9923a9b67620bd34e308f244aa75419","ref":"refs/heads/is_hxl_resource","pushedAt":"2023-12-12T05:10:09.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"mcarans","name":"Mike","path":"/mcarans","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/3799212?s=80&v=4"},"commit":{"message":"Remove use of Reader member variable prefix and explicitly pass file_prefix instead","shortMessageHtmlLink":"Remove use of Reader member variable prefix and explicitly pass file_…"}},{"before":"30495a55145f12c2d6291980561023a2e6f1e785","after":"1fafbbe57a42d498624f5195f83aaaebf1fe7c01","ref":"refs/heads/is_hxl_resource","pushedAt":"2023-12-11T02:15:12.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"mcarans","name":"Mike","path":"/mcarans","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/3799212?s=80&v=4"},"commit":{"message":"Fix and test for: when use_hxl is True, filename is created in the constructor if the ConfigurableScraper, but then when data is read later, the filename is not used","shortMessageHtmlLink":"Fix and test for: when use_hxl is True, filename is created in the co…"}},{"before":"51510c9df447573ad7f2503dd67179fc172fe4d5","after":"30495a55145f12c2d6291980561023a2e6f1e785","ref":"refs/heads/is_hxl_resource","pushedAt":"2023-12-11T01:56:32.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"mcarans","name":"Mike","path":"/mcarans","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/3799212?s=80&v=4"},"commit":{"message":"Move fixtures to fixtures/input","shortMessageHtmlLink":"Move fixtures to fixtures/input"}},{"before":"2bfd20a1c9b9d785afeb591bd925f863be6aa165","after":"51510c9df447573ad7f2503dd67179fc172fe4d5","ref":"refs/heads/is_hxl_resource","pushedAt":"2023-12-11T00:32:51.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"mcarans","name":"Mike","path":"/mcarans","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/3799212?s=80&v=4"},"commit":{"message":"Set filename once available","shortMessageHtmlLink":"Set filename once available"}}],"hasNextPage":true,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"djE6ks8AAAAD78cWpgA","startCursor":null,"endCursor":null}},"title":"Activity · OCHA-DAP/hdx-python-scraper"}