{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":641501862,"defaultBranch":"master","name":"to-data-library","ownerLogin":"timeoutdigital","currentUserCanPush":false,"isFork":false,"isEmpty":false,"createdAt":"2023-05-16T15:47:02.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/738771?v=4","public":true,"private":false,"isOrgOwned":true},"refInfo":{"name":"","listCacheKey":"v0:1719306888.0","currentOid":""},"activityList":{"items":[{"before":"8ca1d102f0c8a64acabcfd2117f9dcc8383f5415","after":null,"ref":"refs/heads/DSS-2462---load-df---Support-job-config-kwargs-for-greater-flexibility","pushedAt":"2024-06-25T08:53:28.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"CeeEffEff","name":"Conor Fehilly","path":"/CeeEffEff","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/87389203?s=80&v=4"}},{"before":"dd2140170aaaa1ccae3f162e4a3371a6da734e23","after":"df4bb2cbdcc7773b815b327afa5fb45bc2ca45d2","ref":"refs/heads/master","pushedAt":"2024-06-25T08:53:28.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"CeeEffEff","name":"Conor Fehilly","path":"/CeeEffEff","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/87389203?s=80&v=4"},"commit":{"message":"DSS-2462 - job_config kwargs support - load_table_from_dataframe (#14)\n\nAdding support for setting arbitrary job_config properties.\r\nReason is that in DSS-2462 I need to add a new column, so would like to\r\nspecify `schema_update_options =\r\n[bigquery.SchemaUpdateOption.ALLOW_FIELD_ADDITION]`.\r\n\r\nDone with kwargs we can easily set other properties too though.\r\n\r\nTest plan:\r\nUpdated unit test","shortMessageHtmlLink":"DSS-2462 - job_config kwargs support - load_table_from_dataframe (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2370576979\" data-permission-text=\"Title is private\" data-url=\"https://github.com/timeoutdigital/to-data-library/issues/14\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/timeoutdigital/to-data-library/pull/14/hovercard\" href=\"https://github.com/timeoutdigital/to-data-library/pull/14\">#14</a>)"}},{"before":null,"after":"8ca1d102f0c8a64acabcfd2117f9dcc8383f5415","ref":"refs/heads/DSS-2462---load-df---Support-job-config-kwargs-for-greater-flexibility","pushedAt":"2024-06-24T15:46:59.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"CeeEffEff","name":"Conor Fehilly","path":"/CeeEffEff","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/87389203?s=80&v=4"},"commit":{"message":"job_config kwargs support - load_table_from_dataframe","shortMessageHtmlLink":"job_config kwargs support - load_table_from_dataframe"}},{"before":"c66a5e6257f570a4a8761ddee7f765bd30efcc5c","after":"dd2140170aaaa1ccae3f162e4a3371a6da734e23","ref":"refs/heads/master","pushedAt":"2024-03-12T16:51:02.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"ghasani","name":"Gzim Hasani","path":"/ghasani","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/77578240?s=80&v=4"},"commit":{"message":"DSS-2326 adding check on write preference (#13)\n\nOnly need both partition field and partition date when using truncate to\r\nidentify which partition to clean.\r\n\r\nTest:\r\n1. Tested new library runs as expected\r\n2. Tested DF with multiple partitions in 1 column gets uploaded into\r\nmultiple partitions","shortMessageHtmlLink":"DSS-2326 adding check on write preference (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2182162850\" data-permission-text=\"Title is private\" data-url=\"https://github.com/timeoutdigital/to-data-library/issues/13\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/timeoutdigital/to-data-library/pull/13/hovercard\" href=\"https://github.com/timeoutdigital/to-data-library/pull/13\">#13</a>)"}},{"before":"b237b94407bf1b9b5536736cf7bb0927820cb209","after":null,"ref":"refs/heads/DSS-2326","pushedAt":"2024-03-12T16:51:02.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"ghasani","name":"Gzim Hasani","path":"/ghasani","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/77578240?s=80&v=4"}},{"before":null,"after":"b237b94407bf1b9b5536736cf7bb0927820cb209","ref":"refs/heads/DSS-2326","pushedAt":"2024-03-12T16:38:30.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"ghasani","name":"Gzim Hasani","path":"/ghasani","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/77578240?s=80&v=4"},"commit":{"message":"adding check on write preference","shortMessageHtmlLink":"adding check on write preference"}},{"before":null,"after":"c66a5e6257f570a4a8761ddee7f765bd30efcc5c","ref":"refs/heads/test","pushedAt":"2023-11-28T10:03:34.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"ghasani","name":"Gzim Hasani","path":"/ghasani","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/77578240?s=80&v=4"},"commit":{"message":"adding partition_date to table name (#12)\n\nAdding partition_date to table names in the load_job config. This is so\r\nwhen data is uploaded to a table it is uploaded to that particular\r\npartition. Also when data is deleted as part of a truncate its deleted\r\nonly from that partition","shortMessageHtmlLink":"adding partition_date to table name (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"1976050617\" data-permission-text=\"Title is private\" data-url=\"https://github.com/timeoutdigital/to-data-library/issues/12\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/timeoutdigital/to-data-library/pull/12/hovercard\" href=\"https://github.com/timeoutdigital/to-data-library/pull/12\">#12</a>)"}},{"before":"c0e1ec142e59fdf54a164c7abd96fd738e3dc1ac","after":null,"ref":"refs/heads/update-partitioned-tables","pushedAt":"2023-11-03T14:52:32.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"Munir93","name":"Munir Welch","path":"/Munir93","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37178013?s=80&v=4"}},{"before":"dd738d7d64903aa7accc32663213e3b07af8625f","after":"c66a5e6257f570a4a8761ddee7f765bd30efcc5c","ref":"refs/heads/master","pushedAt":"2023-11-03T14:52:31.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"Munir93","name":"Munir Welch","path":"/Munir93","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37178013?s=80&v=4"},"commit":{"message":"adding partition_date to table name (#12)\n\nAdding partition_date to table names in the load_job config. This is so\r\nwhen data is uploaded to a table it is uploaded to that particular\r\npartition. Also when data is deleted as part of a truncate its deleted\r\nonly from that partition","shortMessageHtmlLink":"adding partition_date to table name (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"1976050617\" data-permission-text=\"Title is private\" data-url=\"https://github.com/timeoutdigital/to-data-library/issues/12\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/timeoutdigital/to-data-library/pull/12/hovercard\" href=\"https://github.com/timeoutdigital/to-data-library/pull/12\">#12</a>)"}},{"before":null,"after":"c0e1ec142e59fdf54a164c7abd96fd738e3dc1ac","ref":"refs/heads/update-partitioned-tables","pushedAt":"2023-11-03T12:04:12.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"Munir93","name":"Munir Welch","path":"/Munir93","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37178013?s=80&v=4"},"commit":{"message":"adding partition_date to table name","shortMessageHtmlLink":"adding partition_date to table name"}},{"before":"1e3fd5a4b1873240665a90a395e1918ef8826ea0","after":null,"ref":"refs/heads/aws-session-update","pushedAt":"2023-10-31T18:18:33.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"Munir93","name":"Munir Welch","path":"/Munir93","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37178013?s=80&v=4"}},{"before":"d524eb941c6bc9f9a96e2e226e9455cae35bf98f","after":"dd738d7d64903aa7accc32663213e3b07af8625f","ref":"refs/heads/master","pushedAt":"2023-10-31T18:18:32.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"Munir93","name":"Munir Welch","path":"/Munir93","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37178013?s=80&v=4"},"commit":{"message":"aws session update (#11)","shortMessageHtmlLink":"aws session update (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"1971054461\" data-permission-text=\"Title is private\" data-url=\"https://github.com/timeoutdigital/to-data-library/issues/11\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/timeoutdigital/to-data-library/pull/11/hovercard\" href=\"https://github.com/timeoutdigital/to-data-library/pull/11\">#11</a>)"}},{"before":null,"after":"1e3fd5a4b1873240665a90a395e1918ef8826ea0","ref":"refs/heads/aws-session-update","pushedAt":"2023-10-31T18:15:35.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"Munir93","name":"Munir Welch","path":"/Munir93","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37178013?s=80&v=4"},"commit":{"message":"aws session update","shortMessageHtmlLink":"aws session update"}},{"before":"952bda49f3accee0792ec39f0cc226148d9c6ff9","after":null,"ref":"refs/heads/DSS-2064-AWS-session","pushedAt":"2023-10-31T17:13:37.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"ghasani","name":"Gzim Hasani","path":"/ghasani","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/77578240?s=80&v=4"}},{"before":"d97141f50dd970f65ee6511c11508e316007d40b","after":"d524eb941c6bc9f9a96e2e226e9455cae35bf98f","ref":"refs/heads/master","pushedAt":"2023-10-31T17:13:36.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"ghasani","name":"Gzim Hasani","path":"/ghasani","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/77578240?s=80&v=4"},"commit":{"message":"DSS-2064 adding aws session to client calls (#10)\n\nThe data apps all crated a AWS session that is authed to aws.\r\n\r\n---------\r\n\r\nCo-authored-by: Munir93 <munir.welch93@gmail.com>","shortMessageHtmlLink":"DSS-2064 adding aws session to client calls (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"1970560444\" data-permission-text=\"Title is private\" data-url=\"https://github.com/timeoutdigital/to-data-library/issues/10\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/timeoutdigital/to-data-library/pull/10/hovercard\" href=\"https://github.com/timeoutdigital/to-data-library/pull/10\">#10</a>)"}},{"before":"be1ff62251b46cecf61a4e6a675a5314e43db40e","after":"952bda49f3accee0792ec39f0cc226148d9c6ff9","ref":"refs/heads/DSS-2064-AWS-session","pushedAt":"2023-10-31T17:10:39.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"ghasani","name":"Gzim Hasani","path":"/ghasani","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/77578240?s=80&v=4"},"commit":{"message":"pre-commit changes","shortMessageHtmlLink":"pre-commit changes"}},{"before":"f35a657c6eb6bd05061e1f5428e093ff7f00b08f","after":"be1ff62251b46cecf61a4e6a675a5314e43db40e","ref":"refs/heads/DSS-2064-AWS-session","pushedAt":"2023-10-31T17:01:03.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"ghasani","name":"Gzim Hasani","path":"/ghasani","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/77578240?s=80&v=4"},"commit":{"message":"flake 8","shortMessageHtmlLink":"flake 8"}},{"before":"7a7b349e0c8a4d235831b7848392c02e44575850","after":"f35a657c6eb6bd05061e1f5428e093ff7f00b08f","ref":"refs/heads/DSS-2064-AWS-session","pushedAt":"2023-10-31T16:52:13.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"ghasani","name":"Gzim Hasani","path":"/ghasani","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/77578240?s=80&v=4"},"commit":{"message":"refactor s3","shortMessageHtmlLink":"refactor s3"}},{"before":"c6704cf778797a890a9f31e3ef3bf63382d77259","after":"7a7b349e0c8a4d235831b7848392c02e44575850","ref":"refs/heads/DSS-2064-AWS-session","pushedAt":"2023-10-31T16:36:06.000Z","pushType":"push","commitsCount":2,"pusher":{"login":"ghasani","name":"Gzim Hasani","path":"/ghasani","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/77578240?s=80&v=4"},"commit":{"message":"Merge branch 'DSS-2064-AWS-session' of https://github.com/timeoutdigital/to-data-library into DSS-2064-AWS-session","shortMessageHtmlLink":"Merge branch 'DSS-2064-AWS-session' of <a href=\"https://github.com/timeoutdigital/to-data-library\">https://github.com/timeoutdigi…</a>"}},{"before":"d9289a487314831124f03808774e80c561257790","after":"c6704cf778797a890a9f31e3ef3bf63382d77259","ref":"refs/heads/DSS-2064-AWS-session","pushedAt":"2023-10-31T16:32:46.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"Munir93","name":"Munir Welch","path":"/Munir93","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/37178013?s=80&v=4"},"commit":{"message":"fixing flake 8 tests","shortMessageHtmlLink":"fixing flake 8 tests"}},{"before":"f83dfec7b4f08b983178865e7e657a54cd556a82","after":"d9289a487314831124f03808774e80c561257790","ref":"refs/heads/DSS-2064-AWS-session","pushedAt":"2023-10-31T15:23:50.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"ghasani","name":"Gzim Hasani","path":"/ghasani","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/77578240?s=80&v=4"},"commit":{"message":"doc changes","shortMessageHtmlLink":"doc changes"}},{"before":"e9fa937f70dfd05d3dca439823abf23cf52cc4fb","after":"f83dfec7b4f08b983178865e7e657a54cd556a82","ref":"refs/heads/DSS-2064-AWS-session","pushedAt":"2023-10-31T15:22:06.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"ghasani","name":"Gzim Hasani","path":"/ghasani","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/77578240?s=80&v=4"},"commit":{"message":"removed parsed_connection","shortMessageHtmlLink":"removed parsed_connection"}},{"before":"b03e8937c087d0ea9e28bb4918ca3e0b684ddbe7","after":"e9fa937f70dfd05d3dca439823abf23cf52cc4fb","ref":"refs/heads/DSS-2064-AWS-session","pushedAt":"2023-10-31T14:28:44.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"ghasani","name":"Gzim Hasani","path":"/ghasani","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/77578240?s=80&v=4"},"commit":{"message":"refactor client used to get paginator method","shortMessageHtmlLink":"refactor client used to get paginator method"}},{"before":"bb48ad2cfb1f62d2241f9d1c0b7f1512873c0695","after":"b03e8937c087d0ea9e28bb4918ca3e0b684ddbe7","ref":"refs/heads/DSS-2064-AWS-session","pushedAt":"2023-10-31T14:10:27.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"ghasani","name":"Gzim Hasani","path":"/ghasani","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/77578240?s=80&v=4"},"commit":{"message":"comments","shortMessageHtmlLink":"comments"}},{"before":"d97141f50dd970f65ee6511c11508e316007d40b","after":"bb48ad2cfb1f62d2241f9d1c0b7f1512873c0695","ref":"refs/heads/DSS-2064-AWS-session","pushedAt":"2023-10-31T14:08:42.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"ghasani","name":"Gzim Hasani","path":"/ghasani","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/77578240?s=80&v=4"},"commit":{"message":"adding aws session to client calls","shortMessageHtmlLink":"adding aws session to client calls"}},{"before":null,"after":"d97141f50dd970f65ee6511c11508e316007d40b","ref":"refs/heads/DSS-2064-AWS-session","pushedAt":"2023-10-31T13:47:52.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"ghasani","name":"Gzim Hasani","path":"/ghasani","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/77578240?s=80&v=4"},"commit":{"message":"DSS-2031 - Add load from dataframe to BQ (#8)\n\nOfficially adding dataframe -> BQ load to to-data-library as was about\r\nto copy and paste into my second pipeline 😃\r\n\r\nAdded unit test in same style as upload method.\r\nCode ran in previous pipeline adapted for this lib.\r\nI will test once a release version is available using this during\r\ndevelopment of my second pipeline.","shortMessageHtmlLink":"DSS-2031 - Add load from dataframe to BQ (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"1938209853\" data-permission-text=\"Title is private\" data-url=\"https://github.com/timeoutdigital/to-data-library/issues/8\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/timeoutdigital/to-data-library/pull/8/hovercard\" href=\"https://github.com/timeoutdigital/to-data-library/pull/8\">#8</a>)"}},{"before":null,"after":"d97141f50dd970f65ee6511c11508e316007d40b","ref":"refs/heads/DSS---Use-Dataapp-S3-session","pushedAt":"2023-10-31T11:06:18.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"ghasani","name":"Gzim Hasani","path":"/ghasani","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/77578240?s=80&v=4"},"commit":{"message":"DSS-2031 - Add load from dataframe to BQ (#8)\n\nOfficially adding dataframe -> BQ load to to-data-library as was about\r\nto copy and paste into my second pipeline 😃\r\n\r\nAdded unit test in same style as upload method.\r\nCode ran in previous pipeline adapted for this lib.\r\nI will test once a release version is available using this during\r\ndevelopment of my second pipeline.","shortMessageHtmlLink":"DSS-2031 - Add load from dataframe to BQ (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"1938209853\" data-permission-text=\"Title is private\" data-url=\"https://github.com/timeoutdigital/to-data-library/issues/8\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/timeoutdigital/to-data-library/pull/8/hovercard\" href=\"https://github.com/timeoutdigital/to-data-library/pull/8\">#8</a>)"}},{"before":null,"after":"897756c71cdadd8df771221fbc4175fe4e7c48d6","ref":"refs/heads/DSS-2019","pushedAt":"2023-10-27T10:19:55.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"kthhrv","name":"Keith Harvey","path":"/kthhrv","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/4869891?s=80&v=4"},"commit":{"message":"DSS-2019 - S3Client should take aws_session","shortMessageHtmlLink":"DSS-2019 - S3Client should take aws_session"}},{"before":"4a570866da2ecf4e252a2265e9a3f2b35d1efc79","after":null,"ref":"refs/heads/DSS-2031---CrUX-data---add-dataframe-to-TO-DL","pushedAt":"2023-10-12T09:13:21.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"CeeEffEff","name":"Conor Fehilly","path":"/CeeEffEff","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/87389203?s=80&v=4"}},{"before":"b528c4f1f3174df88448e0637a58ab5bd7e459fe","after":"d97141f50dd970f65ee6511c11508e316007d40b","ref":"refs/heads/master","pushedAt":"2023-10-12T09:13:20.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"CeeEffEff","name":"Conor Fehilly","path":"/CeeEffEff","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/87389203?s=80&v=4"},"commit":{"message":"DSS-2031 - Add load from dataframe to BQ (#8)\n\nOfficially adding dataframe -> BQ load to to-data-library as was about\r\nto copy and paste into my second pipeline 😃\r\n\r\nAdded unit test in same style as upload method.\r\nCode ran in previous pipeline adapted for this lib.\r\nI will test once a release version is available using this during\r\ndevelopment of my second pipeline.","shortMessageHtmlLink":"DSS-2031 - Add load from dataframe to BQ (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"1938209853\" data-permission-text=\"Title is private\" data-url=\"https://github.com/timeoutdigital/to-data-library/issues/8\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/timeoutdigital/to-data-library/pull/8/hovercard\" href=\"https://github.com/timeoutdigital/to-data-library/pull/8\">#8</a>)"}}],"hasNextPage":true,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"djE6ks8AAAAEbmIMRQA","startCursor":null,"endCursor":null}},"title":"Activity · timeoutdigital/to-data-library"}