{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":632275206,"defaultBranch":"main","name":"similarity-search-kit","ownerLogin":"ZachNagengast","currentUserCanPush":false,"isFork":false,"isEmpty":false,"createdAt":"2023-04-25T04:35:46.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/1981179?v=4","public":true,"private":false,"isOrgOwned":false},"refInfo":{"name":"","listCacheKey":"v0:1717531536.0","currentOid":""},"activityList":{"items":[{"before":"6f0258006d153d90edd0221b243b8363bd37df98","after":"9bec54706c8124bb9b20a3a06d566066a9f55712","ref":"refs/heads/main","pushedAt":"2024-06-04T20:04:25.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"ZachNagengast","name":"Zach Nagengast","path":"/ZachNagengast","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1981179?s=80&v=4"},"commit":{"message":"Formatting, and replace precondition with default value to prevent crashing","shortMessageHtmlLink":"Formatting, and replace precondition with default value to prevent cr…"}},{"before":"d01772c419a40582d7ad6950d1504da31bb82598","after":"6f0258006d153d90edd0221b243b8363bd37df98","ref":"refs/heads/main","pushedAt":"2024-06-04T19:41:37.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"ZachNagengast","name":"Zach Nagengast","path":"/ZachNagengast","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1981179?s=80&v=4"},"commit":{"message":"Added AMX acceleration (#39)\n\n* Added conformance of SimilarityIndex to Identifiable\r\n\r\n* Added conformance to Hashable\r\n\r\n* Removed print statements.\r\n\r\n* Added AMX acceleration.\r\n\r\n* Removed deprecated methods & added logger for save / load index operations.","shortMessageHtmlLink":"Added AMX acceleration (#39)"}},{"before":"642ba49b92dd2cddd46ce0c7ad9f5b6bd091aaef","after":"d01772c419a40582d7ad6950d1504da31bb82598","ref":"refs/heads/main","pushedAt":"2024-04-18T20:55:56.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"ZachNagengast","name":"Zach Nagengast","path":"/ZachNagengast","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1981179?s=80&v=4"},"commit":{"message":"Use native contextual embedding by default in example","shortMessageHtmlLink":"Use native contextual embedding by default in example"}},{"before":"ef737c4aad3af904a9925c476dc37c0dddc6ad8c","after":"642ba49b92dd2cddd46ce0c7ad9f5b6bd091aaef","ref":"refs/heads/main","pushedAt":"2024-04-18T20:45:52.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"ZachNagengast","name":"Zach Nagengast","path":"/ZachNagengast","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1981179?s=80&v=4"},"commit":{"message":"Fix build issue in example app from #38","shortMessageHtmlLink":"Fix build issue in example app from #38"}},{"before":"eef5ab026094845a11ad4c397cad55a79034e0bd","after":"ef737c4aad3af904a9925c476dc37c0dddc6ad8c","ref":"refs/heads/main","pushedAt":"2024-02-14T05:26:05.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"ZachNagengast","name":"Zach Nagengast","path":"/ZachNagengast","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1981179?s=80&v=4"},"commit":{"message":"\"Refactor loadIndex method to directly assign loaded index items to indexItems\" (#36)","shortMessageHtmlLink":"\"Refactor loadIndex method to directly assign loaded index items to i…"}},{"before":"ddc8e458d0e826b2fe5dbce6f6eac96a8935e8eb","after":"eef5ab026094845a11ad4c397cad55a79034e0bd","ref":"refs/heads/main","pushedAt":"2024-01-26T08:14:19.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"ZachNagengast","name":"Zach Nagengast","path":"/ZachNagengast","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1981179?s=80&v=4"},"commit":{"message":"\"CosineSimilarity\" typo (#33)","shortMessageHtmlLink":"\"CosineSimilarity\" typo (#33)"}},{"before":"224e95ece83f5745bd7b53de263beb2cee098571","after":"ddc8e458d0e826b2fe5dbce6f6eac96a8935e8eb","ref":"refs/heads/main","pushedAt":"2024-01-23T05:29:32.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"ZachNagengast","name":"Zach Nagengast","path":"/ZachNagengast","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1981179?s=80&v=4"},"commit":{"message":"include token_type_ids when building model inputs (#32)\n\n* update BertTokenizer.buildModelInputs to include tokenTypeIds as part of the output\r\n\r\n* leave `buildModelInputs` unchanged and instead add `buildModelInputsWithTokenIds` method\r\n\r\n* add method comment describing parameters and returned type","shortMessageHtmlLink":"include token_type_ids when building model inputs (#32)"}},{"before":"919740d42d25441bd290b1101f8afac47f77b2b2","after":"224e95ece83f5745bd7b53de263beb2cee098571","ref":"refs/heads/main","pushedAt":"2023-12-17T18:56:13.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"ZachNagengast","name":"Zach Nagengast","path":"/ZachNagengast","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1981179?s=80&v=4"},"commit":{"message":"fix: Add completion callback to addItems(_:items) (#31)","shortMessageHtmlLink":"fix: Add completion callback to addItems(_:items) (#31)"}},{"before":"6d78d3051aa35f5701af34f9f49ab793a97c6b76","after":"919740d42d25441bd290b1101f8afac47f77b2b2","ref":"refs/heads/main","pushedAt":"2023-11-23T19:45:46.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"ZachNagengast","name":"Zach Nagengast","path":"/ZachNagengast","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1981179?s=80&v=4"},"commit":{"message":"Update README.md (#29)","shortMessageHtmlLink":"Update README.md (#29)"}},{"before":"61a2e091f13c3e60817e1b4f172112759f38f115","after":"6d78d3051aa35f5701af34f9f49ab793a97c6b76","ref":"refs/heads/main","pushedAt":"2023-11-14T01:57:53.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"ZachNagengast","name":"Zach Nagengast","path":"/ZachNagengast","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1981179?s=80&v=4"},"commit":{"message":"Top k improved search (#25)\n\n* Update NativeEmbeddings.swift\r\n\r\n* update\r\n\r\nupdate\r\n\r\n* test cases and improved topKsearch\r\n\r\n* Update Sources/SimilaritySearchKit/Core/Embeddings/Models/NativeEmbeddings.swift\r\n\r\nCo-authored-by: Zach Nagengast \r\n\r\n* Update Sources/SimilaritySearchKit/Core/Embeddings/Models/NativeEmbeddings.swift\r\n\r\nCo-authored-by: Zach Nagengast \r\n\r\n* getDefault StoragePath is now public\r\n\r\nI didnt see a reason why this function has to be private and i need to access the file location from outside.\r\n\r\n* dokumentation and access to filePath\r\n\r\n* added documentation\r\n\r\nadded documentation\r\n\r\n* performance test case\r\n\r\n* Update DistanceTest.swift\r\n\r\n* Update formatting and naming\r\n\r\n---------\r\n\r\nCo-authored-by: BernhardEisvogel \r\nCo-authored-by: Zach Nagengast \r\nCo-authored-by: ZachNagengast ","shortMessageHtmlLink":"Top k improved search (#25)"}},{"before":"18b52c6a2eb4600e87860c8da68b4984457a3712","after":"61a2e091f13c3e60817e1b4f172112759f38f115","ref":"refs/heads/main","pushedAt":"2023-11-07T19:48:15.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"ZachNagengast","name":"Zach Nagengast","path":"/ZachNagengast","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1981179?s=80&v=4"},"commit":{"message":"Additional options native embeddings (#23)\n\n* Update NativeEmbeddings.swift\r\n\r\n* update\r\n\r\nupdate\r\n\r\n* Update Sources/SimilaritySearchKit/Core/Embeddings/Models/NativeEmbeddings.swift\r\n\r\nCo-authored-by: Zach Nagengast \r\n\r\n* Update Sources/SimilaritySearchKit/Core/Embeddings/Models/NativeEmbeddings.swift\r\n\r\nCo-authored-by: Zach Nagengast \r\n\r\n---------\r\n\r\nCo-authored-by: Zach Nagengast ","shortMessageHtmlLink":"Additional options native embeddings (#23)"}},{"before":"81e176d907297d996a69b8b252ce817f5ad21787","after":"18b52c6a2eb4600e87860c8da68b4984457a3712","ref":"refs/heads/main","pushedAt":"2023-10-15T21:40:33.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"ZachNagengast","name":"Zach Nagengast","path":"/ZachNagengast","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1981179?s=80&v=4"},"commit":{"message":"Support BERT style multilingual native embeddings (#22)\n\n* Add support for BERT style native contextual embeddings\r\n\r\n* Update swift.yml\r\n\r\n* Add flag for building on macos13\r\n\r\n* Fix typo bracket\r\n\r\n* Reduce macos version on ci\r\n\r\n* Use macos13 not latest","shortMessageHtmlLink":"Support BERT style multilingual native embeddings (#22)"}},{"before":"6f129033f0cbb3b905aa7b093739e2ae3fecf840","after":"a753b51c5aa793e218deb22277c3046d62e75d17","ref":"refs/heads/native-contextual-embeddings","pushedAt":"2023-10-15T21:09:57.000Z","pushType":"push","commitsCount":2,"pusher":{"login":"ZachNagengast","name":"Zach Nagengast","path":"/ZachNagengast","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1981179?s=80&v=4"},"commit":{"message":"Merge branch 'main' into native-contextual-embeddings","shortMessageHtmlLink":"Merge branch 'main' into native-contextual-embeddings"}},{"before":"f89baf113db8495647210ed4ab59fcc486af0c94","after":"6f129033f0cbb3b905aa7b093739e2ae3fecf840","ref":"refs/heads/native-contextual-embeddings","pushedAt":"2023-10-11T19:25:49.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"ZachNagengast","name":"Zach Nagengast","path":"/ZachNagengast","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1981179?s=80&v=4"},"commit":{"message":"Use macos13 not latest","shortMessageHtmlLink":"Use macos13 not latest"}},{"before":"f7c6659fcbe1375d734683e2b9680ceb21868308","after":"f89baf113db8495647210ed4ab59fcc486af0c94","ref":"refs/heads/native-contextual-embeddings","pushedAt":"2023-10-11T19:15:19.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"ZachNagengast","name":"Zach Nagengast","path":"/ZachNagengast","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1981179?s=80&v=4"},"commit":{"message":"Reduce macos version on ci","shortMessageHtmlLink":"Reduce macos version on ci"}},{"before":"69b9321b6c6731c9aeb303e1b3476b28970eb5e8","after":"f7c6659fcbe1375d734683e2b9680ceb21868308","ref":"refs/heads/native-contextual-embeddings","pushedAt":"2023-10-11T19:13:59.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"ZachNagengast","name":"Zach Nagengast","path":"/ZachNagengast","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1981179?s=80&v=4"},"commit":{"message":"Fix typo bracket","shortMessageHtmlLink":"Fix typo bracket"}},{"before":"5ad1362fef1f4441d5de5232fa607542fcea016c","after":"69b9321b6c6731c9aeb303e1b3476b28970eb5e8","ref":"refs/heads/native-contextual-embeddings","pushedAt":"2023-10-11T19:12:54.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"ZachNagengast","name":"Zach Nagengast","path":"/ZachNagengast","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1981179?s=80&v=4"},"commit":{"message":"Add flag for building on macos13","shortMessageHtmlLink":"Add flag for building on macos13"}},{"before":"f3125dd5d91057c360262d5f91f0908e1d3b35e2","after":"5ad1362fef1f4441d5de5232fa607542fcea016c","ref":"refs/heads/native-contextual-embeddings","pushedAt":"2023-10-11T18:51:19.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"ZachNagengast","name":"Zach Nagengast","path":"/ZachNagengast","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1981179?s=80&v=4"},"commit":{"message":"Update swift.yml","shortMessageHtmlLink":"Update swift.yml"}},{"before":null,"after":"f3125dd5d91057c360262d5f91f0908e1d3b35e2","ref":"refs/heads/native-contextual-embeddings","pushedAt":"2023-10-11T18:40:49.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"ZachNagengast","name":"Zach Nagengast","path":"/ZachNagengast","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1981179?s=80&v=4"},"commit":{"message":"Add support for BERT style native contextual embeddings","shortMessageHtmlLink":"Add support for BERT style native contextual embeddings"}},{"before":"aef8b0e5200863237362ee826f819c5956358821","after":"81e176d907297d996a69b8b252ce817f5ad21787","ref":"refs/heads/main","pushedAt":"2023-10-04T19:41:40.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"ZachNagengast","name":"Zach Nagengast","path":"/ZachNagengast","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1981179?s=80&v=4"},"commit":{"message":"Update README.md","shortMessageHtmlLink":"Update README.md"}},{"before":"d591adc15ebadfdc43f1fd63fc62863478f9b970","after":"aef8b0e5200863237362ee826f819c5956358821","ref":"refs/heads/main","pushedAt":"2023-08-05T21:41:02.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"ZachNagengast","name":"Zach Nagengast","path":"/ZachNagengast","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1981179?s=80&v=4"},"commit":{"message":"Remove fatalerror on setup + cleanup","shortMessageHtmlLink":"Remove fatalerror on setup + cleanup"}},{"before":"0129d90cb49cab6a6d21fcfcfc7c14b2efacdbe9","after":"d591adc15ebadfdc43f1fd63fc62863478f9b970","ref":"refs/heads/main","pushedAt":"2023-07-30T18:52:58.000Z","pushType":"push","commitsCount":2,"pusher":{"login":"ZachNagengast","name":"Zach Nagengast","path":"/ZachNagengast","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1981179?s=80&v=4"},"commit":{"message":"Update readme with preconverted CoreML models repo","shortMessageHtmlLink":"Update readme with preconverted CoreML models repo"}},{"before":"726f7feac98e0a3c3c464f1d66e000ce2ea82830","after":"0129d90cb49cab6a6d21fcfcfc7c14b2efacdbe9","ref":"refs/heads/main","pushedAt":"2023-07-15T06:25:27.000Z","pushType":"push","commitsCount":4,"pusher":{"login":"ZachNagengast","name":"Zach Nagengast","path":"/ZachNagengast","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1981179?s=80&v=4"},"commit":{"message":"Minor formatting","shortMessageHtmlLink":"Minor formatting"}},{"before":"5f5b00a0dd2b3f702138068d034fecd11e546e51","after":"726f7feac98e0a3c3c464f1d66e000ce2ea82830","ref":"refs/heads/main","pushedAt":"2023-07-15T06:17:40.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"ZachNagengast","name":"Zach Nagengast","path":"/ZachNagengast","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1981179?s=80&v=4"},"commit":{"message":"BinaryStore with compressed binary representation, removed some fatal errors, made saveIndex and loadIndex public methods (#16)\n\n* Felt like fatal errors were overkill here, and added more observability\r\n\r\n* Update SimilarityIndex.swift\r\n\r\n* Update SimilarityIndex.swift\r\n\r\n* Update SimilarityIndex.swift\r\n\r\n* Create BinaryStore.swift\r\n\r\n* public init for BinaryStore\r\n\r\n* Public save and loadIndex for similarityindex\r\n\r\n* Possible fix to EXC_BAD_ACCESS on appending to indexItems\r\n\r\nI think there's a concurrency issue:\r\nThread 2: EXC_BAD_ACCESS (code=1, address=0xfffffffffffffff0)\r\nhappens on indexItems.append(item)\r\nin addItem()\r\n\r\nBy making it all one task, it should avoid that.\r\n\r\nFor more context, here was my chat with GPT-4 ;)\r\n# Memory Access Error\r\n\r\n**USER:**\r\nI'm getting: `Thread 2: EXC_BAD_ACCESS (code=1, address=0xfffffffffffffff0)` when I do indexItems.append(item) in Swift. Why would that be?\r\n\r\n**ASSISTANT:**\r\nThe `EXC_BAD_ACCESS` error in Swift usually means you're trying to access memory that is not allocated, or is no longer allocated. This can happen due to a variety of reasons, such as accessing an array out of its bounds, accessing a deallocated object, or due to a race condition in multi-threaded code.\r\n\r\nIn your case, since the error happens when you're trying to append an item to an array (`indexItems.append(item)`), it's possible that you're dealing with a race condition where `indexItems` is being accessed from multiple threads at the same time.\r\n\r\nSwift's Array is not thread-safe, which means you can't safely read or write to an array from multiple threads without proper synchronization. If you're appending to the array from one thread and reading or writing to it from another thread at the same time, you can get this kind of crash.\r\n\r\nHere's a simple way to make your array access thread-safe using `DispatchQueue`:\r\n\r\n```swift\r\nclass ThreadSafeArray {\r\n private var array: [T] = []\r\n private let accessQueue = DispatchQueue(label: \"com.example.ThreadSafeArray\")\r\n\r\n func append(_ item: T) {\r\n accessQueue.async {\r\n self.array.append(item)\r\n }\r\n }\r\n\r\n var items: [T] {\r\n return accessQueue.sync {\r\n array\r\n }\r\n }\r\n}\r\n```\r\n\r\nThen you can use `ThreadSafeArray` instead of `Array` for `indexItems`.\r\n\r\nPlease note that this is just one possible cause of the error. If this doesn't solve your issue, you might need to debug your program further to find out the exact cause.\r\n\r\n* Add tests\r\n\r\n---------\r\n\r\nCo-authored-by: ZachNagengast ","shortMessageHtmlLink":"BinaryStore with compressed binary representation, removed some fatal…"}},{"before":"f714fa676da6dc0b1574fa1155186b4f4e9e2a23","after":null,"ref":"refs/heads/michaeljelly/main","pushedAt":"2023-07-15T06:05:00.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"ZachNagengast","name":"Zach Nagengast","path":"/ZachNagengast","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1981179?s=80&v=4"}},{"before":null,"after":"f714fa676da6dc0b1574fa1155186b4f4e9e2a23","ref":"refs/heads/michaeljelly/main","pushedAt":"2023-07-15T06:04:20.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"ZachNagengast","name":"Zach Nagengast","path":"/ZachNagengast","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1981179?s=80&v=4"},"commit":{"message":"Add tests","shortMessageHtmlLink":"Add tests"}},{"before":"676dfefeca419fa358fba234b9a89b5f2064164c","after":null,"ref":"refs/heads/LexiestLeszek/main","pushedAt":"2023-07-06T20:30:17.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"ZachNagengast","name":"Zach Nagengast","path":"/ZachNagengast","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1981179?s=80&v=4"}},{"before":"6aec7228a6899965772211d73aaaf3d09dd7704d","after":null,"ref":"refs/heads/ZachNagengast/workflows","pushedAt":"2023-07-06T20:30:15.000Z","pushType":"branch_deletion","commitsCount":0,"pusher":{"login":"ZachNagengast","name":"Zach Nagengast","path":"/ZachNagengast","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1981179?s=80&v=4"}},{"before":"789c72d11c825135ff34c3f9d2f388b031861d3b","after":"5f5b00a0dd2b3f702138068d034fecd11e546e51","ref":"refs/heads/main","pushedAt":"2023-07-05T16:38:56.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"ZachNagengast","name":"Zach Nagengast","path":"/ZachNagengast","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1981179?s=80&v=4"},"commit":{"message":"Added RecursiveCharacterSplitter (#14)\n\n* Added RecursiveCharacterSplitter\r\n\r\nFound the description on LangChain website and created my implementation, seems to be working. It will split documents recursively by different characters - starting with \"\\n\\n\", then \"\\n\", then \" \". This is nice because it will try to keep all the semantically relevant content in the same place for as long as possible.\r\n\r\nOrig from langchain: https://js.langchain.com/docs/modules/indexes/text_splitters/examples/recursive_character\r\n\r\n* Reduce search performance test for CI\r\n\r\n---------\r\n\r\nCo-authored-by: ZachNagengast ","shortMessageHtmlLink":"Added RecursiveCharacterSplitter (#14)"}},{"before":null,"after":"676dfefeca419fa358fba234b9a89b5f2064164c","ref":"refs/heads/LexiestLeszek/main","pushedAt":"2023-07-05T14:55:22.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"ZachNagengast","name":"Zach Nagengast","path":"/ZachNagengast","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1981179?s=80&v=4"},"commit":{"message":"Reduce search performance test for CI","shortMessageHtmlLink":"Reduce search performance test for CI"}}],"hasNextPage":true,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"djE6ks8AAAAEXGsT8AA","startCursor":null,"endCursor":null}},"title":"Activity · ZachNagengast/similarity-search-kit"}