Darktrace timeout exception handled

[thangaraj-ramesh]

Darktrace timeout exception handled #1205.

[codecov]

Codecov Report

Base: 84.66% // Head: 84.67% // Increases project coverage by +0.01% 🎉

Coverage data is based on head (9b21cf5) compared to base (e904922).
Patch coverage: 97.29% of modified lines in pull request are covered.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #1210      +/-   ##
===========================================
+ Coverage    84.66%   84.67%   +0.01%     
===========================================
  Files          557      557              
  Lines        41435    41467      +32     
===========================================
+ Hits         35080    35112      +32     
  Misses        6355     6355              

Impacted Files	Coverage Δ
...odules/darktrace/stix_transmission/error_mapper.py	100.00% <ø> (ø)
...r_modules/darktrace/stix_transmission/connector.py	93.43% <91.66%> (+0.51%)	⬆️
...darktrace/test/stix_transmission/test_darktrace.py	100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

[delliott90]

I tested this out with the execute command. first with a smaller pattern:

"[file:hashes.'SHA-256' IN ('b6c7f878b44c0a074d53e8fec9b65c7dd70844bb67524ff541f17d3d754889ec','519af7038bf8685fbfb228267b5be4c5926970c46af9dcd7d9de456143c816b1','48b48ac4edc40b006f9016ddce39dfbe2f1036338373b6f322795ba06455c668','16c48e52a529ce58bd2e8205c9196d64500b6a4304d8e70040ddb4b1b020bcd2','a4cb8126909f81262142bc478e15e43b5a3253cd3ad9d084e979f7b50d39f6ab','35e243527f5464134e99684437dffa3d88ba54462eacd9179bd11cd8032657ad','1fe7cce7969a0fcee49b03769520c5d61348a08fbf4bcd5a2611bf4afa32eca3','70ae98e4f3aa5f4518d62a1b4eb631728bd7a167d8f3ca42f0dba0ae8e41786b','90470fb5d16be01e8d2bc54488cebfc9ac0ea704c20068b17c1e7199c161efff','b950de924595b49bc861cae1ddd2b05f0e2f5ba1bae6c10b2a0ff27a30557e5b','70370930eb70c6e6c3c13879251ebff88060a1d129cd2d30c0cf940896b27bcb','bf09447beddf7dacb84c8d44ce2e9cd6fd89237059ce82cb4bea70439ee1acd7','237364314fcd23e9fe153a7233564d337b3f8f4357ce10fed75e21d8546a33b6','b81edcbf1a0b56d0f401dcfe4a6ae4d293663b42f120e60579353b6aa86bb105','9cd41ee1fa8156e1ff393ee969da8f14d6c5768d951bea57ac3be444df3416fa','93d7e24385c204fd2afcab10087273d9526d935045c6139c6f709d46bbae6d3b','91954c768c896dc028ae54c11a85def47bb7b83dbfccd3a731d38f141ca9243f','36f517b8125abdd3b03c22d0ea2b6cd9ef9e9e70bc4193a3889156f472d42873','3c1a4c5fa844b69e410e80200829e51c44bc469b0071008ef899e41218a60719','16aad93458ca35b007c5368cc3b22d4c45924ea0318fa3998ee3ba82fdaa1be5','d033dc4c61af720314b52b4b267f14875d191c865d595920ebbe15621ceae2a3','8e1e2ab0d2db8774895094d7785b9684c57749e48289c88d4cadc2a51ca9fa0f')] START t'2022-11-22T21:41:58.000Z' STOP t'2022-11-22T22:41:58.000Z'"

As expected I got the following error handling:

RuntimeError: Fetching results failed; see log for details{'success': False, 'connector': 'darktrace', 'error': 'darktrace connector error => Request Timeout', 'code': 'service_unavailable'}

However I ran a much larger pattern:

"[file:hashes.'SHA-256' IN ('b6c7f878b44c0a074d53e8fec9b65c7dd70844bb67524ff541f17d3d754889ec','519af7038bf8685fbfb228267b5be4c5926970c46af9dcd7d9de456143c816b1','48b48ac4edc40b006f9016ddce39dfbe2f1036338373b6f322795ba06455c668','16c48e52a529ce58bd2e8205c9196d64500b6a4304d8e70040ddb4b1b020bcd2','a4cb8126909f81262142bc478e15e43b5a3253cd3ad9d084e979f7b50d39f6ab','35e243527f5464134e99684437dffa3d88ba54462eacd9179bd11cd8032657ad','1fe7cce7969a0fcee49b03769520c5d61348a08fbf4bcd5a2611bf4afa32eca3','70ae98e4f3aa5f4518d62a1b4eb631728bd7a167d8f3ca42f0dba0ae8e41786b','90470fb5d16be01e8d2bc54488cebfc9ac0ea704c20068b17c1e7199c161efff','b950de924595b49bc861cae1ddd2b05f0e2f5ba1bae6c10b2a0ff27a30557e5b','70370930eb70c6e6c3c13879251ebff88060a1d129cd2d30c0cf940896b27bcb','bf09447beddf7dacb84c8d44ce2e9cd6fd89237059ce82cb4bea70439ee1acd7','237364314fcd23e9fe153a7233564d337b3f8f4357ce10fed75e21d8546a33b6','b81edcbf1a0b56d0f401dcfe4a6ae4d293663b42f120e60579353b6aa86bb105','9cd41ee1fa8156e1ff393ee969da8f14d6c5768d951bea57ac3be444df3416fa','93d7e24385c204fd2afcab10087273d9526d935045c6139c6f709d46bbae6d3b','91954c768c896dc028ae54c11a85def47bb7b83dbfccd3a731d38f141ca9243f','36f517b8125abdd3b03c22d0ea2b6cd9ef9e9e70bc4193a3889156f472d42873','3c1a4c5fa844b69e410e80200829e51c44bc469b0071008ef899e41218a60719','16aad93458ca35b007c5368cc3b22d4c45924ea0318fa3998ee3ba82fdaa1be5','d033dc4c61af720314b52b4b267f14875d191c865d595920ebbe15621ceae2a3','8e1e2ab0d2db8774895094d7785b9684c57749e48289c88d4cadc2a51ca9fa0f','daf081281c524e59e657ba9e9d0422c430ba4c446ecc48a7561fa7577ea93cf9','f71927429a51066fba98010298f7dec7ce3a0a6c2e51d360de85c1020bb44805','d9e442abe9eb34ee7a6279f2c63bc75c860651635d29c5017d34913a42080296','f668ed2bb3bda19310f78be124938610ced3a6c9280eb524c5a78b5008a64c11','dd10c9aed13c64a0be00431cb0d94a088c0abc9714a29034263dea1d8c60ade1','6b6bd65071a853a9c977afcea9e14245b080c4cfcfb8112f9772d5a8cd084a94','4f18caa8a41d71cfbd04baee3432a5e784f6bfa2682806fae29f49e8267dfab0','c067f988a062066d2d001f88c057d6e9508de1db4a448bbba2764b77523bd97a','fcc548e2f25b7a9be097b479ea0947fc37fffd1ca71fef30170b99c65d8e6034','d251c16f84dcb7baedcdbd1b397ffb4f1bbf4bf72bb50b30a775cc653a0b56ce','ddee37a98113d77485e4caf63e59b7fd35352aaf14071aed4aeff50a54219d8e','a16e0c220ec46767ff7b0013cca74259c929e04712afaf1e2ad5cddebe7b0813','f343b23a88b931f7181477dab81cf31d12b47ac89b7faea50a1d36599ba9980d','08a9506250e4cda52e1ae908f10968e858429d8a6cf59da6a41541954a6ee0a9','83c128ddd587d0c533824d89554dc4f21ad3beccd60f1c32e822245703e88d8d','3a7f53fe21ccd5c92be97dc1bbf9254cd76a57823c581bacf3279913d71da32c','d73e2de16e4d57fa264d8cf78ea0e2feec80bc07d251a311c55644dad3d91d2c','f6e04b3710044f76666468559fd2b6688ccac091284d138e461c2257c387d7d3','ffea2ec838fcb2a286f3cc8c92fc9a58117e833956dbb27fa182df34da80b400','afd3ff839eec16933583ec2679281c2aa58b69046e585ec05a3e34fd074539c0','02baaa9eef81fff0a1466022fdf5bf983a399522c092d07e5e71adbfdda4c0e8','0d80b0a30ff598eedd10ee967d2f956244dabbbb7e91af010778ae8f65662ffc','a773aa04412aec61a4771e76a3fb4fbb565c36e2160bfef0d23a996a175f8742','964b08bbb4acda6c3ba935f42d6c2f94d619045ee9baadca949d7cdfed2188a4','eb42a386e090956212ee750e14e3698ff06a21265e0b1afd51d73f57056f2b26','242e8898d85906e5304e5ed4d251522b4743ced603ffa0e55f925b4e2874cfeb','93f816371d53dd0756a27919f075bc2b0de91985910d644682c034428b3db005','7e5c0a40e5387f23eb27dce3d5668c3e4acc84813c3305b206091b98d22970a5','9aaea4b52bcf6a42fff29c859f94981036d80973decd2b5f09a3d4c0391f417a','a27f947d238e05060eb25f2395e7bfdd907529a70b624d7d525cee8669a5b330','0a3d6798e4568b994f5357e9891ab3b66dcd0d80d162ed3c1114f4646015cd5c','ae35c0faa6fe19d20bd42ef291db4df5a166ab482d42a6aeef8716ca0faeab17','9751bf95d354bac6ee9371dcf20a84bdff7ded94f063df034f2683c405827847','45a3744f9dfee0b6c334eb70d5fd4e45a0f80f2fb256be7515a7067b3bb729f3','140569dd1c58481a2466334068a2d698ab80485a5df76a1de34dbb70bb350c86','715d8919d5e17655fcd146ffd308018c86a2406d0b538646907876c5263de3d5','ebadb6988384aba01607d2055277b450eee7060189359a650541183235eb001a','65eeb6d50fbcba5e119cf60336c0cbc6a7244c8493e86fbdddc3d0fae39cd2b5','6d02d463e68f244d2313748085bcf3713ab792aea383d7d8557fb1fe6f2a80da','512f0f656980c3cf3f0bcc4951fe6b52ff5d855c58e6cff4a12ec272a184400f','6e0e7bace3a62a8b0daf1491e0184b92701b5d5fad2a271a5e92406147465731','14fe85da38106d50ab08d4d367d6c3d09be651d9258e68ed6bde846e8c59f16a','dd457f598e084712737a051c61a28b72d726bf6a8dedac8ed5ee17811e177f8f','c324a1366b2245163e9358e0d175fa0e4fe3b7ce6a9319cde5ba1039758e153a','a433e34b458c5850300c58aafa779c266bc6e92a2ce7cd2a585f37d4249dea4a','e2037ff8f8292becdaca2dab6b80db4bd466c76ce0fbe21db4412430d74f6707','8039bf78ae73ca72c3b44a0c264b6fa798dae5d6121f818fada14f7f95f8b33a','36fed73c6ef0fbbf1dc1beba2415d25491ccc486c60818abd7433b7d5f30ba5d','da0bd7c1cc0202db7da33174c090ef1f3fac6abb0578c5a1d4d359070a341bc6','50f413eac2f4d2bdbaa0279510ef74b6844e641ca29cffbb938213d724b443d7','5d4b7640dda619899f8313076fee2602e1cd94813ab98857928ffd90defd5e22','0c253d37110f45825b584dcd62a65eb5d478a13e10849ae18b3700a0f8cd50db','10731c0e46b8cead66efa4ef0c4f4c28f07acc5744caffe230ac26902fe59170','8eb3755790c01e5ae28f5803e0d2c9d95a6ac9c3143a51785a430640e7774449','cc8f59afac88e3d8b8805d3cccdf93711b371518cb20889b2f5d412845089030','0269711f3c608f161ff8cf0e502a518c10ed154397ccae447f5960e8e17d636a','b873fd9f13bca0de227bf0e213579ea8e8e91307d36f6a80f378f0d50ad841b7','e10194b9e777349ace01c140752ef40257d29c39bfe4eb8afc26ebdd5924b341')] START t'2022-11-22T21:41:58.000Z' STOP t'2022-11-22T22:41:58.000Z'"

And I got an authentication error:

RuntimeError: Fetching results failed; see log for details{'success': False, 'connector': 'darktrace', 'error': 'darktrace connector error => Invalid Authentication', 'code': 'authentication_fail'}

I double checked and am using the same auth credentials but I'm wondering if this is still a problem with the API timing out. It's misleading in this case since it's not an auth problem but should still be the same timeout issue this PR addresses.

[thangaraj-ramesh]

I tested this out with the execute command. first with a smaller pattern:

"[file:hashes.'SHA-256' IN ('b6c7f878b44c0a074d53e8fec9b65c7dd70844bb67524ff541f17d3d754889ec','519af7038bf8685fbfb228267b5be4c5926970c46af9dcd7d9de456143c816b1','48b48ac4edc40b006f9016ddce39dfbe2f1036338373b6f322795ba06455c668','16c48e52a529ce58bd2e8205c9196d64500b6a4304d8e70040ddb4b1b020bcd2','a4cb8126909f81262142bc478e15e43b5a3253cd3ad9d084e979f7b50d39f6ab','35e243527f5464134e99684437dffa3d88ba54462eacd9179bd11cd8032657ad','1fe7cce7969a0fcee49b03769520c5d61348a08fbf4bcd5a2611bf4afa32eca3','70ae98e4f3aa5f4518d62a1b4eb631728bd7a167d8f3ca42f0dba0ae8e41786b','90470fb5d16be01e8d2bc54488cebfc9ac0ea704c20068b17c1e7199c161efff','b950de924595b49bc861cae1ddd2b05f0e2f5ba1bae6c10b2a0ff27a30557e5b','70370930eb70c6e6c3c13879251ebff88060a1d129cd2d30c0cf940896b27bcb','bf09447beddf7dacb84c8d44ce2e9cd6fd89237059ce82cb4bea70439ee1acd7','237364314fcd23e9fe153a7233564d337b3f8f4357ce10fed75e21d8546a33b6','b81edcbf1a0b56d0f401dcfe4a6ae4d293663b42f120e60579353b6aa86bb105','9cd41ee1fa8156e1ff393ee969da8f14d6c5768d951bea57ac3be444df3416fa','93d7e24385c204fd2afcab10087273d9526d935045c6139c6f709d46bbae6d3b','91954c768c896dc028ae54c11a85def47bb7b83dbfccd3a731d38f141ca9243f','36f517b8125abdd3b03c22d0ea2b6cd9ef9e9e70bc4193a3889156f472d42873','3c1a4c5fa844b69e410e80200829e51c44bc469b0071008ef899e41218a60719','16aad93458ca35b007c5368cc3b22d4c45924ea0318fa3998ee3ba82fdaa1be5','d033dc4c61af720314b52b4b267f14875d191c865d595920ebbe15621ceae2a3','8e1e2ab0d2db8774895094d7785b9684c57749e48289c88d4cadc2a51ca9fa0f')] START t'2022-11-22T21:41:58.000Z' STOP t'2022-11-22T22:41:58.000Z'"

As expected I got the following error handling:

RuntimeError: Fetching results failed; see log for details{'success': False, 'connector': 'darktrace', 'error': 'darktrace connector error => Request Timeout', 'code': 'service_unavailable'}

However I ran a much larger pattern:

"[file:hashes.'SHA-256' IN ('b6c7f878b44c0a074d53e8fec9b65c7dd70844bb67524ff541f17d3d754889ec','519af7038bf8685fbfb228267b5be4c5926970c46af9dcd7d9de456143c816b1','48b48ac4edc40b006f9016ddce39dfbe2f1036338373b6f322795ba06455c668','16c48e52a529ce58bd2e8205c9196d64500b6a4304d8e70040ddb4b1b020bcd2','a4cb8126909f81262142bc478e15e43b5a3253cd3ad9d084e979f7b50d39f6ab','35e243527f5464134e99684437dffa3d88ba54462eacd9179bd11cd8032657ad','1fe7cce7969a0fcee49b03769520c5d61348a08fbf4bcd5a2611bf4afa32eca3','70ae98e4f3aa5f4518d62a1b4eb631728bd7a167d8f3ca42f0dba0ae8e41786b','90470fb5d16be01e8d2bc54488cebfc9ac0ea704c20068b17c1e7199c161efff','b950de924595b49bc861cae1ddd2b05f0e2f5ba1bae6c10b2a0ff27a30557e5b','70370930eb70c6e6c3c13879251ebff88060a1d129cd2d30c0cf940896b27bcb','bf09447beddf7dacb84c8d44ce2e9cd6fd89237059ce82cb4bea70439ee1acd7','237364314fcd23e9fe153a7233564d337b3f8f4357ce10fed75e21d8546a33b6','b81edcbf1a0b56d0f401dcfe4a6ae4d293663b42f120e60579353b6aa86bb105','9cd41ee1fa8156e1ff393ee969da8f14d6c5768d951bea57ac3be444df3416fa','93d7e24385c204fd2afcab10087273d9526d935045c6139c6f709d46bbae6d3b','91954c768c896dc028ae54c11a85def47bb7b83dbfccd3a731d38f141ca9243f','36f517b8125abdd3b03c22d0ea2b6cd9ef9e9e70bc4193a3889156f472d42873','3c1a4c5fa844b69e410e80200829e51c44bc469b0071008ef899e41218a60719','16aad93458ca35b007c5368cc3b22d4c45924ea0318fa3998ee3ba82fdaa1be5','d033dc4c61af720314b52b4b267f14875d191c865d595920ebbe15621ceae2a3','8e1e2ab0d2db8774895094d7785b9684c57749e48289c88d4cadc2a51ca9fa0f','daf081281c524e59e657ba9e9d0422c430ba4c446ecc48a7561fa7577ea93cf9','f71927429a51066fba98010298f7dec7ce3a0a6c2e51d360de85c1020bb44805','d9e442abe9eb34ee7a6279f2c63bc75c860651635d29c5017d34913a42080296','f668ed2bb3bda19310f78be124938610ced3a6c9280eb524c5a78b5008a64c11','dd10c9aed13c64a0be00431cb0d94a088c0abc9714a29034263dea1d8c60ade1','6b6bd65071a853a9c977afcea9e14245b080c4cfcfb8112f9772d5a8cd084a94','4f18caa8a41d71cfbd04baee3432a5e784f6bfa2682806fae29f49e8267dfab0','c067f988a062066d2d001f88c057d6e9508de1db4a448bbba2764b77523bd97a','fcc548e2f25b7a9be097b479ea0947fc37fffd1ca71fef30170b99c65d8e6034','d251c16f84dcb7baedcdbd1b397ffb4f1bbf4bf72bb50b30a775cc653a0b56ce','ddee37a98113d77485e4caf63e59b7fd35352aaf14071aed4aeff50a54219d8e','a16e0c220ec46767ff7b0013cca74259c929e04712afaf1e2ad5cddebe7b0813','f343b23a88b931f7181477dab81cf31d12b47ac89b7faea50a1d36599ba9980d','08a9506250e4cda52e1ae908f10968e858429d8a6cf59da6a41541954a6ee0a9','83c128ddd587d0c533824d89554dc4f21ad3beccd60f1c32e822245703e88d8d','3a7f53fe21ccd5c92be97dc1bbf9254cd76a57823c581bacf3279913d71da32c','d73e2de16e4d57fa264d8cf78ea0e2feec80bc07d251a311c55644dad3d91d2c','f6e04b3710044f76666468559fd2b6688ccac091284d138e461c2257c387d7d3','ffea2ec838fcb2a286f3cc8c92fc9a58117e833956dbb27fa182df34da80b400','afd3ff839eec16933583ec2679281c2aa58b69046e585ec05a3e34fd074539c0','02baaa9eef81fff0a1466022fdf5bf983a399522c092d07e5e71adbfdda4c0e8','0d80b0a30ff598eedd10ee967d2f956244dabbbb7e91af010778ae8f65662ffc','a773aa04412aec61a4771e76a3fb4fbb565c36e2160bfef0d23a996a175f8742','964b08bbb4acda6c3ba935f42d6c2f94d619045ee9baadca949d7cdfed2188a4','eb42a386e090956212ee750e14e3698ff06a21265e0b1afd51d73f57056f2b26','242e8898d85906e5304e5ed4d251522b4743ced603ffa0e55f925b4e2874cfeb','93f816371d53dd0756a27919f075bc2b0de91985910d644682c034428b3db005','7e5c0a40e5387f23eb27dce3d5668c3e4acc84813c3305b206091b98d22970a5','9aaea4b52bcf6a42fff29c859f94981036d80973decd2b5f09a3d4c0391f417a','a27f947d238e05060eb25f2395e7bfdd907529a70b624d7d525cee8669a5b330','0a3d6798e4568b994f5357e9891ab3b66dcd0d80d162ed3c1114f4646015cd5c','ae35c0faa6fe19d20bd42ef291db4df5a166ab482d42a6aeef8716ca0faeab17','9751bf95d354bac6ee9371dcf20a84bdff7ded94f063df034f2683c405827847','45a3744f9dfee0b6c334eb70d5fd4e45a0f80f2fb256be7515a7067b3bb729f3','140569dd1c58481a2466334068a2d698ab80485a5df76a1de34dbb70bb350c86','715d8919d5e17655fcd146ffd308018c86a2406d0b538646907876c5263de3d5','ebadb6988384aba01607d2055277b450eee7060189359a650541183235eb001a','65eeb6d50fbcba5e119cf60336c0cbc6a7244c8493e86fbdddc3d0fae39cd2b5','6d02d463e68f244d2313748085bcf3713ab792aea383d7d8557fb1fe6f2a80da','512f0f656980c3cf3f0bcc4951fe6b52ff5d855c58e6cff4a12ec272a184400f','6e0e7bace3a62a8b0daf1491e0184b92701b5d5fad2a271a5e92406147465731','14fe85da38106d50ab08d4d367d6c3d09be651d9258e68ed6bde846e8c59f16a','dd457f598e084712737a051c61a28b72d726bf6a8dedac8ed5ee17811e177f8f','c324a1366b2245163e9358e0d175fa0e4fe3b7ce6a9319cde5ba1039758e153a','a433e34b458c5850300c58aafa779c266bc6e92a2ce7cd2a585f37d4249dea4a','e2037ff8f8292becdaca2dab6b80db4bd466c76ce0fbe21db4412430d74f6707','8039bf78ae73ca72c3b44a0c264b6fa798dae5d6121f818fada14f7f95f8b33a','36fed73c6ef0fbbf1dc1beba2415d25491ccc486c60818abd7433b7d5f30ba5d','da0bd7c1cc0202db7da33174c090ef1f3fac6abb0578c5a1d4d359070a341bc6','50f413eac2f4d2bdbaa0279510ef74b6844e641ca29cffbb938213d724b443d7','5d4b7640dda619899f8313076fee2602e1cd94813ab98857928ffd90defd5e22','0c253d37110f45825b584dcd62a65eb5d478a13e10849ae18b3700a0f8cd50db','10731c0e46b8cead66efa4ef0c4f4c28f07acc5744caffe230ac26902fe59170','8eb3755790c01e5ae28f5803e0d2c9d95a6ac9c3143a51785a430640e7774449','cc8f59afac88e3d8b8805d3cccdf93711b371518cb20889b2f5d412845089030','0269711f3c608f161ff8cf0e502a518c10ed154397ccae447f5960e8e17d636a','b873fd9f13bca0de227bf0e213579ea8e8e91307d36f6a80f378f0d50ad841b7','e10194b9e777349ace01c140752ef40257d29c39bfe4eb8afc26ebdd5924b341')] START t'2022-11-22T21:41:58.000Z' STOP t'2022-11-22T22:41:58.000Z'"

And I got an authentication error:

RuntimeError: Fetching results failed; see log for details{'success': False, 'connector': 'darktrace', 'error': 'darktrace connector error => Invalid Authentication', 'code': 'authentication_fail'}

I double checked and am using the same auth credentials but I'm wondering if this is still a problem with the API timing out. It's misleading in this case since it's not an auth problem but should still be the same timeout issue this PR addresses.

@delliott90
Addressed the issue with authentication failure coming for large query.