feat wasm: readPool unlock #1076
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
boddumanohar
requested review from
peterlimg,
dabasov,
guruhubb and
cnlangzi
as code owners
Closed
guruhubb
approved these changes
Jul 2, 2023
|
Manual system tests [failure] with the following config
|
|
Manual system tests [failure] with the following config
|
|
Manual system tests [success] with the following config
|
Closed
peterlimg
added a commit
that referenced
this pull request
Jul 12, 2023
* sprint-july-2: Do not send readmarkers for free reads (#1101) Display correct size values (#1033) Fix exclude path (#1093) cleanup preferred blobbers from chain config (#1099) Resume download (#1096) fix panic (#1090) Add support for sending chunks (#1084) fix interface conversion error (#1088) Change the contract of Sharder and Miner update settings (#1085) Refactor repair (#1004) Add path in thumbnail hash (#1013) Feature/expose multiop winsdk (#1034) Refactor downloads to use sys.File (#1073) Multi download operation (#1045) feat wasm: readPool unlock (#1076)
Kishan-Dhakan
added a commit
that referenced
this pull request
Jul 16, 2023
* feat wasm: readPool unlock (#1076) * readPool unlock * feat (mobile): expose ReadPoolUnLock * show chunk size in listReq response * Revert "show chunk size in listReq response" This reverts commit c62a183. * Multi download operation (#1045) * multi download operation in mobilesdk * add description * set thresh to data shards * multiupload func added to wasm * fix wasm build * create localPath * fix typo * Update proxy.go * Add to response * fix append * increase download workers * decrease number of download workers * set workers back to 20 * use multiUpload demo for multi operation clean up logs * feat wasm: UpdateForbidAllocation * add demo for multiDownload * cleanup * rmv worker semaphore * add objURL in callback * fix build * add description * add packages * add filename to callback * ad localPath to statusBar * add blobber lock * add filename to start and progress * export winsdk functions * Revert "export winsdk functions" This reverts commit eed6cd0. * add multiDownload to winsdk * fix typo * Move min lock demand to allocation (#1046) * min_lock_demand * mobile * Feat/caching get fees estimate (#1069) * added caching instead of the network call * clean up * clean up * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * fixed comments --------- Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: boddumanohar <b.manu199@gmail.com> Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * Rename blobber field to not_available (#1043) * remove unstake total * not_available * added GRAPHNODE_SC * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * run build on all PRs * run on every PR --------- Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: boddumanohar <b.manu199@gmail.com> Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * wasm recover from panic (#1071) * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * recover from panic wasm --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * cleanup * set consensus to threshold for authTicket --------- Co-authored-by: Manohar Reddy <b.manu199@gmail.com> Co-authored-by: Piers Shepperson <piers.shepperson@btinternet.com> Co-authored-by: Yury <yuderbasov@gmail.com> Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * Refactor downloads to use sys.File (#1073) * added memfilechan * added memfilechan * clean up * refactor download methods to accept file handler * fix unit tests * refactor download methods, implement new downloader, fix unit tests * clean up * fix file callback --------- Co-authored-by: dabasov <yuderbasov@gmail.com> * Feature/expose multiop winsdk (#1034) * Expose multioperation in winsdk * Temporary remove callback args from multiupload * Export method * pass callback * test multiop * fix multiUpload * Revert "test multiop" This reverts commit 08ad565. * Multi download operation (#1045) * multi download operation in mobilesdk * add description * set thresh to data shards * multiupload func added to wasm * fix wasm build * create localPath * fix typo * Update proxy.go * Add to response * fix append * increase download workers * decrease number of download workers * set workers back to 20 * use multiUpload demo for multi operation clean up logs * feat wasm: UpdateForbidAllocation * add demo for multiDownload * cleanup * rmv worker semaphore * add objURL in callback * fix build * add description * add packages * add filename to callback * ad localPath to statusBar * add blobber lock * add filename to start and progress * export winsdk functions * Revert "export winsdk functions" This reverts commit eed6cd0. * add multiDownload to winsdk * fix typo * Move min lock demand to allocation (#1046) * min_lock_demand * mobile * Feat/caching get fees estimate (#1069) * added caching instead of the network call * clean up * clean up * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * fixed comments --------- Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: boddumanohar <b.manu199@gmail.com> Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * Rename blobber field to not_available (#1043) * remove unstake total * not_available * added GRAPHNODE_SC * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * run build on all PRs * run on every PR --------- Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: boddumanohar <b.manu199@gmail.com> Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * wasm recover from panic (#1071) * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * recover from panic wasm --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * cleanup * set consensus to threshold for authTicket --------- Co-authored-by: Manohar Reddy <b.manu199@gmail.com> Co-authored-by: Piers Shepperson <piers.shepperson@btinternet.com> Co-authored-by: Yury <yuderbasov@gmail.com> Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> --------- Co-authored-by: Lz <imlangzi@qq.com> Co-authored-by: Manohar Reddy <b.manu199@gmail.com> Co-authored-by: Hitenjain14 <57557631+Hitenjain14@users.noreply.github.com> Co-authored-by: Piers Shepperson <piers.shepperson@btinternet.com> Co-authored-by: Yury <yuderbasov@gmail.com> Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * Add path in thumbnail hash (#1013) * add path in thumbnail hash * lint fix * fix typo * fix input s3_migration_cli_branch system_tests.yml * updated workflow input to s3_migration_cli_branch in system_tests.yml * fix hash calculation * check prevWM * refactor * rmv prevWM check --------- Co-authored-by: shahnawaz-creator <117025384+shahnawaz-creator@users.noreply.github.com> Co-authored-by: Yury <yuderbasov@gmail.com> * Refactor repair (#1004) * add repair in multi op * rmv check status from chunk upload and delete * rmv isRepair flag from worker * rmv fileID from fileMetaHash * add mut for repair callback * repair changes * delete file after upload in repair * add delete mask * add with mask option * fix uint tests * fix multi createDir * add path in filemetahash --------- Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> Co-authored-by: Yury <yuderbasov@gmail.com> * Change the contract of Sharder and Miner update settings (#1085) Co-authored-by: Yury <yuderbasov@gmail.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * fix interface conversion error (#1088) * fix interface conversion * use sdk.multiUpload * Revert "use sdk.multiUpload" This reverts commit 428960e. * use multi upload * Add support for sending chunks (#1084) * add chunk feature * check fileshards * check chunkNum * check fileShards * add check in processUpload * check hash len --------- Co-authored-by: Manohar Reddy <b.manu199@gmail.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * fix panic (#1090) * Resume download (#1096) * resume download * clean up * cleanup preferred blobbers from chain config (#1099) * cleanup preferred blobbers from chain config * fix mobilesdk/CreateAllocation * fix lint * Fix exclude path (#1093) * Fix exclude path * Resolved comment * Display correct size values (#1033) * display correct size values * fix consensus threshold * fix unit tests * fix allocation unit tests * request from all blobbers but read until consensus * fix file meta inconsistency * revert consensus threshold * fix get file meta --------- Co-authored-by: Yury <yuderbasov@gmail.com> * Do not send readmarkers for free reads (#1101) * do not send readmarkers for free reads * remove unused download worker count * add multi op fix (#1077) * add multi op fix * fix lock connection mask * fix op mask * create connObj * add rollback in multi op * fix ctx in streamReader * use multiop ctx in reader * cleanup * fix repair in multi op * fix repair cb --------- Co-authored-by: Yury <yuderbasov@gmail.com> * fix encrypt in multi op (#1097) Co-authored-by: Yury <yuderbasov@gmail.com> * take sharder consensous from wallet config (#1104) * events information, do a consensus among 3 sharders * use a const * init from config.yaml * fix comment * fix UTs * Optimize repair (#1105) * rmv streamreader * optimize repair * fix lint and test * rmv readCtx * fix upload check * Fix (#1110) * Export balance get function (#1103) Co-authored-by: Yury <yuderbasov@gmail.com> * Add getUpdateAllocationMinLock to wasm (#1107) * Add getUpdateAllocationMinLock * Fix demon index.html * fixed lint --------- Co-authored-by: Yury <yuderbasov@gmail.com> * Updating blobber and validator update setting models. (#1095) * Updating blobber and validatory update setting models. * Updating the Terms struct as well. * Updating the StakePoolSettings struct as well. * pass only updted fields in validator --------- Co-authored-by: Yury <yuderbasov@gmail.com> * Accept absolute path when downloading (#1111) * accept abs path also * fix unit tests * check if localpath has ext * fix seek for resume * Updating the validator node which is being passed to 0chain in update settings call (#1114) * fixing json name for the validator id. * fixing the blockchain validator udpate settings to pass to 0chain * fixing the json name for validator * Revert "Optimize repair (#1105)" (#1116) This reverts commit d293d7d. --------- Co-authored-by: Hitenjain14 <57557631+Hitenjain14@users.noreply.github.com> Co-authored-by: Piers Shepperson <piers.shepperson@btinternet.com> Co-authored-by: Yury <yuderbasov@gmail.com> Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> Co-authored-by: Dinmukhammed <52813950+din-mukhammed@users.noreply.github.com> Co-authored-by: Roshan Kumar <56060325+Roshan-Mehta@users.noreply.github.com> Co-authored-by: Lz <imlangzi@qq.com> Co-authored-by: shahnawaz-creator <117025384+shahnawaz-creator@users.noreply.github.com> Co-authored-by: Sunil Kumar <sunilkr.kumar26@gmail.com> Co-authored-by: Jayash Satolia <73050737+Jayashsatolia403@users.noreply.github.com> Co-authored-by: stewartie4 <ryanstewart456@gmail.com> Co-authored-by: peterlimg <54137706+peterlimg@users.noreply.github.com>
dabasov
added a commit
that referenced
this pull request
Jul 18, 2023
* feat wasm: readPool unlock (#1076) * readPool unlock * feat (mobile): expose ReadPoolUnLock * show chunk size in listReq response * Revert "show chunk size in listReq response" This reverts commit c62a183. * Multi download operation (#1045) * multi download operation in mobilesdk * add description * set thresh to data shards * multiupload func added to wasm * fix wasm build * create localPath * fix typo * Update proxy.go * Add to response * fix append * increase download workers * decrease number of download workers * set workers back to 20 * use multiUpload demo for multi operation clean up logs * feat wasm: UpdateForbidAllocation * add demo for multiDownload * cleanup * rmv worker semaphore * add objURL in callback * fix build * add description * add packages * add filename to callback * ad localPath to statusBar * add blobber lock * add filename to start and progress * export winsdk functions * Revert "export winsdk functions" This reverts commit eed6cd0. * add multiDownload to winsdk * fix typo * Move min lock demand to allocation (#1046) * min_lock_demand * mobile * Feat/caching get fees estimate (#1069) * added caching instead of the network call * clean up * clean up * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * fixed comments --------- Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: boddumanohar <b.manu199@gmail.com> Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * Rename blobber field to not_available (#1043) * remove unstake total * not_available * added GRAPHNODE_SC * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * run build on all PRs * run on every PR --------- Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: boddumanohar <b.manu199@gmail.com> Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * wasm recover from panic (#1071) * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * recover from panic wasm --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * cleanup * set consensus to threshold for authTicket --------- Co-authored-by: Manohar Reddy <b.manu199@gmail.com> Co-authored-by: Piers Shepperson <piers.shepperson@btinternet.com> Co-authored-by: Yury <yuderbasov@gmail.com> Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * Refactor downloads to use sys.File (#1073) * added memfilechan * added memfilechan * clean up * refactor download methods to accept file handler * fix unit tests * refactor download methods, implement new downloader, fix unit tests * clean up * fix file callback --------- Co-authored-by: dabasov <yuderbasov@gmail.com> * Feature/expose multiop winsdk (#1034) * Expose multioperation in winsdk * Temporary remove callback args from multiupload * Export method * pass callback * test multiop * fix multiUpload * Revert "test multiop" This reverts commit 08ad565. * Multi download operation (#1045) * multi download operation in mobilesdk * add description * set thresh to data shards * multiupload func added to wasm * fix wasm build * create localPath * fix typo * Update proxy.go * Add to response * fix append * increase download workers * decrease number of download workers * set workers back to 20 * use multiUpload demo for multi operation clean up logs * feat wasm: UpdateForbidAllocation * add demo for multiDownload * cleanup * rmv worker semaphore * add objURL in callback * fix build * add description * add packages * add filename to callback * ad localPath to statusBar * add blobber lock * add filename to start and progress * export winsdk functions * Revert "export winsdk functions" This reverts commit eed6cd0. * add multiDownload to winsdk * fix typo * Move min lock demand to allocation (#1046) * min_lock_demand * mobile * Feat/caching get fees estimate (#1069) * added caching instead of the network call * clean up * clean up * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * fixed comments --------- Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: boddumanohar <b.manu199@gmail.com> Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * Rename blobber field to not_available (#1043) * remove unstake total * not_available * added GRAPHNODE_SC * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * run build on all PRs * run on every PR --------- Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: boddumanohar <b.manu199@gmail.com> Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * wasm recover from panic (#1071) * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * recover from panic wasm --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * cleanup * set consensus to threshold for authTicket --------- Co-authored-by: Manohar Reddy <b.manu199@gmail.com> Co-authored-by: Piers Shepperson <piers.shepperson@btinternet.com> Co-authored-by: Yury <yuderbasov@gmail.com> Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> --------- Co-authored-by: Lz <imlangzi@qq.com> Co-authored-by: Manohar Reddy <b.manu199@gmail.com> Co-authored-by: Hitenjain14 <57557631+Hitenjain14@users.noreply.github.com> Co-authored-by: Piers Shepperson <piers.shepperson@btinternet.com> Co-authored-by: Yury <yuderbasov@gmail.com> Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * Add path in thumbnail hash (#1013) * add path in thumbnail hash * lint fix * fix typo * fix input s3_migration_cli_branch system_tests.yml * updated workflow input to s3_migration_cli_branch in system_tests.yml * fix hash calculation * check prevWM * refactor * rmv prevWM check --------- Co-authored-by: shahnawaz-creator <117025384+shahnawaz-creator@users.noreply.github.com> Co-authored-by: Yury <yuderbasov@gmail.com> * Refactor repair (#1004) * add repair in multi op * rmv check status from chunk upload and delete * rmv isRepair flag from worker * rmv fileID from fileMetaHash * add mut for repair callback * repair changes * delete file after upload in repair * add delete mask * add with mask option * fix uint tests * fix multi createDir * add path in filemetahash --------- Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> Co-authored-by: Yury <yuderbasov@gmail.com> * Change the contract of Sharder and Miner update settings (#1085) Co-authored-by: Yury <yuderbasov@gmail.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * fix interface conversion error (#1088) * fix interface conversion * use sdk.multiUpload * Revert "use sdk.multiUpload" This reverts commit 428960e. * use multi upload * Add support for sending chunks (#1084) * add chunk feature * check fileshards * check chunkNum * check fileShards * add check in processUpload * check hash len --------- Co-authored-by: Manohar Reddy <b.manu199@gmail.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * fix panic (#1090) * Resume download (#1096) * resume download * clean up * cleanup preferred blobbers from chain config (#1099) * cleanup preferred blobbers from chain config * fix mobilesdk/CreateAllocation * fix lint * Fix exclude path (#1093) * Fix exclude path * Resolved comment * Display correct size values (#1033) * display correct size values * fix consensus threshold * fix unit tests * fix allocation unit tests * request from all blobbers but read until consensus * fix file meta inconsistency * revert consensus threshold * fix get file meta --------- Co-authored-by: Yury <yuderbasov@gmail.com> * Do not send readmarkers for free reads (#1101) * do not send readmarkers for free reads * remove unused download worker count * add multi op fix (#1077) * add multi op fix * fix lock connection mask * fix op mask * create connObj * add rollback in multi op * fix ctx in streamReader * use multiop ctx in reader * cleanup * fix repair in multi op * fix repair cb --------- Co-authored-by: Yury <yuderbasov@gmail.com> * fix encrypt in multi op (#1097) Co-authored-by: Yury <yuderbasov@gmail.com> * take sharder consensous from wallet config (#1104) * events information, do a consensus among 3 sharders * use a const * init from config.yaml * fix comment * fix UTs * Optimize repair (#1105) * rmv streamreader * optimize repair * fix lint and test * rmv readCtx * fix upload check * Fix * Fix * Fix update allocation * Fix create allocation * Fix (#1110) * Export balance get function (#1103) Co-authored-by: Yury <yuderbasov@gmail.com> * Add getUpdateAllocationMinLock to wasm (#1107) * Add getUpdateAllocationMinLock * Fix demon index.html * fixed lint --------- Co-authored-by: Yury <yuderbasov@gmail.com> * Updating blobber and validator update setting models. (#1095) * Updating blobber and validatory update setting models. * Updating the Terms struct as well. * Updating the StakePoolSettings struct as well. * pass only updted fields in validator --------- Co-authored-by: Yury <yuderbasov@gmail.com> * Accept absolute path when downloading (#1111) * accept abs path also * fix unit tests * check if localpath has ext * fix seek for resume * Updating the validator node which is being passed to 0chain in update settings call (#1114) * fixing json name for the validator id. * fixing the blockchain validator udpate settings to pass to 0chain * fixing the json name for validator * Fix * Revert "Optimize repair (#1105)" (#1116) This reverts commit d293d7d. --------- Co-authored-by: boddumanohar <b.manu199@gmail.com> Co-authored-by: Hitenjain14 <57557631+Hitenjain14@users.noreply.github.com> Co-authored-by: Piers Shepperson <piers.shepperson@btinternet.com> Co-authored-by: Yury <yuderbasov@gmail.com> Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> Co-authored-by: Dinmukhammed <52813950+din-mukhammed@users.noreply.github.com> Co-authored-by: Roshan Kumar <56060325+Roshan-Mehta@users.noreply.github.com> Co-authored-by: Lz <imlangzi@qq.com> Co-authored-by: shahnawaz-creator <117025384+shahnawaz-creator@users.noreply.github.com> Co-authored-by: Sunil Kumar <sunilkr.kumar26@gmail.com> Co-authored-by: stewartie4 <ryanstewart456@gmail.com> Co-authored-by: peterlimg <54137706+peterlimg@users.noreply.github.com>
dabasov
added a commit
that referenced
this pull request
Jul 21, 2023
* feat wasm: readPool unlock (#1076) * readPool unlock * feat (mobile): expose ReadPoolUnLock * show chunk size in listReq response * Revert "show chunk size in listReq response" This reverts commit c62a183. * Multi download operation (#1045) * multi download operation in mobilesdk * add description * set thresh to data shards * multiupload func added to wasm * fix wasm build * create localPath * fix typo * Update proxy.go * Add to response * fix append * increase download workers * decrease number of download workers * set workers back to 20 * use multiUpload demo for multi operation clean up logs * feat wasm: UpdateForbidAllocation * add demo for multiDownload * cleanup * rmv worker semaphore * add objURL in callback * fix build * add description * add packages * add filename to callback * ad localPath to statusBar * add blobber lock * add filename to start and progress * export winsdk functions * Revert "export winsdk functions" This reverts commit eed6cd0. * add multiDownload to winsdk * fix typo * Move min lock demand to allocation (#1046) * min_lock_demand * mobile * Feat/caching get fees estimate (#1069) * added caching instead of the network call * clean up * clean up * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * fixed comments --------- Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: boddumanohar <b.manu199@gmail.com> Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * Rename blobber field to not_available (#1043) * remove unstake total * not_available * added GRAPHNODE_SC * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * run build on all PRs * run on every PR --------- Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: boddumanohar <b.manu199@gmail.com> Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * wasm recover from panic (#1071) * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * recover from panic wasm --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * cleanup * set consensus to threshold for authTicket --------- Co-authored-by: Manohar Reddy <b.manu199@gmail.com> Co-authored-by: Piers Shepperson <piers.shepperson@btinternet.com> Co-authored-by: Yury <yuderbasov@gmail.com> Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * Refactor downloads to use sys.File (#1073) * added memfilechan * added memfilechan * clean up * refactor download methods to accept file handler * fix unit tests * refactor download methods, implement new downloader, fix unit tests * clean up * fix file callback --------- Co-authored-by: dabasov <yuderbasov@gmail.com> * Feature/expose multiop winsdk (#1034) * Expose multioperation in winsdk * Temporary remove callback args from multiupload * Export method * pass callback * test multiop * fix multiUpload * Revert "test multiop" This reverts commit 08ad565. * Multi download operation (#1045) * multi download operation in mobilesdk * add description * set thresh to data shards * multiupload func added to wasm * fix wasm build * create localPath * fix typo * Update proxy.go * Add to response * fix append * increase download workers * decrease number of download workers * set workers back to 20 * use multiUpload demo for multi operation clean up logs * feat wasm: UpdateForbidAllocation * add demo for multiDownload * cleanup * rmv worker semaphore * add objURL in callback * fix build * add description * add packages * add filename to callback * ad localPath to statusBar * add blobber lock * add filename to start and progress * export winsdk functions * Revert "export winsdk functions" This reverts commit eed6cd0. * add multiDownload to winsdk * fix typo * Move min lock demand to allocation (#1046) * min_lock_demand * mobile * Feat/caching get fees estimate (#1069) * added caching instead of the network call * clean up * clean up * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * fixed comments --------- Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: boddumanohar <b.manu199@gmail.com> Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * Rename blobber field to not_available (#1043) * remove unstake total * not_available * added GRAPHNODE_SC * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * run build on all PRs * run on every PR --------- Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: boddumanohar <b.manu199@gmail.com> Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * wasm recover from panic (#1071) * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * recover from panic wasm --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * cleanup * set consensus to threshold for authTicket --------- Co-authored-by: Manohar Reddy <b.manu199@gmail.com> Co-authored-by: Piers Shepperson <piers.shepperson@btinternet.com> Co-authored-by: Yury <yuderbasov@gmail.com> Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> --------- Co-authored-by: Lz <imlangzi@qq.com> Co-authored-by: Manohar Reddy <b.manu199@gmail.com> Co-authored-by: Hitenjain14 <57557631+Hitenjain14@users.noreply.github.com> Co-authored-by: Piers Shepperson <piers.shepperson@btinternet.com> Co-authored-by: Yury <yuderbasov@gmail.com> Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * Add path in thumbnail hash (#1013) * add path in thumbnail hash * lint fix * fix typo * fix input s3_migration_cli_branch system_tests.yml * updated workflow input to s3_migration_cli_branch in system_tests.yml * fix hash calculation * check prevWM * refactor * rmv prevWM check --------- Co-authored-by: shahnawaz-creator <117025384+shahnawaz-creator@users.noreply.github.com> Co-authored-by: Yury <yuderbasov@gmail.com> * Refactor repair (#1004) * add repair in multi op * rmv check status from chunk upload and delete * rmv isRepair flag from worker * rmv fileID from fileMetaHash * add mut for repair callback * repair changes * delete file after upload in repair * add delete mask * add with mask option * fix uint tests * fix multi createDir * add path in filemetahash --------- Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> Co-authored-by: Yury <yuderbasov@gmail.com> * Change the contract of Sharder and Miner update settings (#1085) Co-authored-by: Yury <yuderbasov@gmail.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * fix interface conversion error (#1088) * fix interface conversion * use sdk.multiUpload * Revert "use sdk.multiUpload" This reverts commit 428960e. * use multi upload * Add support for sending chunks (#1084) * add chunk feature * check fileshards * check chunkNum * check fileShards * add check in processUpload * check hash len --------- Co-authored-by: Manohar Reddy <b.manu199@gmail.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * fix panic (#1090) * Resume download (#1096) * resume download * clean up * cleanup preferred blobbers from chain config (#1099) * cleanup preferred blobbers from chain config * fix mobilesdk/CreateAllocation * fix lint * Fix exclude path (#1093) * Fix exclude path * Resolved comment * Display correct size values (#1033) * display correct size values * fix consensus threshold * fix unit tests * fix allocation unit tests * request from all blobbers but read until consensus * fix file meta inconsistency * revert consensus threshold * fix get file meta --------- Co-authored-by: Yury <yuderbasov@gmail.com> * Do not send readmarkers for free reads (#1101) * do not send readmarkers for free reads * remove unused download worker count * rmv streamreader * optimize repair * fix lint and test * rmv readCtx * fix upload check * add multi op fix (#1077) * add multi op fix * fix lock connection mask * fix op mask * create connObj * add rollback in multi op * fix ctx in streamReader * use multiop ctx in reader * cleanup * fix repair in multi op * fix repair cb --------- Co-authored-by: Yury <yuderbasov@gmail.com> * fix encrypt in multi op (#1097) Co-authored-by: Yury <yuderbasov@gmail.com> * take sharder consensous from wallet config (#1104) * events information, do a consensus among 3 sharders * use a const * init from config.yaml * fix comment * fix UTs * hotfix: fix `upload` and `download` in wasm (#1121) * Fix wasm download (#1117) * fix wasm download * fix syntax * fix reader * reverting reader changes --------- Co-authored-by: Dinmukhammed <52813950+din-mukhammed@users.noreply.github.com> Co-authored-by: din-mukhammed <dimash.r53@gmail.com> * add wasm download changes * fix merge * fix typo * fix repair cb * fix unit test * rmv log * check if env is wasm * cleanup * defer lock --------- Co-authored-by: boddumanohar <b.manu199@gmail.com> Co-authored-by: Piers Shepperson <piers.shepperson@btinternet.com> Co-authored-by: Yury <yuderbasov@gmail.com> Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> Co-authored-by: Dinmukhammed <52813950+din-mukhammed@users.noreply.github.com> Co-authored-by: Roshan Kumar <56060325+Roshan-Mehta@users.noreply.github.com> Co-authored-by: Lz <imlangzi@qq.com> Co-authored-by: shahnawaz-creator <117025384+shahnawaz-creator@users.noreply.github.com> Co-authored-by: Sunil Kumar <sunilkr.kumar26@gmail.com> Co-authored-by: Jayash Satolia <73050737+Jayashsatolia403@users.noreply.github.com> Co-authored-by: stewartie4 <ryanstewart456@gmail.com> Co-authored-by: din-mukhammed <dimash.r53@gmail.com>
YarikRevich
pushed a commit
that referenced
this pull request
Jul 22, 2023
* feat wasm: readPool unlock (#1076) * readPool unlock * feat (mobile): expose ReadPoolUnLock * show chunk size in listReq response * Revert "show chunk size in listReq response" This reverts commit c62a183. * Multi download operation (#1045) * multi download operation in mobilesdk * add description * set thresh to data shards * multiupload func added to wasm * fix wasm build * create localPath * fix typo * Update proxy.go * Add to response * fix append * increase download workers * decrease number of download workers * set workers back to 20 * use multiUpload demo for multi operation clean up logs * feat wasm: UpdateForbidAllocation * add demo for multiDownload * cleanup * rmv worker semaphore * add objURL in callback * fix build * add description * add packages * add filename to callback * ad localPath to statusBar * add blobber lock * add filename to start and progress * export winsdk functions * Revert "export winsdk functions" This reverts commit eed6cd0. * add multiDownload to winsdk * fix typo * Move min lock demand to allocation (#1046) * min_lock_demand * mobile * Feat/caching get fees estimate (#1069) * added caching instead of the network call * clean up * clean up * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * fixed comments --------- Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: boddumanohar <b.manu199@gmail.com> Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * Rename blobber field to not_available (#1043) * remove unstake total * not_available * added GRAPHNODE_SC * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * run build on all PRs * run on every PR --------- Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: boddumanohar <b.manu199@gmail.com> Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * wasm recover from panic (#1071) * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * recover from panic wasm --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * cleanup * set consensus to threshold for authTicket --------- Co-authored-by: Manohar Reddy <b.manu199@gmail.com> Co-authored-by: Piers Shepperson <piers.shepperson@btinternet.com> Co-authored-by: Yury <yuderbasov@gmail.com> Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * Refactor downloads to use sys.File (#1073) * added memfilechan * added memfilechan * clean up * refactor download methods to accept file handler * fix unit tests * refactor download methods, implement new downloader, fix unit tests * clean up * fix file callback --------- Co-authored-by: dabasov <yuderbasov@gmail.com> * Feature/expose multiop winsdk (#1034) * Expose multioperation in winsdk * Temporary remove callback args from multiupload * Export method * pass callback * test multiop * fix multiUpload * Revert "test multiop" This reverts commit 08ad565. * Multi download operation (#1045) * multi download operation in mobilesdk * add description * set thresh to data shards * multiupload func added to wasm * fix wasm build * create localPath * fix typo * Update proxy.go * Add to response * fix append * increase download workers * decrease number of download workers * set workers back to 20 * use multiUpload demo for multi operation clean up logs * feat wasm: UpdateForbidAllocation * add demo for multiDownload * cleanup * rmv worker semaphore * add objURL in callback * fix build * add description * add packages * add filename to callback * ad localPath to statusBar * add blobber lock * add filename to start and progress * export winsdk functions * Revert "export winsdk functions" This reverts commit eed6cd0. * add multiDownload to winsdk * fix typo * Move min lock demand to allocation (#1046) * min_lock_demand * mobile * Feat/caching get fees estimate (#1069) * added caching instead of the network call * clean up * clean up * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * fixed comments --------- Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: boddumanohar <b.manu199@gmail.com> Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * Rename blobber field to not_available (#1043) * remove unstake total * not_available * added GRAPHNODE_SC * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * run build on all PRs * run on every PR --------- Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: boddumanohar <b.manu199@gmail.com> Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * wasm recover from panic (#1071) * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * recover from panic wasm --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * cleanup * set consensus to threshold for authTicket --------- Co-authored-by: Manohar Reddy <b.manu199@gmail.com> Co-authored-by: Piers Shepperson <piers.shepperson@btinternet.com> Co-authored-by: Yury <yuderbasov@gmail.com> Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> --------- Co-authored-by: Lz <imlangzi@qq.com> Co-authored-by: Manohar Reddy <b.manu199@gmail.com> Co-authored-by: Hitenjain14 <57557631+Hitenjain14@users.noreply.github.com> Co-authored-by: Piers Shepperson <piers.shepperson@btinternet.com> Co-authored-by: Yury <yuderbasov@gmail.com> Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * Add path in thumbnail hash (#1013) * add path in thumbnail hash * lint fix * fix typo * fix input s3_migration_cli_branch system_tests.yml * updated workflow input to s3_migration_cli_branch in system_tests.yml * fix hash calculation * check prevWM * refactor * rmv prevWM check --------- Co-authored-by: shahnawaz-creator <117025384+shahnawaz-creator@users.noreply.github.com> Co-authored-by: Yury <yuderbasov@gmail.com> * Refactor repair (#1004) * add repair in multi op * rmv check status from chunk upload and delete * rmv isRepair flag from worker * rmv fileID from fileMetaHash * add mut for repair callback * repair changes * delete file after upload in repair * add delete mask * add with mask option * fix uint tests * fix multi createDir * add path in filemetahash --------- Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> Co-authored-by: Yury <yuderbasov@gmail.com> * Change the contract of Sharder and Miner update settings (#1085) Co-authored-by: Yury <yuderbasov@gmail.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * fix interface conversion error (#1088) * fix interface conversion * use sdk.multiUpload * Revert "use sdk.multiUpload" This reverts commit 428960e. * use multi upload * Add support for sending chunks (#1084) * add chunk feature * check fileshards * check chunkNum * check fileShards * add check in processUpload * check hash len --------- Co-authored-by: Manohar Reddy <b.manu199@gmail.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * fix panic (#1090) * Resume download (#1096) * resume download * clean up * cleanup preferred blobbers from chain config (#1099) * cleanup preferred blobbers from chain config * fix mobilesdk/CreateAllocation * fix lint * Fix exclude path (#1093) * Fix exclude path * Resolved comment * Display correct size values (#1033) * display correct size values * fix consensus threshold * fix unit tests * fix allocation unit tests * request from all blobbers but read until consensus * fix file meta inconsistency * revert consensus threshold * fix get file meta --------- Co-authored-by: Yury <yuderbasov@gmail.com> * Do not send readmarkers for free reads (#1101) * do not send readmarkers for free reads * remove unused download worker count * add multi op fix (#1077) * add multi op fix * fix lock connection mask * fix op mask * create connObj * add rollback in multi op * fix ctx in streamReader * use multiop ctx in reader * cleanup * fix repair in multi op * fix repair cb --------- Co-authored-by: Yury <yuderbasov@gmail.com> * fix encrypt in multi op (#1097) Co-authored-by: Yury <yuderbasov@gmail.com> * take sharder consensous from wallet config (#1104) * events information, do a consensus among 3 sharders * use a const * init from config.yaml * fix comment * fix UTs * Optimize repair (#1105) * rmv streamreader * optimize repair * fix lint and test * rmv readCtx * fix upload check * Fix * Fix * Fix update allocation * Fix create allocation * Fix (#1110) * Export balance get function (#1103) Co-authored-by: Yury <yuderbasov@gmail.com> * Add getUpdateAllocationMinLock to wasm (#1107) * Add getUpdateAllocationMinLock * Fix demon index.html * fixed lint --------- Co-authored-by: Yury <yuderbasov@gmail.com> * Updating blobber and validator update setting models. (#1095) * Updating blobber and validatory update setting models. * Updating the Terms struct as well. * Updating the StakePoolSettings struct as well. * pass only updted fields in validator --------- Co-authored-by: Yury <yuderbasov@gmail.com> * Accept absolute path when downloading (#1111) * accept abs path also * fix unit tests * check if localpath has ext * fix seek for resume * Updating the validator node which is being passed to 0chain in update settings call (#1114) * fixing json name for the validator id. * fixing the blockchain validator udpate settings to pass to 0chain * fixing the json name for validator * Fix * Revert "Optimize repair (#1105)" (#1116) This reverts commit d293d7d. --------- Co-authored-by: boddumanohar <b.manu199@gmail.com> Co-authored-by: Hitenjain14 <57557631+Hitenjain14@users.noreply.github.com> Co-authored-by: Piers Shepperson <piers.shepperson@btinternet.com> Co-authored-by: Yury <yuderbasov@gmail.com> Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> Co-authored-by: Dinmukhammed <52813950+din-mukhammed@users.noreply.github.com> Co-authored-by: Roshan Kumar <56060325+Roshan-Mehta@users.noreply.github.com> Co-authored-by: Lz <imlangzi@qq.com> Co-authored-by: shahnawaz-creator <117025384+shahnawaz-creator@users.noreply.github.com> Co-authored-by: Sunil Kumar <sunilkr.kumar26@gmail.com> Co-authored-by: stewartie4 <ryanstewart456@gmail.com> Co-authored-by: peterlimg <54137706+peterlimg@users.noreply.github.com>
YarikRevich
pushed a commit
that referenced
this pull request
Jul 22, 2023
* feat wasm: readPool unlock (#1076) * readPool unlock * feat (mobile): expose ReadPoolUnLock * show chunk size in listReq response * Revert "show chunk size in listReq response" This reverts commit c62a183. * Multi download operation (#1045) * multi download operation in mobilesdk * add description * set thresh to data shards * multiupload func added to wasm * fix wasm build * create localPath * fix typo * Update proxy.go * Add to response * fix append * increase download workers * decrease number of download workers * set workers back to 20 * use multiUpload demo for multi operation clean up logs * feat wasm: UpdateForbidAllocation * add demo for multiDownload * cleanup * rmv worker semaphore * add objURL in callback * fix build * add description * add packages * add filename to callback * ad localPath to statusBar * add blobber lock * add filename to start and progress * export winsdk functions * Revert "export winsdk functions" This reverts commit eed6cd0. * add multiDownload to winsdk * fix typo * Move min lock demand to allocation (#1046) * min_lock_demand * mobile * Feat/caching get fees estimate (#1069) * added caching instead of the network call * clean up * clean up * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * fixed comments --------- Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: boddumanohar <b.manu199@gmail.com> Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * Rename blobber field to not_available (#1043) * remove unstake total * not_available * added GRAPHNODE_SC * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * run build on all PRs * run on every PR --------- Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: boddumanohar <b.manu199@gmail.com> Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * wasm recover from panic (#1071) * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * recover from panic wasm --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * cleanup * set consensus to threshold for authTicket --------- Co-authored-by: Manohar Reddy <b.manu199@gmail.com> Co-authored-by: Piers Shepperson <piers.shepperson@btinternet.com> Co-authored-by: Yury <yuderbasov@gmail.com> Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * Refactor downloads to use sys.File (#1073) * added memfilechan * added memfilechan * clean up * refactor download methods to accept file handler * fix unit tests * refactor download methods, implement new downloader, fix unit tests * clean up * fix file callback --------- Co-authored-by: dabasov <yuderbasov@gmail.com> * Feature/expose multiop winsdk (#1034) * Expose multioperation in winsdk * Temporary remove callback args from multiupload * Export method * pass callback * test multiop * fix multiUpload * Revert "test multiop" This reverts commit 08ad565. * Multi download operation (#1045) * multi download operation in mobilesdk * add description * set thresh to data shards * multiupload func added to wasm * fix wasm build * create localPath * fix typo * Update proxy.go * Add to response * fix append * increase download workers * decrease number of download workers * set workers back to 20 * use multiUpload demo for multi operation clean up logs * feat wasm: UpdateForbidAllocation * add demo for multiDownload * cleanup * rmv worker semaphore * add objURL in callback * fix build * add description * add packages * add filename to callback * ad localPath to statusBar * add blobber lock * add filename to start and progress * export winsdk functions * Revert "export winsdk functions" This reverts commit eed6cd0. * add multiDownload to winsdk * fix typo * Move min lock demand to allocation (#1046) * min_lock_demand * mobile * Feat/caching get fees estimate (#1069) * added caching instead of the network call * clean up * clean up * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * fixed comments --------- Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: boddumanohar <b.manu199@gmail.com> Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * Rename blobber field to not_available (#1043) * remove unstake total * not_available * added GRAPHNODE_SC * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * run build on all PRs * run on every PR --------- Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: boddumanohar <b.manu199@gmail.com> Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * wasm recover from panic (#1071) * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * recover from panic wasm --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * cleanup * set consensus to threshold for authTicket --------- Co-authored-by: Manohar Reddy <b.manu199@gmail.com> Co-authored-by: Piers Shepperson <piers.shepperson@btinternet.com> Co-authored-by: Yury <yuderbasov@gmail.com> Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> --------- Co-authored-by: Lz <imlangzi@qq.com> Co-authored-by: Manohar Reddy <b.manu199@gmail.com> Co-authored-by: Hitenjain14 <57557631+Hitenjain14@users.noreply.github.com> Co-authored-by: Piers Shepperson <piers.shepperson@btinternet.com> Co-authored-by: Yury <yuderbasov@gmail.com> Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * Add path in thumbnail hash (#1013) * add path in thumbnail hash * lint fix * fix typo * fix input s3_migration_cli_branch system_tests.yml * updated workflow input to s3_migration_cli_branch in system_tests.yml * fix hash calculation * check prevWM * refactor * rmv prevWM check --------- Co-authored-by: shahnawaz-creator <117025384+shahnawaz-creator@users.noreply.github.com> Co-authored-by: Yury <yuderbasov@gmail.com> * Refactor repair (#1004) * add repair in multi op * rmv check status from chunk upload and delete * rmv isRepair flag from worker * rmv fileID from fileMetaHash * add mut for repair callback * repair changes * delete file after upload in repair * add delete mask * add with mask option * fix uint tests * fix multi createDir * add path in filemetahash --------- Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> Co-authored-by: Yury <yuderbasov@gmail.com> * Change the contract of Sharder and Miner update settings (#1085) Co-authored-by: Yury <yuderbasov@gmail.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * fix interface conversion error (#1088) * fix interface conversion * use sdk.multiUpload * Revert "use sdk.multiUpload" This reverts commit 428960e. * use multi upload * Add support for sending chunks (#1084) * add chunk feature * check fileshards * check chunkNum * check fileShards * add check in processUpload * check hash len --------- Co-authored-by: Manohar Reddy <b.manu199@gmail.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * fix panic (#1090) * Resume download (#1096) * resume download * clean up * cleanup preferred blobbers from chain config (#1099) * cleanup preferred blobbers from chain config * fix mobilesdk/CreateAllocation * fix lint * Fix exclude path (#1093) * Fix exclude path * Resolved comment * Display correct size values (#1033) * display correct size values * fix consensus threshold * fix unit tests * fix allocation unit tests * request from all blobbers but read until consensus * fix file meta inconsistency * revert consensus threshold * fix get file meta --------- Co-authored-by: Yury <yuderbasov@gmail.com> * Do not send readmarkers for free reads (#1101) * do not send readmarkers for free reads * remove unused download worker count * rmv streamreader * optimize repair * fix lint and test * rmv readCtx * fix upload check * add multi op fix (#1077) * add multi op fix * fix lock connection mask * fix op mask * create connObj * add rollback in multi op * fix ctx in streamReader * use multiop ctx in reader * cleanup * fix repair in multi op * fix repair cb --------- Co-authored-by: Yury <yuderbasov@gmail.com> * fix encrypt in multi op (#1097) Co-authored-by: Yury <yuderbasov@gmail.com> * take sharder consensous from wallet config (#1104) * events information, do a consensus among 3 sharders * use a const * init from config.yaml * fix comment * fix UTs * hotfix: fix `upload` and `download` in wasm (#1121) * Fix wasm download (#1117) * fix wasm download * fix syntax * fix reader * reverting reader changes --------- Co-authored-by: Dinmukhammed <52813950+din-mukhammed@users.noreply.github.com> Co-authored-by: din-mukhammed <dimash.r53@gmail.com> * add wasm download changes * fix merge * fix typo * fix repair cb * fix unit test * rmv log * check if env is wasm * cleanup * defer lock --------- Co-authored-by: boddumanohar <b.manu199@gmail.com> Co-authored-by: Piers Shepperson <piers.shepperson@btinternet.com> Co-authored-by: Yury <yuderbasov@gmail.com> Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> Co-authored-by: Dinmukhammed <52813950+din-mukhammed@users.noreply.github.com> Co-authored-by: Roshan Kumar <56060325+Roshan-Mehta@users.noreply.github.com> Co-authored-by: Lz <imlangzi@qq.com> Co-authored-by: shahnawaz-creator <117025384+shahnawaz-creator@users.noreply.github.com> Co-authored-by: Sunil Kumar <sunilkr.kumar26@gmail.com> Co-authored-by: Jayash Satolia <73050737+Jayashsatolia403@users.noreply.github.com> Co-authored-by: stewartie4 <ryanstewart456@gmail.com> Co-authored-by: din-mukhammed <dimash.r53@gmail.com>
cnlangzi
added a commit
that referenced
this pull request
Jul 28, 2023
* feat wasm: readPool unlock (#1076) * readPool unlock * feat (mobile): expose ReadPoolUnLock * show chunk size in listReq response * Revert "show chunk size in listReq response" This reverts commit c62a183. * Multi download operation (#1045) * multi download operation in mobilesdk * add description * set thresh to data shards * multiupload func added to wasm * fix wasm build * create localPath * fix typo * Update proxy.go * Add to response * fix append * increase download workers * decrease number of download workers * set workers back to 20 * use multiUpload demo for multi operation clean up logs * feat wasm: UpdateForbidAllocation * add demo for multiDownload * cleanup * rmv worker semaphore * add objURL in callback * fix build * add description * add packages * add filename to callback * ad localPath to statusBar * add blobber lock * add filename to start and progress * export winsdk functions * Revert "export winsdk functions" This reverts commit eed6cd0. * add multiDownload to winsdk * fix typo * Move min lock demand to allocation (#1046) * min_lock_demand * mobile * Feat/caching get fees estimate (#1069) * added caching instead of the network call * clean up * clean up * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * fixed comments --------- Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: boddumanohar <b.manu199@gmail.com> Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * Rename blobber field to not_available (#1043) * remove unstake total * not_available * added GRAPHNODE_SC * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * run build on all PRs * run on every PR --------- Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: boddumanohar <b.manu199@gmail.com> Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * wasm recover from panic (#1071) * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * recover from panic wasm --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * cleanup * set consensus to threshold for authTicket --------- Co-authored-by: Manohar Reddy <b.manu199@gmail.com> Co-authored-by: Piers Shepperson <piers.shepperson@btinternet.com> Co-authored-by: Yury <yuderbasov@gmail.com> Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * Refactor downloads to use sys.File (#1073) * added memfilechan * added memfilechan * clean up * refactor download methods to accept file handler * fix unit tests * refactor download methods, implement new downloader, fix unit tests * clean up * fix file callback --------- Co-authored-by: dabasov <yuderbasov@gmail.com> * Feature/expose multiop winsdk (#1034) * Expose multioperation in winsdk * Temporary remove callback args from multiupload * Export method * pass callback * test multiop * fix multiUpload * Revert "test multiop" This reverts commit 08ad565. * Multi download operation (#1045) * multi download operation in mobilesdk * add description * set thresh to data shards * multiupload func added to wasm * fix wasm build * create localPath * fix typo * Update proxy.go * Add to response * fix append * increase download workers * decrease number of download workers * set workers back to 20 * use multiUpload demo for multi operation clean up logs * feat wasm: UpdateForbidAllocation * add demo for multiDownload * cleanup * rmv worker semaphore * add objURL in callback * fix build * add description * add packages * add filename to callback * ad localPath to statusBar * add blobber lock * add filename to start and progress * export winsdk functions * Revert "export winsdk functions" This reverts commit eed6cd0. * add multiDownload to winsdk * fix typo * Move min lock demand to allocation (#1046) * min_lock_demand * mobile * Feat/caching get fees estimate (#1069) * added caching instead of the network call * clean up * clean up * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * fixed comments --------- Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: boddumanohar <b.manu199@gmail.com> Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * Rename blobber field to not_available (#1043) * remove unstake total * not_available * added GRAPHNODE_SC * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * run build on all PRs * run on every PR --------- Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: boddumanohar <b.manu199@gmail.com> Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * wasm recover from panic (#1071) * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * recover from panic wasm --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * cleanup * set consensus to threshold for authTicket --------- Co-authored-by: Manohar Reddy <b.manu199@gmail.com> Co-authored-by: Piers Shepperson <piers.shepperson@btinternet.com> Co-authored-by: Yury <yuderbasov@gmail.com> Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> --------- Co-authored-by: Lz <imlangzi@qq.com> Co-authored-by: Manohar Reddy <b.manu199@gmail.com> Co-authored-by: Hitenjain14 <57557631+Hitenjain14@users.noreply.github.com> Co-authored-by: Piers Shepperson <piers.shepperson@btinternet.com> Co-authored-by: Yury <yuderbasov@gmail.com> Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * Add path in thumbnail hash (#1013) * add path in thumbnail hash * lint fix * fix typo * fix input s3_migration_cli_branch system_tests.yml * updated workflow input to s3_migration_cli_branch in system_tests.yml * fix hash calculation * check prevWM * refactor * rmv prevWM check --------- Co-authored-by: shahnawaz-creator <117025384+shahnawaz-creator@users.noreply.github.com> Co-authored-by: Yury <yuderbasov@gmail.com> * Refactor repair (#1004) * add repair in multi op * rmv check status from chunk upload and delete * rmv isRepair flag from worker * rmv fileID from fileMetaHash * add mut for repair callback * repair changes * delete file after upload in repair * add delete mask * add with mask option * fix uint tests * fix multi createDir * add path in filemetahash --------- Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> Co-authored-by: Yury <yuderbasov@gmail.com> * Change the contract of Sharder and Miner update settings (#1085) Co-authored-by: Yury <yuderbasov@gmail.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * fix interface conversion error (#1088) * fix interface conversion * use sdk.multiUpload * Revert "use sdk.multiUpload" This reverts commit 428960e. * use multi upload * Add support for sending chunks (#1084) * add chunk feature * check fileshards * check chunkNum * check fileShards * add check in processUpload * check hash len --------- Co-authored-by: Manohar Reddy <b.manu199@gmail.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * fix panic (#1090) * Resume download (#1096) * resume download * clean up * cleanup preferred blobbers from chain config (#1099) * cleanup preferred blobbers from chain config * fix mobilesdk/CreateAllocation * fix lint * Fix exclude path (#1093) * Fix exclude path * Resolved comment * Display correct size values (#1033) * display correct size values * fix consensus threshold * fix unit tests * fix allocation unit tests * request from all blobbers but read until consensus * fix file meta inconsistency * revert consensus threshold * fix get file meta --------- Co-authored-by: Yury <yuderbasov@gmail.com> * Do not send readmarkers for free reads (#1101) * do not send readmarkers for free reads * remove unused download worker count * add multi op fix (#1077) * add multi op fix * fix lock connection mask * fix op mask * create connObj * add rollback in multi op * fix ctx in streamReader * use multiop ctx in reader * cleanup * fix repair in multi op * fix repair cb --------- Co-authored-by: Yury <yuderbasov@gmail.com> * fix encrypt in multi op (#1097) Co-authored-by: Yury <yuderbasov@gmail.com> * take sharder consensous from wallet config (#1104) * events information, do a consensus among 3 sharders * use a const * init from config.yaml * fix comment * fix UTs * Optimize repair (#1105) * rmv streamreader * optimize repair * fix lint and test * rmv readCtx * fix upload check * Fix * Fix * Fix update allocation * Fix create allocation * Fix (#1110) * Export balance get function (#1103) Co-authored-by: Yury <yuderbasov@gmail.com> * Add getUpdateAllocationMinLock to wasm (#1107) * Add getUpdateAllocationMinLock * Fix demon index.html * fixed lint --------- Co-authored-by: Yury <yuderbasov@gmail.com> * Updating blobber and validator update setting models. (#1095) * Updating blobber and validatory update setting models. * Updating the Terms struct as well. * Updating the StakePoolSettings struct as well. * pass only updted fields in validator --------- Co-authored-by: Yury <yuderbasov@gmail.com> * Accept absolute path when downloading (#1111) * accept abs path also * fix unit tests * check if localpath has ext * fix seek for resume * Updating the validator node which is being passed to 0chain in update settings call (#1114) * fixing json name for the validator id. * fixing the blockchain validator udpate settings to pass to 0chain * fixing the json name for validator * Fix * Revert "Optimize repair (#1105)" (#1116) This reverts commit d293d7d. --------- Co-authored-by: boddumanohar <b.manu199@gmail.com> Co-authored-by: Hitenjain14 <57557631+Hitenjain14@users.noreply.github.com> Co-authored-by: Piers Shepperson <piers.shepperson@btinternet.com> Co-authored-by: Yury <yuderbasov@gmail.com> Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> Co-authored-by: Dinmukhammed <52813950+din-mukhammed@users.noreply.github.com> Co-authored-by: Roshan Kumar <56060325+Roshan-Mehta@users.noreply.github.com> Co-authored-by: Lz <imlangzi@qq.com> Co-authored-by: shahnawaz-creator <117025384+shahnawaz-creator@users.noreply.github.com> Co-authored-by: Sunil Kumar <sunilkr.kumar26@gmail.com> Co-authored-by: stewartie4 <ryanstewart456@gmail.com> Co-authored-by: peterlimg <54137706+peterlimg@users.noreply.github.com>
cnlangzi
added a commit
that referenced
this pull request
Jul 28, 2023
* feat wasm: readPool unlock (#1076) * readPool unlock * feat (mobile): expose ReadPoolUnLock * show chunk size in listReq response * Revert "show chunk size in listReq response" This reverts commit c62a183. * Multi download operation (#1045) * multi download operation in mobilesdk * add description * set thresh to data shards * multiupload func added to wasm * fix wasm build * create localPath * fix typo * Update proxy.go * Add to response * fix append * increase download workers * decrease number of download workers * set workers back to 20 * use multiUpload demo for multi operation clean up logs * feat wasm: UpdateForbidAllocation * add demo for multiDownload * cleanup * rmv worker semaphore * add objURL in callback * fix build * add description * add packages * add filename to callback * ad localPath to statusBar * add blobber lock * add filename to start and progress * export winsdk functions * Revert "export winsdk functions" This reverts commit eed6cd0. * add multiDownload to winsdk * fix typo * Move min lock demand to allocation (#1046) * min_lock_demand * mobile * Feat/caching get fees estimate (#1069) * added caching instead of the network call * clean up * clean up * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * fixed comments --------- Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: boddumanohar <b.manu199@gmail.com> Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * Rename blobber field to not_available (#1043) * remove unstake total * not_available * added GRAPHNODE_SC * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * run build on all PRs * run on every PR --------- Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: boddumanohar <b.manu199@gmail.com> Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * wasm recover from panic (#1071) * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * recover from panic wasm --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * cleanup * set consensus to threshold for authTicket --------- Co-authored-by: Manohar Reddy <b.manu199@gmail.com> Co-authored-by: Piers Shepperson <piers.shepperson@btinternet.com> Co-authored-by: Yury <yuderbasov@gmail.com> Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * Refactor downloads to use sys.File (#1073) * added memfilechan * added memfilechan * clean up * refactor download methods to accept file handler * fix unit tests * refactor download methods, implement new downloader, fix unit tests * clean up * fix file callback --------- Co-authored-by: dabasov <yuderbasov@gmail.com> * Feature/expose multiop winsdk (#1034) * Expose multioperation in winsdk * Temporary remove callback args from multiupload * Export method * pass callback * test multiop * fix multiUpload * Revert "test multiop" This reverts commit 08ad565. * Multi download operation (#1045) * multi download operation in mobilesdk * add description * set thresh to data shards * multiupload func added to wasm * fix wasm build * create localPath * fix typo * Update proxy.go * Add to response * fix append * increase download workers * decrease number of download workers * set workers back to 20 * use multiUpload demo for multi operation clean up logs * feat wasm: UpdateForbidAllocation * add demo for multiDownload * cleanup * rmv worker semaphore * add objURL in callback * fix build * add description * add packages * add filename to callback * ad localPath to statusBar * add blobber lock * add filename to start and progress * export winsdk functions * Revert "export winsdk functions" This reverts commit eed6cd0. * add multiDownload to winsdk * fix typo * Move min lock demand to allocation (#1046) * min_lock_demand * mobile * Feat/caching get fees estimate (#1069) * added caching instead of the network call * clean up * clean up * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * fixed comments --------- Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: boddumanohar <b.manu199@gmail.com> Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * Rename blobber field to not_available (#1043) * remove unstake total * not_available * added GRAPHNODE_SC * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * run build on all PRs * run on every PR --------- Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: boddumanohar <b.manu199@gmail.com> Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * wasm recover from panic (#1071) * fixes for update image (#1058) * update wasm demo for download * preserver order in blobber of newallocationrequestg * fixes for update image * use multiUpload * demo for multi operation * clean up logs * init Consensus * feat wasm: UpdateForbidAllocation * expose lockReadPool * fix for unique blobbers * init Consensus in Lock function (#1063) * init Consensus * fix panic in dirworker --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * recover from panic wasm --------- Co-authored-by: Hitenjain14 <hitenjain146@gmail.com> * cleanup * set consensus to threshold for authTicket --------- Co-authored-by: Manohar Reddy <b.manu199@gmail.com> Co-authored-by: Piers Shepperson <piers.shepperson@btinternet.com> Co-authored-by: Yury <yuderbasov@gmail.com> Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> --------- Co-authored-by: Lz <imlangzi@qq.com> Co-authored-by: Manohar Reddy <b.manu199@gmail.com> Co-authored-by: Hitenjain14 <57557631+Hitenjain14@users.noreply.github.com> Co-authored-by: Piers Shepperson <piers.shepperson@btinternet.com> Co-authored-by: Yury <yuderbasov@gmail.com> Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * Add path in thumbnail hash (#1013) * add path in thumbnail hash * lint fix * fix typo * fix input s3_migration_cli_branch system_tests.yml * updated workflow input to s3_migration_cli_branch in system_tests.yml * fix hash calculation * check prevWM * refactor * rmv prevWM check --------- Co-authored-by: shahnawaz-creator <117025384+shahnawaz-creator@users.noreply.github.com> Co-authored-by: Yury <yuderbasov@gmail.com> * Refactor repair (#1004) * add repair in multi op * rmv check status from chunk upload and delete * rmv isRepair flag from worker * rmv fileID from fileMetaHash * add mut for repair callback * repair changes * delete file after upload in repair * add delete mask * add with mask option * fix uint tests * fix multi createDir * add path in filemetahash --------- Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> Co-authored-by: Yury <yuderbasov@gmail.com> * Change the contract of Sharder and Miner update settings (#1085) Co-authored-by: Yury <yuderbasov@gmail.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * fix interface conversion error (#1088) * fix interface conversion * use sdk.multiUpload * Revert "use sdk.multiUpload" This reverts commit 428960e. * use multi upload * Add support for sending chunks (#1084) * add chunk feature * check fileshards * check chunkNum * check fileShards * add check in processUpload * check hash len --------- Co-authored-by: Manohar Reddy <b.manu199@gmail.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> * fix panic (#1090) * Resume download (#1096) * resume download * clean up * cleanup preferred blobbers from chain config (#1099) * cleanup preferred blobbers from chain config * fix mobilesdk/CreateAllocation * fix lint * Fix exclude path (#1093) * Fix exclude path * Resolved comment * Display correct size values (#1033) * display correct size values * fix consensus threshold * fix unit tests * fix allocation unit tests * request from all blobbers but read until consensus * fix file meta inconsistency * revert consensus threshold * fix get file meta --------- Co-authored-by: Yury <yuderbasov@gmail.com> * Do not send readmarkers for free reads (#1101) * do not send readmarkers for free reads * remove unused download worker count * rmv streamreader * optimize repair * fix lint and test * rmv readCtx * fix upload check * add multi op fix (#1077) * add multi op fix * fix lock connection mask * fix op mask * create connObj * add rollback in multi op * fix ctx in streamReader * use multiop ctx in reader * cleanup * fix repair in multi op * fix repair cb --------- Co-authored-by: Yury <yuderbasov@gmail.com> * fix encrypt in multi op (#1097) Co-authored-by: Yury <yuderbasov@gmail.com> * take sharder consensous from wallet config (#1104) * events information, do a consensus among 3 sharders * use a const * init from config.yaml * fix comment * fix UTs * hotfix: fix `upload` and `download` in wasm (#1121) * Fix wasm download (#1117) * fix wasm download * fix syntax * fix reader * reverting reader changes --------- Co-authored-by: Dinmukhammed <52813950+din-mukhammed@users.noreply.github.com> Co-authored-by: din-mukhammed <dimash.r53@gmail.com> * add wasm download changes * fix merge * fix typo * fix repair cb * fix unit test * rmv log * check if env is wasm * cleanup * defer lock --------- Co-authored-by: boddumanohar <b.manu199@gmail.com> Co-authored-by: Piers Shepperson <piers.shepperson@btinternet.com> Co-authored-by: Yury <yuderbasov@gmail.com> Co-authored-by: sarvesh raut <sarveshraut@sarveshs-MacBook-Air.local> Co-authored-by: Manali-Jain-Squareops <manali@squareops.com> Co-authored-by: Kishan Dhakan <42718091+Kishan-Dhakan@users.noreply.github.com> Co-authored-by: Dinmukhammed <52813950+din-mukhammed@users.noreply.github.com> Co-authored-by: Roshan Kumar <56060325+Roshan-Mehta@users.noreply.github.com> Co-authored-by: Lz <imlangzi@qq.com> Co-authored-by: shahnawaz-creator <117025384+shahnawaz-creator@users.noreply.github.com> Co-authored-by: Sunil Kumar <sunilkr.kumar26@gmail.com> Co-authored-by: Jayash Satolia <73050737+Jayashsatolia403@users.noreply.github.com> Co-authored-by: stewartie4 <ryanstewart456@gmail.com> Co-authored-by: din-mukhammed <dimash.r53@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes
ReadPoolUnlockReadPoolUnlockfixes #1079
Tests
Tasks to complete before merging PR:
Associated PRs (Link as appropriate):