Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

go routines and channels #19

Closed
030 opened this issue Apr 29, 2019 · 5 comments · Fixed by #156
Closed

go routines and channels #19

030 opened this issue Apr 29, 2019 · 5 comments · Fixed by #156
Assignees
@030
Labels
enhancement New feature or request pinned

Comments

@030
Copy link
Owner

030 commented Apr 29, 2019 •
edited
Loading

Improve download speed by using go routines and channels.
@030 030 added the enhancement New feature or request label Apr 29, 2019
@030 030 changed the title speed up tests by applying go routines go routines May 14, 2019
@030 030 changed the title go routines go routines and channels May 14, 2019
030 added a commit that referenced this issue Oct 15, 2019
@030
[GH-19] count number of artifacts in a repository
50eb515
030 added a commit that referenced this issue Oct 17, 2019
@030
[GH-19] Increase code coverage upload
2085441
030 added a commit that referenced this issue Oct 17, 2019
@030
[GH-19] Ensure that stringBuilder works again
29d11f0
030 added a commit that referenced this issue Oct 18, 2019
@030
[GH-19] Improve debugging test-zip test in bash
8733be4
030 added a commit that referenced this issue Oct 18, 2019
@030
[GH-19] Backup test failed due to omitted gitkeep
dcfa913
030 added a commit that referenced this issue Oct 20, 2019
@030
[GH-19] Use go-multipart 1.0.1 that contains several bug fixes
45f97d9
@stale
Copy link

stale bot commented Mar 4, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the wontfix This will not be worked on label Mar 4, 2020
@030 030 added the pinned label Mar 6, 2020
@stale stale bot removed the wontfix This will not be worked on label Mar 6, 2020
@030 030 added pinned and removed pinned labels Jul 16, 2020
@030 030 self-assigned this Jul 27, 2020
@030
Copy link
Owner Author

030 commented Jul 27, 2020

https://blog.golang.org/pipelines

Parallel digestion

@030
Copy link
Owner Author

030 commented Jul 27, 2020

waitgroup vs. channel

030 added a commit that referenced this issue Jul 28, 2020
@030
[GH-19] Work in progress.
57476a0
@030
Copy link
Owner Author

030 commented Aug 8, 2020

Difference

without

INFO[0000] n3drHomeDir: '/home/ben/snap/n3dr/23/.n3dr'  
INFO[0000] Using config file: '/home/ben/snap/n3dr/23/.n3dr/config.yml' 
INFO[0000] n3drUser empty. Reading if from '/home/ben/snap/n3dr/23/.n3dr/config.yml' 
INFO[0000] n3drPass empty. Reading if from '/home/ben/snap/n3dr/23/.n3dr/config.yml' 
INFO[0000] n3drURL empty. Reading if from '/home/ben/snap/n3dr/23/.n3dr/config.yml' 
INFO[0000] Temp download dir name: '/tmp/n3dr/download413730042' 
INFO[0019] Assembling downloadURLs '3rdparty-maven'     
 138 / 138 [=========================================================================================================================================================================] 100.00% 18s
Done
INFO[0037] Backing up artifacts '3rdparty-maven'        
 1373 / 1373 [======================================================================================================================================================================] 100.00% 2m3s
Done
INFO[0161] Assembling downloadURLs '3rdparty-maven-gradle-plugins' 
 2 / 2 [==============================================================================================================================================================================] 100.00% 0s
Done
INFO[0161] Backing up artifacts '3rdparty-maven-gradle-plugins' 
 14 / 14 [============================================================================================================================================================================] 100.00% 1s
Done
INFO[0165] Assembling downloadURLs 'releases'           
 21 / 21 [============================================================================================================================================================================] 100.00% 2s
Done
INFO[0168] Backing up artifacts 'releases'              
 203 / 203 [=========================================================================================================================================================================] 100.00% 12s
Done
INFO[0199] Assembling downloadURLs 'snapshots'          
 134 / 134 [=========================================================================================================================================================================] 100.00% 19s
Done
INFO[0218] Backing up artifacts 'snapshots'             
 1333 / 1333 [=====================================================================================================================================================================] 100.00% 1m22s
Done
WARN[0301] Trying to create a zip file in: '/home/ben/dev/github.com/030/n3dr'. Note that this could result in a 'permission denied' issue if N3DR has been installed using snap and is run in a different directory than your own home folder. 
INFO[0301] Zipfile: 'n3dr-backup-08-08-2020T21-28-09.zip' created in '/home/ben/dev/github.com/030/n3dr' 

real    5m1,589s
user    0m31,534s
sys     0m19,819s

channels

INFO[0000] n3drHomeDir: '/home/ben/.n3dr'               
INFO[0000] Using config file: '/home/ben/.n3dr/config.yml' 
INFO[0000] n3drUser empty. Reading if from '/home/ben/.n3dr/config.yml' 
INFO[0000] n3drPass empty. Reading if from '/home/ben/.n3dr/config.yml' 
INFO[0000] n3drURL empty. Reading if from '/home/ben/.n3dr/config.yml' 
INFO[0000] Temp download dir name: '/tmp/n3dr/download597737068' 
Backup:  snapshots
Backup:  3rdparty-maven-gradle-plugins
Backup:  3rdparty-maven
Backup:  releases
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++WARN[0071] Trying to create a zip file in: '/home/ben/dev/github.com/030/n3dr'. Note that this could result in a 'permission denied' issue if N3DR has been installed using snap and is run in a different directory than your own home folder. 
INFO[0071] Zipfile: 'n3dr-backup-08-08-2020T21-22-24.zip' created in '/home/ben/dev/github.com/030/n3dr' 

real    1m13,037s
user    0m24,112s
sys     0m10,107s

~5 vs. 1min

030 added a commit that referenced this issue Aug 8, 2020
@030
[GH-19] Merge conflict in repositories cli resolved.
99cdb16
030 added a commit that referenced this issue Aug 8, 2020
@030
[GH-19] Reuse integration bash gist from n3cac.
8f743a8
030 added a commit that referenced this issue Aug 8, 2020
@030
[GH-19] Removed superfluous docker stop as this occurs in n3cac gist.
c0e0b6d
030 added a commit that referenced this issue Aug 9, 2020
@030
[GH-19] Resolved broken tests due to introduction of channels.
e2c894b
030 added a commit that referenced this issue Aug 9, 2020
@030
[GH-19] Removed unused code.
acec6d9
030 added a commit that referenced this issue Aug 9, 2020
@030
[GH-19] Increased code coverage oci file.
03c0b91
030 added a commit that referenced this issue Aug 9, 2020
@030
Merge pull request #156 from 030/19-channels
3bf77e5 
[GH-19] Work in progress.
@030 030 linked a pull request Aug 9, 2020 that will close this issue
[GH-19] Channels #156
Merged
@030 030 closed this as completed Aug 9, 2020
@030
Copy link
Owner Author

030 commented Aug 9, 2020

5.2.0

030 added a commit that referenced this issue Dec 12, 2020
@030
[GH-19] Merge conflict in repositories cli resolved.
4513bf7
030 added a commit that referenced this issue Dec 12, 2020
@030
[GH-19] Reuse integration bash gist from n3cac.
33457e7
030 added a commit that referenced this issue Dec 12, 2020
@030
[GH-19] Removed superfluous docker stop as this occurs in n3cac gist.
e67dce7
030 added a commit that referenced this issue Dec 12, 2020
@030
[GH-19] Resolved broken tests due to introduction of channels.
0d38645
030 added a commit that referenced this issue Dec 12, 2020
@030
[GH-19] Removed unused code.
982fdf1
030 added a commit that referenced this issue Dec 12, 2020
@030
[GH-19] Increased code coverage oci file.
ab956c5
030 added a commit that referenced this issue Dec 12, 2020
@030
Merge pull request #156 from 030/19-channels
13abcc1 
[GH-19] Work in progress.
030 added a commit that referenced this issue Oct 1, 2022
@030
[GH-19] Merge conflict in repositories cli resolved.
92ea16d
030 added a commit that referenced this issue Oct 1, 2022
@030
[GH-19] Reuse integration bash gist from n3cac.
917b1bd
030 added a commit that referenced this issue Oct 1, 2022
@030
[GH-19] Removed superfluous docker stop as this occurs in n3cac gist.
de5dff5
030 added a commit that referenced this issue Oct 1, 2022
@030
[GH-19] Resolved broken tests due to introduction of channels.
ab56d5e
030 added a commit that referenced this issue Oct 1, 2022
@030
[GH-19] Removed unused code.
f17274f
030 added a commit that referenced this issue Oct 1, 2022
@030
[GH-19] Increased code coverage oci file.
a9a6cbb
030 added a commit that referenced this issue Oct 1, 2022
@030
Merge pull request #156 from 030/19-channels
566d52a 
[GH-19] Work in progress.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@030 030

Labels
enhancement New feature or request pinned
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[GH-19] Channels 030/n3dr
1 participant
@030