Improve handling of Clone Repositories into temporary folders

[plinioh]

Describe the solution you'd like

Currently, each run of git-xargs clones the desired repository into the system's temporary folder. This is true for each time you target a repository running git-xargs.

Here is an example from my machine after experiencing with git-xargs for a couple of days (MacOS 12.5.1):

Command output

➜  ~ cd $TMPDIR
Time: 0h:00m:00s
➜ T find . -type d -name 'git-xargs-*' -exec du -h -d 0 {} + | sort -hr
 80M	./git-xargs-dgx-common-log-streaming678032190
 80M	./git-xargs-dgx-common-log-streaming597087740
 80M	./git-xargs-dgx-common-log-streaming3198864778
 80M	./git-xargs-dgx-common-log-streaming2148875932
 79M	./git-xargs-dgx-common-log-streaming3263876230
 12M	./git-xargs-dgx-myaccount-identity-service1007138129
 11M	./git-xargs-dgx-myaccount-identity-service3008335785
 11M	./git-xargs-dgx-myaccount-identity-service2989643188
 11M	./git-xargs-dgx-myaccount-identity-service2504143130
 11M	./git-xargs-dgx-myaccount-identity-service2399128272
 11M	./git-xargs-dgx-model-search-etl3961058151
 11M	./git-xargs-dgx-model-search-etl3462385865
 11M	./git-xargs-dgx-model-search-etl1808882054
 11M	./git-xargs-dgx-model-search-etl1631063994
 10M	./git-xargs-dgx-model-search-etl4110192775
6.4M	./git-xargs-dgx-myaccount-dashboard-service3331515564
6.4M	./git-xargs-dgx-myaccount-dashboard-service2927083683
6.4M	./git-xargs-dgx-monthlycare-service3226756721
6.2M	./git-xargs-dgx-monthlycare-service1085377643
6.0M	./git-xargs-dgx-pr-service2396586343
6.0M	./git-xargs-dgx-myaccount-dashboard-service522882412
5.7M	./git-xargs-dgx-monthlycare-service3608353224
5.7M	./git-xargs-dgx-monthlycare-service1697101609
5.7M	./git-xargs-dgx-monthlycare-service136264569
5.5M	./git-xargs-dgx-myaccount-dashboard-service762928531
5.5M	./git-xargs-dgx-myaccount-dashboard-service4145730298
5.5M	./git-xargs-dgx-myaccount-dashboard-service3958640074
5.5M	./git-xargs-dgx-myaccount-dashboard-service3647061136
5.0M	./git-xargs-dgx-pr-service538758956
5.0M	./git-xargs-dgx-pr-service461586568
5.0M	./git-xargs-dgx-pr-service3223152605
5.0M	./git-xargs-dgx-pr-service2447754502
4.8M	./git-xargs-dgx-common-services1332210037
4.6M	./git-xargs-dgx-claims-service1042501635
4.3M	./git-xargs-dgx-devops-serverless3712677518
4.3M	./git-xargs-dgx-devops-serverless1130242960
4.3M	./git-xargs-dgx-common-esb-service76139217
4.3M	./git-xargs-dgx-common-esb-service420185571
4.3M	./git-xargs-dgx-common-esb-service2481447801
4.3M	./git-xargs-dgx-common-esb-service2163038516
4.3M	./git-xargs-dgx-common-esb-service1819556725
4.2M	./git-xargs-dgx-devops-serverless492669529
4.2M	./git-xargs-dgx-devops-serverless4002798199
4.2M	./git-xargs-dgx-devops-serverless128927576
4.2M	./git-xargs-dgx-common-services491456596
4.2M	./git-xargs-dgx-common-services3582270430
4.2M	./git-xargs-dgx-common-services2000672389
4.2M	./git-xargs-dgx-common-services1293356187
4.1M	./git-xargs-dgx-claims-service51095670
4.1M	./git-xargs-dgx-claims-service1317238455
4.0M	./git-xargs-dgx-claims-service598378118
4.0M	./git-xargs-dgx-claims-service3887577741
3.6M	./git-xargs-dgx-common-basket-service2783467346
3.4M	./git-xargs-dgx-common-basket-service2102640962
3.2M	./git-xargs-dgx-randc-calendar-service608672353
3.2M	./git-xargs-dgx-common-person-service1016254311
2.8M	./git-xargs-dgx-common-basket-service649762349
2.6M	./git-xargs-dgx-randc-calendar-service1262282764
2.6M	./git-xargs-dgx-common-basket-service497612144
2.6M	./git-xargs-dgx-common-basket-service3018723509
2.5M	./git-xargs-dgx-randc-repair-record-service722209285
2.5M	./git-xargs-dgx-randc-repair-record-service52961552
2.5M	./git-xargs-dgx-randc-repair-record-service2450792822
2.5M	./git-xargs-dgx-randc-repair-record-service2170856959
2.5M	./git-xargs-dgx-randc-repair-record-service1677535228
2.5M	./git-xargs-dgx-randc-calendar-service4082317522
2.5M	./git-xargs-dgx-randc-calendar-service3869472038
2.5M	./git-xargs-dgx-randc-calendar-service2469708160
2.4M	./git-xargs-dgx-common-person-service585502068
2.4M	./git-xargs-dgx-common-person-service1949464635
2.3M	./git-xargs-dgx-common-person-service4179273804
2.3M	./git-xargs-dgx-common-person-service1961217495
2.2M	./git-xargs-dgx-randc-fault-array-service764424756
2.2M	./git-xargs-dgx-randc-fault-array-service3359491842
2.2M	./git-xargs-dgx-randc-fault-array-service335896226
2.2M	./git-xargs-dgx-randc-fault-array-service3317283419
2.2M	./git-xargs-dgx-randc-fault-array-service1527921735
2.0M	./git-xargs-dgx-claims-auth-service883203696
2.0M	./git-xargs-dgx-claims-auth-service4200032768
2.0M	./git-xargs-dgx-claims-auth-service4125151531
2.0M	./git-xargs-dgx-claims-auth-service270423822
2.0M	./git-xargs-dgx-claims-auth-service1939072047
1.9M	./git-xargs-dgx-template-serverless1388691735
1.9M	./git-xargs-dgx-template-serverless1340661728
1.8M	./git-xargs-dgx-template-serverless4110527560
1.8M	./git-xargs-dgx-template-serverless3071265855
1.8M	./git-xargs-dgx-template-serverless1845506612
1.8M	./git-xargs-dgx-skyprotect-plan-service4007101597
1.8M	./git-xargs-dgx-skyprotect-plan-service3697141120
1.8M	./git-xargs-dgx-skyprotect-plan-service3591216090
1.8M	./git-xargs-dgx-skyprotect-plan-service3342834588
1.8M	./git-xargs-dgx-skyprotect-plan-service1474255810
1.8M	./git-xargs-dgx-register-service1516419224
1.7M	./git-xargs-dgx-register-service913907664
1.7M	./git-xargs-dgx-register-service479951696
1.7M	./git-xargs-dgx-register-service2708612804
1.7M	./git-xargs-dgx-register-service2130368497
1.7M	./git-xargs-dgx-randc-model-search-service3287565133
1.7M	./git-xargs-dgx-randc-model-search-service3234429959
1.7M	./git-xargs-dgx-randc-model-search-service2381107305
1.7M	./git-xargs-dgx-randc-model-search-service1821807766
1.7M	./git-xargs-dgx-randc-model-search-service1383130522
1.7M	./git-xargs-dgx-common-test-event-service892877202
1.7M	./git-xargs-dgx-common-test-event-service674811894
1.7M	./git-xargs-dgx-common-test-event-service3637319092
1.7M	./git-xargs-dgx-common-test-event-service3577161244
1.7M	./git-xargs-dgx-common-test-event-service1721180667
1.7M	./git-xargs-dgx-common-catalogue-service983939112
1.7M	./git-xargs-dgx-common-catalogue-service2772470287
1.7M	./git-xargs-dgx-common-catalogue-service2509154114
1.7M	./git-xargs-dgx-common-catalogue-service2099321381
1.7M	./git-xargs-dgx-common-catalogue-service1891326121
1.7M	./git-xargs-dgx-common-auth-service844095786
1.7M	./git-xargs-dgx-common-auth-service834682523
1.7M	./git-xargs-dgx-common-auth-service4055385734
1.7M	./git-xargs-dgx-common-auth-service1811526230
1.7M	./git-xargs-dgx-common-auth-service1161090524
1.6M	./git-xargs-dgx-myaccount-auto-account-etl898797957
1.6M	./git-xargs-dgx-myaccount-auto-account-etl415195308
1.6M	./git-xargs-dgx-myaccount-auto-account-etl3964327510
1.6M	./git-xargs-dgx-myaccount-auto-account-etl2747648088
1.6M	./git-xargs-dgx-myaccount-auto-account-etl1831452060
1.5M	./git-xargs-dgx-model-search-service662120063
1.5M	./git-xargs-dgx-model-search-service3681460599
1.5M	./git-xargs-dgx-model-search-service3580138691
1.5M	./git-xargs-dgx-model-search-service2402385925
1.5M	./git-xargs-dgx-model-search-service1796394386
1.3M	./git-xargs-dgx-common-payment-service4043116693
1.3M	./git-xargs-dgx-common-payment-service3805125252
1.3M	./git-xargs-dgx-common-payment-service315251800
1.3M	./git-xargs-dgx-common-payment-service3049033308
1.3M	./git-xargs-dgx-common-payment-service2832457602
1.2M	./git-xargs-dgx-common-log-alert-service961588218
1.2M	./git-xargs-dgx-common-log-alert-service490772998
1.2M	./git-xargs-dgx-common-log-alert-service339914197
1.2M	./git-xargs-dgx-common-log-alert-service238092585
1.2M	./git-xargs-dgx-common-log-alert-service2337964869
1.2M	./git-xargs-dgx-common-imei-service953757373
1.2M	./git-xargs-dgx-common-imei-service537027263
1.2M	./git-xargs-dgx-common-imei-service2268986149
1.2M	./git-xargs-dgx-common-imei-service1339475111
1.2M	./git-xargs-dgx-common-imei-service1279518698
1.2M	./git-xargs-dgx-common-esb-mock-service3924556738
1.2M	./git-xargs-dgx-common-esb-mock-service367906134
1.2M	./git-xargs-dgx-common-esb-mock-service297756220
1.2M	./git-xargs-dgx-common-esb-mock-service259114694
1.2M	./git-xargs-dgx-common-esb-mock-service175502976
1.1M	./git-xargs-dgx-monthlycare-mocks3789520140
1.1M	./git-xargs-dgx-monthlycare-mocks3313344980
1.1M	./git-xargs-dgx-monthlycare-mocks2236466073
1.1M	./git-xargs-dgx-monthlycare-mocks1441079628
1.1M	./git-xargs-dgx-monthlycare-mocks1243466874
1.0M	./git-xargs-dgx-template-nx-serverless740514672
1.0M	./git-xargs-dgx-template-nx-serverless3962533842
1.0M	./git-xargs-dgx-template-nx-serverless308818190
1.0M	./git-xargs-dgx-template-nx-serverless2986665723
1.0M	./git-xargs-dgx-template-nx-serverless122325124
980K	./git-xargs-dgx-devops-pipeline-slack2687683694
980K	./git-xargs-dgx-devops-pipeline-slack2570664404
968K	./git-xargs-dgx-devops-pipeline-slack2529337648
968K	./git-xargs-dgx-devops-pipeline-slack1113530402
968K	./git-xargs-dgx-devops-pipeline-slack1065848740
964K	./git-xargs-dgx-common-log-queries3822392521
964K	./git-xargs-dgx-common-log-queries3755063549
964K	./git-xargs-dgx-common-log-queries3119748841
964K	./git-xargs-dgx-common-log-queries2477311213
964K	./git-xargs-dgx-common-log-queries2448373359
856K	./git-xargs-dgx-template-glue833851757
856K	./git-xargs-dgx-template-glue4091880147
856K	./git-xargs-dgx-template-glue3470239330
856K	./git-xargs-dgx-template-glue3382504572
856K	./git-xargs-dgx-template-glue2509518460
800K	./git-xargs-dgx-devops-deploy-workflow929715025
800K	./git-xargs-dgx-devops-deploy-workflow3833753504
800K	./git-xargs-dgx-devops-deploy-workflow2820543249
800K	./git-xargs-dgx-devops-deploy-workflow2394489380
800K	./git-xargs-dgx-devops-deploy-workflow1810561314
440K	./git-xargs-dgx-devops-cfn-certificate-provider753957369
440K	./git-xargs-dgx-devops-cfn-certificate-provider3410003968
440K	./git-xargs-dgx-devops-cfn-certificate-provider3186139486
440K	./git-xargs-dgx-devops-cfn-certificate-provider171854214
440K	./git-xargs-dgx-devops-cfn-certificate-provider1533139722

As one can check, there are multiple instances of the same repository, which can lead to extreme sizes after using git-xargs for a long time.

Describe alternatives you've considered

I see a couple of alternatives to resolve this issue:

1. Clean up the repos after each execution from git-xargs. Add a optional flag --keep-cloned-repositories to keep the repo for debugging purposes.

2. Make the CLI check if the repositories that need cloning are not already present in the filesystem. If they are, perform a git pull instead of cloning the repository again. This one will probably need changing quite a lot in the code, but can lead to improvements on performance, specially when dealing with large repositories.

Additional context

N/A

[zackproser]

Great bug report, thank you!

I think both of your suggestions are reasonable.

[zackproser]

@plinioh , I put up a pull request (linked above) to address this - if you have a free moment please give it a shot!

Thanks

[pliniodng]

Looks good, looking forward to the next release (:

Thanks @zackproser