-
Notifications
You must be signed in to change notification settings - Fork 14
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Code ingress with git #611
Closed
Closed
Changes from 2 commits
Commits
Show all changes
164 commits
Select commit
Hold shift + click to select a range
fc6e91f
(broken) Set up two GitLab VMs, "internal" and "external", for code i…
ots22 3c0aecd
Add a basic cloud init template for gitlab external
jack89roberts 6056af1
Remove debugging print statement and unused variable
jack89roberts 8b8d20c
add network security group rule (InboundDenyAll) for nsgGitlabExternal
nbarlowATI 2ab0fce
Add a user with an API key to gitlab internal
jack89roberts 424bbd3
Add a new session server for holding connections to review boxes
jemrobinson 3f04679
Add SSH key and git setup on gitlab external
jack89roberts ff48810
WIP Script to update GitLab projects from whitelist
ots22 8507b63
Updated review session server settings
jemrobinson 7c8d377
Updated RDS deployment configuration
jemrobinson 9a52480
Add gitlab server to gitlab external
jack89roberts b9d5788
Updated SRE teardown script
jemrobinson f8c415a
Removed cw20 configs as we are using testa now
jemrobinson ccdc93f
WIP script for monitoring and accepting merge requests on gitlab exte…
jack89roberts fd62e1f
Merge requests script: Replace print statements with logging
jack89roberts f139b2f
Remove localhost token accidentally added in previous commit
jack89roberts 69df12d
Changed name of test researcher
jemrobinson 49df919
Updated RDS scripts
jemrobinson 23a2733
wait for gitlab server health check before interacting with api
jack89roberts 59d3073
replace bash syntax
jack89roberts ab9de4c
Script to create zipfile from specified commit on a git repo, then up…
nbarlowATI 0881963
couple more shell formatting fixes
jack89roberts b5b2337
Add function to create or push to a repo on GITLAB-INTERNAL
ots22 53485ba
Remove old update from whitelist script
ots22 22f8c4b
add a line break to tidy up logging
jack89roberts b511029
add quotes around url in curl command
nbarlowATI bddd44d
add python script for creating projects and merge request in gitlab-e…
nbarlowATI e3b99ac
don't write curl output to file
nbarlowATI d995c47
add container name for blob storage for zipfiles from git repos
nbarlowATI c831b0e
Move python script content out of cloud-init yaml file into scripts/ …
nbarlowATI 46b778d
substitute scripts into cloud init
nbarlowATI 56cd9ca
temporarily allow 500 status code when creating merge request
nbarlowATI bbb965b
check_merge_requests docstrings, comments and formatting
jack89roberts a3c0323
Add crontab entries for GitLab scripts
ots22 f7dfccc
Switched gitlab airlock container name back to something simple as th…
jemrobinson 3550638
Fixed typo
jemrobinson e4d16e2
Added explicit ConnectionBroker argument as first argument to all RD …
jemrobinson bfe7147
Remove unused Set-RDPublishedName script
jemrobinson f5eb496
Remove hard-coded drive letters
jemrobinson 2e45544
refactor script to upload git project to gitlab-external (git clone b…
nbarlowATI f9b685a
minor fixes after testing locally
nbarlowATI c6c2ca7
Add missing quotation mark
jack89roberts 9efc447
Use the 'artifacts' resource group for the blob storage when uploadin…
nbarlowATI a15a703
protect against non-existing zipfile directory
nbarlowATI 3bc913c
test using webapps resource group for blob upload
nbarlowATI 2906264
revert to using artifacts resource group for gitlab zip upload
nbarlowATI 6182e04
add path to create_gitlab_project to force correct case being preserved
jack89roberts bdd6f65
create branch on approval project after fork to unapproved
nbarlowATI 59761df
Merge branch '264-gitlab-ingress' of https://github.com/alan-turing-i…
nbarlowATI d76ab25
fix path for downloading git repo zipfiles on gitlab-external
nbarlowATI 0fd1356
change ownership of zipfile dir on gitlab-external
nbarlowATI e4a3304
check if project already exists before creating on gitlab-external
nbarlowATI c639e47
simplifications to how disk is mounted, and fix to gitlab datadisk path
nbarlowATI 5bd6752
Force path to match case of name in internal_update_repo
jack89roberts d70ed67
import Security.psm1 to use Resolve-KeyvaultSecret function
nbarlowATI fdef7f2
Robustify zipfile_to_gitlab_project and check_merge_requests to cope …
jack89roberts aebd59a
ssh-keyscan localhost instead of external IP address
jack89roberts 8525dbd
Get gitlab external ssh keys from /etc/ssh/
jack89roberts 9af6278
Change commit message when importing snapshot of requested repo
ots22 8c8b8c5
Use same branch name on source (unapproved) and target (approval) rep…
ots22 f4c0a9e
Change public groups and projects to internal
jack89roberts 162e611
Change default branch and commit a README file to 'approval'
ots22 dfd1dc9
Move docstrings into a separate file, to fit within the character lim…
ots22 837b2da
Get gitlab internal ssh keys by invoking remote script
jack89roberts a4ca583
Remove commented code
ots22 16573fc
Remove unnecessary logging
ots22 7e9ab18
Factor return out of if/else
ots22 1fc2bac
Correct name for the review session host
ots22 74fd45b
No template substitution in cloud-init runcmd for GitLab API tokens
ots22 5c1026b
Merge master into 264-gitlab-ingress
ots22 aa05a61
Factor out deploying empty blob storage container
ots22 3efe4bc
Log file to match script name
ots22 1c927b8
start refactoring gitlab secrets files
jack89roberts 6738f75
Merge branch '264-gitlab-ingress' of https://github.com/alan-turing-i…
jack89roberts c8619c7
Refactor gitlab credentials files and functions
jack89roberts 1b703e5
black python scripts
jack89roberts 7275923
add missing file path argument
jack89roberts 77884e8
fix not updated secrets file location
jack89roberts f6ec3fa
Changes to creating merge request from unzipped repo
ots22 9416fba
Update docstrings (after some functions were renamed)
ots22 db81a4d
Rename Gitlab and Hack MD servers
jack89roberts ebbe624
Merge branch '264-gitlab-ingress' of https://github.com/alan-turing-i…
jack89roberts 36e464d
Add SRE User documentation for the code ingress process
ots22 9406ce2
Merge branch '264-gitlab-ingress' of https://github.com/alan-turing-i…
jack89roberts 1b5248b
Fix overwritten vmName variable for gitlab server names
jack89roberts 3f68aae
Update docs/safe_haven_user_guide.md
ots22 b6cafdd
Update docs/safe_haven_user_guide.md
ots22 6619fc7
Documentation: overall description of the code ingress process
ots22 4d91a42
Remove duplicate git clone
ots22 2f25476
Name 'reviewUsersGroup' consistently with the other group name variables
ots22 cead499
Add (and use) function for clearing a storage container
ots22 a878316
Rename some variables in GitLab upload powershell script
ots22 63e8401
Remove all default IP addresses in the ARM template for RDS hosts/gat…
ots22 18f37db
Delete misleading comment
ots22 6b67674
Exit with status code 1 if any error encountered in check_merge_requests
jack89roberts 60a7ab6
Remove work in progress check
jack89roberts 4e733d5
Add effect of thumbs down to readme docs of approval conditions
jack89roberts 1b5f85f
Give ingress users dummy email addresses rather than using the SHM fqdn
jack89roberts 4a01cd4
Add default value for targetRepoName based on sourceGitURL
ots22 de0c669
Use a temporary container with a unique name to store the repo zipfil…
ots22 fd72517
Merge branch 'master' into 264-gitlab-ingress
jemrobinson c2b01c0
Updated NSG rules. Reordered webapp server deployment. Use a stronger…
jemrobinson ef3ac43
Added gitlab and hackmd daemon users
jemrobinson cf1c8e6
Updated disk provisioning to native cloud-init syntax
jemrobinson ad02e44
Switch to using gitlabdaemon for GitLab review server automation
jemrobinson ae42dc7
Updated to newer version of HackMD. Ensure that service will restart …
jemrobinson e6716ed
Add a rule to allow inbound SSH from VPN admin subnet
jemrobinson a2bc238
Updated NSG rules
jemrobinson 0f512c7
WIP: refactor gitlab ingress py and ps1 scripts
ots22 edbcc1f
Combine review steps scripts in crontab; enforce a single concurrent run
ots22 cde38a9
Move utilities to gitlab_config.py
ots22 5811409
WIP: continue refactor of gitlab ingress scripts
ots22 4514a98
Remove stray pylint headers
ots22 274c93b
Handle 'successful' 500 and 201 returns in a similar way in merge req…
ots22 4416003
Adjust crontab entry for gitlab script
ots22 c147ba7
Lint
ots22 be37161
Add some docstrings back; remove the 'docstrings only' file
ots22 2dab934
Revert removal of gitlab_config.py command-line functionality
ots22 c82dfab
Lint
ots22 90064ac
Lint
ots22 c77453d
Flake8: ignore W503 (line break before binary operator)
ots22 5c858fb
Fix to flake8 config file
ots22 435b33a
Fix NSG rules
ots22 b96fc4e
set subnet to airlock (setup webapp servers)
ots22 9fa64be
Fix to crontab entry (gitlab review scripts)
ots22 4c3dbca
Store both subnets (WebApp servers)
ots22 7fc3f86
Log message before slow key retrieval step
ots22 17ce3b6
Test C region now centralus (was uksouth)
ots22 44b74ee
Test C region centralus (was uksouth) - full config
ots22 7acd3e5
Full path to scripts in crontab
ots22 14a0b9d
Merge branch 'master' into 264-gitlab-ingress
jemrobinson 66964e0
Working ARM template for NSG webapps
jemrobinson 978db16
Added ARM template rules for airlock NSG
jemrobinson 3be6504
Simplified webapp servers deployment flow
jemrobinson a845ba7
Reflect Azure portal change in deployment (Point-to-Site -> User VPN)
ots22 71774f8
Allow (not deny) outbound access to GitLab Internal from Gitlab-Review
ots22 32a4de5
Clarify doc comment
ots22 ccab24d
Style changes for SRE_Upload_Git_Repo_to_GitlabReview
jemrobinson f8ee363
Fix removing storage container: needs context; don't prompt user
ots22 67baafa
Add missing gitlab-rb-host template substitution (fixes inability to …
ots22 cae41f3
containerName -> tmpContainerName
ots22 7b844c3
Fix cleaning up resources
ots22 815f2f0
Fix flag to Remove-AzStorageContainer
ots22 e832a0b
Merge branch 'master' into 264-gitlab-ingress
jemrobinson 86bba46
Merge branch 'master' into 264-gitlab-ingress
jemrobinson 0018a69
Minor fix to blob storage cleaner
jemrobinson 243980d
RDS fixes from redeploying. Includes a fix to make Deploy_RDS_Environ…
jemrobinson 59b5d08
Removed wait for cloud-init as this is included in the deployment scr…
jemrobinson 6d36d72
Updates from webapps redeploy
jemrobinson a80cb82
Pass subprocess.run args as a list
ots22 48d2be3
Use correct LDAP OU for research users on webapp VMs
ots22 2f505d2
Make the GitLab service user the owner of the zipfiles
ots22 8041233
Fix: string interpolation
ots22 a77afc9
Recover from partially-completed run: ensure cloned repo doesn't alre…
ots22 b7ca0da
Fix: argument order
ots22 8d477fc
Lint
ots22 6173b11
Fix typo: gitlabreview -> gitlabReview
ots22 bb6ca85
Check for correct (created) status code
ots22 cc4977b
Inline clone, commit and push
ots22 2ab4d54
gitlabUsername -> gitlabUserIngressUsername for cloud-init substitution
ots22 678be5b
Partial fix to NSG rules
ots22 6e7d2a1
Rename NSG rules template; adjust rules
ots22 3462ba6
Return PSNetworkSecurityGroup object from Add-NetworkSecurityGroupRule
ots22 53f8e63
Add back rule permitting inbound RDP connections to the webapp NSG
ots22 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Didn't we say we were going to put the refactor of this and related functions into it's own PR?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, sorry, this commit probably shouldn't have ended up on this branch - I'll make a separate PR for this.