Implementation of a controlled access system using Bastion Host and Vault

Increasing automation in IT processes and the evolution of software lifecycle processes from a DevOps perspective have meant that there is less and less need to perform direct access to IT systems: typically, access is for the purpose of performing critical operations and extraordinary maintenance.
Access to such systems therefore should be as limited as possible and subject to strict monitoring of the accesses performed.
To this end, it is useful to define a single point of access on which to focus security audits: such a system is typically referred to as a bastion host.
Another critical point in system access is the protection of access keys: these in the enterprise environment are managed in a wide variety of ways, sometimes without any use of protective measures, others with increasingly sophisticated systems such as vaults or HSMs.

HashiCorp Vault in a few words

Manage Secrets & Protect Sensitive Data Secure, store and tightly control access to tokens, passwords, certificates, encryption keys for protecting secrets and other sensitive data using a UI, CLI, or HTTP API.

Thesis puropose

The purpose of this thesis project is to implement a system based on bastion hosts, which through the implementation of an authorization workflow with Vault, allows granting or denying access to remote systems through the automatic use of keys retrieved from a vault by a bastion host.

Development

In this section we will describe the development of the project, starting from the definition of the architecture, the technologies used and the implementation of the Vault plugin.

Architecture

The architecture of the system is shown in the following sequence diagram:

sequenceDiagram
    actor User
    actor Operator
    participant Bastion Host
    participant Vault
    participant API
    participant Target Host
    note over Operator: Vault setup: enable and write plugin policies using root token
    Operator->>Vault: Vault setup
    User->>Bastion Host: User credentials over Bastion Host via Sshwifty
    Bastion Host->>Vault: Bastion Host authentication
    Vault->>API: Bastion Host Credentials
    Note over API: JWT creation to call the other API
    API->>Vault: JWT 
    note over Vault: Store JWT
    note over Vault: Vault Token creation with the plug-in policies
    Vault->>Bastion Host: Bastion Host Vault Token, JWT
    note over Bastion Host: Forwards the user credentials to the Vault with JWT
    Bastion Host->>Vault: User Credentials, JWT
    Vault->>API: User Credentials, JWT
    note over API: JWT checks
    note over API: Check(Username, Password)
    API->>Vault: Valid
    note over Vault: Detaches a valid token for the User
    Vault->>Bastion Host: User Vault Token
    note over Bastion Host: Forwards Bastion Host Vault token to check if the user is authorized to access the Target Host
    Bastion Host->> Vault: Username, Remote Ip, Bastion Host Vault Token
    note over Vault: Check Bastion Host Vault Token
    note over Vault: Forwards JWT
    Vault ->> API: Username, Remote Ip, JWT
    note over API: JWT checks
    note over API: Check(Username, Remote Ip)
    API->>Vault: Authorized
    Vault->>Bastion Host: Authorized
    note over Bastion Host: Uses the user Vault token to request the OTP
    Bastion Host->>Vault: Request OTP, User Vault Token
    note over Vault: User Vault Token checks
    Vault->>Bastion Host: OTP
    Bastion Host->>Target Host: Bastion host connects user to the Target Host using the OTP via Sshwifty
    User->>Target Host: 'echo "Hello World!"'
    Bastion Host->>Vault: Create log 'echo "Hello World!"', Bastion Host Vault Token
    note over Vault: Bastion Host Vault Token checks
    Vault->>API: 'echo "Hello World!"', JWT
    note over API: JWT checks
    note over API: Write('echo "Hello World!"')
    API->>Vault: Log created

Loading

Technologies

The technologies used in the project are:

• Go as programming language
• HashiCorp Vault as a secret management and access control system
• JWT as an additional token system for API authentication and authorization
• Vagrant as a virtual machine manager
• VMWare Fusion as a virtual machine
• Docker as a container manager
• Docker Compose as a container orchestrator
• PostgreSQL as a database
• Swagger as a REST API documentation
• Sshwifty as a SSH web client, with some modifications to allow the communication with the Vault
• Elasticsearch as a search engine for logs and analytics solution
• Kibana as a data visualization platform for Elasticsearch
• Logstash as a data collection and processing pipeline

Auth plugin for Vault

A crucial step in the implementation of the system is the development of an authorization plugin for Vault.
The plugin is based on the plugin development guide.
The api exposed by the plugin are the following:

Each call to the plugin calls another API of the API Vault Helper, that adds another layer of security to the system because it uses a JWT to authenticate the calls.
The second API reflects the Vault API, but it is used to detach the JWT token when the bastion host authenticates itself before requesting the other operations.

Demo

Instructions

You can run the application with Docker or manually.

Docker

To run the application with Docker, you need to have Docker installed on your machine.
To run with Docker, you only need to run the following commands:

1. Get the web client Sshwifty

   $ git submodule init
   $ git submodule update --remote
   

2. Run the Docker compose

   $ docker compose up
   

3. Take a ☕

4. Find the SSH remote host IP address

   $ docker inspect ssh-host | grep IPAddress
   

5. Open the browser at http://localhost:8182 and enjoy with default username elliot and password mrrobot.

Manual

You need to have Go and Node.js installed on your machine.
I also suggest to use Vagrant to setup the remote host.
As virtual machine provider, you can use any one such as VirtualBox.
If you run on macOS Ventura i reccomend to use VMWare.
You also need to have a database installed on your machine, I used PostgreSQL.

To launch the application manually, you need to run the following commands:

1. Get the web client Sshwifty

   $ git submodule init
   $ git submodule update --remote
   

2. Edit the env file with your own values

   $ nano .env
   

3. Build and starts the vault plugin

   $ bash vault-init.sh
   

4. Setup the vault and the plugin

   $ bash vault-setup.sh
   

5. Starts the server API

   $ go run api/cmd/main.go
   

6. Configure the remote host (adapted from SSH Secrets Engine: One-Time SSH Password):

   • Start vagrant

   $ vagrant up
   $ vagrant ssh
   

   • Download and install version 0.2.1 of vault-ssh-helper from releases.hashicorp.com.

   $ wget https://releases.hashicorp.com/vault-ssh-helper/0.2.1/vault-ssh-helper_0.2.1_linux_amd64.zip
   

   • Unzip the binary from the archive into /usr/local/bin.

   $ sudo unzip -q vault-ssh-helper_0.2.1_linux_amd64.zip -d /usr/local/bin
   

   • Set the vault-ssh-helper binary's permissions to executable.

   $ sudo chmod 0755 /usr/local/bin/vault-ssh-helper
   

   • Create a directory to store the configuration file.

   $ sudo mkdir /etc/vault-ssh-helper.d/
   

   • Create a Vault SSH Helper configuration file /etc/vault-ssh-helper.d/config.hcl

   $ sudo tee /etc/vault-ssh-helper.d/config.hcl <<EOF
   vault_addr = "$VAULT_ADDR"
   tls_skip_verify = false
   ssh_mount_point = "ssh"
   allowed_roles = "*"
   EOF
   

   • Edit PAM sshd configuration file.

   $ sudo nano /etc/pam.d/sshd
   

   The common-auth must be commented out or removed to disable the standard Unix authentication and replaced with authentication through vault-ssh-helper. Finally, a workaround for a bug that exists with some versions of pam_exec.so must also be included.

   #PAM configuration for the Secure Shell service
   
   # Standard Un*x authentication.
   # @include common-auth
   auth requisite pam_exec.so quiet expose_authtok log=/var/log/vault-ssh.log /usr/local/bin/vault-ssh-helper -dev -config=/etc/vault-ssh-helper.d/config.hcl
   auth optional pam_unix.so not_set_pass use_first_pass nodelay
   
   
   ...
   

   • Modify the sshd configuration file.

   $ sudo nano /etc/ssh/sshd_config
   

   • Add or set the following options:

   KbdInteractiveAuthentication yes
   UsePAM yes
   PasswordAuthentication yes
   

   • Restart the service

   $ sudo systemctl restart sshd
   

   • Verify that the configuration is correct

   $ vault-ssh-helper -verify-only -dev -config /etc/vault-ssh-helper.d/config.hcl
   

   • Find the ipv4 address of the remote host

   $ ip -4 address show eth1 | grep inet | awk '{print $2}' | cut -d'/' -f1
   

   • Exit from the remote host

   $ exit
   

7. (Optional) If you want to have a Log Management System, you have to install Elastic Stack on your machine.

   • Install Elasticsearch
   • Install Kibana
   • Install Logstash
   • Copy the content of logstash in your Logstash installation directory
   • Edit logstash/pipelines/config.conf.
   • Edit logstash/config/pipelines.yml.
   • Edit ${YOUR_ELASTIC_SEARCH}/conf/elasticsearch.yml adding:

         xpack.security.http.ssl:
         enabled: false
     
         xpack.security.transport.ssl:
         enabled: false 
     

   • Edit ${YOUR_KIBANA}/conf/kibana.yml adding:

         elasticsearch.hosts: ['http://localhost:9200']
         type: elasticsearch, hosts: ['http://localhost:9200']]
     

8. Build and starts the web client over the bastion host

   $ bash sshwifty-run.sh
   

9. (Optional) Run elasticseach, kibana and logstash if you want to have a Log Management System

   $ elasticsearch
   $ kibana
   $ logstash