Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Manage Firewall rules for DB Direct allows IPs #15610

Merged
merged 24 commits into from
Apr 27, 2020
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
305be3d
Firewall Manager WIP
jgoizueta Apr 20, 2020
7ff7f15
Merge branch 'dbdirect_ips_zip' into dbdirect_ips_firewall
jgoizueta Apr 20, 2020
5d813dd
Merge branch 'dbdirect' into dbdirect_ips_firewall
jgoizueta Apr 20, 2020
5cc1988
Sample firewall configuration
jgoizueta Apr 21, 2020
adb1f83
Add configuration parameter
jgoizueta Apr 21, 2020
7b5b5cc
Fixes
jgoizueta Apr 21, 2020
fda03d0
Tests for firewall management
jgoizueta Apr 21, 2020
ae2a882
Merge branch 'dbdirect' into jgoizueta/ch66432/db-direct-allowed-ips-…
jgoizueta Apr 22, 2020
d8c2edf
Show firewall rule name
jgoizueta Apr 22, 2020
6d7d464
Merge branch 'dbdirect' into jgoizueta/ch66432/db-direct-allowed-ips-…
jgoizueta Apr 22, 2020
9d8cfdd
Merge branch 'dbdirect' into jgoizueta/ch66432/db-direct-allowed-ips-…
jgoizueta Apr 23, 2020
ffc8348
Firewall management using Google API client
jgoizueta Apr 23, 2020
67029ee
Merge branch 'master' into jgoizueta/ch66432/db-direct-allowed-ips-ma…
jgoizueta Apr 23, 2020
782ec4e
Ignore errors only for non-existing rules
jgoizueta Apr 23, 2020
9995bc9
Update NEWS
jgoizueta Apr 23, 2020
5c610df
Firewall manager shouldn't be instantiated if not enabled
jgoizueta Apr 24, 2020
b790f38
Fix bug
jgoizueta Apr 24, 2020
57351b5
Show firewall rule name for organization
jgoizueta Apr 24, 2020
8d47d35
Update app/models/carto/dbdirect_ip.rb
jgoizueta Apr 27, 2020
8b749af
Merge branch 'jgoizueta/ch66432/db-direct-allowed-ips-management' of …
jgoizueta Apr 27, 2020
9ba0c55
Move constant definitions to top
jgoizueta Apr 27, 2020
492ddad
Merge branch 'master' into jgoizueta/ch66432/db-direct-allowed-ips-ma…
jgoizueta Apr 27, 2020
fdecb3f
Fix NEWS
jgoizueta Apr 27, 2020
f102662
Merge branch 'master' into jgoizueta/ch66432/db-direct-allowed-ips-ma…
jgoizueta Apr 27, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ sudo make install

### Features
- New internal API for managing DB-Direct certificates & IPs ([#15567](https://github.com/CartoDB/cartodb/pull/15567))
And managing the firewall: ([#15610](https://github.com/CartoDB/cartodb/pull/15610))
- Use Dataservices API client 0.30.0
- Enable deleting Kepler.gl maps ([#15485](https://github.com/CartoDB/cartodb/issues/15485))
- Add Kepler.gl maps to Recent content section in the Dashboard ([#15486](https://github.com/CartoDB/cartodb/issues/15486))
Expand Down
44 changes: 44 additions & 0 deletions app/models/carto/dbdirect_ip.rb
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
require 'ip_checker'
require 'carto/dbdirect/firewall_manager'

module Carto
class DbdirectIp < ActiveRecord::Base
Expand All @@ -12,6 +13,38 @@ class DbdirectIp < ActiveRecord::Base

validate :validate_ips

after_save do
update_firewall(*changes[:ips])
end

after_destroy do
update_firewall(ips, nil)
end


def self.firewall_manager
firewall_manager_class.new(config)
end

def self.firewall_manager_class
Carto::Dbdirect::FirewallManager
end

def firewall_rule_name
return nil unless self.class.firewall_enabled?

rule_id = user.dbdirect_bearer.organization&.name || user.dbdirect_bearer.username
self.class.config['rule_name'].gsub('{{id}}', rule_id)
end

def self.config
Cartodb.get_config(:dbdirect, 'firewall') || {}
end

def self.firewall_enabled?
!!config['enabled']
end

private

MAX_IP_MASK_HOST_BITS = 8
Expand Down Expand Up @@ -46,5 +79,16 @@ def validate_ip(ip)
exclude_loopback: true
)
end

def update_firewall(old_ips, new_ips)
return unless self.class.firewall_enabled?

old_ips ||= []
new_ips ||= []
if old_ips.sort != new_ips.sort
rule_id = user.dbdirect_bearer.organization&.name || user.dbdirect_bearer.username
self.class.firewall_manager.replace_rule(firewall_rule_name, new_ips)
end
jgoizueta marked this conversation as resolved.
Show resolved Hide resolved
end
end
end
8 changes: 6 additions & 2 deletions app/models/carto/user.rb
Original file line number Diff line number Diff line change
Expand Up @@ -310,7 +310,7 @@ def send_password_reset!
end

def dbdirect_effective_ips
dbdirect_bearer.dbdirect_ip&.ips || []
dbdirect_effective_ip&.ips || []
end

def dbdirect_effective_ips=(ips)
Expand All @@ -319,7 +319,9 @@ def dbdirect_effective_ips=(ips)
bearer.create_dbdirect_ip!(ips: ips) if ips.present?
end

private
def dbdirect_effective_ip
dbdirect_bearer.dbdirect_ip
end

def dbdirect_bearer
if organization.present? && organization.owner != self
Expand All @@ -329,6 +331,8 @@ def dbdirect_bearer
end
end

private

def set_database_host
self.database_host ||= ::SequelRails.configuration.environment_for(Rails.env)['host']
end
Expand Down
7 changes: 7 additions & 0 deletions config/app_config.yml.sample
Original file line number Diff line number Diff line change
Expand Up @@ -782,6 +782,13 @@ defaults: &defaults
pgproxy:
host: 'dbconn.carto.com'
port: 5432
firewall:
enabled: false
project_id: ''
rule_name: ''
network: ''
target_tag: ''
ports: 'tcp:5432'

development:
<<: *defaults
Expand Down
81 changes: 81 additions & 0 deletions lib/carto/dbdirect/firewall_manager.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
require 'google/apis/compute_v1'

module Carto
module Dbdirect
# Firewall Rule Manager
class FirewallManager
def initialize(config)
@config = config
@service = Google::Apis::ComputeV1::ComputeService.new
@service.authorization = Google::Auth.get_application_default([AUTH_URL])
end

attr_reader :config

def replace_rule(name, ips)
delete_rule(
project_id: config['project_id'],
name: name
)

if ips.present?
create_rule(
project_id: config['project_id'],
name: name,
network: config['network'],
ips: ips,
target_tag: config['target_tag'],
ports: config['ports']
)
end
end

private

def delete_rule(project_id:, name:)
# `gcloud compute firewall-rules delete "#{name}"`

# we could check existence with @service.list_firewalls(project_id).find {|r| r.name == name}
# but since there could be race conditions anyway we'll just rescue errors
@service.delete_firewall(config['project_id'], name)

rescue Google::Apis::ClientError => error
raise unless error.message =~ /^notFound:/
end

def create_rule(project_id:, name:, network:, ips:, target_tag:, ports:)
# `gcloud -q compute firewall-rules create "#{name}" \
# --network "#{network}" \
# --direction ingress \
# --action allow \
# --source-ranges "#{ips.join(',')}" \
# --target-tags "#{target_tag}" \
# --rules "#{ports}" \
# --no-enable-logging \
# --project "#{project_id}"`

protocol, port = config['ports'].split(':')
allowed = Google::Apis::ComputeV1::Firewall::Allowed.new(
ip_protocol: protocol,
ports: [port.split(',')]
)
rule = Google::Apis::ComputeV1::Firewall.new(
name: name,
allowed: [allowed],
network: network_url(config['project_id'], config['network']),
source_ranges: ips,
target_tags: [config['target_tag']]
)

@service.insert_firewall(config['project_id'], rule)
end

AUTH_URL = 'https://www.googleapis.com/auth/cloud-platform'.freeze
PROJECTS_URL = 'https://www.googleapis.com/compute/v1/projects'.freeze
jgoizueta marked this conversation as resolved.
Show resolved Hide resolved

def network_url(project_id, network)
"#{PROJECTS_URL}/#{project_id}/global/networks/#{network}"
end
end
end
end
1 change: 1 addition & 0 deletions lib/tasks/dbdirect.rake
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ namespace :carto do
if ips.present?
puts "DBDirect IPs for user #{user_id}:"
puts ips
puts "GCP Firewall Rule name: #{user.dbdirect_effective_ip.firewall_rule_name}"
else
puts "No IPs defined for user #{user_id}"
end
Expand Down
Loading