Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[WiP] Refactor common Kontena classes in monorepo form #2559

Closed
wants to merge 6 commits into from
Closed

[WiP] Refactor common Kontena classes in monorepo form #2559

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
09f5c43
kontena-lib skeleton
Jul 5, 2017
2457071
agent: build with and require kontena-lib
Jul 5, 2017
82a89dc
lib: kontena-logging
Jul 5, 2017
f8ebf0d
lib: kontena-wait
Jul 5, 2017
3a51eea
agent: use lib/kontena-wait Kontena::Wait::Helper
Jul 5, 2017
5e0ffbb
rename kontena-lib to kontena-common, using conventional common/lib/.…
Jul 5, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 7 additions & 6 deletions agent/Dockerfile → Dockerfile.agent
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,16 @@ MAINTAINER Kontena, Inc. <info@kontena.io>
RUN apk update && apk --update add tzdata ruby ruby-irb ruby-bigdecimal \
ruby-io-console ruby-json ca-certificates libssl1.0 openssl libstdc++

ADD Gemfile /app/
ADD Gemfile.lock /app/
ADD common/kontena-common.gemspec /opt/kontena/common/
ADD agent/Gemfile agent/Gemfile.lock /opt/kontena/agent/

RUN apk --update add --virtual build-dependencies ruby-dev build-base openssl-dev && \
gem install bundler --no-ri --no-rdoc && \
cd /app ; bundle install --without development test && \
cd /opt/kontena/agent/ ; bundle install --without development test && \
apk del build-dependencies

WORKDIR /app
ADD . /app
ADD common /opt/kontena/common
ADD agent /opt/kontena/agent

CMD ["/app/bin/kontena-agent"]
WORKDIR /opt/kontena/agent
CMD bundle exec /opt/kontena/agent/bin/kontena-agent
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using bundle exec to add the /opt/kontena/common/lib to the $LOAD_PATH... not sure if the best way to do it, but probably the fastest.

  • kontena gem build /opt/kontena/common/kontena-common.gemspec && gem install kontena-common-*.gem?
  • ???

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we keep Dockerfile under agent?

Copy link
Contributor Author

@SpComb SpComb Jul 6, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we could, but I think it's better to have the Dockerfile be in the actual docker build context dir... That way it's apparent that you have to build it with kontena $ docker build -t kontena/agent -f Dockerfile.agent .. If it is just agent/Dockerfile, then the obvious thing of kontena/agent $ docker build -t kontena/agent . will fail.

1 change: 1 addition & 0 deletions agent/Gemfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
source 'https://rubygems.org'

gem 'kontena-common', path: '../common'
gem 'docker-api', '~> 1.32.0'
gem 'eventmachine', '~> 1.2.3'
gem 'faye-websocket', '~> 0.10.7'
Expand Down
8 changes: 7 additions & 1 deletion agent/Gemfile.lock
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
PATH
remote: ../common
specs:
kontena-common (1.4.0.dev)

GEM
remote: https://rubygems.org/
specs:
Expand Down Expand Up @@ -93,6 +98,7 @@ DEPENDENCIES
eventmachine (~> 1.2.3)
faye-websocket (~> 0.10.7)
fluent-logger (~> 0.6.2)
kontena-common!
msgpack (~> 1.0.3)
rake
rspec
Expand All @@ -102,4 +108,4 @@ DEPENDENCIES
webmock

BUNDLED WITH
1.14.3
1.15.1
2 changes: 1 addition & 1 deletion agent/Vagrantfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ Vagrant.configure("2") do |config|
vb.cpus = vm_cpus
end
config.vm.network "private_network", ip: "192.168.66.#{i+100}"
config.vm.synced_folder ".", "/home/core/share", id: "core", :nfs => true, :mount_options => ['nolock,vers=3,udp']
config.vm.synced_folder "..", "/home/core/kontena", id: "core", :nfs => true, :mount_options => ['nolock,vers=3,udp']

if File.exist?(CLOUD_CONFIG_PATH)
config.vm.provision :file, :source => "#{CLOUD_CONFIG_PATH}", :destination => "/tmp/vagrantfile-user-data"
Expand Down
4 changes: 2 additions & 2 deletions agent/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
agent:
container_name: kontena-agent
build: .
dockerfile: Dockerfile
build: ..
dockerfile: Dockerfile.agent
env_file: .env
net: host
environment:
Expand Down
5 changes: 4 additions & 1 deletion agent/lib/kontena-agent.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,16 @@
require 'active_support/core_ext/time'
require 'active_support/core_ext/module/delegation'

gem 'kontena-common'
require 'kontena-logging'
require 'kontena-wait'

require_relative 'ipaddr_helpers'

require_relative 'docker/version'
require_relative 'docker/container'
require_relative 'etcd/health'
require_relative 'etcd/client_patch'
require_relative 'kontena/logging'
require_relative 'kontena/websocket_client'

require_relative 'kontena/models/node'
Expand Down
4 changes: 1 addition & 3 deletions agent/lib/kontena/agent.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
require_relative 'logging'

module Kontena
class Agent
include Logging
include Kontena::Logging

VERSION = File.read('./VERSION').strip

Expand Down
4 changes: 1 addition & 3 deletions agent/lib/kontena/helpers/weave_helper.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,10 @@
require 'docker'
require_relative 'iface_helper'
require_relative 'wait_helper'


module Kontena
module Helpers
module WeaveHelper
include WaitHelper
include Kontena::Wait::Helper

WEAVE_VERSION = ENV['WEAVE_VERSION'] || '1.9.3'
WEAVE_IMAGE = ENV['WEAVE_IMAGE'] || 'weaveworks/weave'
Expand Down
3 changes: 1 addition & 2 deletions agent/lib/kontena/launchers/ipam_plugin.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,12 @@
require_relative '../helpers/image_helper'
require_relative '../helpers/wait_helper'

module Kontena::Launchers
class IpamPlugin
include Celluloid
include Celluloid::Notifications
include Kontena::Logging
include Kontena::Helpers::ImageHelper
include Kontena::Helpers::WaitHelper
include Kontena::Wait::Helper

IPAM_SERVICE_NAME = 'kontena-ipam-plugin'.freeze

Expand Down
69 changes: 0 additions & 69 deletions agent/lib/kontena/logging.rb
View file
Open in desktop

This file was deleted.

2 changes: 0 additions & 2 deletions agent/lib/kontena/network_adapters/ipam_cleaner.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
require_relative '../logging'

module Kontena::NetworkAdapters
class IpamCleaner
include Celluloid
Expand Down
1 change: 0 additions & 1 deletion agent/lib/kontena/network_adapters/weave.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
require_relative '../logging'
require_relative '../helpers/node_helper'
require_relative '../helpers/iface_helper'
require_relative '../helpers/weave_helper'
Expand Down
1 change: 0 additions & 1 deletion agent/lib/kontena/network_adapters/weave_executor.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
require_relative '../logging'
require_relative '../helpers/image_helper'
require_relative '../helpers/iface_helper'

Expand Down
6 changes: 1 addition & 5 deletions agent/lib/kontena/rpc_client.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,8 @@
require_relative 'logging'
require_relative 'helpers/wait_helper'


module Kontena
class RpcClient
include Celluloid
include Kontena::Logging
include Kontena::Helpers::WaitHelper
include Kontena::Wait::Helper

REQUEST_ID_RANGE = 1..2**31

Expand Down
1 change: 0 additions & 1 deletion agent/lib/kontena/rpc_server.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@
require_relative 'rpc/service_pods_api'
require_relative 'rpc/lb_api'
require_relative 'rpc/volumes_api'
require_relative 'logging'

module Kontena
class RpcServer
Expand Down
1 change: 0 additions & 1 deletion agent/lib/kontena/service_pods/creator.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
require 'docker'
require 'celluloid'
require_relative 'common'
require_relative '../logging'
require_relative '../helpers/weave_helper'
require_relative '../helpers/port_helper'
require_relative '../helpers/rpc_helper'
Expand Down
1 change: 0 additions & 1 deletion agent/lib/kontena/service_pods/restarter.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
require 'docker'
require_relative 'common'
require_relative '../logging'

module Kontena
module ServicePods
Expand Down
5 changes: 2 additions & 3 deletions agent/lib/kontena/service_pods/starter.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
require 'docker'
require_relative 'common'
require_relative '../logging'

module Kontena
module ServicePods
Expand Down Expand Up @@ -28,8 +27,8 @@ def perform
)
service_container.restart!('timeout' => service_container.stop_grace_period)
log_service_pod_event(
self.service_id, self.instance_number,
"service:start_instance", "service instance #{service_container.name_for_humans} started successfully"
self.service_id, self.instance_number,
"service:start_instance", "service instance #{service_container.name_for_humans} started successfully"
)
end

Expand Down
1 change: 0 additions & 1 deletion agent/lib/kontena/service_pods/stopper.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
require 'docker'
require_relative 'common'
require_relative '../logging'

module Kontena
module ServicePods
Expand Down
1 change: 0 additions & 1 deletion agent/lib/kontena/service_pods/terminator.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
require 'docker'
require_relative 'common'
require_relative '../logging'

module Kontena
module ServicePods
Expand Down
1 change: 0 additions & 1 deletion agent/lib/kontena/websocket_client.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
require 'msgpack'
require_relative 'logging'
require_relative 'rpc_server'
require_relative 'rpc_client'

Expand Down
2 changes: 1 addition & 1 deletion agent/lib/kontena/workers/service_pod_manager.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ class ServicePodManager
include Celluloid::Notifications
include Kontena::Logging
include Kontena::Observer
include Kontena::Wait::Helper
include Kontena::Helpers::RpcHelper
include Kontena::Helpers::WaitHelper

attr_reader :workers, :node

Expand Down
2 changes: 1 addition & 1 deletion agent/lib/kontena/workers/volumes/volume_manager.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ class VolumeManager
include Celluloid
include Celluloid::Notifications
include Kontena::Logging
include Kontena::Wait::Helper
include Kontena::Helpers::RpcHelper
include Kontena::Helpers::WaitHelper
include Kontena::Observer

class DriverMismatchError < StandardError
Expand Down
1 change: 0 additions & 1 deletion agent/lib/kontena/workers/weave_worker.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
require 'docker'
require_relative '../logging'
require_relative '../helpers/weave_helper'

module Kontena::Workers
Expand Down
6 changes: 2 additions & 4 deletions agent/spec/lib/kontena/observer_spec.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
require_relative '../../../lib/kontena/helpers/wait_helper'

describe Kontena::Observer do
let :observable_class do
Class.new do
Expand Down Expand Up @@ -170,7 +168,7 @@ def crash

it "crashing allows it to re-observe the existing value immediately after restarting", :log_celluloid_actor_crashes => false do
expect{@observer_actor.crash}.to raise_error(RuntimeError)
Kontena::Helpers::WaitHelper.wait_until! { @observer_actor.dead? }
Kontena::Wait::Helper.wait_until! { @observer_actor.dead? }

# simulate supervisor
@observer_actor = Celluloid::Actor[:observer_test] = supervised_observer_class.new(:observable_test)
Expand All @@ -181,7 +179,7 @@ def crash

it "restarts after the observable crashes and waits for it to update", :log_celluloid_actor_crashes => false do
expect{@observable_actor.crash}.to raise_error(RuntimeError)
Kontena::Helpers::WaitHelper.wait_until! { @observable_actor.dead? && @observer_actor.dead? }
Kontena::Wait::Helper.wait_until! { @observable_actor.dead? && @observer_actor.dead? }

# simulate supervisor restart in the wrong order
expect{supervised_observer_class.new(:observable_test)}.to raise_error(Celluloid::DeadActorError)
Expand Down
2 changes: 1 addition & 1 deletion agent/spec/lib/kontena/workers/container_info_worker_spec.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@
it 'logs error on unknown exception' do
event = double(:event, status: 'start', id: 'foo')
expect(Docker::Container).to receive(:get).once.and_raise(StandardError)
expect(subject.wrapped_object.logger).to receive(:error).twice
expect(subject.wrapped_object).to receive(:error).twice
subject.on_container_event('topic', event)
end
end
Expand Down
2 changes: 2 additions & 0 deletions common/.gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
.bundle/
vendor/
1 change: 1 addition & 0 deletions common/.rspec
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
--require spec_helper
6 changes: 6 additions & 0 deletions common/Gemfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
source 'https://rubygems.org'

group :development, :test do
gem 'rspec'
gem 'websocket-driver'
end
30 changes: 30 additions & 0 deletions common/Gemfile.lock
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
GEM
remote: https://rubygems.org/
specs:
diff-lcs (1.3)
rspec (3.6.0)
rspec-core (~> 3.6.0)
rspec-expectations (~> 3.6.0)
rspec-mocks (~> 3.6.0)
rspec-core (3.6.0)
rspec-support (~> 3.6.0)
rspec-expectations (3.6.0)
diff-lcs (>= 1.2.0, < 2.0)
rspec-support (~> 3.6.0)
rspec-mocks (3.6.0)
diff-lcs (>= 1.2.0, < 2.0)
rspec-support (~> 3.6.0)
rspec-support (3.6.0)
websocket-driver (0.6.5)
websocket-extensions (>= 0.1.0)
websocket-extensions (0.1.2)

PLATFORMS
ruby

DEPENDENCIES
rspec
websocket-driver

BUNDLED WITH
1.15.1
Loading