Skip to content
This repository has been archived by the owner on Jul 16, 2020. It is now read-only.

Weekly Meeting 2016 09 08

Jump to bottom
Kristen Carlson Accardi edited this page Sep 8, 2016 · 6 revisions

Agenda

##Minutes

#ciao-project: Weekly Meeting

Meeting started by kristenc at 16:02:30 UTC. The full logs are available at ciao-project/2016/ciao-project.2016-09-08-16.02.log.html .

Meeting summary

Meeting ended at 17:00:41 UTC.

Action Items

  • investigate popular identity/auth schemes, start architecting generic common framework
  • tcpepper to investigate authentication options
  • tcpepper investigate popular identity/auth schemes, start draft architecture for generic common framework
  • mcastelino to draft kubernetes architecture plan.
  • markusry to evaluate impact of moving to 1.7 and file issues

Action Items, by person

  • markusry
    • markusry to evaluate impact of moving to 1.7 and file issues
  • mcastelino
    • mcastelino to draft kubernetes architecture plan.
  • tcpepper
    • tcpepper to investigate authentication options
    • tcpepper investigate popular identity/auth schemes, start draft architecture for generic common framework
  • UNASSIGNED
    • investigate popular identity/auth schemes, start architecting generic common framework

People Present (lines said)

  • markusry (67)
  • kristenc (67)
  • arjan_work (65)
  • tcpepper (49)
  • mcastelino (18)
  • albertom (7)
  • david-lyle (5)
  • mrkz (4)
  • leoswaldo (4)
  • ciaomtgbot (3)
  • _erick0zcr (1)
  • mrkz_afk (1)

Generated by MeetBot_ 0.1.4

.. _MeetBot: http://wiki.debian.org/MeetBot

###Full IRC Log

16:02:30 <kristenc> #startmeeting Weekly Meeting
16:02:30 <ciaomtgbot> Meeting started Thu Sep  8 16:02:30 2016 UTC.  The chair is kristenc. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:02:30 <ciaomtgbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
16:02:30 <ciaomtgbot> The meeting name has been set to 'weekly_meeting'
16:02:59 <kristenc> I updated the meeting bot this week. It might behave differently.
16:03:05 <kristenc> #topic Role call
16:03:08 <tcpepper> o/
16:03:13 <kristenc> o/
16:03:14 <markusry> o/
16:03:17 <leoswaldo> o/
16:03:20 <mrkz_afk> o/
16:03:21 <david-lyle> o/
16:03:32 <_erick0zcr> o/
16:04:38 <kristenc> #topic Opens
16:04:51 <kristenc> does anyone have any opens? I have one.
16:05:05 <markusry> I have one as well
16:05:21 <arjan_work> I have a meta-open I suppose
16:05:27 <kristenc> I'd like to discuss whether we should bother scrubbing P2 and P3 bugs during this meeting. It feels to me like a bit of a waste.
16:05:45 <kristenc> arjan_work, what is your open?
16:05:52 <tcpepper> I agree.  maybe once a month at most
16:06:05 <arjan_work> kristenc: I wonder if this is the place to summarize the recent direction clarifications etc
16:06:31 <kristenc> arjan_work, it sounds like a good idea - let me make notes and I'll set you up a topic.
16:06:55 <kristenc> #agreed we will not scrub P2/P3 bugs regularly in this meeting.
16:07:01 <kristenc> #topic ciao direction
16:07:08 <kristenc> ok arjan_work go ahead.
16:07:25 <mcastelino> o/
16:07:35 <arjan_work> So we have been talking about where ciao needs to go; part of a regular "lets re-evaluate where we go" that got us started in the first place
16:08:00 <arjan_work> also based on feedback from users in both openstack summit and more, linuxcon
16:08:26 <arjan_work> the summary is that we are going to change from "be a drop in replacement for openstack" to being more
16:08:53 <arjan_work> of a generic cloud infrastructure that runs several higher level orchestrators on top of shared infra/shared tennant infra
16:09:08 <arjan_work> users really really like our "containers and VMs on the same level"
16:09:14 <arjan_work> and the tennant separation
16:09:27 <arjan_work> but also tell us they want kubernetes for containers (cattle) as well as having pets
16:09:40 <arjan_work> and potentially wanting something like spark for analytics
16:09:56 <arjan_work> while pretty much nobody cared about openstack (api) compat
16:10:19 <arjan_work> so with that in mind we are all whiteboarding what that means, and the picture looks quite attractive
16:10:26 <arjan_work> where we have a core common layer
16:11:00 <arjan_work> and then hooks into an interface for cattle (kubernetes), an interace for cattle (current ciao UI/cli etc) and an interface for analytics (spark)
16:11:33 <kristenc> the ciao UI/cli would be an interface for pets? or cattle and pets?
16:11:38 <mrkz> while also keeping the pets, right?
16:11:53 <arjan_work> the ciao UI/cli would be primary for pets, but not ruling out also doing cattle
16:12:00 <arjan_work> esp the admin interface will have to do both
16:12:00 <kristenc> ok.
16:12:11 <tcpepper> so it's sort of undercloud++
16:12:13 <arjan_work> yeah
16:12:29 <arjan_work> what I described would be the tennant side/view
16:12:45 <arjan_work> admin side of course will need to be ciao webui, since it needs to understand all 3 types of workloads in a combined view
16:13:03 <arjan_work> so on the one hand this adds a bunch of new things to do
16:13:16 <arjan_work> (adding kubernetes properly etc)
16:13:32 <arjan_work> on the other hand it reduces priorities of other things (adding openstack API)
16:13:39 <arjan_work> the openstack API thing is a bit controversial
16:14:08 <arjan_work> but reality is that in a world where you have these 3 types of tennants, the openstack APIs are a 2nd tier, since they are limited to pretty much VMs
16:14:33 <arjan_work> and in a world where you have containers + VM + analytics, you'd be blind to 2/3rd of your cloud (even within a tennant)
16:15:15 <arjan_work> and if folks want to keep adding openstack APIs, we should absolutely take the pull request
16:15:16 <arjan_work> s
16:15:31 <arjan_work> but in terms of priorities I would say that those are P3 at this point
16:16:17 <tcpepper> makes perfect sense to me based on all of our discussions with users/operators
16:16:36 <kristenc> yes - we'll have a lot more flexibility abandoning api compatibility.
16:16:48 <kristenc> plus we never really did it right in the first place.
16:17:10 <kristenc> well - abandoning isn't the right word.
16:17:11 <david-lyle> how does keystone continue to fit in or is is replaced?
16:17:12 <arjan_work> we can still try to keep api stuff as a module on top of the base
16:17:21 <arjan_work> david-lyle: keystone today is a liability for us
16:17:26 <david-lyle> agree
16:17:28 <arjan_work> it accounts for like 900f the complexity
16:17:32 <tcpepper> and with abstracted interfaces there is always the possibility (ie: as arjan_work notes we accept any PR where somebody does it) of enhanced compat if folks want it and prioritize implementing it
16:17:32 <david-lyle> it's terrible
16:17:42 <arjan_work> so we already were figuring out if we could replace it
16:18:09 <arjan_work> the kubernetes approach for this is nice to be honest, and could even use keystone as a backend if someone would really want it
16:18:26 <david-lyle> more options for sure
16:18:29 <arjan_work> anyway that's one the list of architecture items for us regardless
16:18:36 <kristenc> if we assume kubernetes is only for containers though - we need something for both vm/containers.
16:18:47 <arjan_work> (it's 900f deployment/operation complexity, and we got VERY negative feedback from users about having it still in ciao)
16:18:59 <arjan_work> kristenc: that's the ciao ui/cli
16:19:27 <arjan_work> for authentication specifically we could add the generic authentication framework
16:19:32 <arjan_work> that kubernetes also uses
16:19:40 <arjan_work> just for the "can person X do Y"
16:19:42 <arjan_work> questions
16:20:21 <arjan_work> but just like we have unified networking
16:20:27 <arjan_work> we need unified identity/authentication
16:20:40 <arjan_work> anyway that's still an open
16:22:01 <tcpepper> #link http://kubernetes.io/docs/admin/authentication/
16:22:18 <tcpepper> is pretty in line with what we need
16:22:21 <arjan_work> if we don't end up with unified identity we're not useful
16:22:39 <arjan_work> but most things are plugable in this space so there certainly are ways to get there
16:23:12 <arjan_work> we just need to do an investigation of options and then come up with a proposal
16:23:36 <arjan_work> one criteria for me will be the ease of install/maintenance by the sysadmin
16:23:37 <kristenc> our architecture makes it easy for us to get away from keystone. this should be no problem.
16:25:21 <arjan_work> anyway we can come up with proposals and evaluate in some future meeting
16:25:56 <kristenc> arjan_work, so that was my next question - other than reprioritizing openstack compat, anything immediately actionable for us to do?
16:26:01 <tcpepper> #action investigate popular identity/auth schemes, start architecting generic common framework
16:26:15 <arjan_work> kristenc: 1) authentication future 2) networking into kubernetes
16:26:16 <tcpepper> ^^ is what I heard for actionable
16:26:28 <kristenc> tcpepper, who does the action item belong to?
16:26:36 <kristenc> all of us?
16:26:41 <arjan_work> and manohar was already looking at what glue we need to hook into kubernetes
16:26:58 <arjan_work> which would be 3)
16:27:25 <mcastelino> arjan_work: yes... have started on this this week glue into kubernetes
16:27:31 <arjan_work> kristenc: authetnication/etc fit into security side of things, I'd propose tcpepper as key owner (who delegates stuff)
16:27:34 <tcpepper> kristenc: probably more actionable if we give it dedicated owner(s)...everbody owns becomes nobody owns
16:27:47 <kristenc> tcpepper, yes - that was my feeling too.
16:28:12 <kristenc> #action tcpepper to investigate authentication options
16:28:18 <tcpepper> #action tcpepper investigate popular identity/auth schemes, start draft architecture for generic common framework
16:28:38 <tcpepper> I'm happy to start a draft propose...will of course lean on others heavily over time
16:29:42 <kristenc> #action mcastelino to draft kubernetes architecture plan.
16:29:56 <kristenc> ok - anything else on this subject?
16:30:03 <arjan_work> from me that's it
16:30:20 <kristenc> tcpepper, mcastelino when should we expect a draft?
16:30:20 <tcpepper> is the mcastelino action networking specific or broader too?  (eg: ui?)
16:30:31 <markusry> and maybe scheduler/
16:30:32 <markusry> ?
16:30:53 <tcpepper> oh yeah we need to decide how much call out between them and our scheduler we want
16:30:53 <arjan_work> tcpepper: generic first
16:30:57 * kristenc votes that mcastelino drives the plan for all things kubernetes
16:31:16 <mcastelino> kristenc: I will take a first shot at k8s integration...
16:31:36 <mcastelino> kristenc: focused on ciao being the cloud-provider
16:32:09 <markusry> mcastelino: As soon as I'm done with storage I'd be interested in help on on Kubernetes
16:32:35 <mcastelino> #info https://github.com/kubernetes/kubernetes/tree/master/pkg/cloudprovider
16:32:56 <arjan_work> anyway sorry for taking 30 minutes out of the meeting
16:33:07 <tcpepper> good discussion
16:33:12 <mrkz> refreshing
16:33:16 <kristenc> arjan_work, no problem - we have not much to scrub anyway.
16:33:20 <tcpepper> much better then sequentially agreeing github issues are low priority
16:33:53 <kristenc> are there additional opens?
16:33:57 <markusry> Yes I have one
16:34:12 <leoswaldo> I have one small, next week Thursday and Friday are mexican holidays
16:34:23 <kristenc> #info next week Thursday and Friday are mexican holidays
16:34:38 <kristenc> markusry, go ahead.
16:34:48 <markusry> Should we drop support for Go 1.6
16:34:49 <markusry> ?
16:34:57 <markusry> There are two reasons we might want to do this.
16:35:01 <kristenc> #topic  Should we drop support for Go 1.6
16:35:18 <kristenc> markusry, define "drop support".
16:35:26 <arjan_work> use 1.7 language features
16:35:36 <tcpepper> drop support: does not compile
16:35:39 <markusry> 1. docker no longer compiles on 1.6.  So we cannot update the version of the docker packages we use in ciao
16:36:06 <markusry> 2. It would be nice to use the new context package in the standard lib and the associated functions
16:36:25 <markusry> So what I mean by dropping is we would require people to use 1.7 or greater to build ciao
16:36:33 <markusry> We'd also disable the 1.6 travis build.
16:36:38 <arjan_work> so docker already set the precedent
16:36:43 <markusry> Right.
16:36:45 <arjan_work> which makes the decision a bit easier
16:36:47 <tcpepper> eg: https://travis-ci.org/tpepper/ciao/builds/158278647
16:36:50 <markusry> I agree.
16:36:52 <arjan_work> do distros ship 1.7 enough?
16:36:58 <markusry> No
16:37:00 <tcpepper> distros don't ship 1.6 enough
16:37:01 <kristenc> i have no problems - it will be a bit of a pain is all since most distros don't ship 1.7
16:37:01 <mrkz> not yet I think
16:37:06 <markusry> So you need to install go from the web site.
16:37:10 <kristenc> fc24 specifically does not.
16:37:16 <markusry> But I think we already advise people to do this anyway
16:37:22 <markusry> in the readme.
16:37:27 <kristenc> we did when they were on 1.5 for sure.
16:37:34 <tcpepper> fc23 doesn't have 1.6.  fc25 has 1.7
16:37:34 <kristenc> we refused to use that version.
16:37:36 <tcpepper> it's a moving target
16:37:54 <tcpepper> since November on Fedora we've required non-distro golang
16:38:02 <mcastelino> markusry: should we also start using context and switch directly to go with context support... (assume context support for ciao)
16:38:09 <markusry> Yes I think we should
16:38:11 <tcpepper> briefly in August or now you could go distro probably
16:38:24 <markusry> So if we decide to drop 1.6 I'll enter some tickets to cover the work
16:38:43 <kristenc> anyone object to dropping 1.6?
16:38:49 * tcpepper has started using golang context in some places on branch and has a hundred or so lines of change in controller to make it not complain
16:39:18 <kristenc> it will not be a free move then like 1.5->1.6
16:39:22 <markusry> Note I think that the standard library context package is compatible with the old external package, so we shouldn't need to update our dependencies
16:39:26 <kristenc> but fine with me.
16:39:32 <markusry> Well it is a free move
16:39:43 <markusry> We don't need to update the context package
16:39:47 <markusry> But it makes sense to do so
16:40:09 <kristenc> markusry, maybe you can take a look at the issues tcpepper is having and see if there's an easy way to address them.
16:40:17 <markusry> We can also take advantage of the the new functions like DialContext and CommandContext
16:40:26 <markusry> that allow us to cancel long running tasks easily
16:40:29 <tcpepper> kristenc: it was mostly that context.Foo means something now in the language
16:40:36 <tcpepper> and you have a bunch of context.Foo's
16:40:40 <tcpepper> so just simple gorename
16:40:48 <arjan_work> anyway only real issue would be distro support to make development easier
16:41:01 <arjan_work> but if that has been a mess already and no real new pain, then...
16:41:04 <arjan_work> well docker already does as well
16:41:10 <arjan_work> so they will force the distros to move
16:41:59 <kristenc> installing from the website is pretty straight forward. the only issue is that people won't think to do it. but, our contributors are not vast right now - so we are not in danger here.
16:42:11 <markusry> tcpepper: I can take a look if you feel you need to change too much
16:42:42 <tcpepper> markusry: we either rename kristenc's context or import the language context as a different name and carry that technical debt of confusion
16:42:53 <markusry> I think we may need to revendor the old context package
16:42:53 <mrkz> probably ^ should be mentioned in the contributor guide as well
16:42:55 <arjan_work> lets not create debt
16:43:02 <tcpepper> imho the choice is obvious and quick/easy
16:43:09 <markusry> Ah, I see, kristen has a context package
16:43:14 <arjan_work> anyone opposed to moving to 1.7 ?
16:43:18 <kristenc> markusry, I don't have a context package.
16:43:25 <tcpepper> just a context variable
16:43:29 <markusry> Okay.
16:44:03 <tcpepper> https://github.com/tpepper/ciao/commit/ce982c0ae5f2806bf93c4ccb225c51d44f01c6ad
16:44:10 <markusry> I vote we move to 1.7
16:44:16 <kristenc> #agreed support for 1.6 dropped.
16:44:17 <tcpepper> I vote we move to 1.7
16:44:25 <kristenc> nobody said they were against it.
16:44:39 <markusry> ciao is also smaller and faster when compiled with 1.7
16:44:57 <kristenc> #action markusry to evaluate impact of moving to 1.7 and file issues
16:45:03 <markusry> Th
16:45:05 <markusry> Will do
16:45:09 <kristenc> any other opens?
16:45:48 <kristenc> #topic bugs
16:45:57 <kristenc> there are a few with no priority we need to take a look at.
16:46:22 <kristenc> https://github.com/01org/ciao/issues/518
16:46:37 <kristenc> https://github.com/01org/ciao/issues/526
16:46:51 <kristenc> https://github.com/01org/ciao/issues/528
16:47:08 <mcastelino> kristenc: 518 is a bit worry some... 526 is low priority
16:47:24 <leoswaldo> adding p1 to
16:47:27 <leoswaldo> #link https://github.com/01org/ciao/issues/532
16:48:05 <markusry> 518 is a race condition in the singlemachinevm
16:48:21 <kristenc> I was wondering if 518 was singlevm only.
16:48:45 <markusry> Well, I guess the problem is that it has no way of knowing when it's safe to wipe the ciao state
16:48:56 <mcastelino> markusry: race condition in the launcher or the script
16:48:58 <kristenc> mcastelino, since 526 has a workaround, would you call it a P3?
16:49:07 <markusry> the script.
16:49:07 <mcastelino> yes 526 = P3
16:49:09 <tcpepper> can we go in order and complete one before the next
16:49:17 <markusry> It calls ciao-cli instance delete -all
16:49:24 <markusry> and then wipes the launcher state
16:49:33 <markusry> by doing a rm -rf
16:49:44 <mcastelino> yes...after the CLI returns
16:50:00 <markusry> but ciao-cli instance delete -all doesn't wait until launcher has completed deleting things
16:50:08 <mcastelino> ok.. let me fix the script then
16:50:12 <tcpepper> this argues it should wait
16:50:21 <markusry> That's the other option
16:50:22 <tcpepper> but waiting is also annoying
16:50:36 <tcpepper> can the script delete all, then pause/loop/check that all are deleted, then carry on
16:50:39 <markusry> But I don't think controller has the information to wait
16:50:54 <markusry> I suppose it could use stats
16:51:01 <markusry> But then you'd need to wait for at least 30 secs
16:51:03 <tcpepper> yeah I think having the cli and ciao internally wait is wrong
16:51:03 <mcastelino> tcpepper: I can check that all the qemu's have disappeared
16:51:08 <markusry> Actually, maybe just 6
16:51:10 <tcpepper> but the it that waits should be the script
16:51:32 <tcpepper> wait 6, check, wait 6 check...5 times?
16:51:39 <tcpepper> .sh is good at that
16:51:48 <markusry> The thing is we can just delete the rm -rf
16:51:55 <markusry> As launcher will clean everything up anyway
16:52:12 <mcastelino> markusry: I do not need to do the rm -rf...
16:52:19 <mcastelino> I can do a hard reset on each script start
16:52:22 <mcastelino> that will be better
16:52:25 <mcastelino> let me try that
16:52:43 <tcpepper> yeah I worry about left-overs from a failed shutdown, but a reset on start helps with that
16:53:04 <mcastelino> kristenc: now that we know the root cause it can be a P3
16:53:12 <mcastelino> this will not happen in production
16:53:18 <kristenc> mcastelino, ok.
16:54:03 <kristenc> markusry, you are working on #528?
16:54:23 <markusry> Not let but I will do it in the next two weeks as part of Sprint 3.0
16:54:28 <markusry> Not yet
16:54:43 <tcpepper> on 528, can you put some guidance in the issue re:
16:54:49 <tcpepper> "fix the ceph installations"
16:55:19 <kristenc> markusry, ok - shall I mark it a P1 then?
16:55:21 <markusry> Do the installations need to be fixed?
16:55:27 <markusry> kristenc: Yes why not
16:55:37 <markusry> It only affects volumes that we create
16:55:41 <tcpepper> and...we still have near zero documentation on what we expect for ceph setup.  this is a good reminder we should have some basic something
16:55:42 <kristenc> markusry, should be indepedent of installation - it's a volume thing.
16:56:12 <markusry> agreed.
16:56:24 <kristenc> markusry, I would just edit your description to delete the wording around "fixing installations"
16:56:46 <markusry> Yes sorry
16:56:49 <markusry> that's confusing
16:57:07 <tcpepper> ok so it's literally just s/1/2/ in the ceph block volume create for the --image case?
16:57:20 <albertom> for 528
16:57:26 <albertom> we can use the image format 2
16:57:28 <albertom> and specify
16:57:31 <albertom> --image-feature
16:57:37 <kristenc> tcpepper, yes - except the reason we did that is because of some missing functionality that mark is working on
16:57:38 <albertom> that are supported by the kernel
16:57:38 <markusry> tcpepper: Okay now I remember
16:57:39 <albertom> cvlient
16:57:51 <markusry> The issue was that ceph was not working properly on Ubuntu 16.04 with image format 2
16:57:57 <markusry> I think there's some workaround
16:58:00 <tcpepper> aaah ok
16:58:04 <albertom> there is one or two that are not supported for the ikernel client iirc
16:58:13 <markusry> So fix the installations meant fix the Ubuntu ceph clusters if there are any
16:58:22 <tcpepper> just need to write it up then so we all know what's up
16:58:31 <markusry> I'll clarify
16:58:47 <tcpepper> markusry: do you know if the dockhub ceph(s) have the same issue?
16:59:03 <tcpepper> kristenc: did you have trouble?
16:59:22 <markusry> Now that's a good question
16:59:25 <markusry> I guess it does
16:59:27 <kristenc> tcpepper, wasn't tested.
16:59:34 <kristenc> tcpepper, the issue is with mapping the volume.
16:59:41 <kristenc> which we didn't do till the f2f.
16:59:56 <kristenc> creating the volume has always been fine.
16:59:58 <markusry> But I was trying it locally before the f2f
17:00:08 <markusry> and I needed to use image-format 1
17:00:17 <markusry> anyway, I'll clarify this
17:00:24 <kristenc> ok - we are out of time.

Weekly Meetings

Development

Architecture

Clone this wiki locally