| title | subtitle | date | transition | venue | layout |
|---|---|---|---|---|---|
Action Team |
Working Practices Overview |
2020-04-29 |
None |
DECOVID Team Tour |
talk |
\include{talk-macros.tex}
\section{DRAFT: DELVE Action Team: Working Practices and Philosophy}
\notes{The purpose of this document is to share the working practices of the DELVE Action Team. The Action Team is responsible for delivering scientific insight to SAGE via approval processes through the DELVE Working Group and the DELVE Steering Committee.}
\subsection{Project Structure}
\figure{\includediagram{\diagramsDir/delve/committee-structure}{35%}}{The structure of the DELVE team.}{committee-structure}
\section{Supply Chain of Ideas}
\notes{The main philosphy of the group is to follow the "Supply Chain of Ideas".}
\figure{\includediagram{\diagramsDir/delve/supply-chain-of-ideas}{90%}}{The supply chain of ideas avoids 'coconut science', assuming your scientific solution is the right one to answer a question that's been asked. Our science is directed towards policy questions.}{supply-chain-of-ideas}
\section{Information Coherence}
\notes{Part of forming an efficient supply chain involves bottleneck analysis. In the supply chain of ideas, and across most organisations, a criticial bottleneck is our individual bandwidth.
Humans have a information transmission rate (in spoken form) of around 100 bits per second. This is very low compared to e.g. our computers.
Fortunately we are very intelligent, and investing time in thinking about the project, about those we're working with, what the challenges we're facing, allows us to account for one another in our decision making.
Achieving information coherence is about ensuring that we're working together towards the same objectives. This reduces the need for detailed communication, because we have a better group-wide understanding of the common goals.
That ensures that the supply chain of ideas runs smoothly.}
\figure{\includediagram{\diagramsDir/delve/information-coherence}{70%}}{Our Action team needs to work through trusted connections to the Steering Committee, Working Group and the wider community, while orienting around a policy question and conferring regularly with other members of the Action Team.}{information-coherence}
\subsection{Tenets}
\notes{A keystone of information coherence is our tenets for this project. The Tenets emphasise the prioritisation of our ways of working.
This project has short deadlines, and quick turn around times. This can lead to stress. In moments of stress it can be difficult to decide how to prioritise. At these moments the Tenets can be particularly useful in reminding what the group priorities are.
The Tenets are core, but they are not set in stone. If you have better ideas or improvements, please suggest them!
Tips for good tenets are: avoid platitudes, a good tenet is one that empowers individuals to make better decisions that align with the groups objectives (information coherence).
Our current list of tenets is:}
-
We are steered by policy need.
-
"It is amazing what you can accomplish if you do not care who gets the credit." We're here to be as useful as possible, as fast as possible, as a group.
-
This work is multidisciplinary - everyone should ask questions, and expect to answer them to people from other disciplines in ways they can understand and scrutinise.
\newslide{Tenets}
-
We do not reinvent the wheel: we use existing analysis techniques and datasets wherever possible to answer questions.
-
Being useful is more important than presentee-ism. We should be considerate of people who work different hours and have different commitments.
-
We steer a path between the Hedgehog (too attached to a particular model) and the Fox (always considering worst case outcome).
\subsection{SciOps}
\notes{We're in the unusual position of managing "SciOps". That is scientific operations. The need for science on demand, science-as-a-service or as Neil sometimes called it at Amazon 0800-SCIENCE.
The nature of the questions we are asked is that they will not necessarily sit within our domains of expertise, but they should be at least peripheral. Each of the members of the action team has been chosen to cover sections of the literature we believe will be important.
This means that at times we will be operating outside our comfort zones, but as career scientists or domain experts, you will have the knowledge of the wider network of scientists and literature so that you know who to ask, where to look and what experiments to do. You also have access to members of the Steering Committee, the Working Group and the Advisory Group. Each of whom has shown themselves to be willing to respond rapidly to questions.}
\subsection{Multidisciplinarity}
\notes{Our group is purposely multidisciplinary as reflected in our tenets. That is because we believe that good policy decisions can not be made without considering the range of factors.
Multidisciplinarity brings challenges and opportunities. Among the challenges are the issues with different nomenclature for related technical ideas. The use of jargon in specific fields, and assumptions around what is canoncical knowledge vs what specifics need to be elucidated.
The solution for these challenges is a motto: there are no stupid questions. Each member of the Action Team has been selected for their talents, they should never feel embarassed to ask for clarification or deeper understanding from another member of the team. Naturally, a consequence of this is each member of the team should be prepared to explain their ideas to others clearly, and using different terminologies.
A large bear-trap is assumption by intimidation. The idea that because someone is technically expert, that they can't make a foolish error. In fact, the opposit is often true, by being too focussed on a specific technical idea, we can all miss something that is obvious to those who don't have our individual deep technical understanding. Let's do our very best to avoid this bear-trap.}
\subsection{The Avatar Model}
\notes{As part of the communication bottlenecks, another challenge we face is how to bootstrap the knowledge of our Working Group and Steering Committee. The avatar model is such that each of those individuals should know, and have technical confidence in, at least one member of the Action Team. That member of the Action Team should have a good understanding of their corresponding Steering Group member, and should be able to represent their thought process in the rapid fire Action Team discussions that are resolving the multidisciplinary issues, where many of the tensions and opportunities will exist.
Each member of the Action Team also knows well the mind of members of the Working Group and the Steering Committee. We will be using specific tools to facilitate the different rhythms of communication we require, ensuring we maintain information coherence both in the immediate time period, the short term weekly turn over of projects, and across the entire duration of the DELVE Initiative.}
\section{Explore <-> Exploit}
\figure{\includediagram{\diagramsDir/delve/mccollum-plot}{85%}}{The McCollum Plot shows the manner in which work processes loosen as the Project develops. We use three phases to define our projects. Transition between explore and exploit is occuring at the interaction between Phase 1 and Phase 2.}{mccollum-plot}
\notes{A key mathematical concept we have from areas like reinforcement learning is the evolution of the nature of a project from conception to realisation. At the beginning of a project, there is a need to understand the question, to do research into the issues and develop the best strategy for resolving the question. As the deadline for an answer approaches, it becomes necessary to double down on, and refine, a particular strategy. This transition, from exploration to the refinement of a strategy is known in machine learning as the explore-exploit trade off. In management science, a similar process is known as divergence-convergence.
In rapid management of science cycling through the explore/exploit phases is a critical aspect of a successful project.
A critical challenge in the supply chain of ideas is ensuring we're getting input at the explore stage from all our channels. This type of input is particularly important at the start of the cycle, but new ideas being introduced during the convergence phase can be disruptive.}
\figure{\includediagram{\diagramsDir/delve/mccollum-plot-03}{85%}}{The reality of many projects on the McCollum plot. Projects can drop back phases as part of a successful path. Projects that start in explore with tight process or in exploit with loose process will typically fail. Process needs tightening as the goal nears}{mccollum-plot-03}
\subsection{Three Phase Approach}
\notes{To support the introduction of ideas and the transition between explore and exploit. We have a three phase approach to project management. The phases are within the control of the Action Team Members.}
\notes{The approach is best illustrated by touring our software tools where we have implemented our ideas.}
\notes{Each tool is associated with a particular rhythm of information transfer.}
\subsection{Github}
\figure{\includepng{\diagramsDir/delve/github}{80%}}{Our github repository stores things like working practices and lists of people that we expect to change more slowly over time.}{github-repo}
\notes{Concepts that change most slowly are on github. Like our working practices, our tenets our committee membership, meeting notes, our code bases.}
\subsection{Trello}
\figure{\includepng{\diagramsDir/delve/trello-board-1}{70%}}{The Trello board contains our direct inputs: list of policy questions, our derived input: list of science questions, and the projects we are running to deliver.}{trello-board-1}
\notes{Concepts which change on timescales of days and weeks are stored on Trello. Trello provides us with a system of 'cards' and 'lists'. For us cards represent entities like 'project questions' or 'science questions', or the project stages themselves. Action team members are encouraged to create cards when they have a project concept (Phase 0)
Each project transitions from concept through scoping (Phase 1) to proof of concept (Phase 2). You should check with your 'avatar' what their range of projects is and where they are in stages.
The process is very light touch, except we stipulate each project must have made contact with a Working Group member or Steering Committee member and had a scientific review before moving to Phase 2 (Proof of Concept).
The scientific review will involve working through the idea with the relevant expertise before the hardwork of implementation is done. The aim is to make sure we're getting working group input before any easily avoided errors are made.}
\newslide{Trello}
\figure{\includepng{\diagramsDir/delve/trello-board-2}{70%}}{Also on Trello, for reference, are the list of resource inputs such as data, literature reviews and software. As well as a reminder of our Tenets.}{trello-board-2}
\subsection{Google Docs and Overleaf}
\notes{Google docs and Overleaf allow for collaborative editing of documents. We make use of Google docs for report drafting (e.g. for SAGE) and Overleaf similarly for more technical reports (e.g. modelling of effect of masks on transmission).
We have a google group set up for managing permissions.}
\subsection{Slack}
\figure{\includepng{\diagramsDir/delve/slack-channels}{70%}}{Organising our Slack channels correctly helps keep the focus on the right questions. At the moment we order according to #question- channels, #help- channels, #resource- channels and importantly, for providing support to each other, a #kudos channel.}{slack-channel}
\notes{The Trello board contains our direct inputs: list of policy questions, our derived input: list of science questions, and the projects we are running to deliver.}{trello-board-1}
\notes{Slack is a great tool for in the moment communicaton. Triggering discussions, asking each other, or the group, questions. We use channels on slack to discuss different policy questions, or trigger the creation of new subjects. We also use it to provide help to each other for dealing with technical questions or domain specific questions (we have channels called 'help-virology' and 'help-economics'.}
\subsection{Zoom}
\notes{We have stand-up meetings three times a week to connect via Zoom, as wellinformal meeting between teams. Slack allows for video converstations between individuals.}
\section{Retrospectives}
\notes{We are working together in a highly responsive deep technical team delivering policy insight with very shor tturn around times. While there are aspects of what we are doing present in other roles, e.g. daily insights around Amazon supply chain, or the daily review of the MetOffice weather, this is an unprecedented situation and there is no single right way to do what we're doing.
All our processes are designed to facilitate outcomes. If we are ot achieve the outcome efficiently, we review our processes. We will use weekly retrospectives in our Monday meetings to overview what we can improve and where things are and aren't working.}
\notes{Our first week's retrospective uncovered the following issues with process.}
-
It can be hard to see where to fit in when the Slack channels are moving fast, and if it's not your area of expertise.
-
For the masks report very significant work was put into modelling and statistical data analysis that wasn't included in the final report. It would be good to identify this earlier.
-
Duplication of labour between the Steering Committee and the Action Team. Need better lines of communication.
\newslide{Questions?}