Skip to content

Commit

Permalink
Merge pull request #2 from lsst/drphilmarshall/proof-reading
Browse files Browse the repository at this point in the history
Proof-reading
  • Loading branch information
ranpalgill committed May 2, 2024
2 parents a1b06d7 + 87183b2 commit b4481c6
Show file tree
Hide file tree
Showing 6 changed files with 88 additions and 44 deletions.
8 changes: 7 additions & 1 deletion README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,13 @@ Rubin Observatory Operations: Enabling collaborative ground-up budget planning a
RTN-081
=======

Change is inevitable in large big budget operational programs. Embracing, rather than resisting, change is key to being proactive. It also keeps teams motivated as it’s another avenue for leadership to “listen” to what is going on at the team level. At Rubin Observatory an agile approach to budgeting has been implemented. Annually a bottom up reset, called a scrub, is carried out across all departments of the Rubin Operations organization providing an opportunity to adapt and be nimble to changing situations that can affect resources and budgets. This paper will provide details on the importance for an annual budget scrub, processes followed, tools used and how the cycle continues year on year.
Change is inevitable in large, big-budget operational programs.
Embracing, rather than resisting, change is key to being proactive.
It also keeps teams motivated as it’s another avenue for leadership to "listen" to what is going on at the team level.
At Rubin Observatory, an agile approach to budgeting has been implemented, following related experience in previous High Energy Physics experiments.
Annually, a ground-up review, to address changing needs, priorities and emerging issues, is carried out across all departments of the Rubin Operations organization.
This "annual scrub" provides an opportunity to adapt and be nimble to changing situations that can affect resources and budgets.
This paper provides details on the importance of an annual budget scrub, the processes followed, the tools used, and how the cycle continues year on year.

Links
=====
Expand Down
8 changes: 7 additions & 1 deletion abstract.tex
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@

\begin{abstract}
Change is inevitable in large big budget operational programs. Embracing, rather than resisting, change is key to being proactive. It also keeps teams motivated as it’s another avenue for leadership to “listen” to what is going on at the team level. At Rubin Observatory an agile approach to budgeting has been implemented. Annually a bottom up review to address changing needs, priorities and emerging issues, called a scrub, is carried out across all departments of the Rubin Operations organization. This provides an opportunity to adapt and be nimble to changing situations that can affect resources and budgets. This paper will provide details on the importance for an annual budget scrub, processes followed, tools used, and how the cycle continues year on year.
Change is inevitable in large, big-budget operational programs.
Embracing, rather than resisting, change is key to being proactive.
It also keeps teams motivated as it’s another avenue for leadership to ``listen'' to what is going on at the team level.
At Rubin Observatory, an agile approach to budgeting has been implemented, following related experience in previous High Energy Physics experiments.
Annually, a ground-up review, to address changing needs, priorities and emerging issues, is carried out across all departments of the Rubin Operations organization.
This ``annual scrub'' provides an opportunity to adapt and be nimble to changing situations that can affect resources and budgets.
This paper provides details on the importance of an annual budget scrub, the processes followed, the tools used, and how the cycle continues year on year.
\end{abstract}

11 changes: 9 additions & 2 deletions intro.tex
Original file line number Diff line number Diff line change
@@ -1,3 +1,10 @@
\section{Introduction} \label{sec:intro}
Vera C. Rubin Observatory\cite{2008arXiv0805.2366I} is currently under construction. Once complete, the observatory will consist of an end to end system with the Chile based mountain top summit facility housing an 8.4m telescope and a 3200 megapixel camera, a high bandwidth long haul network, a data processing facility in California, data management systems, a data access platform in the cloud and public engagement programs. Naturally this means the observatory is distributed across a number of locations. The operations program is due to start full operations in 2025; the survey will be carried out over a period of ten years, and a post-operations phase will follow. The operation of Rubin observatory is fairly unique in being funded in equal shares by two US government agencies. Budgets are usually set at high levels years in advance but things rarely stay the same. This paper describes an annual bottom up budgeting process adapted from the one used by the US-ATLAS Experiment. The aim of the annual exercise is to enable change within the budget envelope.
The paper provides the reader with details of all the tools that are used and describes the process that is followed annually.
Vera C. Rubin Observatory\cite{2008arXiv0805.2366I} is currently under construction.
Once complete, the observatory will consist of an end to end system with the mountain top Summit Facility on Cerro Pachon in Chile housing an 8.4m telescope and a 3200 megapixel camera, a high bandwidth long haul network, a system of data processing facilities in California, France and the UK, a data management system, a data access platform in the cloud, and a host of public engagement programs.
Naturally this means the observatory is distributed across a number of locations.
The observatory is due to start full operations in 2025; the survey will be carried out over a period of ten years, and a post-operations phase will follow.
The operation of Rubin Observatory is fairly unique in astronomy for being funded in approximately equal shares by two US government agencies, and operated by two almost equal partner national laboratories (NSF NOIRLab and SLAC National Accelerator Laboratory, funded by the Department of Energy).
Budgets are usually set at high level years in advance, but things rarely stay the same.
This paper describes an annual ground-up budgeting process adapted from the one used by the US-ATLAS operations team, in support of the ATLAS experiment at the Large Hadron Collider at CERN.
The aim of this annual exercise is to enable change within the budget envelope.
The paper provides the reader with details of all the tools that are used, and describes the process that is followed annually.
7 changes: 5 additions & 2 deletions outcomes.tex
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
\section{Outcomes and Conclusions} \label{sec:outcomes}

This annual iterative process enables change to happen in a controlled and transparent manner enabling buy-in at all levels on what the upcoming FY priorities are and the reasons behind difficult decisions which are often inevitable. It should be noted that changes can still happen throughout the year through a process called Request Beyond Target (RBT) which enables team leads to erquest mid year enhancements to their programs. This process is outside of the scope of this paper but is mentioned here to stress the agile nature of planning on the Rubin Program.
This annual iterative process enables change to happen in a controlled and transparent manner, enabling buy-in at all levels on a) what the upcoming FY priorities are and b) the reasons behind difficult decisions which are often inevitable.
It should be noted that changes can still happen throughout the year through a process called Request Beyond Target (RBT), which enables team leads to request mid-year enhancements to their programs.
This process is outside of the scope of this paper, but is mentioned here to stress the agile nature of planning at Rubin.

Year on year, as this process takes place from now through to the end of the ten-year Legacy Survey of Space and Time, expected in 2037, the scrub process is envisaged to evolve. Each iteration is expected to reveal gaps and areas of improvement that can be fed into the design of the process and the tools for the following fiscal year’s scrub.
Year on year, as this process takes place from now through to the end of the ten-year Legacy Survey of Space and Time expected in 2037, the scrub process is envisaged to evolve.
Each iteration is expected to reveal gaps and areas of improvement that can be fed into the design of the process and the tools for the following fiscal year’s scrub.
59 changes: 36 additions & 23 deletions process.tex
Original file line number Diff line number Diff line change
@@ -1,42 +1,55 @@
\section{Process} \label{sec:process}

Throughout the Rubin pre-operations and survey operations phases, annually in May each team will look back at what was planned, what was achieved, do a full review of its activities, and propose a high level plan for the following (US fiscal) year.

Throughout the Rubin pre-operations and operations phases, annually in May each team will look back at what was planned, what was achieved, do a full review of its activities, and propose a high level plan for the following (US fiscal) year.
Such an ``annual scrubbing'' is standard practice in other high energy physics experiments.
The scrub allows the facility to continuously evolve its operating plan, taking critical input from the people that understand best what is really needed.
In Rubin’s case that is the Team Leaders, who are responsible for delivering products.

This is standard practice in other high energy physics experiments as well. The scrub allows the facility to continuously evolve its operating plan, taking critical input from the people that understand best what is really needed. In Rubin’s case that is the Team Leaders.
Following the U.S. National Science Foundation and U.S. Department of Energy joint annual review of Rubin Operations (typically in April) the Rubin Operations Directors office together with department heads sets the major milestones for the next US fiscal year (FY) starting 1st October.
This includes looking at the status of major milestones for the current year and ascertaining whether any of those need to carry over into the next FY.

Following the U.S. National Science Foundation and U.S. Department of Energy joint annual review of Rubin Operations (typically March or April) the Rubin Operations Directors office together with department heads sets the major milestones for the next US fiscal year (FY) starting 1st October. This includes looking at the status of major milestones for the current year and ascertaining whether any of those need to carry over into the next FY.

With the major milestones set, the Director’s office kicks off the month long annual scrub process, see \autoref{fig:timeline} in which the department heads start down stream planning with their teams. This is the “homework” phase of the scrub where teams are looking at:
With the major milestones set, the Director’s office kicks off the month long annual scrub process (see \autoref{fig:timeline}), in which the department heads start downstream planning with their teams.

This is the ``homework'' phase of the scrub, where teams are looking at:
\begin{itemize}
\item status of minor milestones for the current FY
\item setting minor milestones that would contribute to accomplishing the new major milestones for the next FY
\item teams review planned resources both labor and non-labor based on activities needed to achieve the minor milestones and risk mitigation plans
\item if there is a mismatch between resources needed and the resources available the team will propose changes during this scrub period through the tool (described in the next section).

\item status of minor milestones for the current FY;
\item setting minor milestones that would contribute to accomplishing the new major milestones for the next FY;
\item planned resources, both labor and non-labor, based on activities needed to achieve the minor milestones, and risk mitigation plans;
\item whether there is a mismatch between the resources needed and the resources available; if needed, the team will propose changes during this scrub period through the scrub ``sandbox'' tool (described in the next section).
\end{itemize}


\begin{figure}[hb!]
\begin{centering}
\includegraphics[width=1.0\textwidth]{Figure1Scrubtimeline}
\caption{Scrub timeline
\label{fig:timeline}}
\end{centering}
\end{figure}
Having completed the above homework, in the tool provided, the scrub moves into the “pitches” phase. Department heads and team leads prepare short presentations, for which a template is provided, to pitch the needed changes to the directors office. As additional resource requests from one department could impact another, all department leads and team leads are invited and encouraged to attend all pitches. Currently in Rubin the 15-30 minute team level pitches are carried out per department in virtual format. This is in contrast to the US-ATLAS Experiment where the teams come together for an in-person gathering to present pitches (Rubin teams are widely based geographically).

After the pitches phase is complete, a period of iteration takes place between the directors office and department/team to reach an agreement on the changes.

The directors office aggregates the labor and non-labor baseline vs proposed changes across the Rubin departments to ensure the program remains overall within the defined financial envelope. This is one of the reasons for the back and forth iterations and negotiations as priorities have to drive this process.

After agreement the changes are implemented by propagating them throughout the Rubin planning tools. This includes implementation in the accounting systems at SLAC and AURA. This culminates in the spending plan for the next FY and definition of Statements of Work for the contracts enabling requisitions to be input in time for contracts to be placed.

With the resources now updated across all the tools, teams can start realistically planning for the coming FY by defining the tasks and activities that will lead to completion of the defined minor milestones within the boundaries of the available resources. Work then commences at the start of the FY. The activity planning is done in an Atlassian tool called Jira where the milestones are defined enabling downstream and upstream traceability between milestones to tasks. The Jira tool is outside the scope of this paper.

From the start of the FY, the directors office takes inputs, such as overhead rates, escalation rates etc., as defined by the managing organizations and preps the tools for the next annual scrub, and so the cycle continues, see \autoref{fig:cycle}.

Having completed the above homework in the tool provided, the scrub moves into the ``pitches'' phase.
Department Heads and Team Leaders prepare short presentations, for which a structured template is provided, to pitch the proposed changes to the Directors Office.
As additional resource requests from one department could impact another, all Department Heads and Team Leaders are invited and encouraged to attend all teams' pitches, in an observatory-wide 3-day ``scrub workshop.''
Currently in Rubin the 15-30 minute team level pitches are presented department by department and in fully virtual format.
This is in contrast to the US-ATLAS experiment, where the teams come together for an in-person workshop to present their proposals.
(The Rubin teams are widely distributed, geographically).

After the pitches phase is complete, a period of iteration takes place between the Directors Office and each Department/Team, in order to reach an agreement on which changes will be implemented and how.
Some compromise is needed at this point, due to budgetary constraints -- and the Team Leaders understand this, having been briefed at a virtual ``kick-off'' meeting at the start of the homework phase.

The Directors Office aggregates and costs the baseline vs proposed changes, both labor and non-labor, across the Rubin departments to ensure the program remains overall within the defined financial envelope.
This is one of the reasons for the back and forth iterations and negotiations as priorities have to drive this process.
The aggregation is performed automatically and in real-time by the sandbox tool, so that Team Leaders and Deprtament Heads can immediately see the impact of the changes being proposed.

After agreement, the changes are implemented (by the Director's Office staff) by propagating them throughout the Rubin planning tools.
This culminates in implementation of the new spending plan for the next FY in the accounting systems at SLAC and AURA.
Statements of Work for the next year contracts can then be generated (in July) enabling requisitions to be input in time for contracts to be placed.

With the resources now updated across all the observatory's planning tools, the teams can start realistically planning work for the coming FY by defining the tasks and activities that will lead to completion of the defined minor milestones within the boundaries of the available resources.
Work then commences at the start of the FY.
The work planning is done in the Atlassian tool Jira, where the milestones are defined enabling downstream and upstream traceability between milestones to tasks.
(A full appreciation of Jira is beyond the scope of this paper.)

From the start of the FY, the Directors Office collects inputs such as overhead rates, escalation rates etc., as defined by the managing organizations, and prepares the planning tools for the next annual scrub, and so the cycle continues (see \autoref{fig:cycle}).

\begin{figure}[h!]
\begin{centering}
Expand Down

0 comments on commit b4481c6

Please sign in to comment.