README

FFA School Files
================
These files form the main source material for the FFA school held at PSI on
22nd Nov 2019, for tracking an FFA using the OPAL code. This tutorial was
tested against opal version 2.0.1-1

Contact: chris.rogers@stfc.ac.uk


Table of Contents
-----------------
1. Running a first example
2. Finding lattice Parameters
3. Further reading


This tutorial will guide you through the process of installing OPAL, running it
to get tracking output. Later examples use a set of scripts to calculate closed
orbits and tunes.

For more information, please refer to the OPAL website at

  https://gitlab.psi.ch/OPAL/src/wikis/home

1. Running a first example
--------------------------

1.1 Prerequisites
-----------------
For this part of the tutorial, you should be familiar with interacting with the 
command line. You should be comfortable with changing directories, listing 
directories and running executables.

To manipulate lattices, you should be familiar with opening and editing files in 
a text editor.

There is some setup listed in section 1.2 below for installing OPAL.

1.2 Install OPAL
----------------
In order to run the examples, you need to download them and also install OPAL
itself.

  mkdir -p ~/path/to/working/dir
  wget http://amas.web.psi.ch/Downloads/OPAL/package/OPAL-2.0.1-1-x86_64-linux.tar.xz
  tar -xf OPAL-2.0.1-1-x86_64-linux.tar.xz

You need to install the FFA school files

  cd ~/path/to/working/dir
  git clone https://github.com/chrisrogers1234/ffa-school.git

In order to make plots, you will need python3 and the standard python3 
scientific libraries

  sudo apt-get install python3-devel python3-pip g++
  pip3 install numpy matplotlib scipy

I also use xboa library as a backend

  cd ~/path/to/working/dir
  wget http://micewww.pp.rl.ac.uk/maus/xboa/xboa-0.17.0/xboa-0.17.0.tar.gz
  tar -xzf xboa-0.17.0.tar.gz
  cd xboa-0.17.0
  sudo python3 setup.py install

Finally, you need to set a few environment variables

  cd ~/path/to/working/dir/ffa-school
  export OPAL_BUILD_PATH=~/path/to/working/dir/OPAL-2.0.1/
  source env.sh

The env.sh script also checks that the environment is okay and returns an error
message if it detects missing libraries. If the env.sh script ends with

  ERROR: Setup failed

then the setup was not successful. If the env.sh script ends with

  Setup looks okay!

then the setup was successful.


1.3 Running a first example
---------------------------
There is an example FFA lattice file in the first_run directory. In order to run
it, first copy the directory to output, so that you don't write into the 
original directory:

    cp -r first_run output

in this tutorial, all of the output files will be kept in the output directory.
In this directory there are two files.

    disttest.dat FETS_Ring.opal

FETS_Ring.opal contains the main lattice files; disttest.dat contains 
information about the input distribution.

Try running:

    cd output/first_run
    ${OPAL_EXE_PATH}/opal FETS_Ring.opal

You should see output that starts with

    Ippl> CommMPI: Parent process waiting for children ...
    Ippl> CommMPI: Initialization complete.
    >                ____  _____       ___ 
    >               / __ \|  __ \ /\   | | 
    >              | |  | | |__) /  \  | |
    >              | |  | |  ___/ /\ \ | |
    >              | |__| | |  / ____ \| |____
    >               \____/|_| /_/    \_\______|

and ends with

    Timings> Write Stat.......... Wall max =  0.0003146, CPU max =          0
    Timings>                      Wall avg =  0.0003146, CPU avg =          0
    Timings>                      Wall min =  0.0003146, CPU min =          0
    Timings> 
    Timings> -----------------------------------------------------------------

OPAL generates lots of files in the working directory. In these examples, there
are two files that we will look at initially

* FETS_Ring-trackOrbit.dat contains step-by-step tracking information.
* FieldMapXY.dat contains information describing the magnetic field generated by 
the FFA magnets.

There is a script to plot the field map, with the closed orbit superimposed, in
the scripts directory. Try running the plot_orbit.py script

    cd ../../
    python3 scripts/plotting/plot_orbit.py output/first_run/FETS_Ring-trackOrbit.dat

You should get output ending in

    plot_orbit finished okay

You should see a new file,

    closed_orbit_plan_bz.png

Open the file using an image viewing software, like gimp, gwenview, gthumb or
firefox.

The image shows colours (blue through red), which indicate the vertical 
component of the field map and a black wavy line which indicates the trajectory
of the particle that was tracked through the field.

1.4 Input Particle Control
--------------------------
Let's try changing the inputs to our tracking simulation and see how the output
changes.

disttest.dat contains the input position and momentum of a particle relative to
some reference trajectory. Open the file in a text editor (for example, emacs,
vim or nano)

disttest.dat:
1 <--- number of particles in the file
3.997870104214539 -0.007682059555892479 0.0 -0.00041030187886083926 0.0 0.0
 ^                     ^                 ^           ^               ^   ^
radial              radial            vertical      vertical       longitudinal
position [m]        beta gamma        position [m]  beta gamma     position and beta gamma

Here "beta gamma" is the (momentum component)/mass, i.e., 

        (relativistic beta)*(relativistic gamma)

Try increasing the initial radial position to 4.10 m

disttest.dat:
1
4.10 -0.007682059555892479 0.0 -0.00041030187886083926 0.0 0.0

Rerun opal and make the plot of the orbit again.

    cd output/first_run/
    ${OPAL_EXE_PATH}/opal FETS_Ring.opal
    cd ../../
    python3 scripts/plotting/plot_orbit.py output/first_run/FETS_Ring-trackOrbit.dat

and the orbiting particle can be seen to oscillate.

1.5 The Lattice file
--------------------
Open FETS_Ring.opal in a text editor. You will see that the file is about 200 
lines long:

* At the top of the file are some run controls, prefixed by "Option," and
  lattice parameters real variables, prefixed by "REAL".
* // delineates comments
* There are a few elements defined, starting at line 150:
** ringDef, which is a RINGDEFINITION element that holds global ring parameters.
** halfDrift, which is an OFFSET element that introduces an offset between 
   elements.
** magnetD, which is a SCALINGFFAMAGNET element
** magnetF, which is also a SCALINGFFAMAGNET element
* Below the field elements are some field output "elements" of type DUMPFIELDS 
  or DUMPEMFIELDS. You will see that one of them, on line 193, has a filename
  "FieldMapXY.dat". This generates the field map that was used for plotting in 
  the previous section.
* on line 207, there is element cell of type LINE. This holds the beam elements
  in a single cell (a drift, F magnet, D magnet, then another drift).
* on line 208, there is element ring of type LINE, which is the top level line.
** The RingDef must be placed first in the top level line
** A drift is placed to rotate the first element azimuthally (it is also 
   possible to use the RingDef LAT_PHIINIT to do the same thing).
** The 15 cells of the ring are placed.
** The probes are placed. Note probes are in global coordinates, unlike the 
   other elements. 

Try changing the input file. Try changing the FFA_B_MEAN parameter to 0.400 [T].
This will change the mean field on the reference radius, 4.0 metres in this 
example. Now run the simulation and make the plot.

    cd output/first_run/
    ${OPAL_EXE_PATH}/opal FETS_Ring.opal
    cd ../../
    python3 scripts/plotting/plot_orbit.py output/first_run/FETS_Ring-trackOrbit.dat

The orbiting particle is (almost) back on a closed orbit.

Try changing some other things in the input file:
* Remove a cell from the lattice.
* Change the fringe field length of one or both of the magnets by editing the 
  magnetF or magnetD element.
* Change the scale of the field map by editing the DUMPFIELDS command.

You can always recover the original files by copying from ffa-school/first_run


-----------------------------
2. Finding Lattice Parameters
-----------------------------
It is useful to be able to find lattice parameters based on the tracking 
routines. In this section, we will run a series of scripts to find the closed
orbit, tune and dynamic aperture of the ring.

The driver script is scripts/bin/run_one.py. This loads a user-defined
configuration and requests each sub-process to execute; first the script finds
the closed orbit, then the tune, and then the dynamic aperture.

Note that these scripts are included as a convenience; they are not part of the
OPAL core library.

2.1 Prerequisites
-------------------
For this section, knowledge of python, while not essential, may be useful. The
scripts listed below use python3 to drive OPAL.

2.2.1 Running the scripts
-------------------------
To have a go at running the scripts, try

    python3 scripts/bin/run_one.py scripts/config/config_base.py

Now take a cup of tea or coffee. You should get output starting with

    Using configuration module config.config_base
    Using config seed [4100.0, -7.29035424]

and after a couple of minutes, ending with

    Done find da

When the scripts run, they make a directory output/base which contains output
files.

2.2.2 Closed Orbit
------------------
The closed orbit finder uses an iterative process to find closed orbits. It
tracks the beam through a number of cells. Trajectories undergo approximate 
simple harmonic motion. By plotting the trajectory of each particle at PROBE
elements placed at the end of each cell, the trajectory of the tracks can be
studied. Under simple harmonic motion they make an ellipse in phase space, where
the centre of the ellipse is on the closed orbit. Because non-linearities in the
field make the motion not perfectly elliptical, an iterative approach is taken
where the centre of the best fit ellipse is used to seed the next 
iteration in the closed orbit finding.

The first ellipse and last ellipse is plotted in 

    output/base/find_closed_orbits

There are two files; 

* close_orbit-sub-index_0-i_0.png: this shows the phase space coordinates of the
  initial particle, read out at the end of each cell as the particle is tracked
  through one full turn of the ring.
* closed_orbit-sub-index_0-i_5.png: this shows the phase space coordinates of 
  the particle at the end of the closed orbit iterations. The closed orbit is
  found at an accuracy of a few micron and a few keV/c.

The closed orbit is stored in temporary cache output/base/find_closed_orbit.json
which is formatted in json text format.

2.2.3 Tune
----------
The closed orbit is loaded and used as the basis for the tune calculation.
Particles are displaced from the nominal closed orbit by 1 mm and tracked; the
resulting particle trajectories are written out and the elliptical trajectory of
particles is calculated. The elliptical trajectory is decomposed into 
action-angle coordinates by means of cholesky decomposition and the phase 
advance per cell is calculated.

There are four plots in

    output/base/find_tune

The horizontal phase space ellipse and cholesky decomposition is in

    output/base/find_tune/tune_0_x_phase-space.png
    output/base/find_tune/tune_0_x_cholesky-space.png

respectively. The vertical phase space ellipse and cholesky decomposition is in

    output/base/find_tune/tune_0_y_phase-space.png
    output/base/find_tune/tune_0_y_cholesky-space.png

respectively.

2.2.4 Dynamic Aperture
----------------------
The dynamic aperture (DA) is found by tracking trajectories displaced from the 
closed orbit; if the trajectory passes through at least 50 cells, it is 
considered to be within the DA. If the trajectory is lost before it passes 
through 50 cells, it is considered to be outside the dynamic aperture. A 
binomial search is performed to find the particle just on the edge of the DA.

There are four plots in

    output/base/find_da

The horizontal DA is found by displacing a particle in x. The resultant 
phase space trajectories are shown in

    output/base/find_da/da_0_x_da_x_x\'.png
    output/base/find_da/da_0_x_da_y_y\'.png

The first file shows the trajectory in x x' space. Particles inside the DA are
shown in lime, the highest amplitude track that remains in the DA is shown in
black, while trajectories that are outside the DA are shown in grey.

The second file shows the trajectory in y-y' space. As the particle starts on 
the midplane, the y-y' trajectory is always 0.

The vertical DA is found by displacing a particle in y. The resultant 
phase space trajectories are shown in

    output/base/find_da/da_0_y_da_y_y\'.png
    output/base/find_da/da_0_y_da_x_x\'.png

The first file shows the trajectory in y y' space, and a well formed DA is
observed. The second file shows the trajectory in x x' space. It is noted that
there is some coupling between x and y, so even though the particle was 
initially displaced only in the y direction, there is some motion in the x x'
plane.


2.3 Variable Scans
------------------
These scripts work by changing variables in the OPAL lattice file and the
distribution file. During each calculation, the temporary OPAL lattice file and
simulation is run from

    output/base/tmp/find_closed_orbits
    output/base/tmp/find_tune
    output/base/tmp/find_da

for the closed orbit, tune and DA calculation respectively. The file 

    lattice/FETS_Ring.in

is used as a source for the lattice file.

2.3.1 Config file
-----------------
The config file determines the basic parameters of the lattice. The config
file is written in the python programming language. Make a copy of the
config_base file

    cp scripts/config/config_base.py scripts/config/config_school.py

Open the file in a text editor. The first 30 lines show a list of lattice 
parameters. Each of the string keys like "__energy__" corresponds to one or more
strings in lattice/FETS_Ring.in; the key is substituted for a variable value
during run time.

To see how this works, try changing "__b_mean__" in
  
    scripts/config/config_school.py

Try setting, at line 25,

    "__b_mean__":0.400, # was 0.492

Note the syntax - this is an element in a python dictionary, so a comma is
required at the end of the line (before the # sign which marks a comment).

The Config object also contains a number of dictionaries (python hash tables)
that control different aspects of the operation of the code.

* self.find_closed_orbits controls the ClosedOrbitFinder
* self.find_tune controls the tune finder
* self.find_da controls the dynamic aperture finder
* self.substitution_list holds a list of dictionaries for scanning (see 
  section 3.3)
* self.run_control controls the overall execution of the script
* self.tracking controls the tracking

Before running, it may also be useful to disable the tune and DA calculation,
and to change the output directory. This can be achieved by adjusting the python 
dictionary self.run_control defined around line 80:

        #scripts/config/config_school.py line 80:
        self.run_control = {
            "find_closed_orbits":True,
            "find_tune":False, # <----------
            "find_da":False,  # <--------
            "output_dir":os.path.join(os.getcwd(), "output/school"), # <-----
            "clean_output_dir":False,
        }

Now run the example

    python3 scripts/bin/run_one.py scripts/config/config_school.py

You should see output starting with

    Using configuration module config.config_school

and ending with 

    [4113.41356318   -8.7774234 ]
    Done find closed orbits

which is the new closed orbit. The output ellipse plots are, as usual, in

    output/school/find_closed_orbits/

Note that, when changing parameters, the algorithm to find a closed orbit can
fail if the initial particle is outside of the DA of the lattice so that an
elliptical trajectory is never formed; this risk can be mitigated by only
making relatively small changes to the lattice file.

2.3.2 Scan
----------
The config file

    scripts/config/config_scan.py

has been set up to make a scan in d:f ratio. Note that this file uses 

    scripts/config/config_base.py

as a base. If you have changed config_base.py the script may not operate as
expected. 

This config file makes a copy of config_base.py using the python inheritance 
mechanism. The scripts have been set up to iterate over each of the elements in 
config_base.substitution_list, and run the closed orbit finder, tune finder and 
da finder as appropriate.

Run the scan by doing

    python3 scripts/bin/run_one.py scripts/config/config_scan.py

You should see output starting with

    Using configuration module config.config_scan
    Using config seed [4000.0, -7.29035424]

and, after a couple of minutes, ending with

    x_n_cells 15
    x_tune 0.2298781556869427
    x_tune_rms 0.05539687564488001
    y_n_cells 15
    y_tune 0.2527449400248146
    y_tune_rms 0.05320401315694613


The scan is set up to calculate the change in closed orbit and tune. The 
response of the tune to the df ratio can be plotted by doing

    python3 scripts/plotting/plot_tune.py output/scan/find_tune.json

This should produce a plot showing horizontal tune varying by about 2 % and
vertical tune varying by about 20 %, with the df ratio varied between -0.42 and
-0.30.

3. Conclusions
--------------
This has been a brief overview of using OPAL for tracking; and using some 
scripts to calculate lattice parameters and drive parameter scans.