Changes for 1.1.0 (#99)

fsoubelet · Nov 23, 2022 · 85b3ff5 · 85b3ff5
1 parent b6d6e46
commit 85b3ff5
Show file tree

Hide file tree

Showing 56 changed files with 4,093 additions and 2,457 deletions.
diff --git a/.github/workflows/coverage.yml b/.github/workflows/coverage.yml
@@ -33,6 +33,9 @@ jobs:
       - name: Install package with test dependencies
         run: python -m pip install ".[test]"
 
+      - name: Get acc-models-lhc
+        run: git clone https://gitlab.cern.ch/acc-models/acc-models-lhc.git --depth 1
+
       - name: Run all tests
         run: make alltests
 

diff --git a/.github/workflows/cron.yml b/.github/workflows/cron.yml
@@ -41,5 +41,8 @@ jobs:
       - name: Install package with test dependencies
         run: python -m pip install ".[test]"
 
+      - name: Get acc-models-lhc
+        run: git clone https://gitlab.cern.ch/acc-models/acc-models-lhc.git --depth 1
+
       - name: Run Tests
         run: make alltests
diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml
@@ -58,5 +58,8 @@ jobs:
       - name: Install package
         run: python -m pip install ".[test]"
 
+      - name: Get acc-models-lhc
+        run: git clone https://gitlab.cern.ch/acc-models/acc-models-lhc.git --depth 1
+
       - name: Run Tests
         run: make slowtests
diff --git a/.gitignore b/.gitignore
@@ -252,4 +252,6 @@ Temporary Items
 # Sphinx build directory and sphinx-gallery generated documents
 doc_build/
 docs/gallery/
-docs/gen_modules/
+docs/gen_modules/
+
+acc-models-lhc/
diff --git a/docs/api/cpymadtools.rst b/docs/api/cpymadtools.rst
@@ -8,34 +8,46 @@ The ``cpymadtools`` subpackage is a collection of utilities to conveniently hand
 .. automodule:: pyhdtoolkit.cpymadtools.constants
    :members:
 
-.. automodule:: pyhdtoolkit.cpymadtools.correctors
+.. automodule:: pyhdtoolkit.cpymadtools.coupling
    :members:
 
-.. automodule:: pyhdtoolkit.cpymadtools.coupling
+.. automodule:: pyhdtoolkit.cpymadtools.lhc
    :members:
 
-.. automodule:: pyhdtoolkit.cpymadtools.errors
+.. automodule:: pyhdtoolkit.cpymadtools.lhc._coupling
    :members:
 
-.. automodule:: pyhdtoolkit.cpymadtools.generators
+.. automodule:: pyhdtoolkit.cpymadtools.lhc._elements
    :members:
 
-.. automodule:: pyhdtoolkit.cpymadtools.lhc
+.. automodule:: pyhdtoolkit.cpymadtools.lhc._errors
    :members:
 
-.. automodule:: pyhdtoolkit.cpymadtools.matching
+.. automodule:: pyhdtoolkit.cpymadtools.lhc._misc
    :members:
 
-.. automodule:: pyhdtoolkit.cpymadtools.orbit
+.. automodule:: pyhdtoolkit.cpymadtools.lhc._powering
    :members:
 
-.. automodule:: pyhdtoolkit.cpymadtools.parameters
+.. automodule:: pyhdtoolkit.cpymadtools.lhc._queries
    :members:
 
-.. automodule:: pyhdtoolkit.cpymadtools.ptc
+.. automodule:: pyhdtoolkit.cpymadtools.lhc._routines
+   :members:
+
+.. automodule:: pyhdtoolkit.cpymadtools.lhc._setup
    :members:
 
-.. automodule:: pyhdtoolkit.cpymadtools.setup
+.. automodule:: pyhdtoolkit.cpymadtools.lhc._twiss
+   :members:
+
+.. automodule:: pyhdtoolkit.cpymadtools.matching
+   :members:
+
+.. automodule:: pyhdtoolkit.cpymadtools.parameters
+   :members:
+
+.. automodule:: pyhdtoolkit.cpymadtools.ptc
    :members:
 
 .. automodule:: pyhdtoolkit.cpymadtools.track

diff --git a/docs/quickstart.rst b/docs/quickstart.rst
@@ -93,7 +93,7 @@ Then using the `~pyhdtoolkit.cpymadtools` apis goes as:
 In the `~pyhdtoolkit.cpymadtools` one will find modules to:
 
 * Encompass existing ``MAD-X`` commands, such as for example :ref:`matching <cpymadtools-matching>` or :ref:`tracking <cpymadtools-track>`;
-* Perform useful routines with a clean pythonic interface (for instance :ref:`betatron coupling  <cpymadtools-coupling>` calculation and handling, :ref:`errors assignments <cpymadtools-errors>` or :ref:`table querying <cpymadtools-utils>`);
+* Perform useful routines with a clean pythonic interface (for instance :ref:`betatron coupling  <cpymadtools-coupling>` calculation and handling or :ref:`table querying <cpymadtools-utils>`);
 * Run :ref:`(HL)LHC <cpymadtools-lhc>` specific functionality, mostly tailored to my work. 
 
 One can find many examples of the `~pyhdtoolkit.cpymadtools` apis' use in the :ref:`gallery <gallery>` section of this documentation.

diff --git a/docs/release.rst b/docs/release.rst
@@ -3,6 +3,14 @@ Release Notes
 
 The full list of releases can be found in the GitHub repository's `releases page <https://github.com/fsoubelet/PyhDToolkit/releases>`_.
 
+Version 1.1.0
+-------------
+
+.. toctree::
+    :maxdepth: 2
+
+    releases/v1.1.0
+
 Version 1.0.0
 -------------
 

diff --git a/docs/releases/v1.1.0.rst b/docs/releases/v1.1.0.rst
@@ -0,0 +1,36 @@
+.. _release_1.1.0:
+
+1.1.0
+-----
+
+Release `1.1.0` brings some changes to match the adapted extracted API for `beamopticsanalysis/cpymadtools`.
+Code using ``pyhdtoolkit`` will need to be adapted.
+
+Breaking Changes
+~~~~~~~~~~~~~~~~
+
+* The `~pyhdtoolkit.cpymadtools.correctors`, `~pyhdtoolkit.cpymadtools.errors`, `~pyhdtoolkit.cpymadtools.orbit` and `~pyhdtoolkit.cpymadtools.setup` modules has been removed and their functionality included in `~pyhdtoolkit.cpymadtools.lhc`, as it was ``LHC``-specific.
+* The `~pyhdtoolkit.plotting.generators` module has been made private.
+
+Bug Fixes
+~~~~~~~~~
+
+* The `~pyhdtoolkit.cpymadtools.lhc.prepare_lhc_run2` function now properly calls the found sequence file.
+* The `~pyhdtoolkit.cpymadtools.utils.get_table_tfs` function now avoids using the `.dframe()` method of tables and converts them to a dict, then to a DataFrame to export. The previous implementation would sometimes crash on certain element names, or in cases of sliced sequences.
+
+Enhancements
+~~~~~~~~~~~~
+
+* The `~pyhdtoolkit.cpymadtools.lhc.do_kmodulation` function now accepts ``kwargs`` that will be passed down to the ``TWISS`` call.
+
+Maintenance
+~~~~~~~~~~~
+
+* Documentation has been updated for all new APIs. Refer to the gallery for example uses.
+* The `~pyhdtoolkit.cpymadtools._generators` being now private, they have been removed from the public API documentation.
+* Tests and internal API usehave been adapted.
+* Some style parameters in `~pyhdtoolkit.plotting.styles.paper` have been adapted.
+* The `~pyhdtoolkit.cpymadtools.lhc` module has been split up into many smaller *private* modules. The functionality of the APIs is unchanged, and they are still exposed at the `~pyhdtoolkit.cpymadtools.lhc` level.
+
+
+See `v1.1.0 release notes on GitHub <https://github.com/fsoubelet/PyhDToolkit/releases/tag/1.1.0>`_ and the `full changes since v1.0.0 <https://github.com/fsoubelet/PyhDToolkit/compare/1.0.0...1.1.0>`_.
diff --git a/examples/demo_acd_tracking_spectra.py b/examples/demo_acd_tracking_spectra.py
@@ -29,7 +29,7 @@
 from pyhdtoolkit.plotting.styles import _SPHINX_GALLERY_PARAMS
 from pyhdtoolkit.utils import logging
 
-logging.config_logger(level="warning")
+logging.config_logger(level="error")
 plt.rcParams.update(_SPHINX_GALLERY_PARAMS)  # for readability of this tutorial
 
 ###############################################################################
@@ -164,6 +164,6 @@
 #    The use of the following functions, methods, classes and modules is shown
 #    in this example:
 #
-#    - `~.cpymadtools.lhc`: `~.lhc.make_lhc_beams`, `~.lhc.re_cycle_sequence`, `~.lhc.make_lhc_thin`, `~.lhc.install_ac_dipole_as_kicker`
+#    - `~.cpymadtools.lhc`: `~.lhc._setup.make_lhc_beams`, `~.lhc._setup.re_cycle_sequence`, `~.lhc._setup.make_lhc_thin`, `~.lhc._elements.install_ac_dipole_as_kicker`
 #    - `~.cpymadtools.matching`: `~.matching.match_tunes_and_chromaticities`
 #    - `~.cpymadtools.track`: `~.track.track_single_particle`
diff --git a/examples/demo_aperture.py b/examples/demo_aperture.py
@@ -19,7 +19,7 @@
 from pyhdtoolkit.plotting.styles import _SPHINX_GALLERY_PARAMS
 from pyhdtoolkit.utils import logging
 
-logging.config_logger(level="warning")
+logging.config_logger(level="error")
 plt.rcParams.update(_SPHINX_GALLERY_PARAMS)  # for readability of this tutorial
 
 ###############################################################################
@@ -42,9 +42,7 @@
 madx.call("lhc/aper_tol.b1.madx")
 
 madx.command.twiss()
-madx.command.aperture(
-    cor=0.002, dp=8.6e-4, halo="{6,6,6,6}", bbeat=1.05, dparx=0.14, dpary=0.14
-)
+madx.command.aperture(cor=0.002, dp=8.6e-4, halo="{6,6,6,6}", bbeat=1.05, dparx=0.14, dpary=0.14)
 
 ###############################################################################
 # We can now determine the exact position of the IP5 point and plot the LHC
@@ -84,5 +82,5 @@
 #    The use of the following functions, methods, classes and modules is shown
 #    in this example:
 #
-#    - `~.cpymadtools.lhc`: `~.lhc.make_lhc_beams`
+#    - `~.cpymadtools.lhc`: `~.lhc._setup.make_lhc_beams`
 #    - `~.plotting.aperture`: `~.plotting.aperture.plot_aperture`
diff --git a/examples/demo_beam_enveloppe.py b/examples/demo_beam_enveloppe.py
@@ -22,7 +22,7 @@
 from pyhdtoolkit.plotting.styles import _SPHINX_GALLERY_PARAMS
 from pyhdtoolkit.utils import logging
 
-logging.config_logger(level="warning")
+logging.config_logger(level="error")
 plt.rcParams.update(_SPHINX_GALLERY_PARAMS)  # for readability of this tutorial
 
 ###############################################################################
@@ -159,9 +159,7 @@
     title=f"Vertical aperture at {beam_injection.pc_GeV} GeV/c",
     axis=axes[1],
 )
-plot_stay_clear(
-    madx, beam_injection, title=f"Stay-Clear at {beam_injection.pc_GeV} GeV/c", axis=axes[2]
-)
+plot_stay_clear(madx, beam_injection, title=f"Stay-Clear at {beam_injection.pc_GeV} GeV/c", axis=axes[2])
 plt.show()
 
 ###############################################################################

diff --git a/examples/demo_ir_errors.py b/examples/demo_ir_errors.py
@@ -6,7 +6,7 @@
 LHC IR Errors Assignments
 =========================
 
-This example shows how to use the `~.errors.misalign_lhc_ir_quadrupoles` function
+This example shows how to use the `~.lhc.misalign_lhc_ir_quadrupoles` function
 to assign magnet errors in the Insertion Region magnets of the LHC.
 
 .. warning::
@@ -19,11 +19,11 @@
 
 from cpymad.madx import Madx
 
-from pyhdtoolkit.cpymadtools import errors, lhc, matching
+from pyhdtoolkit.cpymadtools import lhc, matching
 from pyhdtoolkit.plotting.styles import _SPHINX_GALLERY_PARAMS
 from pyhdtoolkit.utils import logging
 
-logging.config_logger(level="warning")
+logging.config_logger(level="error")
 plt.rcParams.update(_SPHINX_GALLERY_PARAMS)  # for readability of this tutorial
 
 ###############################################################################
@@ -56,7 +56,7 @@
 # Here let's apply systematic horizontal misalignment errors and tilt errors to the
 # quadrupoles Q1 to Q6 (first to sixth) on both sides of IP1:
 
-errors.misalign_lhc_ir_quadrupoles(
+lhc.misalign_lhc_ir_quadrupoles(
     madx,
     ips=[1],
     beam=1,
@@ -139,6 +139,5 @@
 #    The use of the following functions, methods, classes and modules is shown
 #    in this example:
 #
-#    - `~.cpymadtools.errors`: `~.errors.misalign_lhc_ir_quadrupoles`
-#    - `~.cpymadtools.lhc`: `~.lhc.make_lhc_beams`
+#    - `~.cpymadtools.lhc`: `~.lhc._setup.make_lhc_beams`, `~.lhc._errors.misalign_lhc_ir_quadrupoles`
 #    - `~.cpymadtools.matching`: `~.matching.match_tunes_and_chromaticities`
diff --git a/examples/demo_lattice.py b/examples/demo_lattice.py
@@ -16,13 +16,13 @@
 
 from cpymad.madx import Madx
 
-from pyhdtoolkit.cpymadtools import lhc, matching, orbit
-from pyhdtoolkit.cpymadtools.generators import LatticeGenerator
+from pyhdtoolkit.cpymadtools import lhc, matching
+from pyhdtoolkit.cpymadtools._generators import LatticeGenerator
 from pyhdtoolkit.plotting.lattice import plot_latwiss
 from pyhdtoolkit.plotting.styles import _SPHINX_GALLERY_PARAMS
 from pyhdtoolkit.utils import logging
 
-logging.config_logger(level="warning")
+logging.config_logger(level="error")
 plt.rcParams.update(_SPHINX_GALLERY_PARAMS)  # for readability of this tutorial
 
 ###############################################################################
@@ -56,9 +56,7 @@
 title = rf"Base Lattice, $\mu_{{x, cell}}={mu_x_cell:.3f}, \ \mu_{{y, cell}}={mu_y_cell:.3f}$"
 
 plt.figure(figsize=(18, 11))
-plot_latwiss(
-    madx, title=title, k0l_lim=(-0.15, 0.15), k1l_lim=(-0.08, 0.08), disp_ylim=(-10, 125), lw=3
-)
+plot_latwiss(madx, title=title, k0l_lim=(-0.15, 0.15), k1l_lim=(-0.08, 0.08), disp_ylim=(-10, 125), lw=3)
 plt.tight_layout()
 plt.show()
 
@@ -80,7 +78,7 @@
 
 lhc.re_cycle_sequence(lhc_madx, sequence="lhcb1", start="IP3")
 lhc.re_cycle_sequence(lhc_madx, sequence="lhcb2", start="IP3")
-orbit_scheme = orbit.setup_lhc_orbit(lhc_madx, scheme="flat")
+orbit_scheme = lhc.setup_lhc_orbit(lhc_madx, scheme="flat")
 
 lhc.make_lhc_beams(lhc_madx, energy=7000)
 lhc_madx.command.use(sequence="lhcb1")
@@ -179,8 +177,7 @@
 #    The use of the following functions, methods, classes and modules is shown
 #    in this example:
 #
-#    - `~.cpymadtools.lhc`: `~.lhc.make_lhc_beams`, `~.lhc.re_cycle_sequence`
+#    - `~.cpymadtools.lhc`: `~.lhc._setup.make_lhc_beams`, `~.lhc._setup.re_cycle_sequence`, `~.lhc._setup.setup_lhc_orbit`
 #    - `~.cpymadtools.generators`: `~.generators.LatticeGenerator`
 #    - `~.cpymadtools.matching`: `~.matching.match_tunes_and_chromaticities`
-#    - `~.cpymadtools.orbit`: `~.orbit.setup_lhc_orbit`
 #    - `~.plotting.lattice`: `~.plotting.lattice.plot_latwiss`
diff --git a/examples/demo_lhc_crossing_schemes.py b/examples/demo_lhc_crossing_schemes.py
@@ -21,7 +21,7 @@
 from pyhdtoolkit.plotting.styles import _SPHINX_GALLERY_PARAMS
 from pyhdtoolkit.utils import logging
 
-logging.config_logger(level="warning")
+logging.config_logger(level="error")
 plt.rcParams.update(_SPHINX_GALLERY_PARAMS)  # for readability of this tutorial
 
 
@@ -68,5 +68,5 @@
 #    The use of the following functions, methods, classes and modules is shown
 #    in this example:
 #
-#    - `~.cpymadtools.lhc`: `~.lhc.make_lhc_beams`, `~.lhc.re_cycle_sequence`
+#    - `~.cpymadtools.lhc`: `~.lhc._setup.make_lhc_beams`, `~.lhc._setup.re_cycle_sequence`
 #    - `~.plotting.crossing`: `~.plotting.crossing.plot_two_lhc_ips_crossings`
diff --git a/examples/demo_lhc_rigid_waist_shift.py b/examples/demo_lhc_rigid_waist_shift.py
@@ -31,7 +31,7 @@
 from pyhdtoolkit.plotting.styles import _SPHINX_GALLERY_PARAMS
 from pyhdtoolkit.utils import logging
 
-logging.config_logger(level="warning")
+logging.config_logger(level="error")
 plt.rcParams.update(_SPHINX_GALLERY_PARAMS)  # for readability of this tutorial
 
 ###############################################################################
@@ -184,9 +184,7 @@
     lhc.re_cycle_sequence(madx, sequence=f"lhcb1", start=f"MSIA.EXIT.B1")
     madx.command.use(sequence=f"lhcb1")
     lhc.make_lhc_thin(madx, sequence=f"lhcb1", slicefactor=4)
-    lhc.add_markers_around_lhc_ip(
-        madx, sequence=f"lhcb1", ip=1, n_markers=1000, interval=0.001
-    )
+    lhc.add_markers_around_lhc_ip(madx, sequence=f"lhcb1", ip=1, n_markers=1000, interval=0.001)
     madx.command.twiss()
     initial_twiss = madx.table.twiss.dframe().copy()
 
@@ -287,25 +285,19 @@
 # Manipulating the equation to determine the waist yields:
 # :math:`w = L^{*} - \sqrt{\beta_0 \beta_w - \beta_w^2}`
 
-q1_right_s = twiss_df[twiss_df.name.str.contains(f"mqxa.1r1")].s[
-    0
-]  # to calculate from the right Q1
-q1_left_s = twiss_df[twiss_df.name.str.contains(f"mqxa.1l1")].s[
-    -1
-]  # to calculate from the left Q1
+q1_right_s = twiss_df[twiss_df.name.str.contains(f"mqxa.1r1")].s[0]  # to calculate from the right Q1
+q1_left_s = twiss_df[twiss_df.name.str.contains(f"mqxa.1l1")].s[-1]  # to calculate from the left Q1
 
 L_star = ip_s - q1_left_s  # we calculate from left Q1
 # beta0 = twiss_df[twiss_df.name.str.contains(f"mqxa.1r1")].betx[0]  # to calculate from the right
-beta0 = twiss_df[twiss_df.name.str.contains(f"mqxa.1l1")].betx[
-    -1
-]  # to calculate from the left
+beta0 = twiss_df[twiss_df.name.str.contains(f"mqxa.1l1")].betx[-1]  # to calculate from the left
 betaw = around_ip.betx.min()
 
 ###############################################################################
 # The analytical result (sign will swap depending on if we calculate from left
 # or right Q1) is then easily calculated. We can then compare this value to the
 # one found with the markers we previously added, and they are fairly close.
-waist = L_star - np.sqrt(beta0 * betaw - betaw**2)
+waist = L_star - np.sqrt(beta0 * betaw - betaw ** 2)
 print(f"Analytical: {waist}")
 print(f"Markers: {shift}")
 
@@ -316,6 +308,6 @@
 #    The use of the following functions, methods, classes and modules is shown
 #    in this example:
 #
-#    - `~.cpymadtools.lhc`: `~.lhc.make_lhc_beams`, `~.lhc.re_cycle_sequence`, `~.lhc.apply_lhc_rigidity_waist_shift_knob`, `~.lhc.add_markers_around_lhc_ip`
+#    - `~.cpymadtools.lhc`: `~.lhc._setup.make_lhc_beams`, `~.lhc._setup.re_cycle_sequence`, `~.lhc._powering.apply_lhc_rigidity_waist_shift_knob`, `~.lhc._elements.add_markers_around_lhc_ip`
 #    - `~.cpymadtools.matching`: `~.matching.match_tunes`, `~.matching.match_chromaticities`, `~.matching.match_tunes_and_chromaticities`
 #    - `~.plotting.lattice`: `~.plotting.lattice.plot_latwiss`