Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .flake8
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[flake8]
max-line-length = 120
extend-ignore = E203
82 changes: 82 additions & 0 deletions .github/workflows/mega-linter.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
---
# MegaLinter GitHub Action configuration file
# More info at https://megalinter.github.io
name: MegaLinter

'on': [pull_request_target]

env: # Comment env block if you do not want to apply fixes
# Apply linter fixes configuration
APPLY_FIXES: all # When active, APPLY_FIXES must also be defined as environment variable (in github/workflows/mega-linter.yml or other CI tool)
PR_NUMBER: ${{ github.event.pull_request.number }}

concurrency:
group: ${{ github.ref }}-${{ github.workflow }}
cancel-in-progress: true

jobs:
build:
name: MegaLinter
runs-on: ubuntu-latest
steps:
# Git Checkout
- name: Checkout Code
uses: actions/checkout@v3
with:
fetch-depth: 0

# MegaLinter
- name: MegaLinter
id: ml
# You can override MegaLinter flavor used to have faster performances
# More info at https://megalinter.github.io/flavors/
uses: megalinter/megalinter@v5
env:
# All available variables are described in documentation
# https://megalinter.github.io/configuration/
VALIDATE_ALL_CODEBASE: false # ${{ github.event_name == 'push' && github.ref == 'refs/heads/master' }} # Validates all source when push on master, else just the git diff with master. Override with true if you always want to lint all sources
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# ADD YOUR CUSTOM ENV VARIABLES HERE OR DEFINE THEM IN A FILE .mega-linter.yml AT THE ROOT OF YOUR REPOSITORY
# DISABLE: COPYPASTE,SPELL # Uncomment to disable copy-paste and spell checks

# Create or delete the cleanup branch
- name: Update cleanup branch
if: ${{ github.event.repository.owner.login == 'AliceO2Group' }}
env:
REMOTE_URL: "https://alibuild:${{ secrets.ALIBUILD_GITHUB_TOKEN }}@\
github.com/alibuild/${{ github.event.repository.name }}"
run: |
# An empty CLEANUP_COMMIT means delete the branch.
CLEANUP_COMMIT=""
if [ "${{ steps.ml.outputs.has_updated_sources }}" = 1 ]
then
CLEANUP_COMMIT="HEAD"
git commit -am "MegaLinter fixes"
fi
git push -f "$REMOTE_URL" "$CLEANUP_COMMIT:refs/heads/alibot-cleanup-ml-$PR_NUMBER"

- name: Create pull request with applied fixes
uses: alisw/pull-request@v2
if: steps.ml.outputs.has_updated_sources == 1 && ${{ github.event.repository.owner.login == 'AliceO2Group' }}
with:
source_branch: 'alibuild:alibot-cleanup-ml-${{ env.PR_NUMBER }}'
destination_branch: '${{ github.event.pull_request.head.label }}'
github_token: ${{ secrets.ALIBUILD_GITHUB_TOKEN }}
pr_title: >-
[MegaLinter] Apply linters automatic fixes to
#${{ env.PR_NUMBER }}
pr_body: >-
Your PR ${{ github.event.repository.full_name }}#${{ env.PR_NUMBER }}
cannot be merged as is. You should either run MegaLinter
yourself and update the pull request, or merge this PR in yours.

You can find how to run MegaLinter locally at
<https://megalinter.github.io/latest/mega-linter-runner/>.
continue-on-error: true # We do not create PRs if the branch is not there.

- name: Exit with error if the PR is not clean
run: |
case "${{ steps.ml.outputs.has_updated_sources }}" in
0) echo 'PR clean' ; exit 0 ;;
1) echo 'PR not clean' ; exit 1 ;;
esac
Comment on lines +77 to +82
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is the case handled where MegaLinter finds problems, but does not have automatic fixes for them? In that case, we still want to exit with an error, right?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, in case of errors, the MegaLinter step fails.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In that case, the automatic PR with any fixes that MegaLinter does produce wouldn't be created, right?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is correct. The formatting PR should be created only if MegaLinter did not find any errors, which I think is the intended behaviour, right?

17 changes: 9 additions & 8 deletions .github/workflows/stale.yml
Original file line number Diff line number Diff line change
@@ -1,16 +1,17 @@
---
name: "Close stale PRs"
on:
schedule:
- cron: "0 0 * * *"
- cron: "0 0 * * *"

jobs:
stale:
runs-on: ubuntu-latest
steps:
- uses: actions/stale@v1
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
stale-pr-message: 'This PR has not been updated in the last 30 days. Is it still needed? Unless further action is taken, it will be closed in 5 days.'
stale-pr-label: stale
days-before-stale: 30
days-before-close: 5
- uses: actions/stale@v1
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
stale-pr-message: 'This PR has not been updated in the last 30 days. Is it still needed? Unless further action is taken, it will be closed in 5 days.'
stale-pr-label: stale
days-before-stale: 30
days-before-close: 5
24 changes: 24 additions & 0 deletions .mega-linter.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
---
# Configuration file for Mega-Linter
# See all available variables at https://megalinter.github.io/configuration/ and in linters documentation

APPLY_FIXES: all # all, none, or list of linter keys
DEFAULT_BRANCH: master # Usually master or main
# ENABLE: # If you use ENABLE variable, all other languages/formats/tooling-formats will be disabled by default
# ENABLE_LINTERS: # If you use ENABLE_LINTERS variable, all other linters will be disabled by default
DISABLE:
- C
- COPYPASTE # abusive copy-pastes
- SPELL # spelling mistakes
DISABLE_LINTERS:
- CPP_CPPLINT
- BASH_EXEC
- BASH_SHFMT
- JSON_PRETTIER
- YAML_V8R
- YAML_PRETTIER
SHOW_ELAPSED_TIME: true
FILEIO_REPORTER: false
GITHUB_COMMENT_REPORTER: false
UPDATED_SOURCES_REPORTER: false
PRINT_ALPACA: false # Don't print ASCII alpaca in the log
6 changes: 3 additions & 3 deletions EventFiltering/PWGUD/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,14 @@ Checks for diffractive events and fills table `DiffractionFilters`. The table is

The diffractive event filter is contained in the following set of files:

### cutHolder.cxx, cutHolder.h
## cutHolder.cxx, cutHolder.h
`cutHolder` is a buffer for the various cut parameters. Is used to create a configurable which values can be set by command line options.

### diffHelpers.h
## diffHelpers.h
Contains helper functions for the diffractive event selection and the actual filter logics for the various types of events. Currently implemented are:

- DGSelector: a filter for Double Gap (DG) events

### diffractionFilter.cxx
## diffractionFilter.cxx
Contains the actual filter tasks. Currently implemented are:
- DGFilterRun3: filter task for DG events in Run 3 data
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
# O2Physics

Documentation for the ALICE Analysis framework is available at:

https://aliceo2group.github.io/analysis-framework/
<https://aliceo2group.github.io/analysis-framework/>
109 changes: 71 additions & 38 deletions Scripts/update_ccdb.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,27 +22,35 @@
TODO add support for 3 files update
"""

import argparse
import subprocess
from datetime import datetime

import matplotlib.pyplot as plt
import argparse


def convert_timestamp(ts):
"""
Converts the timestamp in milliseconds in human readable format
"""
return datetime.utcfromtimestamp(ts/1000).strftime('%Y-%m-%d %H:%M:%S')
return datetime.utcfromtimestamp(ts / 1000).strftime("%Y-%m-%d %H:%M:%S")


def get_ccdb_obj(path, timestamp, dest="/tmp/", verbose=0):
"""
Gets the ccdb object from 'path' and 'timestamp' and downloads it into 'dest'
"""
if verbose:
print("Getting obj", path, "with timestamp",
timestamp, convert_timestamp(timestamp))
cmd = f"o2-ccdb-downloadccdbfile --path {path} --dest {dest} --timestamp {timestamp}"
print(
"Getting obj",
path,
"with timestamp",
timestamp,
convert_timestamp(timestamp),
)
cmd = (
f"o2-ccdb-downloadccdbfile --path {path} --dest {dest} --timestamp {timestamp}"
)
subprocess.run(cmd.split())


Expand All @@ -61,7 +69,7 @@ def get_ccdb_obj_validity(path, dest="/tmp/", verbose=0):
print(*output, "\n")
print("err:")
print(error)
result = list(filter(lambda x: x.startswith('Valid-'), output))
result = list(filter(lambda x: x.startswith("Valid-"), output))
ValidFrom = result[0].split()
ValidUntil = result[1].split()
return [int(ValidFrom[-1]), int(ValidUntil[-1])]
Expand All @@ -71,12 +79,18 @@ def upload_ccdb_obj(path, timestamp_from, timestamp_until, dest="/tmp/", meta=""
"""
Uploads a new object to CCDB in the 'path' using the validity timestamp specified
"""
print("Uploading obj", path, "with timestamp", [timestamp_from, timestamp_until],
convert_timestamp(timestamp_from), convert_timestamp(timestamp_until))
print(
"Uploading obj",
path,
"with timestamp",
[timestamp_from, timestamp_until],
convert_timestamp(timestamp_from),
convert_timestamp(timestamp_until),
)
key = path.split("/")[-1]
cmd = f"o2-ccdb-upload -f {dest}{path}/snapshot.root "
cmd += f"--key {key} --path {path} "
cmd += f"--starttimestamp {timestamp_from} --endtimestamp {timestamp_until} --meta \"{meta}\""
cmd += f'--starttimestamp {timestamp_from} --endtimestamp {timestamp_until} --meta "{meta}"'
subprocess.run(cmd.split())


Expand All @@ -87,9 +101,9 @@ def main(path, timestamp_from, timestamp_until, verbose=0, show=False):
Gets the object from CCDB specified in 'path' and for 'timestamp_until+1'
If required plots the situation before and after the update
"""
get_ccdb_obj(path, timestamp_from-1)
get_ccdb_obj(path, timestamp_from - 1)
val_before = get_ccdb_obj_validity(path, verbose=verbose)
get_ccdb_obj(path, timestamp_until+1)
get_ccdb_obj(path, timestamp_until + 1)
val_after = get_ccdb_obj_validity(path, verbose=verbose)
overlap_before = val_before[1] > timestamp_from
overlap_after = val_after[0] < timestamp_until
Expand All @@ -98,48 +112,67 @@ def main(path, timestamp_from, timestamp_until, verbose=0, show=False):
print("Previous objects overalps")
if overlap_after:
print("Next objects overalps")
trimmed_before = val_before if not overlap_before else [
val_before[0], timestamp_from - 1]
trimmed_after = val_after if not overlap_after else [
timestamp_until+1, val_after[1]]
trimmed_before = (
val_before if not overlap_before else [val_before[0], timestamp_from - 1]
)
trimmed_after = (
val_after if not overlap_after else [timestamp_until + 1, val_after[1]]
)
if show:
fig, ax = plt.subplots()
fig

def bef_af(v, y):
return [v[0] - 1] + v + [v[1] + 1], [0, y, y, 0]

if True:
ax.plot(*bef_af(val_before, 0.95), label='before')
ax.plot(*bef_af(val_after, 1.05), label='after')
ax.plot(*bef_af(val_before, 0.95), label="before")
ax.plot(*bef_af(val_after, 1.05), label="after")
if False:
ax.plot(*bef_af(trimmed_before, 0.9), label='trimmed before')
ax.plot(*bef_af(trimmed_after, 1.1), label='trimmed after')
ax.plot(*bef_af([timestamp_from, timestamp_until], 1), label='object')
ax.plot(*bef_af(trimmed_before, 0.9), label="trimmed before")
ax.plot(*bef_af(trimmed_after, 1.1), label="trimmed after")
ax.plot(*bef_af([timestamp_from, timestamp_until], 1), label="object")
xlim = 10000000
plt.xlim([timestamp_from-xlim, timestamp_until+xlim])
plt.xlim([timestamp_from - xlim, timestamp_until + xlim])
plt.ylim(0, 2)
plt.xlabel('Timestamp')
plt.ylabel('Validity')
plt.xlabel("Timestamp")
plt.ylabel("Validity")
plt.legend()
plt.show()


if __name__ == "__main__":
parser = argparse.ArgumentParser(
description="Uploads timestamp non overlapping objects to CCDB."
"Basic example: `./update_ccdb.py qc/TOF/TOFTaskCompressed/hDiagnostic 1588956517161 1588986517161 --show --verbose`")
parser.add_argument('path', metavar='path_to_object', type=str,
help='Path of the object in the CCDB repository')
parser.add_argument('timestamp_from', metavar='from_timestamp', type=int,
help='Timestamp of start for the new object to use')
parser.add_argument('timestamp_until', metavar='until_timestamp', type=int,
help='Timestamp of stop for the new object to use')
parser.add_argument('--verbose', '-v', action='count', default=0)
parser.add_argument('--show', '-s', action='count', default=0)
description="Uploads timestamp non overlapping objects to CCDB. "
"Basic example: `./update_ccdb.py qc/TOF/TOFTaskCompressed/hDiagnostic "
"1588956517161 1588986517161 --show --verbose`"
)
parser.add_argument(
"path",
metavar="path_to_object",
type=str,
help="Path of the object in the CCDB repository",
)
parser.add_argument(
"timestamp_from",
metavar="from_timestamp",
type=int,
help="Timestamp of start for the new object to use",
)
parser.add_argument(
"timestamp_until",
metavar="until_timestamp",
type=int,
help="Timestamp of stop for the new object to use",
)
parser.add_argument("--verbose", "-v", action="count", default=0)
parser.add_argument("--show", "-s", action="count", default=0)

args = parser.parse_args()
main(path=args.path,
timestamp_from=args.timestamp_from,
timestamp_until=args.timestamp_until,
verbose=args.verbose,
show=args.show)
main(
path=args.path,
timestamp_from=args.timestamp_from,
timestamp_until=args.timestamp_until,
verbose=args.verbose,
show=args.show,
)