Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/precommit dev #165

Merged
merged 2 commits into from
Dec 6, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .flake8
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[flake8]
ignore = E501, W503
max-line-length = 88
max-line-length = 79
max-complexity = 18
select = B,C,E,F,W,T4,B9,N
exclude =
Expand Down
14 changes: 7 additions & 7 deletions .github/workflows/deploy.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ jobs:
run: |
poetry run pyinstaller -c -F fair/cli.py --collect-all fair
mv dist/cli.exe dist/fair-cli-${{ steps.previoustag.outputs.tag }}-x64-windows-latest.exe

- uses: actions/upload-artifact@v1
with:
name: windows_faircli
Expand Down Expand Up @@ -72,12 +72,12 @@ jobs:
run: |
poetry run pyinstaller -c -F fair/cli.py --collect-all fair
mv dist/cli dist/fair-cli-${{ steps.previoustag.outputs.tag }}-x64-macos-latest

- uses: actions/upload-artifact@v1
with:
name: macos_faircli
path: dist/fair-cli-${{ steps.previoustag.outputs.tag }}-x64-macos-latest


ubuntu-py39:
name: Deploy Latest Linux Release
Expand Down Expand Up @@ -115,7 +115,7 @@ jobs:
name: ubuntu_faircli
path: dist/fair-cli-${{ steps.previoustag.outputs.tag }}-x64-ubuntu-latest


create-release:
name: Create Release
runs-on: ubuntu-latest
Expand All @@ -134,7 +134,7 @@ jobs:

- name: Install Poetry
run: python -m pip install poetry

- name: Install Module
run: python -m poetry install

Expand Down Expand Up @@ -169,7 +169,7 @@ jobs:
uses: actions/download-artifact@v1
with:
name: windows_faircli

- name: Retrieve macOS Executable
uses: actions/download-artifact@v1
with:
Expand All @@ -187,4 +187,4 @@ jobs:
dist/*.gz
macos_faircli/fair-cli-${{ steps.previoustag.outputs.tag }}-x64-*
windows_faircli/fair-cli-${{ steps.previoustag.outputs.tag }}-x64-*
ubuntu_faircli/fair-cli-${{ steps.previoustag.outputs.tag }}-x64-*
ubuntu_faircli/fair-cli-${{ steps.previoustag.outputs.tag }}-x64-*
1 change: 0 additions & 1 deletion .github/workflows/fair-cli.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -36,4 +36,3 @@ jobs:
files: ./coverage.xml
fail_ci_if_error: false
verbose: true

26 changes: 13 additions & 13 deletions .github/workflows/implementations.yml
Original file line number Diff line number Diff line change
Expand Up @@ -58,27 +58,27 @@ jobs:
- name: Build with Gradle
run: gradle build
working-directory: java_example

- name: install fair-cli
run: |
pip install poetry
poetry install

- name: run javaSimpleModel with fair cli
run: |
poetry run fair registry install
poetry run fair init --ci
poetry run fair pull src/main/resources/seirs-config.yaml --debug
poetry run fair run src/main/resources/seirs-config.yaml --debug
working-directory: java_example
working-directory: java_example

R:
runs-on: ubuntu-20.04
env:
GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
steps:
- uses: actions/checkout@v2

- name: Install memcached
run: sudo apt install -y libmemcached-dev

Expand All @@ -88,19 +88,19 @@ jobs:
repository: FAIRDataPipeline/rDataPipeline
path: r_data_pipeline
ref: 'main'

- name: Clone R Implementation
uses: actions/checkout@v2
with:
repository: FAIRDataPipeline/rSimpleModel
path: r_example
ref: 'main'
ref: 'main'

- uses: r-lib/actions/setup-r@v1
with:
r-version: release
use-public-rspm: true

- name: install remotes
run: |
install.packages("remotes")
Expand All @@ -120,8 +120,8 @@ jobs:
while read -r cmd
do
eval sudo $cmd
done < <(Rscript -e 'cat(remotes::system_requirements("ubuntu", "20.04"), sep = "\n")')
done < <(Rscript -e 'cat(remotes::system_requirements("ubuntu", "20.04"), sep = "\n")')

- name: Install implementation
run: |
install.packages("devtools")
Expand All @@ -142,7 +142,7 @@ jobs:
poetry run fair init --ci
poetry run fair pull inst/extdata/SEIRSconfig.yaml
poetry run fair run inst/extdata/SEIRSconfig.yaml
working-directory: r_example
working-directory: r_example

Julia:
runs-on: ubuntu-latest
Expand All @@ -164,12 +164,12 @@ jobs:

- name: Install Package
run: julia -e 'using Pkg; Pkg.instantiate()'

- name: install fair-cli
run: |
pip install poetry
poetry install

- name: Run SEIRS Model with fair cli
run: |
poetry run fair registry install
Expand Down
4 changes: 2 additions & 2 deletions .isort.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,6 @@ profile= black
multi_line_output = 3
include_trailing_comma = True
force_grid_wrap = 0
use_parentheses = False
use_parentheses = True
ensure_newline_before_comments = True
line_length = 88
line_length = 79
1 change: 1 addition & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ repos:
rev: 21.5b1
hooks:
- id: black
args: ["--line-length","79"]
- repo: https://gitlab.com/pycqa/flake8
rev: 3.9.2
hooks:
Expand Down
4 changes: 2 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,11 @@
# 2021-10-06 [v0.2.2](https://github.com/FAIRDataPipeline/FAIR-CLI/releases/tag/v0.2.2)
- Update to package metadata for PyPi

# 2021-10-06 [v0.2.1](https://github.com/FAIRDataPipeline/FAIR-CLI/releases/tag/v0.2.1)
# 2021-10-06 [v0.2.1](https://github.com/FAIRDataPipeline/FAIR-CLI/releases/tag/v0.2.1)
- Automatic starter `config.yaml` generation.
- Local and Global CLI configurations.
- Start/stop local registry either explicitly or during synchronisations.
- Run logs available via git-like interface.
- Added ability to add/remove files.
- Repository style handling, acts like another git-like tool per project.
- Creation of an interface for `fair` using `click`.
- Creation of an interface for `fair` using `click`.
4 changes: 2 additions & 2 deletions CITATION.cff
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
cff-version: 1.2.0
abstract: "Command Line Interface for the FAIR Data Pipeline system, this software provides commands necessary for integrating analysis and data processing into the FAIR registry."
authors:
authors:
- family-names: Zarebski
given-names: Kristian
affiliation: United Kingdom Atomic Energy Authority
Expand All @@ -18,7 +18,7 @@ authors:
affiliation: United Kingdom Atomic Energy Authority
orcid: https://orcid.org/0000-0003-4359-6337
date-released: 2021-09-13
keywords:
keywords:
- FAIR Data Pipeline
- FAIR
- Data Management
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ fair init --export
the created file can then be re-read at a later point during setup. Alternatively, if creating a configuration from scratch the YAML file should contain the following information:

```yaml
namespaces:
namespaces:
input: testing
output: testing
registries:
Expand Down
34 changes: 17 additions & 17 deletions docs/DEVELOPER_README.md
Original file line number Diff line number Diff line change
Expand Up @@ -167,19 +167,19 @@ Initialising FAIR repository, setup will now ask for basic info:

Checking for local registry
Local registry found
Remote Data Storage Root [http://data.scrc.uk/data/]:
Remote Data Storage Root [http://data.scrc.uk/data/]:
Remote API Token File: $HOME/scrc_token.txt
Local API URL [http://127.0.0.1:8000/api/]:
Local API URL [http://127.0.0.1:8000/api/]:
Local registry is offline, would you like to start it? [y/N]: y
Default Data Store: [/home/joebloggs/.fair/data]:
Default Data Store: [/home/joebloggs/.fair/data]:
Email: jbloggs@noreply.uk
ORCID [None]:
ORCID [None]:
Full Name: Joe Bloggs
Default input namespace [None]: SCRC
Default output namespace [jbloggs]:
Default output namespace [jbloggs]:
Project description: Test project
Local Git repository [/home/joebloggs/Documents/AnalysisProject]:
Git remote name [origin]:
Local Git repository [/home/joebloggs/Documents/AnalysisProject]:
Git remote name [origin]:
Using git repository remote 'origin': git@notagit.com:jbloggs/AnalysisProject.git
Initialised empty fair repository in /home/joebloggs/Documents/AnalysisProject/.fair
```
Expand All @@ -193,14 +193,14 @@ $ fair init
Initialising FAIR repository, setup will now ask for basic info:

Project description: Test Project
Local Git repository [/home/joebloggs/Documents/AnalysisProject]:
Git remote name [origin]:
Local Git repository [/home/joebloggs/Documents/AnalysisProject]:
Git remote name [origin]:
Using git repository remote 'origin': git@nogit.com:joebloggs/AnalysisProject.git
Remote API URL [http://data.scrc.uk/api/]:
Remote API Token File [/home/kristian/scrc_token.txt]:
Local API URL [http://127.0.0.1:8000/api/]:
Default output namespace [jbloggs]:
Default input namespace [SCRC]:
Remote API URL [http://data.scrc.uk/api/]:
Remote API Token File [/home/kristian/scrc_token.txt]:
Local API URL [http://127.0.0.1:8000/api/]:
Default output namespace [jbloggs]:
Default input namespace [SCRC]:
Initialised empty fair repository in /home/joebloggs/Documents/AnalysisProject/.fair
```

Expand All @@ -223,7 +223,7 @@ the user then only needs to add a `script` or `script_path` entry to execute a c
CLI configuration can be read directly from a file which should contain the following:

```yaml
namespaces:
namespaces:
input: testing
output: testing
registries:
Expand Down Expand Up @@ -336,7 +336,7 @@ register:
unique_name: Scottish deaths involving COVID19
file_type: csv
release_date: ${{DATETIME}}
version: 0.${{DATE}}.0
version: 0.${{DATE}}.0
primary: True
```

Expand Down Expand Up @@ -428,7 +428,7 @@ you do not need to specify the full SHA but rather the first few characters:

```text
--------------------------------
Commenced = Wed Jun 30 09:09:30 2021
Commenced = Wed Jun 30 09:09:30 2021
Author = Joe Bloggs <jbloggs@noreply.uk>
Namespace = jbloggs
Command = bash -eo pipefail /home/jbloggs/.fair/data/coderun/2021-06-30_09_09_30_721358/script.sh
Expand Down
1 change: 0 additions & 1 deletion docs/PROPOSAL.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,4 +72,3 @@ $ fair reset --hard

## Removing files
Again, alike to git have `fair rm` and `fair rm --cached`, to remove from tracking and the file system, or just tracking.

6 changes: 3 additions & 3 deletions docs/workflow.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading