Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 12 additions & 5 deletions .github/ISSUE_TEMPLATE/i-t-bug-report.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,19 +7,26 @@ assignees: jordanpadams

---

**Describe the bug**
## Describe the bug

A clear and concise description of what the bug is.

**Expected behavior**

## Expected behavior

A clear and concise description of what you expected to happen.

**Test Data**

## Test Data

Attach test data here

**Related Test Case(s)**
## Related Test Case(s)

Document related test cases here

**Related issues**
## Related issues

Reference relates issues here, e.g.
* for issues in this repo: `#1` (remove back ticks)
* for issues in other repos: NASA-PDS/my_repo#1, NASA-PDS/her_repo#2
4 changes: 2 additions & 2 deletions .github/workflows/unstable-cicd.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# =================================================
#
# Note: for this workflow to succeed, the following secrets must be installed
# in the repository:
# in the repository (or inherited from the repository's organization):
#
# ``ADMIN_GITHUB_TOKEN``
# A personal access token of a user with collaborator or better access to
Expand Down Expand Up @@ -51,7 +51,7 @@ jobs:
lfs: true
-
name: 🤠 Roundup
uses: NASA-PDS/roundup-action@master
uses: NASA-PDS/roundup-action@dev
with:
assembly: unstable
env:
Expand Down
39 changes: 32 additions & 7 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,15 +1,23 @@
# Python build, virtual environments, and buildouts
venv
__pycache__/
dist/
build/
.idea/
*.sublime-workspace
*.egg-info
.*.cfg
develop-eggs/
.python-eggs/
.eggs/
pip-selfcheck.json

# Python testing artifacts
.coverage
htmlcov

# Object files
*.o
*.pkl
*.pyc
*.py[ocd]

# Libraries
*.lib
Expand All @@ -20,15 +28,20 @@ build/
*.project
*.classpath

# IntelliJ files
# Editor support
.idea/
*.iml
.vscode
*.sublime-*


# Shared objects (inc. Windows DLLs)
*.dll
*.so
*.so.*
*.dylib
lib/
lib64/

# Executables
*.exe
Expand All @@ -37,10 +50,22 @@ build/

# Temporary files
*~
.*.swp
var/

# other stuff
*.log
*.xpr
*.swp
/bin/
bin/
.*.swp
*.sublime-*

# OS-specific artifacts
.DS_Store
._*

# Exclusions
!.coveragerc
!.editorconfig
!.gitattributes
!.gitignore
!.gitkeep
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
# Change Log

Use some automated software like [Github Changelog Generator](https://github.com/github-changelog-generator/github-changelog-generator) to update this CHANGELOG.
The [Roundup Action](https://github.com/NASA-PDS/roundup-action) takes care of updating the change log automatically.
5 changes: 3 additions & 2 deletions MANIFEST.in
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
graft src
include versioneer.py
include my_pds_module/_version.py
global-exclude *.py[cod]

# Include package-level files and directories here; e.g.:
#
# include *.png *.jpeg
# exclude *.webp
# graft spice-kernels
# prune my_pds_module/obsolete-dir
# global-exclude *.py[cod]
# global-exclude *.log
2 changes: 1 addition & 1 deletion NOTICE.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
PDS XYZ Tool (update licensing below per specific organization)

Copyright 2020, California Institute of Technology ("Caltech").
Copyright 2021, California Institute of Technology ("Caltech").
U.S. Government sponsorship acknowledged.

All rights reserved.
Expand Down
129 changes: 61 additions & 68 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,59 +1,49 @@
# pds-template-python
Template repository for PDS python developments.
# pds-template-repo-python

This repository aims at being a base for new python repository used in PDS.
This is the template repository for PDS's Python projects.

It guides developers to ease the initialization of a project and recommends preferred options to standardize developments and ease maintenance.
This repository aims at being a base for new python repositories used in PDS. It guides developers to ease the initialization of a project and recommends preferred options to standardize developments and ease maintenance. Simply click the <kbd>Use this template</kbd> button ↑ (or use [this hyperlink](https://github.com/NASA-PDS/pds-template-repo-python/generate)).

## Prerequisites

Any system wide requirements (brew install, apt-get install or yum install ...)
## Prerequisites

**Python3** should be used.
Include any system-wide requirements (`brew install`, `apt-get install`, `yum install`, …) **Python 3** should be used regardless as [Python 2 reached end-of-life on January 1st, 2020](https://pythonclock.org/).


## User quickstart
## User Quickstart

Install
Install with:

pip install my_pds_module

Configure
If possible, make it so that your program works out of the box without any additional configuration—but see the [Configuration](###configuration) section for details.

Update local configuration files is relevant. Ideally a default configuration should work (see [Configuration](###configuration) for detail).
To execute, run:

Use (command line or web service url):
(put your run commands here)

...

## Development

## Developers

**PyCharm** IDE is useful for complex development project.
To develop this project, use your favorite text editor, or an integrated development environment with Python support, such as [PyCharm](https://www.jetbrains.com/pycharm/).


### Packaging

To isolate and be able to re-produce the environment for the project, we use virtualenv:
To isolate and be able to re-produce the environment for this package, you should use a [Python Virtual Environment](https://docs.python.org/3/tutorial/venv.html). To do so, run:

python -m venv venv
source venv/bin/activate


Dependencies for development are stored in file requirements.txt, they are installed in the virtualenv as follow:

pip install -r requirements.txt


Use setup tools to package your code:
Then exclusively use `venv/bin/python`, `venv/bin/pip`, etc. (It is no longer recommended to use `venv/bin/activate`.)

pip install setuptools
Dependencies for development are stored in requirements.txt; they are installed into the virtual environment as follows:

venv/bin/pip install --requirement requirements.txt

All the source code is in a sub-directory named after the developed module, for example my_pds_module.
If the project is complex, we might have different sub-modules in this directory.
All the source code is in a sub-directory under `src`.

You should update the `setup.py` file with:

Update the setup.py file:
- name of your module
- version
- license, default apache, update if needed
Expand All @@ -64,96 +54,99 @@ Update the setup.py file:
- install_requires, add the dependencies of you module
- entry_points, when your module can be called in command line, this helps to deploy command lines entry points pointing on scripts in your module

You can update the setup.cfg file which describes how setup.py can be called, with which directives and options. This is useful for pypi publication.
Eventually, we should move to putting everything into `setup.cfg`, as having package metadata in `setup.py` is passé.

For the packaging details, see https://packaging.python.org/tutorials/packaging-projects/ as a reference.


### Configuration

It is convenient to use ConfigParser package to manage configuration.
It allows to have a default configuration which can be overwritten by the user in a specific file in their environment.
See https://pymotw.com/2/ConfigParser/
It is convenient to use ConfigParser package to manage configuration. It allows a default configuration which can be overwritten by the user in a specific file in their environment. See https://pymotw.com/2/ConfigParser/

For example:

candidates = ['my_pds_module.ini',
'my_pds_module.ini.default']
candidates = ['my_pds_module.ini', 'my_pds_module.ini.default']
found = parser.read(candidates)


### Logs

You should not use `print()`vin the purpose of logging information on the execution of your code. Depending on where the code runs these information could be redirected to specific log files.

To make that work, have a the beginning of your python file:
To make that work, start each Python file with:

```python
import logging

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
```

To log a message:

logger.info("my message")

In your `main` routine, include:

logging.basicConfig(level=logging.INFO)

to get a basic logging system configured.


### Code style
### Code Style

So that your code is readable, you should comply with the [PEP8 style guide](https://www.python.org/dev/peps/pep-0008/). It is automatically enforced in PyCharm IDE.

So that your code is readable, you must comply with the PEP8 style guide, see https://www.python.org/dev/peps/pep-0008/
Note that several PEP8 guidelines are rather old fashioned; trust your judgment.

It is automatically enforced in PyCharm IDE.

### Recommended libraries
### Recommended Libraries

Python offers a large variety of libraries. In PDS scope, for the most current usage we should use:

| Library | Usage |
|------------|-----------------------------|
| configparser | manage and parse configuration files |
| argparse | command line argument documentation and parsing |
| requests | interact with web APIs |
| lxml | read/write XML files |
| json | read/write JSON files |
| pyyaml | read/write YAML files |
| pystache | generate files from templates |
| Library | Usage |
|--------------|------------------------------------------------ |
| configparser | manage and parse configuration files |
| argparse | command line argument documentation and parsing |
| requests | interact with web APIs |
| lxml | read/write XML files |
| json | read/write JSON files |
| pyyaml | read/write YAML files |
| pystache | generate files from templates |

Some of these are built into Python 3; others are open source add-ons you can include in your `requirements.txt`.


### Tests

This section describes testing for your package.


#### Unit tests

Your project should have built-in unit tests and validation tests.
Your project should have built-in unit tests, functional, validation, acceptance, etc., tests.

The package to be used for unit testing is unittest, see https://docs.python.org/3/library/unittest.html
For unit testing, check out the [unittest](https://docs.python.org/3/library/unittest.html) module, built into Python 3.

Tests objects must be in packages 'test' subdirectories or preferably in project 'tests' directory which mirrors the project package structure.
Tests objects should be in packages `test` modules or preferably in project 'tests' directory which mirrors the project package structure.

Unit tests are launched with command:

python setup.py test

#### Integration/behavioral tests

One shoud use the `behave package` and push the test results to testrail.
#### Integration/Behavioral Tests

See example in https://github.com/NASA-PDS/pds-doi-service#behavioral-testing
One should use the `behave package` and push the test results to "testrail".

## Build
See an example in https://github.com/NASA-PDS/pds-doi-service#behavioral-testing-for-integration--testing

pip install wheel
python sdist bdist_wheel

### Publication

You can publish your module on PyPi (you need a pypi account):

pip install twine
twine upload dist/*

You can also use github actions, see example provided in `.github/workflows/publish.yml.example`

## Build

pip install wheel
python setup.py sdist bdist_wheel


### Publication

NASA PDS packages can publish automatically using the [Roundup Action](https://github.com/NASA-PDS/roundup-action), which leverages GitHub Actions to perform automated continuous integration and continuous delivery. A default workflow that includes the Roundup is provided in the `.github/workflows/unstable-cicd.yaml` file. (Unstable here means an interim release.)
16 changes: 10 additions & 6 deletions SECURITY.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,20 @@
# Security Policy

This document describes the security policy of this package.


## Supported Versions

Use this section to tell people about which versions of your project are
currently being supported with security updates.
Use this section to tell people about which versions of your project are currently being supported with security updates.

| Version | Supported |
| ------- | ------------------ |
| Version | Supported |
| -------- | ------------------ |
| 1.18.2 | :white_check_mark: |
| 1.17.5 | :white_check_mark: |
| < 1.17.5 | :x: |
| < 1.17.5 | :x: |


## Reporting a Vulnerability

Please create a new issue using the [Vulnerability Issue template](https://github.com/NASA-PDS-Incubator/validate/issues/new?assignees=jordanpadams&labels=triage-needed%2C+security&template=vulnerability-issue.md&title=%5BSECURITY%5D+Title+Here)
To report a vulnerability in this package, please [submit a new isssue using the Vulnerability Issue template](https://github.com/NASA-PDS-Incubator/validate/issues/new?assignees=jordanpadams&labels=triage-needed%2C+security&template=vulnerability-issue.md&title=%5BSECURITY%5D+Title+Here).

Loading