neuston_util
Sidney Batchelder edited this page Jun 23, 2022
·
3 revisions
The neuston_util.py
module allows users to perform some useful tasks related to the creation of a CNN model.
The commands are:
-
MAKE_DATASET_CONFIG
: Create a default dataset-combining configuration file. -
MAKE_CLASS_CONFIG
: Creates a default class-config csv file. -
CALC_IMG_NORM
: Calculate the MEAN and STD of dataset for image normalizing.
These tools all run on the CPU. Usages for these different commands are as follows.
usage: neuston_util.py MAKE_DATASET_CONFIG [-h] [-o OUTFILE] PATH [PATH ...]
positional arguments:
PATH List of dataset paths. Space deliminated.
You may optionally prefix the paths with "n:" where n is an integer priority value.
optional arguments:
-h, --help show this help message and exit
-o OUTFILE Specify an output file. If unset, outputs to stdout.
usage: neuston_util.py MAKE_CLASS_CONFIG [-h] [-o OUTFILE] PATH
positional arguments:
PATH path to a dataset directory or dataset configuration csv file.
optional arguments:
-h, --help show this help message and exit
-o OUTFILE Specify an output file. If unset, outputs to stdout.
usage: neuston_util.py CALC_IMG_NORM [-h] [--resize N] [--class-config CSV COL]
[--class-min MIN] [--class-max MAX] [--batch-size B] SRC
positional arguments:
SRC
optional arguments:
-h, --help show this help message and exit
--resize N Default is 299 (for inception_v3)
--class-config CSV COL Skip/combine classes as defined by column COL of a CSV configuration file.
--class-min MIN Exclude classes with fewer than MIN instances. Default is 2.
--class-max MAX Limit classes to a MAX number of instances. If multiple datasets are specified via a
dataset-configuration csv, classes from lower-priority datasets are truncated first.
--batch-size B Number of images per minibatch
- Overview
- Installation
- local
- whoi hpc
- Training a Model
- Running a Model
- Utilities
- SLURM SBATCH Tool ⊛
- Dupes Training ⊛
- Tips
- HPC Patch Notes