Skip to content
Awesome Continuous Integration - Lot's of tools for git, file and static source code analysis.
Shell Dockerfile Makefile HTML Perl Python
Branch: master
Clone or download
cytopia Merge pull request #9 from cytopia/docs
Add related projects
Latest commit 343d731 Jun 26, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
bin Fix #7: Ensure CRLF is not falsy treated as trailing whitespace Dec 8, 2018
dependencies Removing inline-js dependencies Nov 1, 2016
etc New check: file-cr (Carriage Return only) Dec 13, 2016
test Restructure tests Jun 12, 2018
.editorconfig
.gitignore Fix various syntax error checkers Sep 14, 2016
.travis.yml Push to dockerhub Jun 12, 2018
Dockerfile Add Docker image Jun 11, 2018
EXAMPLES.md Examples Sep 3, 2016
LICENSE.md Adding license Apr 9, 2018
Makefile Restructure tests Jun 12, 2018
README.md Add related projects Jun 26, 2019
REGEX_CONTRIBUTIONS.md Fix anchor Sep 15, 2016
composer.json Adding composer Sep 3, 2016
configure Add Makefile for easy installation Aug 31, 2016

README.md

awesome-ci

Build Status Latest Stable Version Total Downloads Docker image License

All #awesome-ci Docker images

ansible ansible-lint awesome-ci black checkmake eslint file-lint gofmt golint jsonlint phpcs pycodestyle pylint terraform-docs terragrunt yamllint

All #awesome-ci Makefiles

Visit cytopia/makefiles for seamless project integration, minimum required best-practice code linting and CI.

Docker image

Runs on

Linux FreeBSD OSX

Continuous Integration command line tools for git repositories, file characteristics, syntax errors and static source code analysis.

Awesome-CI is capable of finding various problems in your code repository as well as fixing them automatically.


Table of Contents

  1. Tools
  2. Learn / Validate
  3. Fix
  4. Custom regex contributions
  5. General usage
  6. Installation
    1. Requirements
    2. Install OSX
    3. Install Linux/BSD
  7. Awesome CI Docker image
  8. Documentation
  9. License

Tools

All checks have the option to only check by one or more file extensions, by shebang as well as to exclude one or more folders from the whole search.

Note: Fixable options are currently in testing phase. Please report any bugs.

Type Tool Fixable Description
Git git-conflicts Scan files and check if they contain git conflicts.
Git git-ignored Scan git directory and see if ignored files are still in git cache.
File file-cr Scan files and check if they contain CR (Carriage Return only).
File file-crlf Scan files and check if they contain CRLF (Windows Line Feeds).
File file-empty Scan files and check if they are empty (0 bytes).
File file-nullbyte-char Scan files and check if they contain a null-byte character (\x00).
File file-trailing-newline Scan files and check if they contain a trailing newline.
File file-trailing-single-newline Scan files and check if they contain exactly one trailing newline.
File file-trailing-space Scan files and check if they contain trailing whitespaces.
File file-utf8 Scan files and check if they have a non UTF-8 encoding.
File file-utf8-bom Scan files and check if they contain BOM (Byte Order Mark): <U+FEFF>.
Syntax syntax-bash Scan shell files for bash syntax errors.
Syntax syntax-css Scan CSS files for CSS syntax errors.
Syntax syntax-js Scan JS files for JS syntax errors.
Syntax syntax-json Scan files for JSON syntax errors.
Syntax syntax-markdown Scan files for Markdown syntax errors.
Syntax syntax-perl Scan Perl files for Perl syntax errors.
Syntax syntax-php Scan files for PHP syntax errors.
Syntax syntax-python Scan Python files for Python syntax errors.
Syntax syntax-ruby Scan Ruby files for Ruby syntax errors.
Syntax syntax-scss Scan SCSS files for SCSS syntax errors.
Syntax syntax-sh Scan shell files for /bin/sh syntax errors.
Code Conventions inline-css Scan files and check if they contain inline css code.
Code Conventions inline-js Scan files and check if they contain inline javascript code.
Regex regex-grep egrep (grep -E) regex version to scan files for an occurance.
Regex regex-perl perl regex version to scan files for an occurance.

Learn / validate

All of the above scripts offer the --dry option which will only show you the built command without actually executing it:

$ regex-grep --path=. --ignore=".git,.svn" --shebang=sh --size --text \
--custom="if[[:space:]]*\[\[" --dry

find . -type f -not \( -path "./.git*" -o -path "./.svn*" \) ! -size 0 -print0 | \
   xargs -0 -P 8 -n1 grep -Il '' | \
   tr '\n' '\0' | \
   xargs -0 -P 8 -n1 awk '/^#!.*(\/sh|[[:space:]]+sh)/{print FILENAME}' | \
   tr '\n' '\0' | \
   xargs -0 -P 8 -n1  sh -c 'if [ -f "${1}" ]; then grep --color=always -inHE "if[[:space:]]*\[\[" "$1" || true; fi' --

Fix

Some of the above scripts offer the --fix option (see table above), with which you are actually able to fix the problem. You can also combine it with --dry to see how the actual fix command looks like:

$ file-utf8 --path=dump.sql --fix --dry

find dump.sql -type f -print0 | \
   xargs -0 -P 8 -n1  sh -c 'if [ -f "${1}" ]; then isutf8 "$1" >/dev/null || (TERM=vt100 vi -u NONE -n -es -c "set fileencoding=utf8" -c "wq" "$1" > /dev/tty && echo "Fixing: $1" || echo "FAILED: $1"); fi' --

Custom regex contributions

regex-grep and regex-perl have a lot of potential for doing custom project validation.

In order to give you an idea, have a look at the compiled Regex Contributions.

Please use pull requests to add useful checks.

General Usage

  • All tools share the same pattern (except git-ignored) and can be used with the same command line arguments.
  • Some tools have an additional option --custom="" to overwrite the command itself (this is explained and shown in detail in each command's --help option).

Options:

# Required:
  --path            # Specify the path where to scan

# Optional pattern (each option is logically and-ed):
  --fix             # Fix the problems for the specified files (not every check)

  --text            # Only scan non-binary files
  --size            # Only scan non-empty files (greater 0 bytes)
  --shebang         # Only scan files (shell scripts) that match a certain shebang
  --extension       # Only scan files with these file extensions
  --ignore          # Ignore files/folders

# Optional misc:
  --config          # Specify configuration file
  --confpre         # Alter configuration directive prefix for this check
  --verbose         # Show files and commands as being processed
  --debug           # Show additional debug messages
  --list            # Only show files that would be processed (no processing)
  --dry             # Show command that would be executed (no processing)

# System
  --help            # Show help
  --info            # Show version of required binaries
  --version         # Show tool version

Installation

Requirements

Awesome-ci requires the following tools to be installed:

  • dos2unix
  • eslint
  • file
  • git
  • jsonlint
  • mdl
  • perl
  • php
  • python
  • ruby
  • scss_lint
  • shellcheck

Install OSX

brew tap cytopia/tap
brew install awesome-ci

Install Linux/BSD

# Install to /usr/bin
./configure
make install

# Instal to /usr/local/bin
./configure --prefix=/usr/local
make install

# Install to /opt/bin
./configure --prefix=/opt
make install

Awesome CI Docker image

Docker image

Instead of installing awesome-ci and all its required dependencies locally on your computer, you can also use the bundled Docker image cytopia/awesome-ci which has everything pre-installed and is built nightly by travis-ci.

docker run -v ${PWD}:/ac cytopia/awesome-ci file-crlf --path=/ac

The above example is using file-crlf to scan the current directory for files containing Windows newlines:

  • ${PWD} (the current host directory) is mounted into the container's /ac directoy
  • file-crlf path then points (inside the container) to /ac (which is the current host directory)
  • /ac can actually be named by whatever name you want

If you use an awesome-ci configuration which is not inside the directory you want to check, you will also have to mount that into the container:

docker run \
    -v /host/path/to/awesome-ci.conf:/etc/awesome-ci.conf \
    -v ${PWD}:/ac cytopia/awesome-ci file-crlf --path=/ac --config=/etc/awesome-ci.conf

Documentation

To find out more about awesome-ci, have a look at the following links.

License

MIT License

Copyright (c) 2018 cytopia

You can’t perform that action at this time.