Skip to content

Commit

Permalink
Add suggestion to run wget with --progress=dot:giga
Browse files Browse the repository at this point in the history
Fixes #546

Warn when runnign wget without without --progress flag unless
other flags are present (such as quiet/no-verbose/output/append),
which usually leads to insane amouts of useless logs when downloaded
files are big (usually anything over 5MB becomes annoying)

Adding --progress=dot:giga makes output much shorter even for very large
files such as AM/ML models etc.

See wget manual, search for --progress=dot:giga

Signed-off-by: Michał Sochoń <kaszpir@gmail.com>
  • Loading branch information
nvtkaszpir committed Mar 15, 2021
1 parent 4db396c commit 51ee880
Show file tree
Hide file tree
Showing 5 changed files with 103 additions and 1 deletion.
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -225,6 +225,7 @@ Please [create an issue][] if you have an idea for a good rule.
| [DL3044](https://github.com/hadolint/hadolint/wiki/DL3044) | Do not refer to an environment variable within the same `ENV` statement where it is defined. |
| [DL3045](https://github.com/hadolint/hadolint/wiki/DL3045) | `COPY` to a relative destination without `WORKDIR` set. |
| [DL3046](https://github.com/hadolint/hadolint/wiki/DL3046) | `useradd` without flag `-l` and high UID will result in excessively large Image. |
| [DL3047](https://github.com/hadolint/hadolint/wiki/DL3047) | `wget` without flag `--progress` will result in excessively bloated build logs when downloading larger files. |
| [DL4000](https://github.com/hadolint/hadolint/wiki/DL4000) | MAINTAINER is deprecated. |
| [DL4001](https://github.com/hadolint/hadolint/wiki/DL4001) | Either use Wget or Curl but not both. |
| [DL4003](https://github.com/hadolint/hadolint/wiki/DL4003) | Multiple `CMD` instructions found. |
Expand Down
3 changes: 2 additions & 1 deletion hadolint.cabal
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ cabal-version: 2.0
--
-- see: https://github.com/sol/hpack
--
-- hash: 0894de2541e49590657147243a298ff0585a0acb41aa2dbaf9cf5666f1b25348
-- hash: c7489925f7c60f72f7b137aaa8665b6eaa1e6ca5c4d8785d1f31bc5a55573914

name: hadolint
version: 2.0.0
Expand Down Expand Up @@ -90,6 +90,7 @@ library
Hadolint.Rule.DL3044
Hadolint.Rule.DL3045
Hadolint.Rule.DL3046
Hadolint.Rule.DL3047
Hadolint.Rule.DL4000
Hadolint.Rule.DL4001
Hadolint.Rule.DL4003
Expand Down
2 changes: 2 additions & 0 deletions src/Hadolint/Process.hs
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ import qualified Hadolint.Rule.DL3043
import qualified Hadolint.Rule.DL3044
import qualified Hadolint.Rule.DL3045
import qualified Hadolint.Rule.DL3046
import qualified Hadolint.Rule.DL3047
import qualified Hadolint.Rule.DL4000
import qualified Hadolint.Rule.DL4001
import qualified Hadolint.Rule.DL4003
Expand Down Expand Up @@ -162,6 +163,7 @@ failures RulesConfig {allowedRegistries} =
<> Hadolint.Rule.DL3044.rule
<> Hadolint.Rule.DL3045.rule
<> Hadolint.Rule.DL3046.rule
<> Hadolint.Rule.DL3047.rule
<> Hadolint.Rule.DL4000.rule
<> Hadolint.Rule.DL4001.rule
<> Hadolint.Rule.DL4003.rule
Expand Down
36 changes: 36 additions & 0 deletions src/Hadolint/Rule/DL3047.hs
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
module Hadolint.Rule.DL3047 (rule) where

import Hadolint.Rule
import Hadolint.Shell (ParsedShell)
import qualified Hadolint.Shell as Shell
import Language.Docker.Syntax

rule :: Rule ParsedShell
rule = simpleRule code severity message check
where
code = "DL3047"
severity = DLWarningC
message =
"Avoid use of wget without progress bar. Use `wget --progress=dot:giga <url>`.\
\Or consider using `-q` or `-nv` (shorthands for `--quiet` or `--no-verbose`)."

check (Run (RunArgs args _)) = foldArguments (Shell.noCommands forgotProgress) args
check _ = True

forgotProgress cmd = isWget cmd && (not (hasProgressOption cmd) && not (hasSpecialFlags cmd))
isWget (Shell.Command name _ _) = name == "wget"
hasProgressOption cmd = Shell.hasFlag "progress" cmd

hasSpecialFlags cmd =
hasQuietFlag cmd
|| hasOutputFlag cmd
|| hasAppendOutputFlag cmd
|| hasNoVerboseFlag cmd

hasQuietFlag cmd = Shell.hasAnyFlag ["q", "quiet"] cmd
hasOutputFlag cmd = Shell.hasAnyFlag ["o", "output-file"] cmd
hasAppendOutputFlag cmd = Shell.hasAnyFlag ["a", "append-output"] cmd
hasNoVerboseFlag cmd =
Shell.hasAnyFlag ["no-verbose"] cmd
|| Shell.cmdHasArgs "wget" ["-nv"] cmd
{-# INLINEABLE rule #-}
62 changes: 62 additions & 0 deletions test/Spec.hs
Original file line number Diff line number Diff line change
Expand Up @@ -1507,6 +1507,68 @@ main =
(Text.unlines dockerFile)
(failsWith 2 "DL4001")
--
describe "Wget no progress" $ do
it "warns when using wget without --progress option" $
let dockerFile =
[ "FROM node as foo",
"RUN wget my.xyz"
]
in ruleCatches "DL3047" $ Text.unlines dockerFile
it "does not warn when running with --progress option" $
let dockerFile =
[ "FROM node as foo",
"RUN wget --progress=dot:giga my.xyz"
]
in ruleCatchesNot "DL3047" $ Text.unlines dockerFile
it "does not warn when running with -q (quiet short option) and without --progress option" $
let dockerFile =
[ "FROM node as foo",
"RUN wget -q my.xyz"
]
in ruleCatchesNot "DL3047" $ Text.unlines dockerFile
it "does not warn when running with --quiet (quiet long option) and without --progress option" $
let dockerFile =
[ "FROM node as foo",
"RUN wget --quiet my.xyz"
]
in ruleCatchesNot "DL3047" $ Text.unlines dockerFile
it "does not warn when running with -nv (no-verbose short option) and without --progress option" $
let dockerFile =
[ "FROM node as foo",
"RUN wget -nv my.xyz"
]
in ruleCatchesNot "DL3047" $ Text.unlines dockerFile
it "does not warn when running with --no-verbose (no-verbose long option) and without --progress option" $
let dockerFile =
[ "FROM node as foo",
"RUN wget --no-verbose my.xyz"
]
in ruleCatchesNot "DL3047" $ Text.unlines dockerFile
it "does not warn when running with --output-file (output-file long option) and without --progress option" $
let dockerFile =
[ "FROM node as foo",
"RUN wget --output-file=/tmp/wget.log my.xyz"
]
in ruleCatchesNot "DL3047" $ Text.unlines dockerFile
it "does not warn when running with -o (output-file long option) and without --progress option" $
let dockerFile =
[ "FROM node as foo",
"RUN wget -o /tmp/wget.log my.xyz"
]
in ruleCatchesNot "DL3047" $ Text.unlines dockerFile
it "does not warn when running with --append-output (append-output long option) and without --progress option" $
let dockerFile =
[ "FROM node as foo",
"RUN wget --append-output=/tmp/wget.log my.xyz"
]
in ruleCatchesNot "DL3047" $ Text.unlines dockerFile
it "does not warn when running with -a (append-output long option) and without --progress option" $
let dockerFile =
[ "FROM node as foo",
"RUN wget -a /tmp/wget.log my.xyz"
]
in ruleCatchesNot "DL3047" $ Text.unlines dockerFile
--
describe "ONBUILD" $ do
it "error when using `ONBUILD` within `ONBUILD`" $
let dockerFile =
Expand Down

0 comments on commit 51ee880

Please sign in to comment.