Skip to content

Commit

Permalink
Add local run possibility (#33)
Browse files Browse the repository at this point in the history
Signed-off-by: Sascha Grunert <mail@saschagrunert.de>
  • Loading branch information
saschagrunert committed Jun 29, 2019
1 parent b579b3a commit fc4871c
Show file tree
Hide file tree
Showing 5 changed files with 118 additions and 25 deletions.
90 changes: 87 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ During the run, Performabot needs some information about the local test
environment before actually commenting to GitHub pull requests. These parameters
can be specified either via local environment variables or the command line,
where test runs within [CircleCI](https://circleci.com) should work out of the
box (except for the token). The needed information are:
box, except for the token. The needed environmental information are:

| Flag | Environment Variable | Description |
| --------------------- | ------------------------------------------- | --------------------------- |
Expand All @@ -65,7 +65,91 @@ settings](https://github.com/settings/tokens), whereas only `public_repo`
(Access public repositories) has to be activated in case of an open source
project.

## Depdendencies
## Benchmark runs

As already mentioned, right now only
[ginkgo](https://onsi.github.io/ginkgo/#benchmark-tests) benchmarks are
supported. The tests have to be executed with `--succinct` to let the parser
finally work. Succinct silences much of Ginkgo’s more verbose output. Test
suites that succeed basically get printed out on just one line! Succinct is
turned off, by default, when running tests for one package. It is turned on by
default when Ginkgo runs multiple test packages.

Feel free to open an issue or pull request for additional parsing support.

## Try it out

If you want to give the latest master branch a quick try, then install a
container runtime like [podman](https://podman.io) and test it with the
[`test/sample`](test/sample) file:

```shell
> cat test/sample | podman run -i saschagrunert/performabot -l
...
[NOTE]: Welcome to performabot!
...
[NOTE]: Processing done, found 16 results
[NOTE]: Local run specified
[NOTE]: The report:

### Performabot Result 🤖

Nothing to compare against.

@@ Performance Diff @@
## Ø ± × ##
==========================================================
create PodSandbox 0.404s 0.471s 20
PodSandbox status 0.000s 0.000s 20
stop PodSandbox 0.240s 0.015s 20
remove PodSandbox 0.056s 0.012s 20
create PodSandbox and container 0.405s 0.049s 20
list Container 0.002s 0.005s 20
pull Image 6.498s 0.478s 20
Image status 0.002s 0.001s 20
remove Image 0.068s 0.044s 20
create Container 0.122s 0.035s 20
start Container 0.030s 0.012s 20
Container status 0.002s 0.001s 20
stop Container 0.048s 0.014s 20
remove Container 0.026s 0.004s 20
list PodSandbox 0.000s 0.000s 20
list Container 0.000s 0.000s 20
==========================================================
```

The `-l` `--local` flag specifies a local run. This means that there will be no
validation of the local environment variables nor any GitHub interaction at all.
Nevertheless, a local `performabot.sqlite` should now be available within the
current directory which contains the run.

Further command line arguments are:

```
Usage: performabot [-c|--commit COMMIT] [-p|--pull-request PULL_REQUEST]
[-r|--repository REPOSITORY] [-o|--owner OWNER]
[-t|--token TOKEN] [-v|--verbose] [-l|--local] [--version]
Available options:
-c,--commit COMMIT Commit hash - fallback environment variables:
$PB_COMMIT, $CIRCLE_SHA1
-p,--pull-request PULL_REQUEST
Pull request number - fallback environment variables:
$PB_PULL_REQUEST, $CIRCLE_PR_NUMBER
-r,--repository REPOSITORY
GitHub repository - fallback environment variables:
$PB_REPOSITORY, $CIRCLE_PROJECT_REPONAME
-o,--owner OWNER GitHub owner - fallback environment variables:
$PB_OWNER, $CIRCLE_PROJECT_USERNAME
-t,--token TOKEN Token - fallback environment variable: $PB_TOKEN
-h,--help Show this help text
-v,--verbose Logging verbosity, can be specified up to 2x
-l,--local Run only locally and do not upload anything
--version Print the current version
-h,--help Show this help text
```

## Build dependencies

There is only one dependency needed to get started with this project:
[nix](https://nixos.org/nix)
Expand Down Expand Up @@ -100,7 +184,7 @@ included in `$PATH`, then run:
```

There are other useful targets within the [Makefile](Makefile), which are used
by the CI and could be worth a look. If you don't want to use nix, then you're
by the CI and could be worth a look. If you dont want to use nix, then youre
free to build the project with [stack](https://haskellstack.org) or
[cabal](https://www.haskell.org/cabal) as well.

Expand Down
20 changes: 10 additions & 10 deletions app/Main.hs
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,8 @@ import Options.Applicative
prefShowHelpOnEmpty, prefShowHelpOnError, prefBacktrack,
prefColumns)
, customExecParser, flag', footer, fullDesc, header, help
, helper, infoOption, internal, long, many, metavar, short
, short, strOption, switch, value )
, helper, infoOption, long, many, metavar, short, short
, strOption, switch, value )

import Result
( amount, initParserStep, parseStepIO, save )
Expand Down Expand Up @@ -64,7 +64,7 @@ parser =
++ "<https://github.com/saschagrunert/performabot>"))

arguments :: Parser Args
arguments = Args <$> environment <*> verbosity <*> devel
arguments = Args <$> environment <*> verbosity <*> local

environment :: Parser Environment
environment = Environment <$> strOption (long "commit" <> short 'c'
Expand All @@ -91,31 +91,31 @@ verbosity :: Parser Priority
verbosity = priority . length
<$> many (flag' ()
(long "verbose" <> short 'v'
<> help ("the logging verbosity,"
++ " can be specified up to 2x")))
<> help "Logging verbosity, can be specified up to 2x"))
where
priority a
| a == 0 = NOTICE
| a == 1 = INFO
| otherwise = DEBUG

devel :: Parser Bool
devel = switch (internal <> long "devel" <> short 'd')
local :: Parser Bool
local = switch (long "local" <> short 'l'
<> help "Run only locally and do not upload anything")

version :: Parser (a -> a)
version =
infoOption "v0.1.0" (long "version" <> help "Print the current version")

-- | The entry function after argument parsing
run :: Args -> IO ()
run (Args e v d) = do
run (Args e v l) = do
-- Setup logging
initLogger v
notice "Welcome to performabot!"
notice . printf "The logging verbosity is set to: %s" $ show v

-- Prepare environment
env <- fillEnvironment e d
env <- fillEnvironment e l
L.info . printf "Using commit: %s" $ env ^. commit
L.info . printf "Using pull request: %s" $ env ^. pullRequest
L.info . printf "Using repository: %s" $ env ^. repository
Expand All @@ -132,7 +132,7 @@ run (Args e v d) = do
notice . printf "Processing done, found %d result%s" c $
if c == 1 then "" else "s" :: String
if c /= 0
then save r env
then save l r env
else do
warn "Not saving anything because no results found"
exitFailure
4 changes: 2 additions & 2 deletions src/Environment.hs
Original file line number Diff line number Diff line change
Expand Up @@ -40,15 +40,15 @@ $(makeLenses ''Environment)

-- | Try to fill the environment from local variables
fillEnvironment :: Environment -> Bool -> IO Environment
fillEnvironment e d = do
fillEnvironment e l = do
c <- getEnv (e ^. commit) "commit" commitEnvVars
p <- getEnv (e ^. pullRequest) "pull request" pullRequestEnvVars
r <- getEnv (e ^. repository) "repository" repositoryEnvVars
o <- getEnv (e ^. owner) "owner" ownerEnvVars
t <- getEnv (e ^. token) "token" tokenEnvVars

-- Validate the other environment variables
if not d && any null [ c, p, r, t ]
if not l && any null [ c, p, r, t ]
then exitFailure
else return $ token .~ t $ pullRequest .~ p $ commit .~ c $ owner .~ o $
repository .~ r $ e
Expand Down
7 changes: 5 additions & 2 deletions src/Github.hs
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ import Environment
( Environment, owner, pullRequest, repository, token )

import GitHub ( Auth(OAuth) )
import GitHub.Data.Comments ( Comment, commentUrl )
import GitHub.Data.Comments ( Comment, commentHtmlUrl )
import GitHub.Data.Definitions
( Error, IssueNumber(IssueNumber), Owner )
import GitHub.Data.Id ( Id(Id) )
Expand Down Expand Up @@ -82,7 +82,10 @@ handleCommentErr x t = case x of
debug $ show f
exitFailure
Right c -> notice $
printf "Comment %s successful: %s" t (getUrl $ commentUrl c)
printf "Comment %s successful: %s" t (urlOf $ commentHtmlUrl c)
where
urlOf Nothing = "Not found."
urlOf (Just u) = getUrl u

-- | Finds comments from performabot
isFromPerformabot :: [IssueComment] -> Maybe IssueComment
Expand Down
22 changes: 14 additions & 8 deletions src/Result.hs
Original file line number Diff line number Diff line change
Expand Up @@ -72,16 +72,22 @@ debugResult :: Benchmarks -> IO ()
debugResult r = debug . printf "Current result: %s" $ show r

-- | Sen the provided data to the given url including the environment
save :: Step -> Environment -> IO ()
save (_, b) e = do
c <- baseCommit e
info "Base commit retrieval successful"
save :: Bool -> Step -> Environment -> IO ()
save l (_, b) e = do
insertInDB e b
info "Database insertion successful"
pb <- entryForCommit c
let r = prettyPrint b pb
notice $ printf "The report: %s" r
comment e r
r <- if not l
then do
c <- baseCommit e
info "Base commit retrieval successful"
pb <- entryForCommit c
let r = prettyPrint b pb
comment e r
return r
else do
notice "Local run specified"
return $ prettyPrint b Nothing
notice $ printf "The report:\n\n%s" r

-- | The database name
db :: Text
Expand Down

0 comments on commit fc4871c

Please sign in to comment.