Add expression parser evaulator for build filter

[praxist]

continuation of #2171 (comment), using the expression library from https://github.com/drone/expr

[praxist]

Thoughts on moving this logic into the fn declaration? That would reduce the need for having the extra variable declarations. Or is catching a potential error from the selector initialization important?

[bradrydzewski]

yes, for caching, otherwise it would re-parse the sql string every time :)

[bradrydzewski]

we can vendor this repository, and then remove this line. Will add a comment with instructions shortly

[bradrydzewski]

ok, quick guide to vendoring ...

download the govendor tool

go get -u github.com/kardianos/govendor

and make sure you have a local copy of the expr library

go get -u github.com/drone/expr

and then you can run this command:

govendor add github.com/drone/expr

this will add the code to the vendor directory, which gets checked into the repository.

[bradrydzewski]

Thoughts on moving this logic into the fn declaration? That would reduce the need for having the extra variable declarations. Or is catching a potential error from the selector initialization important?

I'm guessing you were looking to clean up / simplify the code. That file is definitely getting a bit unruly after a lot of changes and years of evolution ...

If you wanted to clean things up a bit, you could break out the function like this:

func createFilterFunc(filter rpc.Filter) (queue.Filter, error) {
	var st *expr.Selector
 	var err error

 	if filter.Expr != "" {
 		st, err = expr.ParseString(filter.Expr)
 		if err != nil {
 			return nil, err
 		}
 	}

	return func(task *queue.Task) bool {
 		if st != nil {
 			match, _ := st.Eval(expr.NewRow(task.Labels))
 			return match
 		}
 
  		for k, v := range filter.Labels {
  			if task.Labels[k] != v {
  				return false
  			}
  		}
  		return true
	}, nil
}

You could then invoke this above function to get the filter, and pass to the poller:

fn, err := createFilterFunc(filter)
if err != nil {
	return nil, err
}
task, err := s.queue.Poll(c, fn)

I am fine either way, so I'll leave the decision up to you :)

[praxist]

good recommendation!

[ozbillwang]

Nice job, well done!

@praxist & @bradrydzewski

Could you provide some instructions on how to use this feature?

[bradrydzewski]

I am planning on writing official docs, but in the mean time, here is a quick overview.

The first step is to define a query used to filter which builds the agent will accept. The query expression is similar to sql. For example:

platform = 'linux/amd64' AND repo GLOB octocat/*

There are two fields available by default:

• repo is the repository name
• platform is the target platform for the build

The user can define additional fields in the labels section of the .drone.yml file. These fields can then be evaluated by the query expression:

pipeline:
  build:
    image: golang
    commands: go test

label:
  - ram=32
  - cpu=16

For example:

platform = 'linux/amd64' AND ram >= 32 AND cpu >= 16

The query expression string is passed to the agent using the DRONE_FILTER environment variable. The agent will only process builds when the expression evaluates to true. In the previous example, the agent would only process builds for linux, with user-defined labels specifying 32 GB ram or higher and 16 cores or higher.

[yoanisgil]

@bradrydzewski is it possible to define labels at each step so individual steps are executed by a given agent? It does not look like is possible right now but I just wanted to confirm.

[bradrydzewski]

It does not look like is possible right now but I just wanted to confirm

It is not possible, but perhaps this is something that would be possible when we launch the .drone.js file, which will have more advanced scripting and conditional capabilities.

[yoanisgil]

Quick question though: this new upcoming functionality in .drone.js, is something that can be tested? Like we're in the process of choosing our CI tool and I personally like Drone and features such as this one would really help in the decision process (as well as the monorepo one ;)).

[bradrydzewski]

There should be a release candidate for 0.9 end of month, which will include mono-repo support. The .drone.js will probably be part of 0.10. So will be a while.