Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for creating GitLab issues #37

Merged
merged 8 commits into from Aug 10, 2021
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion .buildkite/lint.sh
Expand Up @@ -6,6 +6,6 @@ cd `dirname $0`/..

go get golang.org/x/lint/golint
go install golang.org/x/tools/go/analysis/passes/shadow/cmd/shadow
go get github.com/fzipp/gocyclo
go get github.com/fzipp/gocyclo/cmd/gocyclo

./scripts/lint.sh
2 changes: 1 addition & 1 deletion README.md
Expand Up @@ -2,7 +2,7 @@

Web service which collects and serves bug reports.

rageshake requires Go version 1.11 or later.
rageshake requires Go version 1.15 or later.

To run it, do:

Expand Down
1 change: 1 addition & 0 deletions changelog.d/37.feature
@@ -0,0 +1 @@
Add support for creating GitLab issues. Contributed by @tulir.
1 change: 1 addition & 0 deletions changelog.d/37.misc
@@ -0,0 +1 @@
Update minimum Go version to 1.15.
16 changes: 5 additions & 11 deletions go.mod
@@ -1,17 +1,11 @@
module github.com/matrix-org/rageshake

go 1.15

require (
cloud.google.com/go v0.0.0-20170406015231-675fad27ef35
github.com/golang/protobuf v0.0.0-20170331031902-2bba0603135d
github.com/google/go-genproto v0.0.0-20170404132009-411e09b969b1
github.com/google/go-github v0.0.0-20170401000335-12363ffc1001
github.com/google/go-querystring v0.0.0-20170111101155-53e6ce116135
github.com/googleapis/gax-go v0.0.0-20170321005343-9af46dd5a171
github.com/jordan-wright/email v4.0.1-0.20200824153738-3f5bafa1cd84+incompatible
github.com/pkg/errors v0.0.0-20171018195549-f15c970de5b7
golang.org/x/net v0.0.0-20170329170435-ffcf1bedda3b
golang.org/x/oauth2 v0.0.0-20170321013421-7fdf09982454
golang.org/x/text v0.0.0-20170401064109-f4b4367115ec
google.golang.org/grpc v0.0.0-20170405173540-b5071124392b
gopkg.in/yaml.v2 v2.0.0-20170407172122-cd8b52f8269e
github.com/xanzy/go-gitlab v0.50.2
golang.org/x/oauth2 v0.0.0-20181106182150-f42d05182288
gopkg.in/yaml.v2 v2.2.2
)
61 changes: 45 additions & 16 deletions go.sum
@@ -1,20 +1,49 @@
cloud.google.com/go v0.0.0-20170406015231-675fad27ef35/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
github.com/golang/protobuf v0.0.0-20170331031902-2bba0603135d/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/google/go-genproto v0.0.0-20170404132009-411e09b969b1/go.mod h1:3Rcd9jSoLVkV/osPrt5CogLvLiarfI8U9/x78NwhuDU=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/golang/protobuf v1.2.0 h1:P3YflyNX/ehuJFLhxviNdFxQPkGK5cDcApsge1SqnvM=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/google/go-github v0.0.0-20170401000335-12363ffc1001 h1:OK4gfzCBCtPg14E4sYsczwFhjVu1jQJZI+OEOpiTigw=
github.com/google/go-github v0.0.0-20170401000335-12363ffc1001/go.mod h1:zLgOLi98H3fifZn+44m+umXrS52loVEgC2AApnigrVQ=
github.com/google/go-github v17.0.0+incompatible h1:N0LgJ1j65A7kfXrZnUDaYCs/Sf4rEjNlfyDHW9dolSY=
github.com/google/go-querystring v0.0.0-20170111101155-53e6ce116135 h1:zLTLjkaOFEFIOxY5BWLFLwh+cL8vOBW4XJ2aqLE/Tf0=
github.com/google/go-querystring v0.0.0-20170111101155-53e6ce116135/go.mod h1:odCYkC5MyYFN7vkCjXpyrEuKhc/BUO6wN/zVPAxq5ck=
github.com/googleapis/gax-go v0.0.0-20170321005343-9af46dd5a171/go.mod h1:SFVmujtThgffbyetf+mdk2eWhX2bMyUtNHzFKcPA9HY=
github.com/google/go-querystring v1.0.0 h1:Xkwi/a1rcvNg1PPYe5vI8GbeBY/jrVuDX5ASuANWTrk=
github.com/google/go-querystring v1.0.0/go.mod h1:odCYkC5MyYFN7vkCjXpyrEuKhc/BUO6wN/zVPAxq5ck=
github.com/hashicorp/go-cleanhttp v0.5.1 h1:dH3aiDG9Jvb5r5+bYHsikaOUIpcM0xvgMXVoDkXMzJM=
github.com/hashicorp/go-cleanhttp v0.5.1/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtngrth3wmdIIUrZ80=
github.com/hashicorp/go-hclog v0.9.2 h1:CG6TE5H9/JXsFWJCfoIVpKFIkFe6ysEuHirp4DxCsHI=
github.com/hashicorp/go-hclog v0.9.2/go.mod h1:5CU+agLiy3J7N7QjHK5d05KxGsuXiQLrjA0H7acj2lQ=
github.com/hashicorp/go-retryablehttp v0.6.8 h1:92lWxgpa+fF3FozM4B3UZtHZMJX8T5XT+TFdCxsPyWs=
github.com/hashicorp/go-retryablehttp v0.6.8/go.mod h1:vAew36LZh98gCBJNLH42IQ1ER/9wtLZZ8meHqQvEYWY=
github.com/jordan-wright/email v4.0.1-0.20200824153738-3f5bafa1cd84+incompatible h1:d60x4RsAHk/UX/0OT8Gc6D7scVvhBbEANpTAWrDhA/I=
github.com/jordan-wright/email v4.0.1-0.20200824153738-3f5bafa1cd84+incompatible/go.mod h1:1c7szIrayyPPB/987hsnvNzLushdWf4o/79s3P08L8A=
github.com/pkg/errors v0.0.0-20171018195549-f15c970de5b7/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
golang.org/x/net v0.0.0-20170329170435-ffcf1bedda3b h1:Co3zyosPfwWowmu8+roHGC+aDgizpCPH3ukhubZ0Ttg=
golang.org/x/net v0.0.0-20170329170435-ffcf1bedda3b/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/oauth2 v0.0.0-20170321013421-7fdf09982454 h1:qH7SPXL1bLgpFB+ycaFjqQ2lI54cG8OGelAQGpmZSnc=
golang.org/x/oauth2 v0.0.0-20170321013421-7fdf09982454/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/text v0.0.0-20170401064109-f4b4367115ec/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
google.golang.org/grpc v0.0.0-20170405173540-b5071124392b/go.mod h1:yo6s7OP7yaDglbqo1J04qKzAhqBH6lvTonzMVmEdcZw=
gopkg.in/yaml.v2 v2.0.0-20170407172122-cd8b52f8269e h1:o/mfNjxpTLivuKEfxzzwrJ8PmulH2wEp7t713uMwKAA=
gopkg.in/yaml.v2 v2.0.0-20170407172122-cd8b52f8269e/go.mod h1:JAlM8MvJe8wmxCU4Bli9HhUf9+ttbYbLASfIpnQbh74=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.4.0 h1:2E4SXV/wtOkTonXsotYi4li6zVWxYlZuYNCXe9XRJyk=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/xanzy/go-gitlab v0.50.2 h1:Qm/um2Jryuqusc6VmN7iZYVTQVzNynzSiuMJDnCU1wE=
github.com/xanzy/go-gitlab v0.50.2/go.mod h1:Q+hQhV508bDPoBijv7YjK/Lvlb4PhVhJdKqXVQrUoAE=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20201021035429-f5854403a974 h1:IX6qOQeG5uLjB/hjjwjedwfjND0hgjPMMyO1RoIXQNI=
golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/oauth2 v0.0.0-20181106182150-f42d05182288 h1:JIqe8uIcRBHXDQVvZtHwp80ai3Lw3IJAeJEs55Dc1W0=
golang.org/x/oauth2 v0.0.0-20181106182150-f42d05182288/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9 h1:SQFwaSi55rU7vdNs9Yr0Z324VNlrF+0wMqRXT4St8ck=
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/time v0.0.0-20191024005414-555d28b269f0 h1:/5xXl8Y5W96D+TtHSlonuFqGHIWVuyCkGJLwGh9JJFs=
golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
google.golang.org/appengine v1.3.0 h1:FBSsiFRMz3LBeXIomRnVzrQwSDj4ibvcRexLG0LZGQk=
google.golang.org/appengine v1.3.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
23 changes: 21 additions & 2 deletions main.go
Expand Up @@ -30,9 +30,10 @@ import (
"time"

"github.com/google/go-github/github"
"github.com/xanzy/go-gitlab"
"golang.org/x/oauth2"

yaml "gopkg.in/yaml.v2"
"gopkg.in/yaml.v2"
)

var configPath = flag.String("config", "rageshake.yaml", "The path to the config file. For more information, see the config file in this repository.")
Expand All @@ -51,6 +52,13 @@ type config struct {

GithubProjectMappings map[string]string `yaml:"github_project_mappings"`

GitlabURL string `yaml:"gitlab_url"`
GitlabToken string `yaml:"gitlab_token"`

GitlabProjectMappings map[string]int `yaml:"gitlab_project_mappings"`
GitlabProjectLabels map[string][]string `yaml:"gitlab_project_labels"`
GitlabIssueConfidential bool `yaml:"gitlab_issue_confidential"`

SlackWebhookURL string `yaml:"slack_webhook_url"`

EmailAddresses []string `yaml:"email_addresses"`
Expand Down Expand Up @@ -102,6 +110,17 @@ func main() {
ghClient = github.NewClient(tc)
}

var glClient *gitlab.Client
if cfg.GitlabToken == "" {
fmt.Println("No gitlab_token configured. Reporting bugs to gitlab is disaled.")
} else {
glClient, err = gitlab.NewClient(cfg.GitlabToken, gitlab.WithBaseURL(cfg.GitlabURL))
if err != nil {
// This probably only happens if the base URL is invalid
log.Fatalln("Failed to create GitLab client:", err)
}
}

var slack *slackClient

if cfg.SlackWebhookURL == "" {
Expand All @@ -127,7 +146,7 @@ func main() {
}
log.Printf("Using %s/listing as public URI", apiPrefix)

http.Handle("/api/submit", &submitServer{ghClient, apiPrefix, slack, cfg})
http.Handle("/api/submit", &submitServer{ghClient, glClient, apiPrefix, slack, cfg})

// Make sure bugs directory exists
_ = os.Mkdir("bugs", os.ModePerm)
Expand Down
17 changes: 17 additions & 0 deletions rageshake.sample.yaml
Expand Up @@ -17,6 +17,23 @@ github_token: secrettoken
github_project_mappings:
my-app: octocat/HelloWorld

# a GitLab personal access token (https://gitlab.com/-/profile/personal_access_tokens), which
# will be used to create a GitLab issue for each report. It requires
# `api` scope. If omitted, no issues will be created.
gitlab_token: secrettoken
# the base URL of the GitLab instance to use
gitlab_url: https://gitlab.com

# mappings from app name (as submitted in the API) to the GitLab Project ID (not name!) for issue reporting.
gitlab_project_mappings:
my-app: 12345
# mappings from app name to a list of GitLab label names for issue reporting.
gitlab_project_labels:
my-app:
- client::my-app
# whether GitLab issues should be created as confidential issues
tulir marked this conversation as resolved.
Show resolved Hide resolved
gitlab_issue_confidential: true

# a Slack personal webhook URL (https://api.slack.com/incoming-webhooks), which
# will be used to post a notification on Slack for each report.
slack_webhook_url: https://hooks.slack.com/services/TTTTTTT/XXXXXXXXXX/YYYYYYYYYYY
Expand Down
66 changes: 58 additions & 8 deletions submit.go
Expand Up @@ -39,6 +39,7 @@ import (

"github.com/google/go-github/github"
"github.com/jordan-wright/email"
"github.com/xanzy/go-gitlab"
)

var maxPayloadSize = 1024 * 1024 * 55 // 55 MB
Expand All @@ -47,6 +48,7 @@ type submitServer struct {
// github client for reporting bugs. may be nil, in which case,
// reporting is disabled.
ghClient *github.Client
glClient *gitlab.Client

// External URI to /api
apiPrefix string
Expand Down Expand Up @@ -467,6 +469,10 @@ func (s *submitServer) saveReport(ctx context.Context, p parsedPayload, reportDi
return nil, err
}

if err := s.submitGitlabIssue(p, listingURL, &resp); err != nil {
return nil, err
}

if err := s.submitSlackNotification(p, listingURL); err != nil {
return nil, err
}
Expand Down Expand Up @@ -509,6 +515,29 @@ func (s *submitServer) submitGithubIssue(ctx context.Context, p parsedPayload, l
return nil
}

func (s *submitServer) submitGitlabIssue(p parsedPayload, listingURL string, resp *submitResponse) error {
if s.glClient == nil {
return nil
}

glProj := s.cfg.GitlabProjectMappings[p.AppName]
glLabels := s.cfg.GitlabProjectLabels[p.AppName]

issueReq := buildGitlabIssueRequest(p, listingURL, glLabels, s.cfg.GitlabIssueConfidential)

issue, _, err := s.glClient.Issues.CreateIssue(glProj, issueReq)

if err != nil {
return err
}

log.Println("Created issue:", issue.WebURL)

resp.ReportURL = issue.WebURL

return nil
}

func (s *submitServer) submitSlackNotification(p parsedPayload, listingURL string) error {
if s.slack == nil {
return nil
Expand Down Expand Up @@ -541,7 +570,7 @@ func buildReportTitle(p parsedPayload) string {
return trimmedUserText
}

func buildReportBody(p parsedPayload, quoteChar string) *bytes.Buffer {
func buildReportBody(p parsedPayload, newline, quoteChar string) *bytes.Buffer {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is getting a bit confusing. A comment explaining what exactly the parameters do would be helpful.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm also not entirely convinced this needs to be different between email and github/gitlab?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Email isn't markdown and I didn't want to accidentally break something with extra spaces at the end of each line, but I guess it might be fine to add the spaces for emails too. (GitHub is markdown, but doesn't seem to require the two spaces for newlines. GitLab does require spaces to make a newline)

var bodyBuf bytes.Buffer
fmt.Fprintf(&bodyBuf, "User message:\n\n%s\n\n", p.UserText)
var dataKeys []string
Expand All @@ -551,17 +580,17 @@ func buildReportBody(p parsedPayload, quoteChar string) *bytes.Buffer {
sort.Strings(dataKeys)
for _, k := range dataKeys {
v := p.Data[k]
fmt.Fprintf(&bodyBuf, "%s: %s%s%s\n", k, quoteChar, v, quoteChar)
fmt.Fprintf(&bodyBuf, "%s: %s%s%s%s", k, quoteChar, v, quoteChar, newline)
}

return &bodyBuf
}

func buildGithubIssueRequest(p parsedPayload, listingURL string) github.IssueRequest {
bodyBuf := buildReportBody(p, "`")
func buildGenericIssueRequest(p parsedPayload, listingURL string) (title, body string) {
bodyBuf := buildReportBody(p, " \n", "`")

// Add log links to the body
fmt.Fprintf(bodyBuf, "[Logs](%s)", listingURL)
fmt.Fprintf(bodyBuf, "\n[Logs](%s)", listingURL)

for _, file := range p.Files {
fmt.Fprintf(
Expand All @@ -572,9 +601,15 @@ func buildGithubIssueRequest(p parsedPayload, listingURL string) github.IssueReq
)
}

title := buildReportTitle(p)
title = buildReportTitle(p)

body = bodyBuf.String()

return
}

body := bodyBuf.String()
func buildGithubIssueRequest(p parsedPayload, listingURL string) github.IssueRequest {
title, body := buildGenericIssueRequest(p, listingURL)

labels := p.Labels
// go-github doesn't like nils
Expand All @@ -588,6 +623,21 @@ func buildGithubIssueRequest(p parsedPayload, listingURL string) github.IssueReq
}
}

func buildGitlabIssueRequest(p parsedPayload, listingURL string, labels []string, confidential bool) *gitlab.CreateIssueOptions {
title, body := buildGenericIssueRequest(p, listingURL)

if p.Labels != nil {
labels = append(labels, p.Labels...)
}

return &gitlab.CreateIssueOptions{
Title: &title,
Description: &body,
Confidential: &confidential,
Labels: labels,
}
}

func (s *submitServer) sendEmail(p parsedPayload, reportDir string) error {
if len(s.cfg.EmailAddresses) == 0 {
return nil
Expand All @@ -604,7 +654,7 @@ func (s *submitServer) sendEmail(p parsedPayload, reportDir string) error {

e.Subject = fmt.Sprintf("[%s] %s", p.AppName, buildReportTitle(p))

e.Text = buildReportBody(p, "\"").Bytes()
e.Text = buildReportBody(p, "\n", "\"").Bytes()

allFiles := append(p.Files, p.Logs...)
for _, file := range allFiles {
Expand Down