[Jira] Project name should be in project.json, not in mordred.cfg

[jgbarah]

Perceval Jira backend allow user to specify a project:

$ perceval jira 'https://tickets.puppetlabs.com' --project PUP

In this case, only issues for that project will be retrieved from the Jira API.

The current way of specifying this for Mordred is in mordred.cfg:

[jira]
raw_index = jira_test-raw
enriched_index = jira_test
project = PUP
max-issues = 10

But this is weird, and allows only for a single project to be filtered. This should in fact be in projects.json, and should be specified in a way that any list of projects from the Jira API endpoint are specified. For example, it could be like:

"jira": [
     "https://tickets.puppetlabs.com --project project1",
     "https://tickets.puppetlabs.com --project project2",
     "https://tickets.puppetlabs.com --project project3",
     ...
],

See some more details in a recent question, #121.

[filmaj]

Ooo perhaps this could be one that, with a little bit of help and pointing me in the right direction, I could try taking a stab at preparing a PR for?

[jgbarah]

Of course! If you want, have a look at how projects.json is loaded by Mordred. Let me know if you want more specific directions.

[filmaj]

I've filed #125 first so I can figure out how to run the tests first before I attempt to change anything :)

[filmaj]

I think I will take a closer look at this issue. I would also require that we expose some manner of authentication, e.g. Basic Auth, for JIRA (our corporate JIRA instance requires either OAuth or Basic Auth).

[filmaj]

I am able to get perceval to talk to my corporate JIRA instance and get past basic auth, etc. I promise to look into this issue soon 😄

[filmaj]

Question: would configuring the JIRA backend via the mordred config to a specific JIRA project somehow result in the issues pulled in from that JIRA project be associated with the correct (git/github-backed) project defined in projects.json? Or would that functionality be part of the changes we want to implement in this issue?

For example, we have an internal project called RSP. I could set the mordred config file as follows:

[jira]
raw_index = jira_test-raw
enriched_index = jira_test
project = RSP
user = me
password = mypassword

Then my projects.json looks something like this:

    "RSP": {
        "github": [
            "https://git.corp.adobe.com/React/RSP"
        ],
        "git": [
            "git@git.corp.adobe.com:React/RSP.git"
        ],
        "jira": [
            "https://jira.corp.adobe.com"
        ]
    }

Would this setup be sufficient for the JIRA issues under the RSP JIRA project to be associated with the RSP project as defined in projects.json? If so, great, that unblocks my immediate work. If not, then would that feature be part of the solution for this issue?

[filmaj]

It looks to me like some tests in sirmordred pass command line flags/parameters via the main (URL?) string inside projects.json: https://github.com/chaoss/grimoirelab-sirmordred/blob/master/tests/test-projects.json#L28

However, when I try this w/ JIRA (as in your original example above @jgbarah), I get errors as it seems the perceval JIRA backend assumes everything in the string to be a URL.

I dug through the code a bit, and I'm not sure whether this would necessitate a change in either perceval or mordred, but perhaps grimoire_elk? Here's my reasoning:

1. The mechanism within mordred that kicks off data collection from various backends seems to be controlled via the Collection task.
2. "URLs" for backends are then filtered through a couple of Task parent class methods called compose_p2o_params and compose_perceval_parameters_from_url.
3. These methods themselves rely on method that exist on the backend "ocean" classes - which are part of grimoirelab-elk.
4. Over in grimoirelab-elk, the JiraOcean class inherits from the ElasticOcean class - which has a very simple implementation for get_perceval_params_from_url, essentially utilizing the entire string as a URL.

So my thinking is: why not implement the get_p2o_params_from_url and get_perceval_params_from_url methods for the JiraOcean class? And make it create proper objects that provide support for the full suite of parameters that perceval's JIRA backend class accepts, like project, user, password, etc.?

Let me know if that makes sense and I can open a new issue in elk and attempt to put a PR together.

[valeriocos]

Sorry for the late reply!

The current implementation should be fine, in a nutshell all data stored in a Jira is reflected to the raw index, while the enrichment phase can be set up to enrich only some projects. To achieve that, you should have a projects.json like the one below (more details about the filter-raw option can be found at: https://github.com/chaoss/grimoirelab-sirmordred#projectsjson).

{
    "acme.A": {
        "jira": [
            "https://acme.jira.com --filter-raw=data.fields.project.key:<project-key-of-your-project>"
        ],
        "meta": {
            "title": "A Documentation"
        }
    },
    "acme.B": {
        "jira": [
            "https://acme.jira.com --filter-raw=data.fields.project.name:<project-name-of-your-project>"
        ],
        "meta": {
            "title": "B Documentation"
        }
    },
    "unknown": {
        "jira": [
            "https://acme.jira.com"
        ]
    }
}