Create a ML model for the patch recipients based on the recipients of sent patches

[bulwahn]

We have a lot of data from patches and we can use this for creating a ML model to predict the recipients a patch would be sent to, based on the existing data.

Various questions immediately arise:

1. Prepare the existing data to learn the model.
2. Do we pass the whole patch content as-is for learning, or do we already preselect specific information, such as files, significant keywords in commit message and diff?
3. What is the precision and recall of the learned model?
4. Do certain points, e.g., the sender of a patch, have an impact for the model?
5. Can we relate this model and the information in get_maintainers.pl?

[rsarky]

2: if we do go for a deep learning based technique, it would be better to pass the entire patch as the model would discover optimal higher level features from those.
3. I am assuming false positives (output person is not actually a maintainer) is more acceptable as compared to false negatives (a maintainer is not amongst the outputs). So precision should be important as an output parameter as compared to recall.

I am not sure if this is indeed a good usecase for a ML model as the problem statement isnt that fuzzy. There are a finite and tractable number of rules that can be written down that will give a definite answer to who are the maintainers of a patch.

[fun-akhil]

It would be helpful if the data available could be made public @bulwahn . If not the entire data set, I guess sufficient amount of data to train and experiment various models might be helpful. Also a metadata file would be highly useful

[bulwahn]

All data is available in this repository here: https://github.com/lfd/PaStA-resources

[bulwahn]

@rsarky Your hypothesis (2:) needs to be proved or disproved; I would not state that as a matter of fact. The task here would be to investigate that.
To the point about finite and tractable rules, I will create a new task.

[rsarky]

I agree. I was feeling a bit off about point 2, because if we do indeed choose a subset of features such as those you mentioned, each of them has some rule that you could state that would help in determining the maintainers if I am not wrong. But I would be interested in seeing if this indeed works.

Another question to consider is how do we form a good ground truth. If it is driven by get maintainers script then the model will be inherently limited. A manual ground truth is ideal but quite involved.

[bulwahn]

Here the assumption is that we simply take the full email data as ground truth. How reliably can we predict who a patch will be sent to based on the previous observations?

[rsarky]

So if I understand correctly the assumption is that the existing email data gives a good approximation of who patches should be sent to?

[bulwahn]

Yes, that would be the assumption of the first investigation. We may refine this by giving more weight on "confidence of its correctness" to patches that have been accepted vs. the ones being ignored, or based on the sender's known involvement, e.g., active for many years, known maintainer, etc.

[bulwahn]

@rsarky I have recorded the task on finding simple rules for this investigation in #66