-
Notifications
You must be signed in to change notification settings - Fork 184
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
GSoC idea: Reporting of CHAOSS Metrics #70
Comments
Hey @jgbarah I'd like to work on this project for Google Summer of Code 2018. |
Hi, @apsknight have a look at the description of this issue, and let me know if you need further clarifications. |
Hi @jgbarah |
Hi @vishnoitanuj. For Elasticsearch, I've added in the description of this issue two refernces, to the most usual Python packages to access it. I would recoment going through them, and maybe some tutorial for starters on Elasticsearch. WRT Mordred, please let me know what issues you're facing, I will try to produce documentation to help you address them, if I can. |
Hi @jgbarah , is it suitable for you if I create my private repo with the work and share it with you and your colleagues once I finished? Sorry if the question is trivial, but I was a bit confused by saying "open a GitHub repository, and upload to it:" Best, |
Hello @jgbarah , i am quite interested in contributing in this project . Can you guide me through ? |
@jsdabrowski: the idea is that you open a public repository, and work in it. We will publish soon how to inform us about the url of the repository. |
@gopal86, did you have a look at the microtasks and guidelines in the description of this issue? |
I need to ask that to work on this project for Google summer of code, I need to compete those microtask provided?? |
Hey @sreetamparida, these tasks are for you to understand the libraries and how to analyse repositories using GrimoireLab tools and ELasticsearch. You can also complete these tasks to show that you are capable and comfortable in working with the above listed technologies.
|
@sreetamparida we need to evaluate candidates. If we have people fulfilling the microtasks, that will help us a lot to know about their skills. |
@aswanipranjal thanks for your comment! Yes, it is as you say. In fact, it will not only something that will help us to decide, but also you to know about the project, and about your own skills in this area. |
@jgbarah How to update you about the repository ? |
@apoorvkhare07 we're about to publish a message in the mailing list about that. Please, stay tuned! |
@jgbarah Was the message published? , because I joined the mailing list recently. |
@f2014169 Not yet. |
@jgbarah , could you please include in the mail some information related to a project proposal (what do you expect from it, and what should be included) ? |
@jsdabrowski @apoorvkhare07 @f2014169 you can see the message at: https://lists.linuxfoundation.org/pipermail/oss-health-metrics/2018-February/000517.html |
@jgbarah I am interested in this project. I have installed the necessary tools and have gone through the tutorials and also ran some code. However I had one question in my mind, which may be trivial. The question is: Thank you |
hi @jgbarah , @germonprez . I have completed the microtask-1 . can you guide as to what to do next ? |
@jgbarah do you have any specific format for proposal?? |
Not for now. But let me know if the elements that should be included in it are not clear. |
The raw index is intended to store data as similar as possible to the original data source. Raw indexes are usually rich JSON documents, with all the details obtained from the service API or similar. The enriched index is produced from the raw index, for being visualized with Kibiter/Kibana. It is usually a flat JSON document (because nested documents have some trouble with Kibana), which is in part a summary of the documents in the raw index (because only those properties relevant for visualizing are included), and the result of some computing for some parameters (eg, time to close a ticket, which is not directly available from the API, but has to be computed from the time the ticket was opened, to the time it was closed).
Yes. p2o.py is just a Python script with uses the grimoire_elk module to fetch data (using Perceval), enrich it, and upload it to Elasticsearch. |
@jgbarah what would you like to be Included in the proposal? Would you like a candidate to propose how to approach the project? If so, then more detail are needed. Otherwise, cover letter is enough? |
Just wondering would you be able to use Linux Bash Shell on Windows 10 for the things that will be installed for example Elasticsearch. Or would you use something like a virtual machine like Oracle VM? |
@jsdabrowski the proposal could be as much detailed as you feel convenient. If you have doubts on the scope, or on what is asked for, that's my fault: let me know what you're missing, and we'll complete the description of this issue as much as needed. If you're doubting between different approaches, that's no problem: either explain the alternatives you see, or just select one and expose why you think it is better. That does not mean we're going to decide to go that way, but it is going to tell us about how you deal with selecting choices... But again, if you feel you need more info, let us know which info you are missing, please. |
@kmn5409 I'm not sure what do you mean. If that is if you can run Elasticsearch from a bash (or a powershell) in Windows, I assume you can, although I'm not familiar with Windows, sorry. If that is something else, please let me know. However, for clarifying, almost all the development for GrimoireLab is done in Linux-based systems. That means that using Linux for development is certainly more friendly, and we can provide much more support. |
@aswanipranjal said in chaoss/grimoirelab-tutorial#19:
We may need too add code, or to reuse some code and write the rest from scratch. We welcome your ideas and proposals on how to tackle that, based on your understanding of the code. With respect to specific metrics, please have in mind that not all of them can be retrieved automatically from software development repositories. Some of them could be, for example, collected manually, and written into some properly formatted YAML file, which Manuscripts would read to produce the relevant metrics. But again, we're open to your ideas.
It is the latter, and the number is all of them. Please remember that the idea is that this should run on GrimoireLab indexes, which means the real answer is "all repositories in the indexes considered". In other words, the code should just look at the indexes, find all the repositories in them, and produce numbers with their activity during the last three months (including those with zero activity). Is it more clear now? |
I'm not sure but when I try to create the Kibana dashboard I'm getting the error |
@valeriocos Yeah so this was what I had used to try and fix my problem but I still kept getting other issues |
@kmn5409, can you tell more about the other issues (or the other issues are the ones above?) |
So @valeriocos I'm trying to follow the tutorial for https://grimoirelab.gitbooks.io/tutorial/grimoireelk/a-simple-dashboard.html and when I try to create the dashboard I get the issue |
@Deepayan-Ghosh said:
In short, yes. But details could be different. For example, we could use a combination of scripts and Python notebooks for producing the result. But first step will be doing that with the current Manuscripts machinery, the rest will depend on progress and ideas. |
@kmn5409 said:
Which file are you using for |
@Deepayan-Ghosh said:
The only thing I see different from my setup is that you're using Kibana 6.2.x, and the latest release I've tried is Kibana 6.1.x. Would you mind trying with Kibana (and Elasticsearch) 6.1.x? Meanwhile, I will try with 6.2.x, to see if we have some trouble with it.
Maybe this means something changed in ES 6.1x to 6.2.x... Not sure about that. I will check. BTW, please ensure that you have the latest GrimoireLab pip packages, since those available one week ago were known not to work with ES 6.x. (If you don't mind, tell me which version of the grimoire-mordred package are you using). |
@kmn5409 I'm not able to replicate your error. Below, the steps I did:
Let me know if this helps to solve your issue. |
@jgbarah The same modred config file works and does not give any error, when I used Kibana (and elasticsearch) 6.1.0 and 6.1.3 and 6.1.4 versions. However there is an error with the default index pattern that I previously mentioned here: #70 (comment) and hence I moved to using lastest 6.2.2 versions. On the other hand, I had mordred version 0.1.15 and I upgraded and reinstalled using pip to get version 0.1.16. However, I see in the github repo that version.py of mordred was just updated to 0.1.19. Trying to use the mordred config file using mordred 0.1.16 still gives error:
Also I successfully created report using manuscripts for
|
@valeriocos I don't know if I'm saving the git-dashboard.json file in the wrong place, when you say the /tmp/ folder where is that exactly. I get the same issue as before when I run the command |
Also I would like to know about how Microtask #5, so when it comes to providing things like documentation, it says that we can do it on any of the GrimoireLab tools so would it also include repositories like chaoss/grimoirelab-perceval and even the repository we're in now chaoss/grimoirelab? |
@kmn5409 , open a shell and write cd /tmp/ (you will be in the /tmp/ folder). In a similar way you could store the git-dashboard.json in another directory (for instance with cd ~/, you will be in your home directory). |
Okay thank you for the responses @valeriocos so I saved it in the tmp folder now and when I run the command I now have this error |
|
I was running this command as given in the tutorial
Can anyone help me out ? |
Hi @gopal86, have you imported/installed the sigils/panels package? |
yes @valeriocos , i had installed it . it also shows |
It's possible that you are using an old version of grimoire elk. Please @gopal86, try with the latest version. |
@kmn5409, I'm not able to replicate your issue. Do you have the latest version of kidash, ELK? |
@valeriocos I saw that it might have been the memory on my computer was filled up so I had to make room on it, I assume that was the issue because when I tried to upload things to elasticsearch it was not working, so I think I got it out, and also to answer your question I use docker to get my instance of kibana and elasticsearch. |
Hey @jgbarah I'd like to work on this project for Google Summer of Code 2018. |
Thanks for interest. We do have tasks that we ask potential students to complete. They can be found here: https://lists.linuxfoundation.org/pipermail/oss-health-metrics/2018-February/000517.html You should also note that the student application deadline is: March 27 16:00 UTC. |
GSoC is well over, so I'm closing this issue. |
Hi@jgbarah what would you say ? |
@jgbarah needed your guidance regarding GSoC-19 project contribution. |
@urstrulykkr sorry for being so late with this. In case you're still interested: this issue was for GSoC'18. For '19, we're still in the stage of proposing our project for participate in GSoC'19. We will open issues with the new tasks in due time. Thanks a lot for your interest! |
Hi Jesus,
I have a question to ask,should I wait for the GSoC'19 tasks (micro tasks )
and issues or continue solving the previous year issues.
I think issues related to a project would be good.
Secondly, what would you recommend me to do, so as to get selected into
GSoC'19, to contribute for CHAOSS and learn a lot of stuff with you guys.
Best,
K. Kaushik Reddy.
…On Wed, Jan 30, 2019 at 2:44 AM Jesus M. Gonzalez-Barahona < ***@***.***> wrote:
@urstrulykkr <https://github.com/urstrulykkr> sorry for being so late
with this. In case you're still interested: this issue was for GSoC'18. For
'19, we're still in the stage of proposing our project for participate in
GSoC'19. We will open issues with the new tasks in due time. Thanks a lot
for your interest!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#70 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AohxGGtdjU-zIQOczp2ldnlQ5tTaE_Ffks5vILnKgaJpZM4SI2AM>
.
|
[ This issue for addressing questions and comments related to this GSoC idea, which is one of the ideas proposed by the CHAOSS group for the 2018 edition of GSoC ]
[Edited, 2018-03-03: Added procedure for uploading information for application.]
[Edited, 2018-03-12: Added link to guidelines for proposals]
Currently, GrimoireLab includes a tool for reporting: Manuscripts. This tool reads data from a GrimoireLab ElasticSearch database, and produces with it a PDF report with relevant metrics for a set of analyzed projects. Internally, Manuscripts uses some Python code to produce charts and CSV tables, which are integrated into a LaTeX document to produce the final PDF. Other approaches, such as producing Jupyter notebooks, will be explored too.
This idea is about adding support to Manuscripts to produce reports based on the work of the CHAOSS Community. Since Manuscripts is still a moving target, this will be also a chance to participate in the general development of the tool itself, to convert it into a generic reporting system for GrimoireLab data.
The aims of the project are as follows:
Other aims, such as producing Jupyter notebooks as a final result or an intermediate step are completely within scope.
Microtasks
For becoming familiar with the GitLab technology, it is useful if you produce an analysis (Elasticsearch indexes and dashboard) for a GitHub project (git repos and GitHub issues and pull requests). Have a look at the "Before you start" chapter in the GrimoireLab tutorial to learn how to install the needed infrastructure (ElasticSearch, Kibiter, MariaDB) and GrimoireLab itself as Python packages, or how to deploy everything as a Docker container.
Once you're familiar with producing analysis, you can exploit the information in the indexes via a Python script, preferably presented as a Python Jupyter Notebook:
Microtask 1: Produce a listing of the number of new committers per month, and the number of commits for each of them, as a table and as a CSV file. Use the GrimoireLab enriched index for git.
Microtask 2: Produce a chart showing the distribution of time-to-close (using the corresponding field in the GrimoireLab enriched index for GitHub issues) for issues already closed, and opened during the last six months.
Microtask 3: Produce a listing of repositories, as a table and as CSV file, with the number of commits authored, issues opened, and pull requests opened, during the last three months, ordered by the total number (commits plus issues plus pull requests).
Microtask 4: Perform any other analysis you may find interesting, based on GrimoireLab enriched indexes for git and GitHub repositories.
If you want, you can also:
Of course, there is no need to do all the microtasks, you only need to show that your skills are in good standing for working in this project.
Showing the work you did
If you want to show the work you did, open a GitHub repository, and upload to it:
A
README.md
file explaining what you did, and linking to the results (which will be in the same repository, see below). This will be the main file to show your skills and interest on the project, so try to make it organized and clear, in a way that we can easily understand what you did.Screenshots of the dashboard(s) you produced, and configuration files you used for them (if any). Please remember removing passwords and/or auth tokens that could be in them, before uploading. In the
REAME.md
file, link all those files, telling about which GitHub projects you analyzed.Python scripts and/or Python Jupyter notebooks you produced, with enough information (in comments and/or in the
README.md
file) so that we can run them if needed. Upload Python scripts ready to work assuming GrimoireLab packages are already installed. Upload Jupyter notebooks ready to be seen via the GitHub web interface.Links to the pull request you did (if any), along with any comment you may have about it.
Submitting information for the application process
You must complete at least one micro-task related to the idea you are interested on.
Once you completed at least one micro-task, go to the governance repository and create a pull request to add yourself, your information, and a link to your repository with the completed micro-task(s) in the GSoC-interest.md file (see above for the contents of the repository).
You are welcome to include in your repository other information that could be of interest, such as open issues or pull requests submitted to the project to which you intend to contribute during GSoC, contributions to other projects, skills, and other related information.
You must complete these things by March 27 16:00 UTC. Make sure to also submit the information required by GSoC for applicants (i.e., project proposal), linking to it from your pull request in the GSoC-interest.md file.
Message to the mailing list with guidelines on proposals.
Useful documentation
GrimoireLab Tutorial
CHAOSS Metrics (work in progress)
Elasticsearch Python module: for accessing Elasticsearch from Python, using its REST interface.
Elasticsearch DSL Python module: for accessing Elasticsearch from Python, higher lever (more Pythonic) module.
Asking for help
If you need help, please use the following channels.
For issues related to GrimoireLab:
For general issues related to CHAOSS or CHAOSS metrics
The text was updated successfully, but these errors were encountered: