Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
[REVIEW]: GAMA: Genetic Automated Machine learning Assistant #1132
Status badge code:
Reviewers and authors:
Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)
Reviewer instructions & questions
@jsgalan, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:
Review checklist for @jsgalan
Conflict of interest
Code of Conduct
referenced this issue
Dec 17, 2018
If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews
To fix this do the following two things:
For a list of things I can do to help you, just type:
I have a few suggestions to be made to the authors:
1- add a few examples (not just the minimal example, but all five examples) to the Github repository. Also please check that the examples is wrongly edited, missing the last part of the statement
2- Provide a more detailed examples and explanations of the following sections: Using ARFF files, Logging, Log Visualization and GAMA Search Space Configuration.
3- The authors could add in the manuscript a few lines describing in detai
l logging and log visualization aspects, which can be interesting for a user/reader.
I changed log levels of some statements so that anything of at least
Thanks again for your time.
Sorry for being so vague on my descriptions. I will reformulate:
1- add a few examples (not just the minimal example, but all five examples) to the Github repository.
1.1-Also please check that the examples is wrongly edited, missing the last part of the statement <-- this was done! [check!]
2- What more detailed explanation are you missing for the log visualization? <- can this link or this information be found somewhere on the initial Github webpage
3- We mentioned the log visualization and type of questions you can currently answer with it in our paper submission. What in particular were you missing or expected to be changed?
The article currently reads:
I think is important to include in those lines regarding the aspects such as:
a) the figures/plots it produces. (a.e Fitness over number of evaluations vs No. evaluations, Pipeline length over number of evaluations vs No. evaluations and more importantly how Fitness over number of evaluations by main learner vs No. evaluations )
b) all the methods/models that can be learned (a.e NB, Decision Tree,Boosting, Random Forest, KNN, SVC, LogReg). This is not stated clearly in the documentation.
Sorry for being stubborn but I think this should be main focus of the article and be very well described in the Github website, showing all the full capabilities of the software and all the models that can be tested using the implementation.
After revising I think some extra homework...
4- There were .csv files generated that are not well described anywhere
4.1- There were folders empty folders created with the same name of the .csv files.
Hope this helps.
Could you elaborate what you mean specifically? I made three of them into separate scripts in the example folder and linked them from the
It is referred to in the documentation but I will add it to the
I turned it off for examples now, as I don't think examples should produce extra files. I will add documentation regarding the files should you wish to produce them.
Yes, using the 'cache_dir' hyperparameter when initializing a
I'll make the changes tomorrow.
No news since last comment.
Added link and example image to the
I added a plot example to the paper and also explicitly named some algorithms as well as emphasize using scikit-learn algorithms (see the latest article proof).
I find the adjustments I made are a reasonable balance between providing too little information and all information. Please let me know if you disagree.
They now have their section in the documentation.
I just revised and all the changes suggested were made in documents and pdf, respectively. I am happy to say that from my part all the requirements were fulfilled.
Answering to this comment, after my initial thoughts and replies, I imagined that the package can be (and will be extended) to include different types of tests or can be put into a greater pipeline using various packages, so i concur with your reasoning.
Happy to revise and test this novel tool that will be beneficial for the ML community.
Great work everyone! Thanks for the review @jsgalan!
@PGijsbers : your paper is ready to be processed for acceptance.
Could you please create an archive of the current state of your software (e.g., by creating a tag/version of the software and uploading that to zenodo).
Once you have a DOI for the software, please post it here, so we can add the archive to the paper.
Here's what you must now do:
Any issues? notify your editorial technical team...
If you would like to include a link to your paper from your README use the following code snippets:
This is how it will look in your documentation:
We need your help!
Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following: