-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Request for a tutorial demonstrating simple implementation of cookie-cutter-data-science framework #59
Comments
I have similar questions about |
@pgr-me @mnarayan Thanks for raising this — if you're finding it confusing, there are probably others who are too. In terms of how to improve the documentation/comments and potentially add to the content about how to use this repo, it'd be helpful if you could share some specifics here about what you found confusing or difficult. |
Sorry about the delay in responding - I've been on holiday the past two weeks. I recommend generalizing this example so that it leverages all the functionality of the cookie-cutter-data-science framework. I was able to use this example to meet my needs, but it may be useful for others if you provide step-by-step instructions showing how users can take the default cookie-cutter-data-science framework and make it into said example. This could mean showing users how to:
|
with the make_dataset.py main function it self i found that understanding the Click library was enough. This webcast made by the developer of Click is quite good https://youtu.be/kNke39OZ2k0 . I remember you mentioned in the tutorial/presentation/documentation that the idea of using make was inspired by the necessity of building data pipelines. I am an avid user of airflow so pipelines are natural to me. So maybe a practical example should be enough, like getting a standard modelling tutorial and pipeing the analysis through. It could even be a completed or iconic DrivenData competition/tutorial. |
Like the others here, I have problems figuring out how to get started and integrate my workflow. Regarding your question, @isms: I think a small sample project with all the necessary steps (from preprocessing to model) already implemented would be very beneficial for beginners. No need to do anything fancy, just derive some features and train a decision tree. For example by using the Titanic Kaggle challenge most people should be familiar with: https://www.kaggle.com/c/titanic/data. Or even easier: by integrating an example withing the base project. If you're new it helps to get started, if you're experienced, you'll have no problems to delete it. Once I understand it well enough and am able to use the project, I'm going to give it a shot. A simple example repository should only take me a few hours. Things that I could not figure out right away:
Proposed steps for a tutorial:
|
You can have a look at https://github.com/hackalog/bus_number/ which was from a tutorial we just gave at PyData NYC. There's a sizable framework in src/, but you should be able to see the basic linkage between the Makefile and the various scripts. |
@lorey Any update for this? Frankly, i couldn't understand how is it possible if we download data from internet with make_dataset script. Also how the data could be passed into interim folder. |
hey @isspek, great timing. I actually released a project containing a minimum working example last weekend (although extended/adapted to LaTeX generation but this shouldn't matter). Should be quite easy to grasp along the example implemented. You can find it here: https://github.com/lorey/data-intensive-latex-documents BTW: I have the sad feeling that this project has been neglected by the authors. I've been using this for the last two years on several occasions and there has not been any significant update since. I have found no better alternative though. |
You could check out the work we are doing with
https://github.com/hackalog/cookiecutter-easydata which we maintain quite
actively. It started as our experimental fork of this project.
Also see the tutorial at https://github.com/hackalog/bus_number/
…
BTW: I have the sad feeling that this project has been neglected by the
authors. I've been using this for the last two years on several occasions
and there has not been any significant update since. I have found no better
alternative though.
|
For context, there is a massive tension between most contributors' wish list ("Feature _______ should be added because in my work I do _____") and keeping the project general. We tend to keep issues open to promote discussion, but there is a strong rationale for not adding complications, and we encourage people to fork the project for particular use cases. |
Preface: Data scientist, not a software engineer. I wrote up the first steps of using cookiecutter datascience here. If there's some way to make an open document (like a gist?), I wouldn't mind contributing the perspective of someone who has no idea what they're doing. Some future steps I'd like to do are adding a git init, setting up some logins in .env, pip freezing requirements and putting in requirements.txt, and using an S3 bucket. Maybe commands for Mac (all the other datascience students used Mac) would be nice. Maybe some instructions for venv/conda people. |
This is now included in the docs: |
Hello,
I'd like to use the cookiecutter-data-science framework for the project that I'm working on, but unfortunately I'm having trouble getting started. Specifically, I'm having trouble figuring out how to configure the make_dataset.py file to execute any python data-making scripts. I'm sure the fix is pretty basic, but I've been spinning my wheels for awhile trying to figure this issue out.
It would be great if you could provide a basic tutorial demonstrating a simple implementation of your framework that people like me could use to get started.
Thanks!
The text was updated successfully, but these errors were encountered: