Paper: Teaching numerical methods with IPython notebooks and inquiry-based learning (Ketcheson)

[ketch]

No description provided.

[stefanv]

This looks great, thanks! We'll start the reviews soon after the deadline.

By the way, you may be interested to see https://conference.scipy.org/scipy2014/schedule/presentation/23/

There are some announcements on Google+ and elsewhere for more info.

[stefanv]

Dear author and reviewers: please note that the latest PDF version of this paper is always available at http://bit.ly/scipy2014_proc

[minrk]

Dear @ketch, I look forward to reviewing your paper.

[ketch]

@minrk Thanks for volunteering! I pushed some minor changes just now, so if you haven't started yet then please use the latest version. If you already started, that's fine too. ws

[dfolch]

@ketch I'm looking forward to reading your paper. I will get comments back to you early next week.

[stefanv]

Thanks @minrk, @dfolch!

[stefanv]

Any update on this?

[stefanv]

Dear author and reviewers, the final deadline for paper changes is set for the 5th. We're going to extend that by an extra 3 days for one final review + edit cycle, thereby making the sign-off deadline the 8th. Papers that have not been signed off by a reviewer on this date will unfortunately not be included in this year's proceedings.

Thanks for all your hard work so far!

[stefanv]

@dfolch Did you get a chance to look at the paper?

[dfolch]

This paper offers very good pedagogic advice that would be valuable to any instructor who is trying to integrate python into a theory (or methods) course. Although this paper is focused on numerical methods, I would say that its bigger contribution is that it builds a solid argument for the IBL approach. Overall I liked this paper. I plan to implement a similar approach this fall is, and this paper gave me many ideas to integrate into my own class.

High-level comment: Since I see this as an important contribution beyond just the numerical methods community, many of my comments relate to giving additional detail on numerical analysis for the reader with no background in this topic.

• Consider spelling out KAUST the first time it's used.
• Consider starting the paper with the "Inquiry-based learning" section. The current first section is a little jarring without the grounding found in the following section.
• Consider adding a brief introduction to numerical methods. Maybe something that explains that they are computationally intensive and thus require thoughtful coding to get them to give answers in reasonable time using a reasonable amount of memory.
• When you devote time to hands-on learning, do you need to reduce the amount of theory that can be taught? Are you making a trade-off? -> I found the answer near the end of the paper, but it was a question I had on the first page.
• Do the optimization tools, like cplex and gurobi, have a place in the list at the bottom-left of page 2? They are especially interesting since they can both be accessed using python.
• Consider adding multiprocessing to the list of Cython and f2py. multiprocessing is an easy way to use all the cores on the machine, but still use standard python syntax. IPython also has some nice tools for parallel computing.
• Can you expand on the 1-2 hour lab session to get the students up to speed on Python basics. Would it be reasonable to assume that 100% of your students already understand the basics of programming (loops, boolean logic, conditionals, variable types, etc.), and so this short session is simply to communicate python syntax?
• The partial-flipped approach seems like a good hybrid of the two teaching extremes.
• I like the gradual ramp-up of difficulty in the lab sessions using the notebooks.
• What has been your success rate with paired programming? Do you make the pairs or let them team-up on their own?
• Tailoring difficulty to a diverse audience seems like a big challenge for the instructor. If you have more to say on this topic it would be appreciated.
• Consider briefly explaining what multigrid is.
• Are your course materials posted online? I would like to see how your homework notebooks evolve over the semester. Also the animations sound very interesting.
• I'm going to checkout SMC.
• The "lengthy programs" problem is a concern of mine for a pure IPython notebook pedagogic style. I would guess in a work environment the text editor would be the more common tool.
• How do you grade this course (i.e., what is the distribution of points to homework, quizzes, etc.)?

[holtzhau]

I really enjoyed reading this paper, well written and definitely has something to say.

It would be helpful to include a few well chosen examples that highlight the ideas behind Inquiry Based Learning, or at least point to a notebook that demonstrates the concept.

The article would definitely benefit from a few images, the Animations section especially. An article about hands on exploration would feel much more tangible with a bit more visual impact.

[holtzhau]

Fix merge artifact

[ketch]

@dfolch Thanks for the excellent, detailed review. Point-by-point responses below.

High-level comment: Since I see this as an important contribution beyond just the numerical methods community, many of my comments relate to giving additional detail on numerical analysis for the reader with no background in this topic.

Good point; this is the first conference paper I've written for an audience that is not primarily composed of mathematicians. I've tried to make it more accessible to the general Scipy audience, incorporating your suggestions.

Consider spelling out KAUST the first time it's used.

Done.

Consider starting the paper with the "Inquiry-based learning" section. The current first section is a little jarring without the grounding found in the following section.

I tried this, but couldn't get it to work. Hopefully it is less jarring due to my implementation of your next comment.

Consider adding a brief introduction to numerical methods. Maybe something that explains that they are computationally intensive and thus require thoughtful coding to get them to give answers in reasonable time using a reasonable amount of memory.

Done.

When you devote time to hands-on learning, do you need to reduce the amount of theory that can be taught? Are you making a trade-off? -> I found the answer near the end of the paper, but it was a question I had on the first page.

Near the beginning of the paper, I added a forward reference to the Drawbacks section.

Do the optimization tools, like cplex and gurobi, have a place in the list at the bottom-left of page 2? They are especially interesting since they can both be accessed using python.

The things in that list are fairly general programming languages (the mathematical ones are specialized, but you can still do a very broad range of things with them). I don't think cplex and gurobi are general programming languages.

Consider adding multiprocessing to the list of Cython and f2py. multiprocessing is an easy way to use all the cores on the machine, but still use standard python syntax. IPython also has some nice tools for parallel computing.

There are a huge number of things that could be added to this list; I certainly didn't mean for it to be comprehensive. I think a complete list would also include SWIG, Weave, PyCUDA, Numba, and others. I don't think that making such a list is central to this paper, and I don't want to try because I'd inevitably leave something important out.

Can you expand on the 1-2 hour lab session to get the students up to speed on Python basics. Would it be reasonable to assume that 100% of your students already understand the basics of programming (loops, boolean logic, conditionals, variable types, etc.), and so this short session is simply to communicate python syntax?

I added links to two examples of notebooks for such lab sessions. And yes, I generally assume that students understand programming basics. This is a luxury I can afford in a graduate course, that probably wouldn't be valid in an undergraduate course.

What has been your success rate with paired programming? Do you make the pairs or let them team-up on their own?

It's challenging because students may feel uncomfortable pairing up or intimidated at having someone else read their code. But I think it's worthwhile for them to overcome those things. I usually tell them to pair with someone next to them. If anyone is left unpaired, I assign them.

Tailoring difficulty to a diverse audience seems like a big challenge for the instructor. If you have more to say on this topic it would be appreciated.

It's something I'm just learning to do. I added a link to an example.

Consider briefly explaining what multigrid is.

I've added some explanation, which hopefully makes some sense to those who aren't familiar with it.

Are your course materials posted online? I would like to see how your homework notebooks evolve over the semester. Also the animations sound very interesting.

I've added links to my course materials, near the beginning of the paper.

The "lengthy programs" problem is a concern of mine for a pure IPython notebook pedagogic style. I would guess in a work environment the text editor would be the more common tool.

Yes, that's true. In my own research, I use a text editor much more than the notebook, although my usage of the notebook is increasing. A primary reason for this is the lack of Vim keybindings in the notebook, which I think will be remedied soon.

How do you grade this course (i.e., what is the distribution of points to homework, quizzes, etc.)?

I don't think this is essential to the paper, but I added a link to the syllabus, which contains that information.

[ketch]

Thanks for the review and comments, @holtzhau .

It would be helpful to include a few well chosen examples that highlight the ideas behind Inquiry Based Learning, or at least point to a notebook that demonstrates the concept.

I've added links to several of my notebooks.

The article would definitely benefit from a few images, the Animations section especially. An article about hands on exploration would feel much more tangible with a bit more visual impact.

I added a couple of notebook screenshots. It's a bit difficult to make good use of them because the text in them is so small when inserted in the paper. If you have suggestions for other figures, they would be welcome.

[stefanv]

Feel free to make the figures wide if they aren't already.

[minrk]

computational

[minrk]

I agree with most of @dfolch's review, and think it's largely addressed by revisions.

I think that one of the notebook's most often criticized (rightly so) weaknesses as a tool for teaching students programming is that notebooks are not libraries, and notebook-based teaching can often fail to convey practices for building and testing code as libraries. The article mentions 'lengthy programs' as something that benefits from a traditional text editor, but discussion of reusable library code would probably be prudent.

The fourth bullet in the initial list of goals is 'Test', but no discussion is made of traditional code testing tools (unit tests, etc.). Do you not do any formal testing of code? Some discussion of your experience with testing and verification would be appreciated.

[ketch]

@minrk Thanks; I added the following near the end of the paper:

Placing code in notebooks also makes it harder to reuse code, since
functions from one notebook cannot be used in another with copying.
Furthermore, for the reasons already give, the notebook is poorly suited to
development of library code. Exclusive use of the notebook for coding
may thus encourage poor software development practices. This can be
countered by teaching students to place reusable functions in files and
then importing them in the notebook.

Regarding testing: I don't have students do any formal testing. I do make them design and run convergence tests and other checks in a less formal way, as exercises. I'm toying with the idea of including some formal testing next time.

[minrk]

Thanks, sounds good. One typo:

Furthermore, for the reasons already give[n]

[stefanv]

Merged into the 2014 branch.