Minutes

Mon 11/15/2010 (Joe, Aleks, Kee-Siong, Florian, Eugene, Caleb)

• Joe's thoughts on package management (Package Manager Notes) >> will use RPM and port it to Mac OS
  • thinking through DB migrations
  • polishing approximate counting
• Florian replaced Cephes library by own implementation for computing Student-t distribution (nice!)
  • is working on at Naive-Bayes interface and SQL only code
  • will look at logistic regression plpgsql code by Steven & Luke
• Eugene to look at the Decision Tree
• Kee Siong will talk to Jeff and Caleb about SVEC road map and dev plan.
• Caleb made github accessible from internal network (Yay!)
  • will be reviewing code as soon as reviews are created.
• Aleks to schedule initial reviews using Fisheye/Crucible
  • finishing up k-means clustering (multi-user support, doc)
  • need to "materialize" RelEng resource anytime now
  • start discussing cooperation with GP QA

Mon 11/8/2010 (Joe, Aleks, Kee-Siong, Florian, Eugene, Gavin)

• Development Status page
• Joe to pursue pkg mgr experiments
• Joe to ponder: python is for what? loops, 2nd order, testing harness.
• Gavin points out: API levels, documentation, unit testing (discussion postponed until we have the initial code).
• Aleks to coordinate setup of Fisheye/Crucible/Github environment for the team.

Mon 10/25/2010 (Joe, Aleks, Kee-Siong, Florian, Eugene, Gavin)

• Status
  • Joe: CountMin mods for Frequent Values in the works, challenge is parallelization
  • Florian to begin port/checkin of linear regression
  • Kee-Siong to begin port/checkin of sparse vectors, linking to lapack
  • Eugene to begin port/checkin of decision trees (and to send out a technical ref to the algorithm)
  • Gavin to send out a preliminary package mgr design doc
  • Aleks to begin port/checking of k-means
  • General note: as a group we need to gain experience/design pointers on writing "vanilla SQL" with external drivers in lieu of PL/PGSQL.
• Test harness to be postponed until more experience gathered
• Code review/QA process:
  • Gavin has 4 hours/week to do code reviews (once he returns)
  • When Gavin in San Mateo (week of 11/22) he will drive import of MadLib into GP QA process
• Questions remain on deploying PL/C to GP customers
• Brief discussion of API, including the Oracle way of passing parameters via table insertion. Some sense that we don't like that but it merits more discussion and experience
• GitHub seems week on issue tracking, etc. We need a solution to that before we release to public.

Mon 10/18/2010 (Gavin,Aleks,Steven)

• talk to Caleb about the internal functions vs madlib code changes
• methods for ver 1.0:
  • approximate counting (in works - Joe)
  • k-means (in works - Aleks)
  • random sampling (in works - Aleks)
  • linear regression - shared code GP/madlib (in queue)
  • naive-bayes - shared code GP/madlib (in queue)
  • sparse vector - shared code GP/madlib (in queue)
  • logistic regression (in queue)
• ongoing code review (critical!)
  • all of the above methods should be reviewed by at least two of Aleks/Gavin/Caleb
  • review for performance, code standards, code structure, file/directory structure, tests, documentation
• code review software (github plugin ?)
• package manager ver1.0 specs (Gavin)

Tues 10/5/2010

• Joe to drive todo list
  • build tools (Gavin drives)
  • directories (Joe drives)
  • API (Aleks drives)
• Aleks to set up meeting for team
• Discussion on API:
  • keep working through examples to understand tradeoffs
  • try to boil out distinguishing features: e.g. via type signatures of the code, e.g. via computational patterns like iteration/recursion, etc.
  • eventually we need a set of Guiding Principles for the next person writing a routine: help them figure out which style to use.