comment on post-release buts

[timm]

Reviewer3:

• You compare Figure 8 to Figure 1 to point out that they don't match. These are graphs of different things. The y-axis of Figure 1 is cost and the y-axis of Figure 8 is number of defects. You've pointed out that the past data ignored the phase where the defect was introduced - so I don't see the point of Figure 9. (Overall I don't see the point of investigating H1, but if you are going to include it, this is a big problem with the analysis.)

Reviewer 1:

• The claims are not commensurate with evidence. The projects in
  the sample are homogeneous, small, and do not contain post-release
  defects.
• To note a few weaker sides, the study is based on mostly small to
  medium projects, but the authors themselves pointed out (in section
  3, Figure 4) that Phase Delay effect and the cost to fix an error
  are most important for larger projects. More importantly, one would
  assume requirement errors would have more effect for larger
  projects, since they are supposed to contain more functionality and
  also to be general, in contrast with the small or medium projects
  that are specifically designed for some particular process/ intended
  for a specific group of users in many cases. So it may not be
  justified to claim that hypothesis 3 , "Requirements errors are the
  most expensive to fix", is wrong, as mentioned in the paper, when
  considering only small & medium projects and ignoring post-release
  issues. Moreover, the study accounts for the phases in which some
  defect was injected and the phase in which it was fixed, but no
  information is provided about the absolute time scale. One would
  assume a long running project is supposed to have more serious
  impact in terms of cost-to-fix.
• The blanket claims are not justified by the narrow domain size and practices
• The problem is supposed to be most prominent in large projects
• No post release information: the critical concern of phase delay
• Confusing terminology