stricter pre-release test plan

[totaam]

Issue migrated from trac ticket # 660

component: core | priority: major | resolution: fixed

2014-08-23 04:21:09: totaam created the issue

---

What we have right now does not work, we have too many tickets causing regressions (#614), regressions that don't get tested (#658, #655 / #656), then I ended up messing up #651.

We need to find all these issues before the release, not after.

I think I will cut the version from trunk roughly 2 weeks before the actual release date, make a release-candidate build, raise all outstanding tickets to release-blockers and create a tracker ticket to ensure a number of test scenarios are tested properly (platforms, OS / Python / toolkit versions, backwards compatibility, encodings, etc..) - see Testing.

[totaam]

2014-08-23 06:41:47: onlyjob commented

---

I'd like to suggest introducing some post-build unit tests. Long time ago I expressed some concerns about automated testing in #177 so perhaps it could be useful to run at least some tests on every build. We can use xvfb-run to run tests that require X.

[totaam]

2014-08-24 05:42:25: totaam commented

---

The problem with the tests is that very few of the problems we encountered recently would have been found using automated tests or unit tests.

That said, it can't hurt. So I am going to try to deal with #177.

[totaam]

2015-03-14 03:38:01: totaam changed status from new to assigned

[totaam]

2015-03-14 03:38:01: totaam changed owner from antoine to totaam

[totaam]

2015-03-14 03:38:01: totaam commented

---

This one should have been caught: #822 (just start shadow and attach)

[totaam]

2015-04-30 16:15:42: antoine changed status from assigned to new

[totaam]

2015-04-30 16:15:42: antoine changed owner from totaam to afarr

[totaam]

2015-04-30 16:15:42: antoine commented

---

The more complete integration tests will be dealt with in #847, #177 is now closed.

I have tagged 0.15.x in 9202 and I am building updated beta packages. Unless something major shows up, I will release 0.15.0 in about 5 days.

Of all the [/query?status=assigned&status=new&status=reopened&group=status&milestone=0.15 remaining 0.15 tickets], there is only one "blocker" (#770) and even this one will not hold up the release.

@afarr / testers: please take a look at [/milestone/0.15] and add to this ticket with the various end-to-end tests that you've run to verify that the new features do work, and that we haven't borken any platforms or existing functionality.
One tool that is worth using during testing is the "box painting": #760

I've done quite a bit of testing today between total power cuts.. including:

• sound speaker forwarding may cause server to hang #669
• osx builds migrate osx build to using jhbuild and modulesets #533
• printer forwarding support for printing via a cups backend script #598
  etc, and I'll do more over the next few days.
  But ultimately, I would like the go-ahead decision to be out of my hands.. (you can call me a coward!)

[totaam]

2015-05-01 08:52:04: onlyjob commented

---

I don't want to hold release but 5 days (not even 5 working days) is too little even to schedule testing... I have most of the next week already booked so it looks like I will be testing 0.15 only after release (which is OK I just wish the notice wouldn't be so short)... Congratulations for new milestone and good luck with release, Antoine. Thanks for your much appreciated hard work on Xpra.

[totaam]

2015-05-05 13:28:52: antoine commented

---

Actually, I am quite happy to delay things a bit more to ensure more tickets get closed.
Tagging 0.15 has allowed me to look at new things in the meantime, which is also very helpful in identifying bugs. And we even have one ticket re-opened: #626

[totaam]

2015-07-28 07:53:53: antoine changed status from new to closed

[totaam]

2015-07-28 07:53:53: antoine set resolution to fixed

[totaam]

2015-07-28 07:53:53: antoine commented

---

Not heard back, closing - will follow up in #847.