Opt Out of A/B Experiments

[ddurham2]

I believe I'm seeing the effects of an A/B experiment. I've containerized my application which involves an install of the vscode-python extension. After installing I make a modification to the debugger launcher script.

Initially it was ptvsd_launcher.py, then it switched to debugpy's built-in launcher and I had to completely rework the launcher patch.. but recently it seems to have switched back to ptvsd_launcher.py despite pulling the exact same versions of vscode, and the plugin. Digging into the extension's code a bit, I believe I'm seeing it do this due to a A/B experiment.

That was quite disappointing to learn as I've wasted a lot of time, and it probably explains why I've gotten inconsistent behavior in other environments which I've yet to investigate.

Is there any way to disable the experiments so that the internals stay the same until I'm ready to have them change (as version locking is supposed to allow)?

And/or is there a way to make it choose debugpy's launcher specifically? .. since that seems to be the future anyhow.

TIA

[karrtikr]

Apologies, there was a bug which opted users out of experiments. It should be fixed now with da85a76
Reload to activate the extension again, which will download the fresh experiments. Then reload again and the extension will be activated using the new experiments.

[int19h]

If you want to opt out of all experiments, "python.experiments.enabled" setting lets you do this. Or you can use "python.experiments.optOutFrom" to have fine-grained control. Conversely, you can use "python.experiments.optIn" to force some experiments that you want.

In this case, though, the problem was the reverse. We've had the new debugger (debugpy) in the extension for a long time now, and the A/B experiment that enables it has been directing everybody to the new debugger for a while (i.e. it was a 100% / 0%). But we still kept the experiment to allow anybody who runs into issues to opt out from the new debugger and fall back to the old one (ptvsd), until we fix those issues.

Unfortunately, we accidentally changed the ratio over the weekend, forcing fallback to ptvsd for everybody (and thus surfacing all the bugs that have since been fixed in debugpy) - #12481. This has since been fixed, so you shouldn't need to force opt-in anymore - restarting VSCode should fetch the correct experiment values.

[ddurham2]

It's working for me again. Thanks.