dials.generate_mask trusted_range bug

[dagewa]

The behaviour of dials.generate_mask appears to be to combine untrusted area definitions with the untrusted_range mask of the first image of a sequence. This causes overloaded spots on image 1 to be persistently masked throughout a dataset, like a flash burn. This example makes it obvious:

cat > mask.phil <<+
untrusted {
  rectangle = 0 1375 1246 1300
}
+
dials.import $(dials.data get -q x4wide)/*.cbf trusted_range=-1,100
dials.generate_mask imported.expt mask.phil
dials.import $(dials.data get -q x4wide)/X4_wide_M1S4_2_00*.cbf mask=pixels.mask

Now see the strong pixels from the first image masked throughout the whole dataset:

[dagewa]

Probable culprit

dials/util/masking.py

Lines 144 to 145 in 9780dfd

	# Get the first image
	image = imageset.get_raw_data(0)

[graeme-winter]

Good spot. Suspect that this was an attempt in $past to find and mask hot pixels. Ouch.

[graeme-winter]

Verified bug but pointy finger of blame misplaced... print(1/0) just after this line does not cause crash.

[graeme-winter]

Silver-Surfer util :) [master] $ rm -rfv masking
masking/__pycache__/__init__.cpython-36.pyc
masking/__pycache__
masking/__init__.pyc
masking

• explains condusion - d'oh

[graeme-winter]

Branched to trusted-pixel-range-mask for fix.

[graeme-winter]

We should discuss how this interacts with multi-panel detectors. Also for large image sets could have better way of doing this e.g. with random sampling?

[dagewa]

I don't understand - I still see the same behaviour with the test case above while on the trusted-pixel-range-mask branch.

[graeme-winter]

  551  dials.import $(dials.data get -q x4wide)/*.cbf trusted_range=-1,100
  552  dials.generate_mask imported.expt mask.phil
  553  dials.import $(dials.data get -q x4wide)/X4_wide_M1S4_2_00*.cbf mask=pixels.mask
  554  dials.image_viewer imported.expt 

While this may not be helpful, it appears to work for me.

[dagewa]

I had old .pyc files too... Working now, thanks