Simulator architecture and API in diffsims

[dnjohnstone]

In #111 @hakonanes asked about where the new ReciprocalLatticePoint or CrystalPlane class will be used in diffsims. This reminded me that we need to look at the overall architecture for different types of diffraction simulation in diffsims, which follows on from an offline conversation with @pc494 and @JoonatanL.

To summarize the current types of simulators in diffsims, we have:

The calculate_ed_data method of DiffractionGenerator computes a spot electron diffraction pattern by evaluating the Structure Factor Fhkl for relevant crystal planes, code is here:

diffsims/diffsims/generators/diffraction_generator.py

Line 93 in 8482796

def calculate_ed_data(

The calculate_profile_data method of DiffractionGenerator computes a one dimensional electron (powder type) electron diffraction pattern based on evaluating the Structure Factor Fhkl for relevant families of crystal planes, code is here:

diffsims/diffsims/generators/diffraction_generator.py

Line 179 in 8482796

def calculate_profile_data(

The calculate_ed_data method of AtomicDiffractionGenerator computes a single crystal spot/disk like electron diffraction pattern for a specified probe function and crystal structure by evaluating the FFT of the crystal potential and probe function. This is similar to one slice of a multislice code, code is here:

diffsims/diffsims/generators/diffraction_generator.py

Line 326 in 8482796

def calculate_ed_data(

The simulate_kinematic_scattering function computes a single-crystal electron diffraction pattern for an arbitrary arrangement of atoms under either plane wave or Gaussian probe illumination by evaluating the structure factor of the arbitrary atomic arrangment. In othe words the atomic scattering factor of every atom in the user defined object is summed to simulate diffraction, code is here:

diffsims/diffsims/utils/sim_utils.py

Line 296 in 8482796

def simulate_kinematic_scattering(

The last two approaches are very helpful because they take account of the object shape properly if you're simulating diffraction from a nanocrystal and they can also be used to simulate scattering from non-crystalline or defective materials. The FFT approach has some advantages in terms of computational efficiency, but direct summation of atomic scattering factors has the advantage that arbitrarily small displacements of atomic coordinates are easy to incorporate.

In short, I think we want all of these as well as adding:

• Kikuchi diffraction
• A multislice simulator
• A bloch wave simulator

Clearly the API here is a total mess and I think we should tear it up and start again after #111 is merged and we re-evaluate these simulators in any case.

I think the people mentioned above have some interest in this and hopefully this helps let everyone know some stuff that is probably only in my head right now.

In writing this down, it strikes me that @CSSFrancis might have some interest too.

[dnjohnstone]

In other work with @robtovey we used MULTEM (https://github.com/Ivanlh20/MULTEM) for some multislice simulations via its matlab interface. I think if we do go with providing access to multislice simulations through diffsims we might consider doing it by writing a python wrapper for that.

[hakonanes]

In short, I think we want all of these as well as adding:

* Kikuchi diffraction

My plans for this:

• Locally I have, using the ReciprocalLatticePoint class, a geometrical model (not using structure factors) for projecting lines ("Kikuchi bands") on an EBSD detector.
• My plan was to create a kikuchipy.detectors module with EBSDDetector and ProjectionCentre (pattern centre) classes, and a kikuchipy.projections module with a Gnomonic class. The EBSDDetector.projection attribute would be set to Gnomonic.

Do these modules and classes overlap with any ideas anyone else have? I'm not interested in writing code specifically to project bands onto a TEM detector, however I can work with someone who is.

[pc494]

Yes, this gets to the point nicely. I think my ideal architecture (right now, subject to change as with everything) for the kinematical case would be:

calculate_orientated (or similar) which does what calculate_ed_data does at the moment, for a single crystal, deals with shape functions etc.
calculate_powder (or similar) which does what calculate_profile_data does, doesn't have a shape factor term, but could (in future) be expanded to deal with textured data

The key point is that internally these should both rely on the same kinematical (structure factor) model, which we can then commit some real effort to making effective. I think this should be what is currently simulate_kinematical_scattering which should also be offered to users for when they have more interesting cases (that they still want to simulate kinematically).

I think AtomicDiffractionGenerator is probably okay as is right now.

Then in terms of dynamic simulations, (okay @hakonanes has dealt with Kikuchi already) I think we should be wrapping things. I've got a fair sense of what's out there for multislice codes (although they all come with their own installation challenges) but in terms of Bloch wave I would need to do some investigation.

I think getting the kinematical case sorted is the first thing though.

[pc494]

My plans for this:

Locally I have, using the ReciprocalLatticePoint class, a geometrical model (not using structure factors)
for projecting lines ("Kikuchi bands") on an EBSD detector.

My plan was to create a kikuchipy.detectors module with EBSDDetector and ProjectionCentre (pattern centre) classes, and > a kikuchipy.projections module with a Gnomonic class. The EBSDDetector.projection attribute would be set to Gnomonic.

Do these modules and classes overlap with any ideas anyone else has? I'm not interested in writing code specifically to project bands onto a TEM detector, however I can work with someone who is.

I would suggest in the medium term having something, however qualitative in is such a big improvement that it's still worth doing. If it's built in a way that makes it extensible, then as/when people want more, more will happen.

[dnjohnstone]

• My plan was to create a kikuchipy.detectors module with EBSDDetector and ProjectionCentre (pattern centre) classes, and a kikuchipy.projections module with a Gnomonic class. The EBSDDetector.projection attribute would be set to Gnomonic.

Do these modules and classes overlap with any ideas anyone else has? I'm not interested in writing code specifically to project bands onto a TEM detector, however I can work with someone who is.

The only thing that comes to mind about this is that we also have a pyxem.detectors module, which defines pyFAI Detector objects for azimuthal integration.

It sounds like your EBSDDetector might incorporate aspects of the experimental geometry, which is something we need to decide how to do best in both transmission ED and XRD - it would be nice if this were harmonized, but I wouldn't let it hold you up!

[dnjohnstone]

The key point is that internally these should both rely on the same kinematical (structure factor) model, which we can then commit some real effort to making effective. I think this should be what is currently simulate_kinematical_scattering which should also be offered to users for when they have more interesting cases (that they still want to simulate kinematically).

Hopefully that's what #102 is going to do, right? At least at first pass, or you mean something else?

And you're saying that we should have one diffraction generator that takes both the crystalline and non-crystalline cases mentioned above where a structure factor is evaluated?

I think AtomicDiffractionGenerator is probably okay as is right now.

My only issue with this one is the name, all of our simulation options are based on atomistic models... FFTDiffractionGenerator ?

[pc494]

The key point is that internally these should both rely on the same kinematical (structure factor) model, which we can then commit some real effort to making effective. I think this should be what is currently simulate_kinematical_scattering which should also be offered to users for when they have more interesting cases (that they still want to simulate kinematically).

Hopefully that's what #102 is going to do, right? At least at first pass, or you mean something else?

And you're saying that we should have one diffraction generator that takes both the crystalline and non-crystalline cases mentioned above where a structure factor is evaluated?

Yeah, #102 is the first step in this certainly. I think what I'm suggesting is that we start thinking in the terms laid out by #102. So yes, there will be a kinematical simulator that deals with gaussian/planar probes and any arrangements of atoms, then we wrap this to give the "common" cases, here just single-crystal and powder.

I think AtomicDiffractionGenerator is probably okay as is right now.

My only issue with this one is the name, all of our simulation options are based on atomistic models... FFTDiffractionGenerator ?

Yep, I will do that alongside #98

[dnjohnstone]

Yeah, #102 is the first step in this certainly. I think what I'm suggesting is that we start thinking in the terms laid out by #102. So yes, there will be a kinematical simulator that deals with gaussian/planar probes and any arrangements of atoms, then we wrap this to give the "common" cases, here just single-crystal and powder.

Ok - just don't forget that we evaluate the Structure Factor for k = g only in the current crystalline / powder case, and you could convolve the answer with a top hat to get a "Bessel probe" solution. But in the non-crystalline case you have to evaluate the Structure Factor for all k on the detector and we've done it by multiplying in by the probe function keeping everything in direct space.

They probably could all use the same code though as you say.

[hakonanes]

I haven't fully understood the "generator architecture"... but I think I have an idea of how to set up a geometrical/kinematical EBSD simulation in kikuchipy (or diffsims):

1. Define objects orix.crystal_map.Phase, ReciprocalLatticePoint (Miller indices) and EBSDDetector (width, height, projection/pattern center (PC), detector tilt, sample tilt [fits best here, I think]). The detector needs to be a class, since it stores so many parameters conveniently.
2. Use these to initialize a DiffractionGenerator object. Acceleration voltage for kinematical simulation, this should be obtained from the ReciprocalLatticePoint object.
3. Pass some orix.quaternion.Orientation to a method to obtain a DiffractionSimulation, storing gnomonic coordinates of Miller indices on the detector as pole lines, also zone axes.
4. Project these gnomonic coordinates onto the detector in kikuchipy either as HyperSpy markers and/or numpy arrays, similar to how it is done with sim_as_signal in pyxem. Preferrably HyperSpy markers should be easily created from the DiffractionSimulation to overlay experimental and/or dynamically simulated (with EMsoft/Bruker DynamicS) EBSD patterns.

What I would like to facilitate is easy adjustements of stuff and immediately visualize the change by just a few lines of code, like changing the detector parameters like PC in the Detector class, the orientations etc.

The detector class is important for coordinate transformations of planes in the unit cell to lines on the detector. Is there some part of the code for projecting spots onto a TEM diffraction detector that could be abstracted out as its own detector class, so that we could replace the EBSDDetector with this TEMDetector (or whatever) and project lines onto this instead?

[pc494]

The detector class is important for coordinate transformations of planes in the unit cell to lines on the detector. Is there some part of the code for projecting spots onto a TEM diffraction detector that could be abstracted out as its own detector class, so that we could replace the EBSDDetector with this TEMDetector (or whatever) and project lines onto this instead?

Not yet, but there is likely to be in future. Just one small point, I think acclerating_voltage should be provided to a DiffractionGenerator directly rather than as be attached to a given ReciprocalLatticePoint

[hakonanes]

It seems to me like the code and peoples interests have moved on beyond this discussion, especially after the introducting of a new simulations module. I'm closing for now, please re-open if I shouldn't have.