Backend abstraction

[stavros11]

This is a toy implementation of what is discussed in #300 regarding the numpy/tensorflow backend abstraction and switcher. @scarrazza I used the idea you proposed for the switcher with a few simplifications. I tested with the following script:

import numpy as np
from qibo import set_computation_backend, K

x = np.random.random((10, 5))
y = np.random.random((5, 8))

set_computation_backend("numpy")
z = K.einsum("ab,bc->ac", np.copy(x), np.copy(y))
w = K.sum(np.copy(y), axis=1)
print()
print(type(z), z.shape) # should be numpy arrays
print(w)

set_computation_backend("tensorflow")
z = K.einsum("ab,bc->ac", np.copy(x), np.copy(y))
w = K.sum(np.copy(y), axis=1)
print()
print(type(z), z.shape) # should be tensorflow Tensors
print(w)


set_computation_backend("numpy")
z = K.einsum("ab,bc->ac", np.copy(x), np.copy(y))
w = K.sum(np.copy(y), axis=1)
print()
print(type(z), z.shape) # should be numpy arrays
print(w)

and the switcher seems to work as expected.

Please have a look and if you agree I can add the rest of methods we need in the backends and then think how to integrate it with the rest of Qibo. Ideally we would like almost the whole Qibo code (perhaps except gates) to be free of tf calls and use K instead.

[scarrazza]

@stavros11 thanks for this. It implements exactly what I had in mind and I hope it simplifies the code.
Moving the implementation to the backend/interface.py sounds reasonable, however shall we move the BaseBackend to base?

[stavros11]

Moving the implementation to the backend/interface.py sounds reasonable, however shall we move the BaseBackend to base?

That's a good suggestion. My idea is to first try and refactor the Qibo code (circuits, models, Hamiltonians, etc.) to use K instead of tf. Depending on how this works it may be possible to simplify our current structure where we have a different folder for each backend.

Right now I am not sure what is the best approach in terms of module structure. I think this should be built in terms of dependencies. One possibility is to have the base independent of all external libraries and have a different folder for the models that actually do the calculation (similar to what we have now, although currently base depends on numpy and I am not sure if it is a good idea to remove that dependency).

[scarrazza]

Ok, that's a good point. So ideally the code is trying to do:

• use math for constants and expected python numerical types
• use numpy (from qibo import numpy) for objects which must be or are expected to be numpy objects
• use K in places where objects rely on the typo of the state.

The idea is good, however in places like here, it supposes the parameters will be as numpy arrays or compatible formats (tf which casts to numpy) or python lists. I am fine with these choices, but would be great to have some place where we summarize how to use K, qibo.numpy and math.

[scarrazza]

For example here, if we implement jax we may prefer to use K.kron in order to use the GPU allocation.

[scarrazza]

This is just a comment, from a practical perspective we don't need this level of flexibility for these objects.

[scarrazza]

@stavros11 thank you very much for this masterpiece!
The implementation looks very good to me, and matches the layout needed by our projects.

You can find some minor comments below, but in general looks pretty good, my only concern is about the usage documentation of math, K, qnp.

This PR with #293 and eventually a new state interface #300 should be sufficient for the time being.

[stavros11]

Thank you for the detailed review. I fixed some of the above comments. I also removed measurements.py and fusion.py from base. These objects are not used by any other abstract objects so it is not required to have them in base and I believe the code is simplified if we keep them only in core. If in the future we add a completely different backend (eg. hardware) that requires to abstract some of this (for example measurements) we can do it accordingly.

I will continue working on the remaining comments and will try to also improve documentation.

[stavros11]

I believe I fixed most of the above comments except the following two:

First, regarding the initial state custom operator in the defaulteinsum/matmuleinsum, as I wrote above we could probably do the state initialization using Tensorflow primmitives for these backends so that we keep them completely custom operator free.

Second, regarding the math/qnp/K usage.

math is used only in very simple cases where we really need just a scalar mathematical operation (like log or square root) for which we don't need to import a full linear algebra library. This is mostly done to keep abstractions free from any backend (numpy, Tensorflow, etc.). I don't think this would be useful for external users, it is up to them if they prefer to use numpy even for these operations as it is usually more convenient.

Regarding qnp, the reason it exists is that in some cases (eg. gate matrix creation or Trotter Hamiltonians) we force using numpy for some specific operations even when the user selected backend K is Tensorflow. This is mainly because numpy is faster for small matrix operations. An alternative and possibly simpler design that I currently thought is to completely remove qnp and just have a "small operation" backend within K, which we could call as K.np. Then we would replace for example qnp.kron with K.np.kron.

The difference between K.matmul and K.np.matmul would be that the first would be either numpy or tf, while the second will be forced to numpy for all K. If we add a different backend (eg. Jax) that is fast for both small and big operations, we could define its K.np to be Jax and in this case both K.matmul and K.np.matmul would be Jax.

[scarrazza]

@stavros11 thanks for the updates.

First, regarding the initial state custom operator in the defaulteinsum/matmuleinsum, as I wrote above we could probably do the state initialization using Tensorflow primmitives for these backends so that we keep them completely custom operator free.

Lets keep the custom init for all tf backends.

The difference between K.matmul and K.np.matmul would be that the first would be either numpy or tf, while the second will be forced to numpy for all K. If we add a different backend (eg. Jax) that is fast for both small and big operations, we could define its K.np to be Jax and in this case both K.matmul and K.np.matmul would be Jax.

I tend to prefer this idea, so we reduce the number of objects which may refer to numpy but maybe not, say with jax, or tf.numpy.

Some other comments when testing the code:

• would be great to have a more helpful error message for set_backend(backend) if backend is not supported. Ideally we could simply print the available backend names.
• if qibo.K.name == 'numpy' and I try to set_device('/device:CPU:0') (same for GPU), I get "Device /device:CPU:0" does not exist". This error message may be misleading, so we could simply raise a warning message, saying that numpy does not support CPU:0 e GPU:0, or keep the default not implemented error.

[stavros11]

I tend to prefer this idea, so we reduce the number of objects which may refer to numpy but maybe not, say with jax, or tf.numpy.

I replaced qnp with K.qnp and I removed it from __init__ so that from qibo import numpy is no longer possible to avoid confusing users. Note that K.qnp is still the Qibo numpy backend and not the full numpy that we get with import numpy as np. The reason I am using the Qibo backend is that it has the proper dtypes coded in it.

• would be great to have a more helpful error message for set_backend(backend) if backend is not supported. Ideally we could simply print the available backend names.

I updated this error message and now it prints all the available backend names.

• if qibo.K.name == 'numpy' and I try to set_device('/device:CPU:0') (same for GPU), I get "Device /device:CPU:0" does not exist". This error message may be misleading, so we could simply raise a warning message, saying that numpy does not support CPU:0 e GPU:0, or keep the default not implemented error.

I disabled set_device completely for the numpy backend. Now if this is used when K is the numpy backend it will only show a warning that device placement is not supported by numpy and will not make any changes.

[scarrazza]

@stavros11 thank you, everything looks good. Let's wait for the test and then merge.