WIP: Qrack GPU-accelerated simulator framework as backend #283
Conversation
|
Hi Thanks for this PR and your work. It will take us a few days to have a look at it and we will write our comments as soon as possible. By quickly looking over the files, I have noticed that Qrack is under a GPL license. While this is not a problem, it will require us to decide on a legal solution because GPL prohibit us from providing an integration into ProjectQ without our code adapting the GPL license, too. However, in order to keep ProjectQ as open as possible we insist on keeping the Apache 2 license. A few possible solution came to my mind:
There might be more solutions so I will get back once I have had time to look at everything. It is probably much easier to find the best solution then. I just wanted to let you know so you have time to think about the license issues as well. |
|
I completely understand that licensing compatibility is an issue. If it would help, I can move the Qrack wrapper C++ files into the Qrack repo itself, and include them as part of Qrack's installation process. I am not a legal expert, and I do not know whether this is technically satisfactory. However, if we can think of a way to maintain our respective licensing, I can implement it, and my intent is for both projects to keep their respective licenses. Benn Bollay (https://github.com/orgs/vm6502q/people/bennbollay) and I are the original maintainers of Qrack. I would like to keep Qrack under the GPL, but I can do whatever footwork is necessary to protect your licensing, as well. Thank you. |
|
Thanks for the fast response. I have dealt with these licenses in the past and so I will write a longer explanation soon once I have had time to look at everything. The important point is that it is absolutely possible for you to keep your GPL license. In order for us to keep our Apache 2 license, we cannot include your GPL code or code interfacing to your GPL code into ProjectQ. But the other way around works fine. We can put all the interfacing code into either your current GPL repo or a separate GPL repo. In that GPL repo we are allowed to import any Apache 2 licensed code from ProjectQ (the Apache 2 license only requires to keep our headers and add our notice file). Important Apache 2 code into a GPL repo is allowed an keeps that repo under a GPL license which is more restrictive. User can install this additional backend from that Repo and use it together with ProjectQ... |
|
Thank you for the fast response, as well. I think it's important that a selection of good quantum computing and simulation resources are maintained under the (strong) GPL, but it's not Benn's and my intent to prevent you folks from maintaining Apache 2. It wouldn't be hard to move the C++ code and whatever extent of the wrapper into our base repo, for installation from our end, if that would suffice. Otherwise, I could copy my personal fork of ProjectQ under the same vm6502q organization Qrack is under (named so for our other project, which is a quantum super set of the MOS-6502 instruction set, using Qrack). By the way, I'm trying to fix the coverage in this branch, based on your CI scripts, but I seem to be running into errors outside of the Qrack wrapper. I think I accidentally forked from your "development" branch instead of the master, and I'm not sure if that's at the root of the issue. In any case, please don't hesitate to ask me for any workaround that satisfies your licensing requirements, or work to improve the implementation in the PR. |
|
Damian, take your time responding, I understand you're busy, but I'm thinking about how to get Qrack as a backend while maintaining your licensing. The PR summary says I touched 12 files, but this is basically just an independent extension of ProjectQ with _qracksimulator.py, _qracksimulator_test.py, and the associated C++ code. Without modification to your repo, it should possible to move that code into a submodule under the Qrack repo, with a setup.py that just compiles the feature, and coverage for just _qracksimulator_test.py. If a user installs ProjectQ, they can separately install Qrack and our submodule to enable Qrack as a backend. Any further work that Qrack could use on, for example, the math library, I can open separate PRs for that don't explicitly depend on any Qrack code. I'll start this refactor tonight, unless you folks come back with a different option that you would prefer. Thanks again! |
|
Hi folks - after a bit of discussion, we're going to move the qrack project over to the LGPL license. Would that satisfy your very reasonable considerations and provide a minimum-friction interface between the two components? We'll get on this and sort out the CI issues. |
|
Long and detail answer is coming once I know what your code does ;-) I am a bit short in time in the next days but will try my best to have a look... Both license (LGPL and GPL) work so it is totally up to you what you like best for your code. In both cases we would need to move the interfacing code into a separat repo. So that users need to install it and then tehy could do sth like To achieve this, you can import/copy ProjectQ code into your interfacing code as GPL/LGPL allows to include Apache 2 code (and Apache 2 allows it if you keep the header and notice file so no problem) As a general note: |
|
Regarding the branches:
|
|
Damian, to be clear, there's no library code or binary from Qrack in this PR. Not a file and not a binary of Qrack is in this branch. The intent was, the user separately does a "make install" with our Qrack repo, which puts headers and a library binary in default include and library folders for the user's compiler and system. (The lack of a Qrack binary installed separately by the CI is part of the reason we're failing, at the moment.) Once Qrack is installed this way, the branch in this PR has an optional installation feature in ProjectQ, to compile a wrapper (which is not directly part of the Qrack library) that references a separately installed Qrack header and compiles with the separately installed Qrack binary. I could be wrong, but my impression was that that's hypothetically acceptable use of an LGPL compiled library in a project under a less restrictive or commercial license. Benn and I figuring out the CI, now. We'll get this passing pytest, while we figure out licensing compatibility. Thanks again for your attention. |
|
Damian, I've obviously been hacking away at this since you've been busy. By testing a SimulaQron -> ProjectQ -> Qrack integration, I've been able to get a sense of context for how a back end should fit into ProjectQ in a larger stack, maybe. The ProjectQ unit tests pass with Qrack, as written. The Qrack simulator has direct implementation for some operations the ProjectQ simulator doesn't, and vice versa. If the SimulaQron and ProjectQ unit test float equality tolerances are widened a reasonable amount, most of the SimulaQron unit tests pass on float accuracy, except for state preparation and some of the register merging. Qrack can also be compiled with a cmake flag that enables double accuracy throughout, and I'm working on fixing support for this flag in ProjectQ. As I've already stated, Qrack floors values on order of the residual that tends to be left behind in the other permutations on a sequence like H(0) H(0), which should return the register to its exact original state, ideally. Additionally, when a gate isn't composed purely of exactly (+/-) 0, 1, and i components, Qrack calculates the length of the state vector during each single bit gate application and then normalizes on the next single bit gate. This level of error is basically unavoidable, as far as we can see, and it's about 10^(-6) scale for amplitude components. For double accuracy, it's much lower than this. I had originally intended, back when I opened this PR, for the Qrack simulator to go into ProjectQ as an optional component that was kind of "off to the side." Since playing with SimulaQron, though, I have a sense of how Qrack should (optionally) respond to a compilation flag to be substituted throughout for the default ProjectQ simulator. This has become a somewhat ambitious integration, and I'm marking this a work-in-progress, for now. I estimate that I could have the stack fully functioning by the end of the week. In addition to suppporting the double accuracy compilation option for Qrack in ProjectQ, some of the decompositions and filters don't work with Qrack, yet. These seem to be the primary reason SimulaQron unit tests are failing, within expected tolerances. I will fix support for all the decompositions, and I will finish double accuracy support. By the time I am done with both these, the double accuracy compilation just might pass within the original test tolerances, but I suggest widening the tolerances for the much faster float compilation, which works with many more OpenCL devices. Thank you! |
WrathfulSpatula
changed the title
|
Per #302 I was able to make some scripts that isolate some of the remaining issues. The Gist there points to a problem while setting wave function amplitudes with a "QFusion" gate fusion layer. (Qrack was written to be probability amplitude agnostic.) Even if this passes unit tests, it's not entirely safe until I remove the work-in-progress designation. |
|
This has been debugged, but this PR itself has admittedly become a totally unmanageable mess, months later. Apologies for the thrashing. I'm going to squash this whole thing, over on my personal fork, and give you one succinct comment, with one tidied commit, to start. We can work from there, if Qrack integration is something you'd like to optionally support under the Apache 2.0 license. Thank you again. |
Over at https://github.com/vm6502q/qrack, we've been working on an open source, (OpenCL) GPU-accelerated quantum computer simulator framework, for about a year. Qrack::QUnit also tries to explicitly Schmidt-decompose its global wave function into subsystems, which can majorly improve both memory usage and speed, (per Pednault et al., https://arxiv.org/abs/1710.05867).
I've taken the liberty of wrapping our framework into an optional ProjectQ "backend." To use it, you install Qrack, and then passing the global option "--with-qracksimulator" to the ProjectQ setup.py should compile the backend wrapper with the Qrack installation.
The Qrack framework doesn't allow the application of arbitrary-qubit-count gates, more than one qubit, but I've expanded our project's API to accept arbitrary numbers of control qubits, as your ProjectQ is designed. I've also wrapped optimized, controlled addition/subtraction, multiplication, and division gates, based on your math library. I support your simulator's interface for directly accessing the wave function amplitudes of the simulator, but you should note that Qrack::QUnit Schmidt decomposes the global state into separable subsystems, so these methods in your interface only spit out the representational equivalent of a fully entangled permutation basis wave function, which is not the raw state of Qrack::QUnit.
I've included a _qracksimulator_test.py script based on your cppsim test script, and all tests pass. Your documentation also builds. Anything else I can help with, please let me know!