feat: Encode Stan writer output using protobuf in C++#286
feat: Encode Stan writer output using protobuf in C++#286ahartikainen merged 9 commits intostan-dev:masterfrom riddell-stan:feature/protobuf-shared-library
Conversation
|
This is not ready to be merged. |
|
@ahartikainen How do I tell gcc on Windows about a library and include directories I have installed with Or is this not going to work because the shared libraries will not be compatible? edit: add correction |
|
I think lib should on the PATH |
|
This contains a lot of useful information: microsoft/vcpkg#8577 . I'm still not sure precisely what to do. Testing is hard without a Windows computer. It seems like we're 95% of the way there. |
Previously Python was used. Doing everything directly in C++ is much faster and we eliminate the dependency on the Python protobuf library. This version requires that the user install protobuf system-wide. Once this solution is verified as working, we can remove this limitation by copying the shared library into place.
It should be possible to get things working on Windows. With luck we can revert this change when someone finds a solution.
|
@ahartikainen Have a look. Everything important happens in the first commit, 4cd2fff. The file httpstan/callbacks_writer_parser.py is removed. The parsing work in that file is moved into C++ files, in httpstan/queue_logger.hpp and httpstan/queue_writer.hpp. Encoding the protobuf messages in C++ means we need to link with the protobuf shared library, which we have to compile and place into the Python package (so subsequent model compilations can use it). |
ahartikainen
left a comment
There was a problem hiding this comment.
Looks good.
I wonder if there is a way to hack docker so it could be used for fast development?
Previously Python was used. Doing everything directly in C++ is much
faster and we eliminate the dependency on the Python protobuf library.
This version requires that the user install protobuf system-wide. Oncethis solution is verified as working, we can remove this limitation by
copying the shared library into place.
Before this is merged:
This version is about 36% faster on longer fits (lots of draws).
Closes #282
Closes #102