Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade protobuf3-cpp to 24.4 and fix up its dependencies (mosh, grpc, #21170

Closed
wants to merge 5 commits into from

Conversation

blair
Copy link
Contributor

@blair blair commented Nov 1, 2023

Description

Protobuf has dropped support for C++11: https://protobuf.dev/news/2022-08-03/#c11-support . So patch dependencies to force them to use C++14.

Type(s)
  • bugfix
  • enhancement
  • security fix
Tested on

macOS x.y
Xcode x.y / Command Line Tools x.y.z

Verification

Have you

  • followed our Commit Message Guidelines?
  • squashed and minimized your commits?
  • checked that there aren't other open pull requests for the same change?
  • referenced existing tickets on Trac with full URL?
  • checked your Portfile with port lint --nitpick?
  • tried existing tests with sudo port test?
  • tried a full install with sudo port -vst install?
  • tested basic functionality of all binary files?
  • checked that the Portfile's most important variants haven't been broken?

@macportsbot
Copy link

Notifying maintainers:
@mascguy for port protobuf3-cpp.
@quentinmit for port mosh.

@macportsbot macportsbot added maintainer: open Affects an openmaintainer port by: member Created by a member with commit rights labels Nov 1, 2023
@mascguy mascguy marked this pull request as draft November 1, 2023 13:01
@mascguy
Copy link
Member

mascguy commented Nov 1, 2023

Converting to draft for now, until I've had a chance to test/validate locally.

@blair
Copy link
Contributor Author

blair commented Nov 1, 2023

Some notes.

For py27-protobuf3

Reenabling the 27 build and trying it quickly fails. I didn't look into fixing the failure.

--->  Patching setup.py: s|@@MACPORTS_STDLIB@@|-stdlib=libc++|g
--->  Patching setup.py: s|@@MACPORTS_EXTRAARG@@||g
--->  Configuring py27-protobuf3
--->  Building py27-protobuf3
Executing:  cd "/opt/local/var/macports/build/python_py-protobuf3/py27-protobuf3/work/protobuf-24.4/python" && /opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7 setup.py --no-user-cfg --cpp_implementation build 
  File "setup.py", line 291
    return False
SyntaxError: 'return' with argument inside generator
Command failed:  cd "/opt/local/var/macports/build/python_py-protobuf3/py27-protobuf3/work/protobuf-24.4/python" && /opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7 setup.py --no-user-cfg --cpp_implementation build 
Exit code: 1

For grpc

Not upgrading grpc gets this compile error, so it was easier just to upgrade to the latest version:

[ 94%] Building CXX object CMakeFiles/grpc_plugin_support.dir/src/compiler/cpp_generator.cc.o
/usr/bin/clang++ -Dgrpc_plugin_support_EXPORTS -I/opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_devel_grpc/grpc/work/grpc-1.48.4/include -I/opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_devel_grpc/grpc/work/grpc-1.48.4 -I/opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_devel_grpc/grpc/work/grpc-1.48.4/third_party/address_sorting/include -I/opt/local/libexec/openssl3/include -I/opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_devel_grpc/grpc/work/grpc-1.48.4/src/core/ext/upb-generated -I/opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_devel_grpc/grpc/work/grpc-1.48.4/src/core/ext/upbdefs-generated -I/opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_devel_grpc/grpc/work/grpc-1.48.4/third_party/upb -I/opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_devel_grpc/grpc/work/grpc-1.48.4/third_party/xxhash -I/opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_devel_grpc/grpc/work/build/gens -isystem /opt/local/include -pipe -Os -DNDEBUG -I/opt/local/include -D__STDC_FORMAT_MACROS -stdlib=libc++ -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk  -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk -mmacosx-version-min=14.0 -fPIC -std=c++14 -MD -MT CMakeFiles/grpc_plugin_support.dir/src/compiler/cpp_generator.cc.o -MF CMakeFiles/grpc_plugin_support.dir/src/compiler/cpp_generator.cc.o.d -o CMakeFiles/grpc_plugin_support.dir/src/compiler/cpp_generator.cc.o -c /opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_devel_grpc/grpc/work/grpc-1.48.4/src/compiler/cpp_generator.cc
In file included from /opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_devel_grpc/grpc/work/grpc-1.48.4/src/compiler/cpp_generator.cc:19:
In file included from /opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_devel_grpc/grpc/work/grpc-1.48.4/src/compiler/cpp_generator.h:30:
In file included from /opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_devel_grpc/grpc/work/grpc-1.48.4/src/compiler/config.h:24:
In file included from /opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_devel_grpc/grpc/work/grpc-1.48.4/src/compiler/config_protobuf.h:22:
/opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_devel_grpc/grpc/work/grpc-1.48.4/include/grpcpp/impl/codegen/config_protobuf.h:93:9: error: no type named 'Status' in namespace 'google::protobuf::util'
typedef GRPC_CUSTOM_UTIL_STATUS Status;
        ^~~~~~~~~~~~~~~~~~~~~~~
/opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_devel_grpc/grpc/work/grpc-1.48.4/include/grpcpp/impl/codegen/config_protobuf.h:72:59: note: expanded from macro 'GRPC_CUSTOM_UTIL_STATUS'
#define GRPC_CUSTOM_UTIL_STATUS ::google::protobuf::util::Status
                                ~~~~~~~~~~~~~~~~~~~~~~~~~~^
In file included from /opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_devel_grpc/grpc/work/grpc-1.48.4/src/compiler/cpp_generator.cc:19:
In file included from /opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_devel_grpc/grpc/work/grpc-1.48.4/src/compiler/cpp_generator.h:30:
In file included from /opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_devel_grpc/grpc/work/grpc-1.48.4/src/compiler/config.h:24:
/opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_devel_grpc/grpc/work/grpc-1.48.4/src/compiler/config_protobuf.h:53:10: fatal error: 'google/protobuf/compiler/csharp/csharp_names.h' file not found
#include <google/protobuf/compiler/csharp/csharp_names.h>
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2 errors generated.
make[2]: *** [CMakeFiles/grpc_plugin_support.dir/src/compiler/cpp_generator.cc.o] Error 1

For protobuf3-cpp

I took out the mention of py-onnx here

-# NOTE:   Revbump et, protobuf-c, mosh and py-onnx

Since this comment removed an explicit dependency on protobuf3-cpp: 76ad740

@blair
Copy link
Contributor Author

blair commented Nov 6, 2023

Hi @mascguy, any chance to look at this?

Also, I noticed that this PR upgraded py311-protobuf3 from 4.21.12_2 to 24.4_0. Or should we keep the leading 4. in the version?

Thanks,
Blair

@blair blair force-pushed the protobuf3-cpp-24.4 branch 2 times, most recently from 49b9104 to fceab18 Compare November 13, 2023 06:46
@blair
Copy link
Contributor Author

blair commented Nov 13, 2023

Hi @mascguy, some more work on this.

The previous version of devel/grpc/files/patch-avoid-overlinking-abseil.diff for some reason applied cleanly on my system failed on the buildbots.

Regardless, I updated it to

--- ./cmake/pkg-config-template.pc.in.orig	2023-11-12 21:50:04.134766426 -0800
+++ ./cmake/pkg-config-template.pc.in	2023-11-12 21:57:26.986222807 -0800
@@ -7,7 +7,6 @@
 Description: @PC_DESCRIPTION@
 Version: @PC_VERSION@
 Cflags: -I${includedir}
-Requires: @PC_REQUIRES@
-Requires.private: @PC_REQUIRES_PRIVATE@
+Requires.private: @PC_REQUIRES@ @PC_REQUIRES_PRIVATE@
 Libs: -L${libdir} @PC_LIB@
 Libs.private: @PC_LIBS_PRIVATE@

But it builds the Bear build:

[ 96%] Linking CXX executable wrapper
cd /opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_devel_Bear/Bear/work/build/subprojects/Build/BearSource/intercept && /opt/local/bin/cmake -E cmake_link_script CMakeFiles/wrapper.dir/link.txt --verbose=ON
/usr/bin/clang++ -pipe -Os -DNDEBUG -I/opt/local/include -stdlib=libc++ -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk -fno-exceptions -DSPDLOG_NO_EXCEPTIONS -DGOOGLE_PROTOBUF_NO_RTTI -Wall -Wextra -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk -mmacosx-version-min=14.0 -Wl,-search_paths_first -Wl,-headerpad_max_install_names -L/opt/local/lib -Wl,-headerpad_max_install_names -Wl,-syslibroot,/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk CMakeFiles/wrapper.dir/source/report/wrapper/main.cc.o ../libmain/CMakeFiles/main_a.dir/source/ApplicationLogConfig.cc.o ../libmain/CMakeFiles/main_a.dir/source/ApplicationFromArgs.cc.o ../libmain/CMakeFiles/main_a.dir/source/SubcommandFromArgs.cc.o ../libflags/CMakeFiles/flags_a.dir/source/Flags.cc.o CMakeFiles/wrapper_a.dir/source/report/wrapper/EventFactory.cc.o CMakeFiles/wrapper_a.dir/source/report/wrapper/EventReporter.cc.o CMakeFiles/wrapper_a.dir/source/report/wrapper/RpcClients.cc.o CMakeFiles/wrapper_a.dir/source/report/wrapper/Application.cc.o CMakeFiles/domain_a.dir/source/Domain.cc.o CMakeFiles/domain_a.dir/source/Convert.cc.o proto/CMakeFiles/rpc_a.dir/supervise.pb.cc.o proto/CMakeFiles/rpc_a.dir/supervise.grpc.pb.cc.o proto/CMakeFiles/rpc_a.dir/intercept.pb.cc.o proto/CMakeFiles/rpc_a.dir/intercept.grpc.pb.cc.o ../libsys/CMakeFiles/sys_a.dir/source/Os.cc.o ../libsys/CMakeFiles/sys_a.dir/source/Guard.cc.o ../libsys/CMakeFiles/sys_a.dir/source/Errors.cc.o ../libsys/CMakeFiles/sys_a.dir/source/Path.cc.o ../libsys/CMakeFiles/sys_a.dir/source/Process.cc.o ../libsys/CMakeFiles/sys_a.dir/source/Signal.cc.o -o wrapper  /opt/local/lib/libfmt9/libfmt.9.1.0.dylib /opt/local/lib/libprotobuf.dylib /opt/local/lib/libabsl_log_internal_check_op.dylib /opt/local/lib/libabsl_leak_check.dylib /opt/local/lib/libabsl_die_if_null.dylib /opt/local/lib/libabsl_log_internal_conditions.dylib /opt/local/lib/libabsl_log_internal_message.dylib /opt/local/lib/libabsl_examine_stack.dylib /opt/local/lib/libabsl_log_internal_format.dylib /opt/local/lib/libabsl_log_internal_proto.dylib /opt/local/lib/libabsl_log_internal_nullguard.dylib /opt/local/lib/libabsl_log_internal_log_sink_set.dylib /opt/local/lib/libabsl_log_sink.dylib /opt/local/lib/libabsl_log_entry.dylib /opt/local/lib/libabsl_flags.dylib /opt/local/lib/libabsl_flags_internal.dylib /opt/local/lib/libabsl_flags_marshalling.dylib /opt/local/lib/libabsl_flags_reflection.dylib /opt/local/lib/libabsl_flags_private_handle_accessor.dylib /opt/local/lib/libabsl_flags_commandlineflag.dylib /opt/local/lib/libabsl_flags_commandlineflag_internal.dylib /opt/local/lib/libabsl_flags_config.dylib /opt/local/lib/libabsl_flags_program_name.dylib /opt/local/lib/libabsl_log_initialize.dylib /opt/local/lib/libabsl_log_globals.dylib /opt/local/lib/libabsl_log_internal_globals.dylib /opt/local/lib/libabsl_raw_hash_set.dylib /opt/local/lib/libabsl_hash.dylib /opt/local/lib/libabsl_city.dylib /opt/local/lib/libabsl_low_level_hash.dylib /opt/local/lib/libabsl_hashtablez_sampler.dylib /opt/local/lib/libabsl_statusor.dylib /opt/local/lib/libabsl_status.dylib /opt/local/lib/libabsl_cord.dylib /opt/local/lib/libabsl_cordz_info.dylib /opt/local/lib/libabsl_cord_internal.dylib /opt/local/lib/libabsl_cordz_functions.dylib /opt/local/lib/libabsl_exponential_biased.dylib /opt/local/lib/libabsl_cordz_handle.dylib /opt/local/lib/libabsl_crc_cord_state.dylib /opt/local/lib/libabsl_crc32c.dylib /opt/local/lib/libabsl_crc_internal.dylib /opt/local/lib/libabsl_crc_cpu_detect.dylib /opt/local/lib/libabsl_bad_optional_access.dylib /opt/local/lib/libabsl_str_format_internal.dylib /opt/local/lib/libabsl_strerror.dylib /opt/local/lib/libabsl_synchronization.dylib /opt/local/lib/libabsl_graphcycles_internal.dylib /opt/local/lib/libabsl_kernel_timeout_internal.dylib /opt/local/lib/libabsl_stacktrace.dylib /opt/local/lib/libabsl_symbolize.dylib /opt/local/lib/libabsl_debugging_internal.dylib /opt/local/lib/libabsl_demangle_internal.dylib /opt/local/lib/libabsl_malloc_internal.dylib /opt/local/lib/libabsl_time.dylib /opt/local/lib/libabsl_civil_time.dylib /opt/local/lib/libabsl_time_zone.dylib /opt/local/lib/libabsl_bad_variant_access.dylib /opt/local/lib/libutf8_validity.a /opt/local/lib/libutf8_range.a /opt/local/lib/libabsl_strings.dylib /opt/local/lib/libabsl_string_view.dylib /opt/local/lib/libabsl_strings_internal.dylib /opt/local/lib/libabsl_base.dylib /opt/local/lib/libabsl_spinlock_wait.dylib /opt/local/lib/libabsl_int128.dylib /opt/local/lib/libabsl_throw_delegate.dylib /opt/local/lib/libabsl_raw_logging_internal.dylib /opt/local/lib/libabsl_log_severity.dylib /opt/local/lib/libgrpc++.dylib 
ld: Undefined symbols:
  _gpr_assertion_failed, referenced from:
      grpc::CompletionQueue::Pluck(grpc::internal::CompletionQueueTag*) in supervise.grpc.pb.cc.o
      grpc::internal::CallOpSet<grpc::internal::CallOpSendInitialMetadata, grpc::internal::CallOpSendMessage, grpc::internal::CallOpRecvInitialMetadata, grpc::internal::CallOpRecvMessage<google::protobuf::MessageLite>, grpc::internal::CallOpClientSendClose, grpc::internal::CallOpClientRecvStatus>::ContinueFillOpsAfterInterception() in supervise.grpc.pb.cc.o
      grpc::internal::CallOpSet<grpc::internal::CallOpSendInitialMetadata, grpc::internal::CallOpSendMessage, grpc::internal::CallOpRecvInitialMetadata, grpc::internal::CallOpRecvMessage<google::protobuf::MessageLite>, grpc::internal::CallOpClientSendClose, grpc::internal::CallOpClientRecvStatus>::ContinueFinalizeResultAfterInterception() in supervise.grpc.pb.cc.o
      grpc::internal::InterceptorBatchMethodsImpl::Proceed() in supervise.grpc.pb.cc.o
      grpc::internal::InterceptorBatchMethodsImpl::Hijack() in supervise.grpc.pb.cc.o
      grpc::internal::InterceptorBatchMethodsImpl::Hijack() in supervise.grpc.pb.cc.o
      grpc::internal::InterceptorBatchMethodsImpl::GetSerializedSendMessage() in supervise.grpc.pb.cc.o
      grpc::internal::InterceptorBatchMethodsImpl::GetSerializedSendMessage() in supervise.grpc.pb.cc.o
      ...

Bear can build without the patch, so I've currently disabled it in the build for you to look at and decide if we keep it or modify the patch somehow.

@pmetzger
Copy link
Member

@mascguy I'm going to ask you to remove your lock on this unless you're willing to finish it in the next few days. Please let other people finish the work.

@pmetzger
Copy link
Member

@blair Do you think you could resolve the conflicts and push a new version?

@mascguy
Copy link
Member

mascguy commented Jan 17, 2024

@mascguy I'm going to ask you to remove your lock on this unless you're willing to finish it in the next few days. Please let other people finish the work.

I have ongoing professional and personal obligations at the moment, so this week isn't an option.

Perhaps next week, if I'm able to carve out some time.

@blair
Copy link
Contributor Author

blair commented Jan 17, 2024

I rebased onto HEAD and did a push. I made minor changes, e.g. not deleting py27-protobuf3. Besides that, I didn't spend any time looking at if this works or not.

BTW, the py3*protobuf3 version number went from 4.21.12 to 24.4. Let me know if that's OK.

@mascguy
Copy link
Member

mascguy commented Jan 17, 2024

I rebased onto HEAD and did a push. I made minor changes, e.g. not deleting py27-protobuf3. Besides that, I didn't spend any time looking at if this works or not.

BTW, the py3*protobuf3 version number went from 4.21.12 to 24.4. Let me know if that's OK.

I'll try to take a look at all of this soon. I might be able to find some time this week, or this weekend.

More to follow.

@pmetzger
Copy link
Member

@mascguy I'm going to have to ask you to step aside. There are a lot of ports where you mark the thing "Draft", insist that you are going to shepherd it, and do nothing for months or longer. If you insist only you can do the work, then you have to finish things in a reasonable period of time. If you have other obligations, then allow other people to finish the work.

@pmetzger
Copy link
Member

@blair What's the reason the CI is failing?

@pmetzger
Copy link
Member

@blair Let's try to get this moving.

@pmetzger
Copy link
Member

@blair ?

@reneeotten
Copy link
Contributor

Feel free to reopen if/when someone had the time/interest to finish this up.

@reneeotten reneeotten closed this Apr 17, 2024
@FlyingSamson
Copy link
Contributor

I would like to revive this PR and directly move on to protobuf 26.1.

Unfortunately, protobuf no longer ships with its own setup.py prohibiting building py-protobuf directly from the github sources. Is there something that speaks against using the prebuilt source package made available through pypi instead?

@reneeotten
Copy link
Contributor

reneeotten commented May 12, 2024

We don't want to do that presumably it ships now with a pyproject.toml and that will work just fine.

@FlyingSamson
Copy link
Contributor

Sorry, but I can't find a pyproject.toml either. From the python related README, it sounds to me as one has to build the python source package first.

@reneeotten
Copy link
Contributor

okay, that's too bad. You'll have to spend some time then to figure out how to do that.

@FlyingSamson
Copy link
Contributor

I now managed to first build that wheel using Bazel and then installing that into destroot with pip.

@reneeotten Can you advice on how I can add my changes to this PR? Or will I have to create a new PR from my branch that is on top of blair's changes?

@ryandesign
Copy link
Contributor

Can you advice on how I can add my changes to this PR?

https://trac.macports.org/wiki/WorkingWithGit#WorkingwithsomeoneelsespullrequestthroughitsID

@ryandesign
Copy link
Contributor

protobuf no longer ships with its own setup.py prohibiting building py-protobuf directly from the github sources. Is there something that speaks against using the prebuilt source package made available through pypi instead?

Per https://protobuf.dev/news/2022-08-03/#lang-specific-distros, "To reduce dependence on autotools and minimize the number of artifacts we release, we plan to stop publishing language-specific source distributions on our GitHub release page. Instead, we advise users to download the source code distribution automatically generated by GitHub on the release page." In MacPorts terms, that means switching from github.tarball_from releases to github.tarball_from archive.

@ryandesign
Copy link
Contributor

the py3*protobuf3 version number went from 4.21.12 to 24.4

That is not so. When you wrote this comment, the version number of the python module was 4.24.4. Now, the version number of the python module is 5.26.x. See https://protobuf.dev/support/version-support/#python. This was clearer when they released separate distfiles for the python module.

@ryandesign
Copy link
Contributor

Per https://protobuf.dev/support/version-support/#cpp, as of protobuf 22.x, the version number for the C++ bindings goes from 3 to 4. The protobuf3-cpp port, given the 3 in its name, is intended to track the 3.x version of the C++ bindings so it cannot be updated to protobuf 22.x or later. Providing C++ bindings for protobuf 22.x and later will require the creation of a new protobuf4-cpp port. As of protobuf 26.x, the version number of the C++ bindings has increased to 5 which will require the creation of a new protobuf5-cpp port.

@ryandesign
Copy link
Contributor

When you wrote this comment, the version number of the python module was 4.24.4

This makes me wonder if py-protobuf3 should ever have been updated to 4.x in 6d94469. Should a new py-protobuf4 port should have been created instead? Should the version number in the python module port's name refer to the version number of the python module or the version number of the C++ library it uses?

The use of a new major version number indicates backward-incompatible changes. It seems likely that old projects would want to continue to use version 3 of the python module that they were designed with while new projects could move to version 4.

According to https://github.com/protocolbuffers/protobuf/tree/main/python#implementation-backends the python module has three different backends, as of version 4.21.0: one based on upb, one based on protobuf-cpp, and one in pure python, with the one based on protobuf-cpp no longer built or installed by default, suggesting the port shouldn't even have a dependency on protobuf3-cpp anymore and the python module port's name really shouldn't be tied to the protobuf3-cpp port's verison.

It looks like our current py-protobuf3 doesn't install into paths that include the python library version number so a hypothetical new py-protobuf4 and py-protobuf5 would conflict with the existing py-protobuf3. The conclusion I must reach is that I have no idea how the developer intent for their software to be packaged.

@ryandesign
Copy link
Contributor

Is there something that speaks against using the prebuilt source package made available through pypi instead?

We don't want to do that

@reneeotten Why not? It's acceptable for python modules to download their source code from pypi, so much so that it's the default behavior of the python portgroup to do so:

default master_sites {pypi:[string index ${python.rootname} 0]/${python.rootname}}

@FlyingSamson
Copy link
Contributor

FlyingSamson commented May 20, 2024

Thank you for gathering all that information.

The conclusion I must reach is that I have no idea how the developer intent for their software to be packaged.

I was just about to say the same. That versioning strategy is a real nightmare.

@FlyingSamson
Copy link
Contributor

suggesting the port shouldn't even have a dependency on protobuf3-cpp anymore

Indeed, the protobuf3-cpp port alone is sufficient to "compile" proto files into python specific code, while the py-protobuf3 port alone suffices to import that python specific code. This, in particular, might suggest that we can use the pypi versions of the source package.

@reneeotten
Copy link
Contributor

@reneeotten Why not? It's acceptable for python modules to download their source code from pypi, so much so that it's the default behavior of the python portgroup to do so:

Yes, source code is of course fine. Just as for all packages we usually don't use pre-compiled packages (i.e., ".whl" files) but prefer to download the source files and compile things within the MacPorts framework as we do with pretty much all other Python packages.

@FlyingSamson
Copy link
Contributor

FlyingSamson commented May 20, 2024

Could we maybe also use the protoc version (aka. the minor version that is kept constant across languages) instead of the major version (currently 3) to name the ports? My thinking here is, that the protobuf releases are taged minor.patch, meaning that any major update would also require a minor update for them to be able to create a new release.

Of course that does not fix the problem that we cannot install different versions in parallel.

@FlyingSamson
Copy link
Contributor

Yes, source code is of course fine. Just as for all packages we usually don't use pre-compiled packages (i.e., ".whl" files) but prefer to download the source files and compile things within the MacPorts framework as we do with pretty much all other Python packages.

OK, then I believe this has been a misunderstanding. I meant do download the source package from pypi not the wheel. Sorry for miscommunicating that.

@FlyingSamson
Copy link
Contributor

FlyingSamson commented May 20, 2024

Another (maybe to drastic) option might be to drop the major version currently encoded in the port name all together (I know there already is a protobuf-cpp and corresponding python bindings, but maybe these are also obsolete, since there is only one dependent shogun-devel which seems also rather outdated as it still offers python 2.7 bindings) and have ports protobuf-cpp, py-protobuf, and protobuf-java with consistent minor.patch versions (and corresponding language-major versions) that are kept rather up to date. Not sure how heavily the dependent of protobuf rely on volatile parts of its API and how well these projects adapt to changes in protobuf, though.

@reneeotten
Copy link
Contributor

Yes, source code is of course fine. Just as for all packages we usually don't use pre-compiled packages (i.e., ".whl" files) but prefer to download the source files and compile things within the MacPorts framework as we do with pretty much all other Python packages.

OK, then I believe this has been a misunderstanding. I meant do download the source package from pypi not the wheel. Sorry for miscommunicating that.

If you meant to use the "source distribution (i.e., .tar.gz) file instead of the " rebuilt source package", which to me sounded clearly like the .whl file - then it was indeed a misunderstanding!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
by: member Created by a member with commit rights maintainer: open Affects an openmaintainer port
7 participants