Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Aggregate Metadata Files #7155

Merged
merged 11 commits into from Feb 10, 2020
Merged

Conversation

@TeBoring
Copy link
Contributor

TeBoring commented Jan 30, 2020

Instead of calling initOnce of dependencies, initialize metadata of dependencies in the same file.

Needs to pass aggregate_metadata option to protoc to trigger, e.g.:
--php_out=aggregate_metadata=foo#bar:generated_dir
For each input file, transitive dependencies (including itself), whose package name has the prefix of foo or bar, will be aggregated, in which their metadata string will be aggregated in the same internalAddGeneratedFile call. For other dependencies, initOnce is called as before.

This feature is EXPERIMENTAL. DO NOT USE!!!

TeBoring added 10 commits Dec 5, 2019
Only packages with specified prefix will be aggregated
1) Needs to pass aggregate_metadata option to protoc to trigger, e.g.:
     --php_out=aggregate_metadata=GPBMetadata\\\\Aggregate:generated_dir
   This will generate an aggregated metadata file in GPBMetadata/Aggregate.php
   Metadata file of all input files will reference to this aggregated metadata file.

2) Can specify package prefixes to explicitly specify which files to aggregate:
     --php_out=aggregate_metadata=GPBMetadata\\\\Aggregate#foo#bar:generated_dir
   Among all input files, only those whose packages have prefixes (foo or bar) will
   be aggregated. All the other input files will be treated as previous.
1) No need to specify aggregate class name
2) Each input files will use the same prefix list to decide whether to unwrap
   its dependencies. If prefix list is not specified, all dependencies will be unwrapped.
Fix bugs
1) No longer split metadata string
2) Do not initialize metadata for descriptor.proto
@googlebot googlebot added the cla: yes label Jan 30, 2020
@TeBoring TeBoring added php and removed cla: yes labels Jan 30, 2020
@googlebot googlebot added the cla: yes label Jan 30, 2020
@TeBoring TeBoring added release notes: no and removed cla: yes labels Jan 30, 2020
@googlebot googlebot added the cla: yes label Jan 30, 2020
@TeBoring TeBoring added kokoro:run and removed cla: yes labels Jan 30, 2020
@googlebot googlebot added the cla: yes label Jan 30, 2020
@dankurka

This comment has been minimized.

Copy link
Contributor

dankurka commented Feb 3, 2020

Can we please squash this into one commit?

@TeBoring

This comment has been minimized.

Copy link
Contributor Author

TeBoring commented Feb 10, 2020

When merging, it will be squashed.

Copy link
Contributor

stanley-cheung left a comment

Discussed offline. LGTM.

@TeBoring TeBoring merged commit ac70b7c into protocolbuffers:master Feb 10, 2020
57 of 58 checks passed
57 of 58 checks passed
MacOS Ruby 2.3 Kokoro build failed
Details
Bazel Kokoro build successful
Details
Dist artifact installation Kokoro build successful
Details
Linux 32-bit Kokoro build successful
Details
Linux C# Kokoro build successful
Details
Linux C++ Distcheck Kokoro build successful
Details
Linux C++ TC Malloc Kokoro build successful
Details
Linux Golang Kokoro build successful
Details
Linux Java Compatibility Kokoro build successful
Details
Linux Java JDK 7 Kokoro build successful
Details
Linux Java Linkage Monitor Kokoro build successful
Details
Linux Java Oracle 7 Kokoro build successful
Details
Linux JavaScript Kokoro build successful
Details
Linux PHP Kokoro build successful
Details
Linux Python Kokoro build successful
Details
Linux Python 2.7 Kokoro build successful
Details
Linux Python 2.7 C++ Kokoro build successful
Details
Linux Python 3.3 Kokoro build successful
Details
Linux Python 3.3 C++ Kokoro build successful
Details
Linux Python 3.4 Kokoro build successful
Details
Linux Python 3.4 C++ Kokoro build successful
Details
Linux Python 3.5 Kokoro build successful
Details
Linux Python 3.5 C++ Kokoro build successful
Details
Linux Python 3.6 Kokoro build successful
Details
Linux Python 3.6 C++ Kokoro build successful
Details
Linux Python 3.7 Kokoro build successful
Details
Linux Python 3.7 C++ Kokoro build successful
Details
Linux Python 3.8 Kokoro build successful
Details
Linux Python 3.8 C++ Kokoro build successful
Details
Linux Python C++ Kokoro build successful
Details
Linux Python Compatibility Kokoro build successful
Details
Linux Python Release Kokoro build successful
Details
Linux Ruby 2.3 Kokoro build successful
Details
Linux Ruby 2.4 Kokoro build successful
Details
Linux Ruby 2.5 Kokoro build successful
Details
Linux Ruby 2.6 Kokoro build successful
Details
Linux Ruby Release Kokoro build successful
Details
MacOS C++ Kokoro build successful
Details
MacOS C++ Distcheck Kokoro build successful
Details
MacOS JavaScript Kokoro build successful
Details
MacOS Obj-C CocoaPods Integration Kokoro build successful
Details
MacOS Obj-C OS X Kokoro build successful
Details
MacOS Obj-C iOS Debug Kokoro build successful
Details
MacOS Obj-C iOS Release Kokoro build successful
Details
MacOS PHP5.6 Kokoro build successful
Details
MacOS PHP7.0 Kokoro build successful
Details
MacOS Python Kokoro build successful
Details
MacOS Python C++ Kokoro build successful
Details
MacOS Python Release Kokoro build successful
Details
MacOS Ruby 2.4 Kokoro build successful
Details
MacOS Ruby 2.5 Kokoro build successful
Details
MacOS Ruby 2.6 Kokoro build successful
Details
MacOS Ruby Release Kokoro build successful
Details
Mergeable Mergeable Run has been Completed!
Details
Windows C# Kokoro build successful
Details
Windows Csharp Release Kokoro build successful
Details
Windows Python Release Kokoro build successful
Details
cla/google All necessary CLAs are signed
TeBoring added a commit to TeBoring/protobuf that referenced this pull request Feb 10, 2020
Instead of calling initOnce of dependencies, initialize metadata of dependencies in the same file.

Needs to pass aggregate_metadata option to protoc to trigger, e.g.:
--php_out=aggregate_metadata=foo#bar:generated_dir
For each input file, transitive dependencies (including itself), whose package name has the prefix of foo or bar, will be aggregated, in which their metadata string will be aggregated in the same internalAddGeneratedFile call. For other dependencies, initOnce is called as before.

This feature is EXPERIMENTAL. DO NOT USE!!!
TeBoring added a commit that referenced this pull request Feb 11, 2020
Instead of calling initOnce of dependencies, initialize metadata of dependencies in the same file.

Needs to pass aggregate_metadata option to protoc to trigger, e.g.:
--php_out=aggregate_metadata=foo#bar:generated_dir
For each input file, transitive dependencies (including itself), whose package name has the prefix of foo or bar, will be aggregated, in which their metadata string will be aggregated in the same internalAddGeneratedFile call. For other dependencies, initOnce is called as before.

This feature is EXPERIMENTAL. DO NOT USE!!!
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

5 participants
You can’t perform that action at this time.