Skip to content

Commit

Permalink
Bug 1749604 - simplified moz.build generation for libwebrtc. r=ng
Browse files Browse the repository at this point in the history 
Updated README.md to reflect simplified setup for generation.
Updated generate-gn-build-files.sh:
  - remove dependencies on MOZ_LIBWEBRTC environment variable
  - add checking for GN environment variable

Differential Revision: https://phabricator.services.mozilla.com/D136030
  • Loading branch information
@mfromanmoz
mfromanmoz committed Jan 18, 2022
1 parent aba641e commit e6f459c
Show file tree
Hide file tree
Showing 2 changed files with 65 additions and 98 deletions.
120 changes: 60 additions & 60 deletions dom/media/webrtc/third_party_build/gn-configs/README.md
View file
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
# Generate new gn json files and moz.build files for building libwebrtc in our tree

1. If generating on macOS for Apple Silicon (cross-compiling), make sure to have at least
[Xcode 12.2](https://download.developer.apple.com/Developer_Tools/Xcode_12.2/Xcode_12.2.xip).
1. If generating on macOS for Apple Silicon (cross-compiling), make sure to have
at least [Xcode 12.2](https://download.developer.apple.com/Developer_Tools/Xcode_12.2/Xcode_12.2.xip).

In addition the aarch64 Rust target will need to be installed via: `rustup target add aarch64-apple-darwin`
The aarch64 Rust target will need to be installed via:
`rustup target add aarch64-apple-darwin`

2. If generating on Windows 10, Visual Studio 2019 is required. Please follow
the install instructions from [here](https://firefox-source-docs.mozilla.org/setup/windows_build.html)
Expand All @@ -14,86 +15,85 @@
- MSVC v142 - VS2019 C++ ARM64 build tools

"Debugging Tools for Windows" is also required.
- Under `Settings -> Apps` search for "Windows Software Development Kit" with the version
number 10.0.19041.685.
- Under `Settings -> Apps` search for "Windows Software Development Kit" with
the version number 10.0.19041.685.
- select Modify (and allow the installer to modify)
- select Change and then click Next
- select "Debugging Tools for Windows" and then click Change.

And the aarch64 Rust target will need to be installed via: `rustup target add aarch64-pc-windows-msvc`
The aarch64 Rust target will need to be installed via:
`rustup target add aarch64-pc-windows-msvc`

3. See information in `third_party/libwebrtc/README.mozilla` for the proper revision of libwebrtc
3. See information in `third_party/libwebrtc/README.mozilla` for the proper
revision of libwebrtc

libwebrtc updated from commit https://github.com/mozilla/libwebrtc/archive/149d693483e9055f574d9d65b01fe75a186b654b.tar.gz on 2020-11-30T15:48:48.472088.
third_party updated from commit https://chromium.googlesource.com/chromium/src/third_party/+archive/5dc5a4a45df9592baa8e8c5f896006d9193d8e45.tar.gz on 2020-11-30T17:00:15.612630.

In our current case, the revision is `149d693483e9055f574d9d65b01fe75a186b654b` which
corresponds to:
In our current case, the revision is `149d693483e9055f574d9d65b01fe75a186b654b`
which corresponds to:

mozilla-modifications-rel86

4. Clone Mozilla's version of libwebrtc from [libwebrtc](https://github.com/mozilla/libwebrtc)
This commit was made on `Thu Nov 19 14:14:00 2020`.

git clone https://github.com/mozilla/libwebrtc moz-libwebrtc
export MOZ_LIBWEBRTC=`pwd`/moz-libwebrtc
(cd moz-libwebrtc ; git checkout mozilla-modifications-rel86)
4. Download a version of the `gn` executable that corresponds to
`Thu Nov 19 14:14:00 2020`. In our case, that is version `1889 (8fe02009)`.

- [Win](https://chrome-infra-packages.appspot.com/p/gn/gn/windows-amd64/+/e_UmTHedzuu4zJ2gdpW8jrFFNnzIhThljx3jn3RMlVsC)
- [Linux](https://chrome-infra-packages.appspot.com/p/gn/gn/linux-amd64/+/bvBFKgehaepiKy_YhFnbiOpF38CK26N2OyE1R1jXof0C)
- [macOS](https://chrome-infra-packages.appspot.com/p/gn/gn/mac-amd64/+/nXvMRpyJhLhisAcnRmU5s9UZqovzMAhKAvWjax-swioC)

Note that branch was made on `Thu Nov 19 14:14:00 2020`
Find the downloaded `.zip` file, unzip and export the location of the
executable:

5. Clone `depot_tools` from [depot_tools](https://chromium.googlesource.com/chromium/tools/depot_tools.git)
unzip gn-mac-amd64.zip && export GN=`pwd`/gn
unzip gn-windows-amd64.zip && export GN=`pwd`/gn.exe
unzip gn-linux-amd64.zip && export GN=`pwd`/gn

git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git
export DEPOT_TOOLS=`pwd`/depot_tools
On platforms that don't have pre-built `gn` executables, `ninja` and `gn` can
be easily built:

If generating files on Windows, the following steps must be completed
from a traditional Windows Cmd prompt (cmd.exe) launched from the start
menu or search bar. This allows `gclient` to properly bootstrap the
required python setup.
git clone https://github.com/ninja-build/ninja.git
git clone https://gn.googlesource.com/gn
(cd gn && git checkout 8fe02009)
(cd ninja && ./configure.py --bootstrap)
(export NINJA=`pwd`/ninja/ninja ; cd gn && python build/gen.py && $NINJA -C out)
export GN=`pwd`/gn/out/gn

cd {depot_tools directory}
set PATH=%CD%;%PATH%
set DEPOT_TOOLS_WIN_TOOLCHAIN=0
set vs2019_install="c:\Program Files (x86)\Microsoft Visual Studio\2019\Community"
gclient
exit
On OpenBSD, a slightly newer version of `gn` is needed in order to build:

Now, we need to checkout a revision of `depot_tools` that corresponds to the date of
our libwebrtc branch. The closest `depot_tools` commit to `Thu Nov 19 14:14:00 2020` is
`e7d1862b155ac3ccbef72c4d70629b5c88ffcb32`. There is additional information on how to
more automatically determine this [here](https://chromium.googlesource.com/chromium/src/+/master/docs/building_old_revisions.md).
(cd gn && git checkout 31f2bba8)

(cd depot_tools ; git checkout e7d1862b155ac3ccbef72c4d70629b5c88ffcb32 )
5. Clone `depot_tools` from [depot_tools](https://chromium.googlesource.com/chromium/tools/depot_tools.git)

6. It is necessary to let `depot_tools` pull information into the `libwebrtc` tree as well. This can take a while.
git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git
export DEPOT_TOOLS=`pwd`/depot_tools

(cd moz-libwebrtc ; \
export PATH=$DEPOT_TOOLS:$PATH ; \
export DEPOT_TOOLS_UPDATE=0 ; \
export DEPOT_TOOLS_WIN_TOOLCHAIN=0 ; \
gclient config https://github.com/mozilla/libwebrtc && \
gclient sync -D --force --reset --with_branch_heads \
)

Note that if one uses `gclient` sync with a different output directory `$MOZ_LIBWEBRTC_GIT`
must be set to the original clone directory, and `$MOZ_LIBWEBRTC` needs to be set to the
directory created by `gclient sync`.
Now, we need to checkout a revision of `depot_tools` that corresponds to the
date of our libwebrtc branch. The closest `depot_tools` commit to
`Thu Nov 19 14:14:00 2020` is `e7d1862b155ac3ccbef72c4d70629b5c88ffcb32`.
There is additional information on how to more automatically determine this
[here](https://chromium.googlesource.com/chromium/src/+/master/docs/building_old_revisions.md).

(cd depot_tools ; git checkout e7d1862b155ac3ccbef72c4d70629b5c88ffcb32 )

7. Now it is time to generate the build files. The script should be run from the top
directory of our firefox tree.
6. It is time to generate the build files. The script should be run from the
top directory of our firefox tree.

./dom/media/webrtc/third_party_build/gn-configs/generate-gn-build-files.sh
bash ./dom/media/webrtc/third_party_build/gn-configs/generate-gn-build-files.sh

Debugging the generate script itself may prove useful, and one can do this by setting the DEBUG_GEN environment
variable to a non-empty value. This will print everything that the script executes.
Debugging the generate script itself may prove useful, and one can do this by
setting the DEBUG_GEN environment variable to a non-empty value. This will
print everything that the script executes.

8. Checkin all the generated/modified files and try your build!
7. Checkin all the generated/modified files and try your build!

# Adding new configurations to the build

- Each new platform/architecture will require 2 new mozconfig files,
one for the debug build and one for the non-debug build. The
filenames follow the same pattern as the generated json files,
`a-b-c-d.mozconfig` where:
- Each new platform/architecture will require 2 new mozconfig files, one for the
debug build and one for the non-debug build. The filenames follow the same
pattern as the generated json files, `a-b-c-d.mozconfig` where:
- a = generating cpu (example: x64)
- b = debug (True / False)
- c = target cpu (example: x64 / arm64)
Expand All @@ -103,9 +103,9 @@ directory of our firefox tree.
- The new configs must be added to the appropriate platform section in
`generate-gn-build-files.sh`.

**Note:** when adding new mozconfig files, especially for linux/android
configs, it is important to include the `ac_add_options
--enable-bootstrap`. This ensures switching archtectures for
"cross-compiled" generation works properly. For example, when generating
`x86` or `arm64` linux json files, it would be necessary to install additional
libraries in order for the configure step to complete.
**Note:** when adding new mozconfig files, especially for linux/android configs,
it is important to include the `ac_add_options --enable-bootstrap`. This
ensures switching archtectures for "cross-compiled" generation works properly.
For example, when generating `x86` or `arm64` linux json files, it would be
necessary to install additional libraries in order for the configure step to
complete.
43 changes: 5 additions & 38 deletions dom/media/webrtc/third_party_build/gn-configs/generate-gn-build-files.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,33 +8,15 @@ if [ ! "x$DEBUG_GEN" = "x" ]; then
set -x
fi

if [ "x$MOZ_LIBWEBRTC" = "x" ]; then
echo "MOZ_LIBWEBRTC is not defined, see README.md"
if [ "x$GN" = "x" ]; then
echo "GN is not defined, see README.md"
exit
fi

if [ -d $MOZ_LIBWEBRTC ]; then
echo "MOZ_LIBWEBRTC is $MOZ_LIBWEBRTC"
if [ -f $GN ]; then
echo "GN is $GN"
else
echo "Path $MOZ_LIBWEBRTC is not found, see README.md"
exit
fi

# git clone and gclient checkout may be in different places
if [ "x$MOZ_LIBWEBRTC_GIT" = "x" ]; then
MOZ_LIBWEBRTC_GIT=$MOZ_LIBWEBRTC
fi

if [ ! -d $MOZ_LIBWEBRTC_GIT/.git ]; then
echo "No .git directory is found in the libwebrtc checkout, see README.md"
exit
fi

if [ ! -d $MOZ_LIBWEBRTC/src/buildtools ]; then
echo "Path $MOZ_LIBWEBRTC/src/buildtools is not found, see README.md"
echo "Please run the following commands from inside $MOZ_LIBWEBRTC:"
echo "\tgclient config https://github.com/mozilla/libwebrtc"
echo "\tgclient sync -D --force --reset --with_branch_heads # this make take a while"
echo "Path $GN is not found, see README.md"
exit
fi

Expand Down Expand Up @@ -115,21 +97,6 @@ fi
export PATH=$DEPOT_TOOLS:$PATH
export DEPOT_TOOLS_UPDATE=0

# Symlink in the buildtools and .git directories from our copy of libwebrtc.
if [ -L ./third_party/libwebrtc/buildtools ]; then
rm ./third_party/libwebrtc/buildtools
elif [ -d ./third_party/libwebrtc/buildtools ]; then
rm -rf ./third_party/libwebrtc/buildtools
fi
ln -s $MOZ_LIBWEBRTC/src/buildtools ./third_party/libwebrtc/

if [ -L ./third_party/libwebrtc/.git ]; then
rm ./third_party/libwebrtc/.git
elif [ -d ./third_party/libwebrtc/.git ]; then
rm -rf ./third_party/libwebrtc/.git
fi
ln -s $MOZ_LIBWEBRTC_GIT/.git ./third_party/libwebrtc/

CONFIG_DIR=dom/media/webrtc/third_party_build/gn-configs
echo "CONFIG_DIR is $CONFIG_DIR"

Expand Down

0 comments on commit e6f459c

Please sign in to comment.