Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix uriparser build #150

Closed
wants to merge 999 commits into from
Closed

Fix uriparser build #150

wants to merge 999 commits into from

Conversation

DemiMarie
Copy link
Contributor

A path was wrong.

cblichmann and others added 30 commits July 14, 2021 08:00
@cblichmann
Rename static singleton accessor
7b711b8 
PiperOrigin-RevId: 384699374
Change-Id: I674baffc77bc6b3815f94512058a14d37d164c6f
@cblichmann
Revert memfd file sealing for embeded files
9c21744 
Ideally, we'd seal the embedded SAPI binary using fcntl(). However, in rare
cases, adding the file seals `F_SEAL_SEAL | F_SEAL_SHRINK | F_SEAL_GROW |
F_SEAL_WRITE` results in `EBUSY` errors.

This is likely because of an interaction of `SEAL_WRITE` with pending writes
to the mapped memory region (see `memfd_wait_for_pins()` in Linux'
`mm/memfd.c`). Since `fsync()` is a no-op on memfds, it doesn't help to
ameliorate the problem.

On systems where it is enabled, ksmd might also be a source of pending writes.

PiperOrigin-RevId: 385741435
Change-Id: I21bd6a9039be4b6298774e837ce3628180ed91a8
@junyer
Introduce AllowRestartableSequencesWithProcFiles() and tidy up.
bb6ae1d 
1. In many cases, sandboxes need to allow /proc/stat and /proc/cpuinfo so that
get_nprocs(3) will work; otherwise, per-CPU logic can't determine how many CPUs
there are. Unfortunately, some of those sandboxes also disable namespaces. The
solution is to provide two functions: AllowRestartableSequencesWithProcFiles(),
which allows syscalls and files; and AllowRestartableSequences(), which allows
syscalls only. Sandboxes should usually call the former; sandboxes that disable
namespaces should instead call the latter and are responsible for allowing the
files via the deprecated Fs mechanism.

2. Make the mmap(2) policy evaluate prot AND flags, not prot OR flags.

3. Order the code and the comments identically for better readability.

PiperOrigin-RevId: 386414028
Change-Id: I016b1854ed1da9c9bcff7b351c5e0041093b8193
@cblichmann
Introduce sapi::OsErrorMessage() for error handling
ccd7b03 
This should make handling OS error less repetetive.

PiperOrigin-RevId: 387074642
Change-Id: I09b8c5e37e7f7b08341e22ba01ccda21a916a4bc
@cblichmann
Reduce logspam: Log Tomoyo LSM check only with VLOG
85c58dc 
PiperOrigin-RevId: 387114844
Change-Id: Ib670799e3327fcc991ad012ccee20b96089c2f48
@cblichmann
Internal change.
f14aeee 
PiperOrigin-RevId: 387565158
Change-Id: I7b5293b614fae74abae1f9a347b0ef414028b8ea
@cblichmann
Reorder error logging before Terminate()
fd20eb0 
Calling `Terminate()` issues additional syscalls that may clobber the `errno`
value. Reordering the log statements ensures we actually log the initial error
in `read()`/`write()`.

PiperOrigin-RevId: 387576942
Change-Id: I0f9c8c6001e6dc4ca098abe02cd251029f92a737
@cblichmann
Fix factory method sapi::v::Proto<>::FromMessage
8b1dfd7 
This was missing a friend declaration in order to actually compile.
It's now being used in the "stringop" example, so we test it as well.

Drive-by:
- Do not copy the proto's bytes the constructor, but use `std::move`
PiperOrigin-RevId: 387774353
Change-Id: Ic8824af911ac744e2e68130e1f4673c4dddd4939
@happyCoder92
Replace a CHECK with a warning
80ad7bb 
PiperOrigin-RevId: 388893117
Change-Id: I0b0ccf2045aea09d31ae1605b205aab456bd8550
@happyCoder92
Use FDCloser in Executor extensively
e887552 
Also really own `exec_fd_` as previously if the executor is destructed without calling `StartSubProcess` the file descriptor would leak.

PiperOrigin-RevId: 388901766
Change-Id: I6bbb15ced37a0a832ec5a5228452a3d54ef46ee9
@happyCoder92
Wait for global forkserver when shutting it down
fe70950 
Otherwise starting forkserver multiple times will result in zombie processes lingering around.

PiperOrigin-RevId: 388926497
Change-Id: Ia9947cce3d9e909edd709b0d3525e1ae8b8bbc51
@happyCoder92
Allow recovering from global forkserver failure
0621e06 
PiperOrigin-RevId: 389164847
Change-Id: I40bc3b6d3bea28ee8954ea2a11a0427a6c05da35
Delete deprecated sapi::Sandbox::GetPid and its remaining call sites.
3f08757 
PiperOrigin-RevId: 389713115
Change-Id: I1832e759016a581e10bf5bd8b5b70244b40ecd69
Delete deprecated sapi::Sandbox::GetComms and its remaining call si…
165c155 
…tes.

PiperOrigin-RevId: 389716023
Change-Id: I092bc37f3f3bb40554b627f9dd528525b60d67a1
@happyCoder92
Do not fail-hard in global forkserver startup
773dc6b 
PiperOrigin-RevId: 389816114
Change-Id: Icd672028ff224cf01095d6590fe1cc2adb312316
Delete deprecated sapi::Sandbox::GetRpcChannel and its remaining ca…
d631154 
…ll sites.

PiperOrigin-RevId: 389968873
Change-Id: Ia72e0064fa57679180f9c406f96266473f8461c2
Fix Symbolize* tests.
dae91ff 
PiperOrigin-RevId: 390372065
Change-Id: I1ddc9dd9238795eb0674e04c20a5c91a68582027
Delete deprecated sapi::Sandbox::IsActive and its remaining call si…
7b31dea 
…tes.

PiperOrigin-RevId: 390412024
Change-Id: Iab3853b3c40dd4e9b0ff31532e8c41c2583ebc4e
@happyCoder92
Allow collecting stacktraces on normal process exit
59f5fa8 
This mainly a debugging facility.
It makes diagnosing problems where sandboxed process just randomly exits whereas unsandboxed one runs to completion due to differences in the setup/environment much easier.

PiperOrigin-RevId: 391005548
Change-Id: Ia19fe6632748da93c1f4291bb55e895f50a4e2b0
@cpatulea
'Map' symbols: add pgoff to disambiguate multiple mappings on same ob…
9ab330d 
…ject.

PiperOrigin-RevId: 391520785
Change-Id: Icb05e60f778acfb9fe6f519911ce54bec65fc4ff
@cblichmann
Internal change.
289adcf 
For OSS, this change should be mostly a no-op. Visible edits are due to
changed order of code and/or includes.

PiperOrigin-RevId: 394177395
Change-Id: I1d32f9fd175579e8f05c051b1307953b249d139d
@fluxchief
Rework GetListOfFDs API
eb2c5a6 
PiperOrigin-RevId: 395043959
Change-Id: I77ce13f0c786d3644971ed239f3106319667e979
@fluxchief
Use absl::flat_hash_set + Status in favor of std::set in the sani…
4b01875 
…tizer API

PiperOrigin-RevId: 395061068
Change-Id: I31548eb6fc9f27f55acf25bd6d3d0b941a529e63
Automated rollback of commit 4b01875.
2036f5b 
PiperOrigin-RevId: 395067992
Change-Id: I5db335ed881aa81748a0fc8082091b160fe83e86
@fluxchief
Automated rollback of commit 2036f5b.
aea8bb2 
PiperOrigin-RevId: 395893427
Change-Id: Iabd32de9cd83de5cc8567834e1f91e48c521ac60
@cblichmann
(Mostly) internal change. Add pid() accessor.
c400f92 
PiperOrigin-RevId: 397070773
Change-Id: I9ebac9078f3866ef3e0061ec79da5c9f71e5f480
@cpatulea
Fix formatting of pgoff.
b5fb483 
PiperOrigin-RevId: 397763298
Change-Id: I027ef4cd381247521ee2bcce57a17c9d480efb22
@happyCoder92
Make the fd cleanup test less brittle
b470a6e 
PiperOrigin-RevId: 398229418
Change-Id: If8af43f33b07839ea8d46b85ff77efa8557a31a8
@happyCoder92
Log when global forkserver is started and its exit status
c29c510 
PiperOrigin-RevId: 398232735
Change-Id: Ia0628cf2dee51a94938dae82bcb392384feeb74c
Enable mmap for msan (it's already enabled for asan and tsan)
448f393 
PiperOrigin-RevId: 399163710
Change-Id: I2cebb6136adb00a53e4baf18d343cf80191efcb0
okunz and others added 19 commits March 14, 2022 07:28
@okunz
Migrate forkserver.proto to proto3 syntax
2065475 
PiperOrigin-RevId: 434458725
Change-Id: I277f76a1a5ebd3eed15c6b3f3e7f849bf6edacea
Merge pull request google#141 from DemiMarie:fix-pffft-build
6d51497 
PiperOrigin-RevId: 434707632
Change-Id: I384cd7275cfe8f80931a9ca3108ee6324a7df175
@cblichmann
Merge branch 'main' into zip
071cb79
@oshogbo
Copybara import of the project:
6d5f257 
--
74c7f66 by Mariusz Zaborski <oshogbo@invisiblethingslab.com>:

Sandbox uriparser

COPYBARA_INTEGRATE_REVIEW=google#130 from oshogbo:uriparser 74c7f66
PiperOrigin-RevId: 434755972
Change-Id: I90f97229122acd47354327c0ae9e58a02cb7bd20
@okunz
Migration of remaining protobufs from proto2 to proto3
ee11d9f 
PiperOrigin-RevId: 434973223
Change-Id: I5518aa3944cab94d33ce0538bed8ee82f90d4b3a
@oshogbo
Introduce SAPI_ASSERT_OK
5416cbb 
We have a SAPI_ASSERT_OK_AND_ASSIGN which corespondents to
SAPI_ASSIGN_OR_RETURN.

We also have SAPI_RETURN_IF_ERROR but we don't have a coresponding
macro for ASSERT.

I think that this completes the API and makes writting tests a little
bit simpler.
Merge pull request google#123 from oshogbo:zip
5cab501 
PiperOrigin-RevId: 435067254
Change-Id: Id5cd97edfd1ad2fcfdbb530ea3ab20e627ceb3be
@oshogbo
Drop Uriparser CMAKE patch
6fa8424 
Upstream fixed the issue with object visiblity in static version
of library. [1]

[1] uriparser/uriparser@a259209
Merge pull request google#147 from oshogbo:matcher2
97b3a9e 
PiperOrigin-RevId: 435267982
Change-Id: I8f877da70282df9192be4cfe43d74d1539b9824f
@okunz
Instruct copybara to exclude contrib/** when changing absl/flags:flag
dedcdba 
PiperOrigin-RevId: 435276975
Change-Id: Ide53847ca2d2b2d280c3cb66b557cc885f38872e
@cblichmann
clang_generator: Emit the correct enum names in SAPI variables
80b325a 
This is a follow up to fa9e6e8.

Drive-by:
- Replace deprecated calls to `getNameAsString()`
PiperOrigin-RevId: 435287759
Change-Id: I81d8c2f93b1ab23c781421b114779b7a241e4a7e
Merge pull request google#151 from oshogbo:uriparser_version
6c6bc1e 
PiperOrigin-RevId: 435288973
Change-Id: I2ea3f83384287c0cf0c56f7b39fad47bed4fbc8a
@cblichmann
clang_generator: Handle intrinsics directly
d5ebb81 
PiperOrigin-RevId: 435289903
Change-Id: I68a37bb7b25a7b77c046d00a2740aa9de2fcaa89
@cblichmann
clang_generator: Use a btree_map for fixed iteration order
b8579e4 
PiperOrigin-RevId: 435291910
Change-Id: I198247409d095183849a221af9c3be21b5bb859b
@cblichmann
clang_generator: Correctly emit nested C-like structs
92ccfea 
- Add more tests for this

PiperOrigin-RevId: 435296715
Change-Id: I7b42dbc58dc054d2565af9ad22498d98416b7af7
@cblichmann
clang_generator: Do not collect structs/unions if declared in another…
4e71f1d 
… record

The enclosing type is enough to reconstruct the AST when writing the header and this
change avoids emitting the same struct twice.

PiperOrigin-RevId: 435300029
Change-Id: I34bd660db5ba5c68b64cce73ecf2f026727ac57b
@cblichmann
clang_generator: Enable mixed header processing
e8fe398 
This implements a custom compilation database to conditionally add the correct
language flags to the compiler frontend. Otherwise, a C header might receive
`--std=c++17` and fail.

Note: All headers are always processed in C++ mode. We expect that headers of
well-behaved C libraries contain `#ifdef __cplusplus`/`extern "C" {}` guards.
PiperOrigin-RevId: 435302048
Change-Id: Ib84e6e1f301ba434999846a012b3f8c16884648e
@cblichmann
clang_generator: Improve formatting of function prototype comments
6d3adc4 
This uses the Google formatting style to format the prototype comments, with an
internal line length of 75, which accomodates the indentation in the generated
API class.

PiperOrigin-RevId: 435303665
Change-Id: I4dcdf0ed773a79ebc55ead3843f07ca8556fd985
@okunz
Patch sandbox2/comms module to build for Android.
d0f5f54 
PiperOrigin-RevId: 435318451
Change-Id: If0e40bab30f3cb68d7e79f26d2336c638742f1ac
cblichmann
cblichmann reviewed Mar 18, 2022
View reviewed changes
Copy link
Member

@cblichmann cblichmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM pending internal review.

oshogbo and others added 4 commits March 18, 2022 02:01
@oshogbo
Copybara import of the project:
cc4b5f3 
--
8160f26 by Mariusz Zaborski <oshogbo@invisiblethingslab.com>:

zopfli: fix a small typo
COPYBARA_INTEGRATE_REVIEW=google#137 from oshogbo:doc3 a80d3e7
PiperOrigin-RevId: 435576465
Change-Id: Ia6b4e95e251b425edad4bca31baf9b65ae0809fd
@DemiMarie
Copybara import of the project:
fd823df 
--
2019d66 by Demi Marie Obenour <demi@invisiblethingslab.com>:

Fix uriparser build

A path was wrong.

COPYBARA_INTEGRATE_REVIEW=google#150 from DemiMarie:fix-uriparser-build 2019d66
PiperOrigin-RevId: 435587012
Change-Id: I70e392f86f796c5ebeccc8e920110aecb8c40b42
Merge pull request google#134 from oshogbo:c-blosc-header
16a6c37 
PiperOrigin-RevId: 435587139
Change-Id: I4cdc6a5d1c66f17fa5074f187badcaee69bdf618
@DemiMarie
Use PROJECT_BINARY_DIR for binary path
5f63e00 
It is more stable than CMAKE_CURRENT_BINARY_DIR.

Suggested-by: Christian Blichmann <cblichmann@google.com>
happyCoder92 pushed a commit that referenced this pull request May 4, 2022
@DemiMarie
Copybara import of the project:
41a35ce 
--
2019d66 by Demi Marie Obenour <demi@invisiblethingslab.com>:

Fix uriparser build

A path was wrong.

COPYBARA_INTEGRATE_REVIEW=#150 from DemiMarie:fix-uriparser-build 2019d66
PiperOrigin-RevId: 435587012
Change-Id: I70e392f86f796c5ebeccc8e920110aecb8c40b42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cblichmann cblichmann cblichmann left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet