Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fortify scan of some Abseil libraries #940

Closed
tombmantle opened this issue Apr 14, 2021 · 8 comments
Closed

Fortify scan of some Abseil libraries #940

tombmantle opened this issue Apr 14, 2021 · 8 comments
Labels
question

Comments

@tombmantle
Copy link

Hello abseil devs,
My company has been testing some various logging, string manip libraries and abseil has come up in the discussion. We are required to run fortify scans on our code and to use C++11 for a certain project, and can not have and "CAT I" issues in our code.

Small disclaimer, I'm aware that fortify is far from perfect, and that abseil has ported some C++11 compatible C++17.

I scanned several abseil libraries and more than a few CAT I's (and CAT II, can share if there is interest) have come up. Is this known by the developers/has this been brought up before? Is this a concern in your opinions?

Best,
Tom

Below are several examples, there are about a dozen more:

CAT I's
Format string buffer overflow vulnerabilities:
abseil/abseil-cpp/absl/synchronization/mutex.cc:433 Sink: snprintf() Enclosing Method: PostSynchEvent()

Type mismatch overflow vulnerability:
1 abseil/abseil-cpp/absl/numeric/bits.h:136 Sink: ReturnStatement Enclosing Method: bit_width()
2 abseil/abseil-cpp/absl/strings/charconv.cc:249 Sink: ReturnStatement Enclosing Method: bit_width()
3 abseil/abseil-cpp/absl/strings/escaping.cc:314 Sink: AssignmentStatement Enclosing Method: CEscapeInternal()
4 abseil/abseil-cpp/absl/strings/escaping.cc:378 Sink: AssignmentStatement Enclosing Method: CEscapedLength()
5 abseil/abseil-cpp/absl/strings/escaping.cc:394 Sink: AssignmentStatement Enclosing Method: CEscapeAndAppendInternal()
6 abseil/abseil-cpp/absl/strings/escaping.cc:503 Sink: AssignmentStatement Enclosing Method: Base64UnescapeInternal()
7 abseil/abseil-cpp/absl/strings/internal/charconv_parse.cc:193 Sink: ReturnStatement Enclosing Method: ToDigit()
8 abseil/abseil-cpp/absl/base/internal/low_level_alloc.cc:335 Sink: ReturnStatement Enclosing Method: GetPageSize()
9 abseil/abseil-cpp/absl/hash/internal/city.cc:233 Sink: AssignmentStatement Enclosing Method: HashLen0to16()
10 abseil/abseil-cpp/absl/hash/internal/city.cc:234 Sink: AssignmentStatement Enclosing Method: HashLen0to16()
11 abseil/abseil-cpp/absl/hash/internal/city.cc:235 Sink: AssignmentStatement Enclosing Method: HashLen0to16()
12 abseil/abseil-cpp/absl/strings/numbers.cc:218 Sink: ReturnStatement Enclosing Method: FastHexToBufferZeroPad16()
13 abseil/abseil-cpp/absl/strings/internal/str_format/extension.cc:62 Sink: AssignmentStatement Enclosing Method: PutPaddedString()
14 abseil/abseil-cpp/absl/strings/internal/str_format/arg.cc:168 Sink: AssignmentStatement Enclosing Method: without_neg_or_zero()
15 abseil/abseil-cpp/absl/time/internal/cctz/src/time_zone_info.cc:625 Sink: AssignmentStatement
Enclosing Method: Open()
16 abseil/abseil-cpp/absl/time/internal/cctz/src/time_zone_info.cc:674 Sink: AssignmentStatement
Enclosing Method: Open()
17 abseil/abseil-cpp/absl/synchronization/mutex.cc:348 Sink: strcpy() Enclosing Method: EnsureSynchEvent
18 abseil/abseil-cpp/absl/strings/numbers.cc:495 Sink: strcpy() Enclosing Method: SixDigitsToBuffer()
19 abseil/abseil-cpp/absl/strings/numbers.cc:495 Sink: strcpy() Enclosing Method: SixDigitsToBuffer()
@tituswinters
Copy link

tituswinters commented Apr 14, 2021 via email

@tombmantle
Copy link
Author

Titus, I appreciate your response. I'll add that I just also realized that the gcc we are using is apparently more advanced than the max version that fortify officially supports, so you can consider this as a potential DOUBLE false positive. I'm getting gcc 5. installed on our scanning system ASAP and will re-post the results at that point.

Fortify is blunt, and I gave the full reports that are given for each issue... there is a developer workbook that give automated long and generic responses about "this is a stack overflow attack" with simple examples and explanations as well. The previous message is as precise as the output gets for the absl issues.

Assuming that things are somewhat okay in our scan considering the compiler issue, the precise details of the scan are pretty straight forward. I compiled our code with the static analysis flags relevant for running fortify, with adding absl::strings to target_link_libraries of the main executable. I was just interested in what fortify would have to say about the lib, I didn't test any functionality. I'll post an update as soon as I rescan with a definitely compatible compiler.

@tituswinters
Copy link

tituswinters commented Apr 14, 2021 via email

@Mbellsudteen

This comment has been minimized.

Sign in to view
@tombmantle
Copy link
Author

I don't think there's much difference in the results for gcc > 4.9 in the scan. Here are the remaining issues that were pointed out -- which I completely agree may be the result of an ill designed check, but may be of some mild interest to devs.

At the very least it may be worth considering that people would like to use this library and have certain requirements for code conformance from "larger customers" -- and the use cases of abseil may increase if reports on such static analyses were available directly on some site or repo. This isn't a suggestion, more of a comment.

Anyway -- here are the remaining issues:
CAT II:
DOS risks:
abseil-cpp/absl/flags/usage.cc:49 Sink: program_usage_message = operator new(...)
Enclosing Method: SetProgramUsageMessage()

abseil-cpp/absl/flags/internal/program_name.cc:53 Sink: program_name = operator new(...)
Enclosing Method: SetProgramInvocationName()

abseil-cpp/absl/random/internal/pool_urbg.cc:197 Sink: memory = malloc(...)
Enclosing Method: PoolAlignedAlloc()

abseil-cpp/absl/flags/internal/flag.h:102 Sink: Dereferenced : 0
Enclosing Method: Alloc()

abseil-cpp/absl/random/internal/seed_material.cc:132 Sink: dev_urandom = open(...)
Enclosing Method: ReadSeedMaterialFromOSEntropyImpl()

abseil-cpp/absl/time/internal/cctz/
src/time_zone_info.cc:647 Sink: dev_urandom = open(...)
Enclosing Method: ReadSeedMaterialFromOSEntropyImpl()

abseil-cpp/absl/time/internal/cctz/src/time_zone_info.cc:647 Sink: fp = FOpen(...)
Enclosing Method: Open()

abseil-cpp/absl/random/internal/pool_urbg.cc:197 Sink: memory = malloc(...) : malloc may return NULL
Enclosing Method: PoolAlignedAlloc()

abseil-cpp/absl/time/internal/cctz/src/time_zone_format.cc:912 Sink: ParseTM(data, ?, ?) : ParseTM may return NULL Enclosing Method: parse()

abseil/abseil-cpp/absl/synchronization/mutex.cc:2057 Sink: Dereferenced : waitp
Enclosing Method: UnlockSlow()

CAT I "input handling vulnerabilities"
abseil-cpp/absl/strings/internal/charconv_bigint.h:115 Sink: Assignment to this->words_
Enclosing Method: ShiftLeft()

abseil-cpp/absl/base/internal/low_level_alloc.cc:162 Sink: Assignment to prev[]
Enclosing Method: LLA_SkiplistSearch()
Source: mmap() from absl::base_internal.DoAllocWithArena() /abseil-cpp/absl/base/int
ernal/low_level_alloc.cc:562

abseil-cpp/absl/base/internal/low_level_alloc.cc:162 Sink: Assignment to prev[] Enclosing Method: LLA_SkiplistSearch() Source: sysconf() from absl::debugging_internal. SymbolizerSize() In abseil/abseil-cpp/absl/debugging/symbolize_elf.inc:355

abseil-cpp/absl/debugging/symbolize_elf.inc:419 Sink: read() Enclosing Method: ReadPersistent()
Source: read() from absl::debugging_internal.ReadPersistent() In abseil-cpp/absl/debugging/symbolize_elf.inc:419

abseil-cpp/absl/synchronization/internal/graphcycles.cc:96 Sink: Assignment to this->ptr_[] Enclosing Method: push_back() Source: mmap() from absl::base_internal.DoAllocWithArena() In /abseil-cpp/absl/base/internal/low_level_alloc.cc:562

all cases of strcpy, stlen, strmp, snprintf, GetPageSize()

@derekmauro
Copy link
Member

abseil-cpp/absl/random/internal/pool_urbg.cc:197 Sink: memory = malloc(...) : malloc may return NULL
Enclosing Method: PoolAlignedAlloc()

I've patched this one. The fix should be appearing on GitHub soon.

The rest of the issues do not appear to be legitimate bugs. As @tituswinters mentioned, we will accept patches for the rest of the warning if the patches do not complicate or pessimize the code. However, we are unable to prioritize silencing the rest of the warning ourselves.

@tombmantle
Copy link
Author

Hey Derek, thanks for checking out the issues. I'm glad that the fortify report was worth reading over for you all -- and I fully understand that engineering fixes for all of those is not a priority. It's something that our company may at some point in the future consider making a branch for.

Best,
Tom

absl-federation-github pushed a commit that referenced this issue Apr 20, 2021
@dinord
Export of internal Abseil changes
1ae9b71 
--
ac1df60490c9583e475e22de7adfc40023196fbf by Martijn Vels <mvels@google.com>:

Change Cord constructor(string_view) to explicit make_tree and Cordz tracking

This CL changes the ctor to use an easier to maintain model where Cord code explicitly invokes Cordz update or new / tree logic, which avoids the ambiguity of the 'branched' InlineRep::set_tree code. This removes the need to equip InlineRep with 'MethodIdentifier' or other necessary call info, and also is a cleaner model: InlineRep is carrying too much code now that should plainly sit in Cord, especially with all internal abstractions having moved to InlineData.

See child CL(s) for desired state

PiperOrigin-RevId: 369433619

--
b665af7f586e6c679a8b27d4f78d5a1d2b596058 by Abseil Team <absl-team@google.com>:

Rename the 'Compare' template type to 'LessThan', as the passed-in function is expected to act like operator<. It is worth avoiding confusion with std::compare, which returns an int (-1/0/1), as due to implicit casting this can lead to hard-to-spot bugs.

PiperOrigin-RevId: 369391118

--
c3c775269cad0f4982ec63f3616dd78bb9e52dca by Martijn Vels <mvels@google.com>:

Integrate CordzUpdateTracker into CordzInfo

PiperOrigin-RevId: 369348824

--
771d81ed357496c117179e1daec76eba5155932d by Martijn Vels <mvels@google.com>:

Replace mutex() with Lock() / Unlock() function

Mini design future tracking of CordzInfo sampled cords: CordzInfo holds a CordRep* reference without a reference count. Cord is responsible for synchronizing updates for sampled cords such that the CordRep* contained in CordzInfo is at all times valid. This is done by scoping Lock() and Unlock() calls around the code modifying the code of a sampled cord. For example (using the future CL CordzUpdateScope()):

CordzInfo* cordz_info = get_cordz_info();
CordzUpdateScope scope(cordz_info, CordzUpdateTracker::kRemovePrefix);
CordRep* rep = RemovePrefixImpl(root);
set_tree(rep);
if (cordz_info) {
  cordz_info->SetCordRep(rep);
}

On CordzInfo::Unlock(), if the internal rep is null, the cord is no longer sampled, and CordzInfo will be deleted. Thus any update resulting in the Cord being inlined will automatically no longer be sampled.

PiperOrigin-RevId: 369338802

--
5563c12df04a1e965a03b50bdd032739c55c0706 by Martijn Vels <mvels@google.com>:

Add UpdateTracker to CordzStatistics

PiperOrigin-RevId: 369318178

--
6b4d8463722a3e55a3e8f6cb3741a41055e7f83e by Martijn Vels <mvels@google.com>:

Add kClear, kConstructor* and kUnknown values and fix typo

PiperOrigin-RevId: 369297163

--
041adcbc929789d6d53371a8236840fc350e1eeb by Derek Mauro <dmauro@google.com>:

Switch from malloc to operator new in pool_urbg.cc
so it can only fail by throwing/aborting

PiperOrigin-RevId: 369274087

--
5d97a5f43e3f2d02d0a5bbe586d93b5751812981 by Benjamin Barenblat <bbaren@google.com>:

Correct Thumb function bound computation in the symbolizer

On 32-bit ARM, all functions are aligned to multiples of two bytes, and
the lowest-order bit in a function’s address is ignored by the CPU when
computing branch targets. That bit is still present in instructions and
ELF symbol tables, though; it’s repurposed to indicate whether the
function contains ARM or Thumb code. If the symbolizer doesn’t ignore
that bit, it will believe Thumb functions have boundaries that are off
by one byte, so instruct the symbolizer to null out the lowest-order bit
after retrieving it from the symbol table.

PiperOrigin-RevId: 369254082

--
462bb307c6cc332c1e2c3adb5f0cad51804bf937 by Derek Mauro <dmauro@google.com>:

Add a check for malloc failure in pool_urbg.cc
GitHub #940

PiperOrigin-RevId: 369238100
GitOrigin-RevId: ac1df60490c9583e475e22de7adfc40023196fbf
Change-Id: Ic6ec91c62cd3a0031f6a75a43a83da959ece2d25
@tituswinters
Copy link

tituswinters commented Apr 26, 2021 via email

razmser pushed a commit to razmser/abseil-cpp that referenced this issue Sep 12, 2023
@razmser
Export of internal Abseil changes
cd514fa 
--
ac1df60490c9583e475e22de7adfc40023196fbf by Martijn Vels <mvels@google.com>:

Change Cord constructor(string_view) to explicit make_tree and Cordz tracking

This CL changes the ctor to use an easier to maintain model where Cord code explicitly invokes Cordz update or new / tree logic, which avoids the ambiguity of the 'branched' InlineRep::set_tree code. This removes the need to equip InlineRep with 'MethodIdentifier' or other necessary call info, and also is a cleaner model: InlineRep is carrying too much code now that should plainly sit in Cord, especially with all internal abstractions having moved to InlineData.

See child CL(s) for desired state

PiperOrigin-RevId: 369433619

--
b665af7f586e6c679a8b27d4f78d5a1d2b596058 by Abseil Team <absl-team@google.com>:

Rename the 'Compare' template type to 'LessThan', as the passed-in function is expected to act like operator<. It is worth avoiding confusion with std::compare, which returns an int (-1/0/1), as due to implicit casting this can lead to hard-to-spot bugs.

PiperOrigin-RevId: 369391118

--
c3c775269cad0f4982ec63f3616dd78bb9e52dca by Martijn Vels <mvels@google.com>:

Integrate CordzUpdateTracker into CordzInfo

PiperOrigin-RevId: 369348824

--
771d81ed357496c117179e1daec76eba5155932d by Martijn Vels <mvels@google.com>:

Replace mutex() with Lock() / Unlock() function

Mini design future tracking of CordzInfo sampled cords: CordzInfo holds a CordRep* reference without a reference count. Cord is responsible for synchronizing updates for sampled cords such that the CordRep* contained in CordzInfo is at all times valid. This is done by scoping Lock() and Unlock() calls around the code modifying the code of a sampled cord. For example (using the future CL CordzUpdateScope()):

CordzInfo* cordz_info = get_cordz_info();
CordzUpdateScope scope(cordz_info, CordzUpdateTracker::kRemovePrefix);
CordRep* rep = RemovePrefixImpl(root);
set_tree(rep);
if (cordz_info) {
  cordz_info->SetCordRep(rep);
}

On CordzInfo::Unlock(), if the internal rep is null, the cord is no longer sampled, and CordzInfo will be deleted. Thus any update resulting in the Cord being inlined will automatically no longer be sampled.

PiperOrigin-RevId: 369338802

--
5563c12df04a1e965a03b50bdd032739c55c0706 by Martijn Vels <mvels@google.com>:

Add UpdateTracker to CordzStatistics

PiperOrigin-RevId: 369318178

--
6b4d8463722a3e55a3e8f6cb3741a41055e7f83e by Martijn Vels <mvels@google.com>:

Add kClear, kConstructor* and kUnknown values and fix typo

PiperOrigin-RevId: 369297163

--
041adcbc929789d6d53371a8236840fc350e1eeb by Derek Mauro <dmauro@google.com>:

Switch from malloc to operator new in pool_urbg.cc
so it can only fail by throwing/aborting

PiperOrigin-RevId: 369274087

--
5d97a5f43e3f2d02d0a5bbe586d93b5751812981 by Benjamin Barenblat <bbaren@google.com>:

Correct Thumb function bound computation in the symbolizer

On 32-bit ARM, all functions are aligned to multiples of two bytes, and
the lowest-order bit in a function’s address is ignored by the CPU when
computing branch targets. That bit is still present in instructions and
ELF symbol tables, though; it’s repurposed to indicate whether the
function contains ARM or Thumb code. If the symbolizer doesn’t ignore
that bit, it will believe Thumb functions have boundaries that are off
by one byte, so instruct the symbolizer to null out the lowest-order bit
after retrieving it from the symbol table.

PiperOrigin-RevId: 369254082

--
462bb307c6cc332c1e2c3adb5f0cad51804bf937 by Derek Mauro <dmauro@google.com>:

Add a check for malloc failure in pool_urbg.cc
GitHub abseil#940

PiperOrigin-RevId: 369238100
GitOrigin-RevId: ac1df60490c9583e475e22de7adfc40023196fbf
Change-Id: Ic6ec91c62cd3a0031f6a75a43a83da959ece2d25
razmser pushed a commit to razmser/abseil-cpp that referenced this issue Sep 12, 2023
@razmser
Export of internal Abseil changes
935bf97 
--
ac1df60490c9583e475e22de7adfc40023196fbf by Martijn Vels <mvels@google.com>:

Change Cord constructor(string_view) to explicit make_tree and Cordz tracking

This CL changes the ctor to use an easier to maintain model where Cord code explicitly invokes Cordz update or new / tree logic, which avoids the ambiguity of the 'branched' InlineRep::set_tree code. This removes the need to equip InlineRep with 'MethodIdentifier' or other necessary call info, and also is a cleaner model: InlineRep is carrying too much code now that should plainly sit in Cord, especially with all internal abstractions having moved to InlineData.

See child CL(s) for desired state

PiperOrigin-RevId: 369433619

--
b665af7f586e6c679a8b27d4f78d5a1d2b596058 by Abseil Team <absl-team@google.com>:

Rename the 'Compare' template type to 'LessThan', as the passed-in function is expected to act like operator<. It is worth avoiding confusion with std::compare, which returns an int (-1/0/1), as due to implicit casting this can lead to hard-to-spot bugs.

PiperOrigin-RevId: 369391118

--
c3c775269cad0f4982ec63f3616dd78bb9e52dca by Martijn Vels <mvels@google.com>:

Integrate CordzUpdateTracker into CordzInfo

PiperOrigin-RevId: 369348824

--
771d81ed357496c117179e1daec76eba5155932d by Martijn Vels <mvels@google.com>:

Replace mutex() with Lock() / Unlock() function

Mini design future tracking of CordzInfo sampled cords: CordzInfo holds a CordRep* reference without a reference count. Cord is responsible for synchronizing updates for sampled cords such that the CordRep* contained in CordzInfo is at all times valid. This is done by scoping Lock() and Unlock() calls around the code modifying the code of a sampled cord. For example (using the future CL CordzUpdateScope()):

CordzInfo* cordz_info = get_cordz_info();
CordzUpdateScope scope(cordz_info, CordzUpdateTracker::kRemovePrefix);
CordRep* rep = RemovePrefixImpl(root);
set_tree(rep);
if (cordz_info) {
  cordz_info->SetCordRep(rep);
}

On CordzInfo::Unlock(), if the internal rep is null, the cord is no longer sampled, and CordzInfo will be deleted. Thus any update resulting in the Cord being inlined will automatically no longer be sampled.

PiperOrigin-RevId: 369338802

--
5563c12df04a1e965a03b50bdd032739c55c0706 by Martijn Vels <mvels@google.com>:

Add UpdateTracker to CordzStatistics

PiperOrigin-RevId: 369318178

--
6b4d8463722a3e55a3e8f6cb3741a41055e7f83e by Martijn Vels <mvels@google.com>:

Add kClear, kConstructor* and kUnknown values and fix typo

PiperOrigin-RevId: 369297163

--
041adcbc929789d6d53371a8236840fc350e1eeb by Derek Mauro <dmauro@google.com>:

Switch from malloc to operator new in pool_urbg.cc
so it can only fail by throwing/aborting

PiperOrigin-RevId: 369274087

--
5d97a5f43e3f2d02d0a5bbe586d93b5751812981 by Benjamin Barenblat <bbaren@google.com>:

Correct Thumb function bound computation in the symbolizer

On 32-bit ARM, all functions are aligned to multiples of two bytes, and
the lowest-order bit in a function’s address is ignored by the CPU when
computing branch targets. That bit is still present in instructions and
ELF symbol tables, though; it’s repurposed to indicate whether the
function contains ARM or Thumb code. If the symbolizer doesn’t ignore
that bit, it will believe Thumb functions have boundaries that are off
by one byte, so instruct the symbolizer to null out the lowest-order bit
after retrieving it from the symbol table.

PiperOrigin-RevId: 369254082

--
462bb307c6cc332c1e2c3adb5f0cad51804bf937 by Derek Mauro <dmauro@google.com>:

Add a check for malloc failure in pool_urbg.cc
GitHub abseil#940

PiperOrigin-RevId: 369238100
GitOrigin-RevId: ac1df60490c9583e475e22de7adfc40023196fbf
Change-Id: Ic6ec91c62cd3a0031f6a75a43a83da959ece2d25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@derekmauro @tituswinters @Mbellsudteen @tombmantle