Add other libs #25

ajabep · 2020-08-15T12:28:34Z

Add:

Monitoring of DiectLibc
Monitoring of Musl
Monitoring of Alpine (has only musl)
Had to change the regex to extract symbols (no @@GLIBC after symbols). To do so, I pulled the one of @blukat29 (blukat29@287ca62#diff-6f1488814a51063192c9aabb59112ef1R11)

Requirements:

Require merging Add monitoring of others OS #24

niklasb · 2020-08-16T19:24:21Z

please rebase on top of master

ajabep · 2020-08-16T20:47:36Z

I've rebased on top of #24, because it use some sutff of it

ajabep · 2020-08-18T16:59:23Z

Ok, will rebase it when #24 will be merged.

niklasb · 2020-08-20T10:31:13Z

can you rebase this?

@blukat29

Regex from @blukat29 (blukat29@287ca62#diff-6f1488814a51063192c9aabb59112ef1R11)

niklasb · 2020-08-31T11:23:55Z

common/libc.sh

@@ -8,7 +8,7 @@ die() {
 }

 dump_symbols() {
-  readelf -Ws $1 | perl -n -e '/: (\w*).*?(\w+)@@GLIBC_/ && print "$2 $1\n"'
+  readelf -Ws $1 | perl -n -e '/: (\w*)[\s]*[\w]*[\s]*(FUNC|OBJECT)[\s]*[\w]*[\s]*[\w]*[\s]*[\w]*[\s]*(\w*)/ && print "$3 $1\n"'


wait what does this do. I cannot understand :D can you give examples of what this matches that was not matched before?

Hi there,

This line matches libs which does not append @@GLIBC_*** after symbols (i.e. all libc which are not glibc):

$ readelf -Ws musl-1.2.1-r0.so | head La table de symboles « .dynsym » contient 1698 entrées : Num: Valeur Tail Type Lien Vis Ndx Nom 0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND 1: 000000000002b510 332 FUNC GLOBAL DEFAULT 8 y0f 2: 000000000001f373 27 FUNC GLOBAL DEFAULT 8 copy_file_range 3: 000000000004faea 75 FUNC GLOBAL DEFAULT 8 strlen 4: 00000000000183a0 124 FUNC GLOBAL DEFAULT 8 ctanf 5: 000000000004bcf2 403 FUNC GLOBAL DEFAULT 8 vfprintf 6: 00000000000463ce 23 FUNC GLOBAL DEFAULT 8 mkdirat $ readelf -Ws dietlibc_0.31-1ubuntu3_i386.so | head La table de symboles « .dynsym » contient 961 entrées : Num: Valeur Tail Type Lien Vis Ndx Nom 0: 00000000 0 NOTYPE LOCAL DEFAULT UND 1: 00000000 0 NOTYPE GLOBAL DEFAULT UND main 2: 0000bc99 4 FUNC GLOBAL DEFAULT 8 chown 3: 000158d0 1288 FUNC GLOBAL DEFAULT 8 iconv 4: 0000bd91 7 FUNC GLOBAL DEFAULT 8 mlockall 5: 0000bbe5 4 FUNC GLOBAL DEFAULT 8 ioctl 6: 00010c98 61 FUNC GLOBAL DEFAULT 8 rewinddir $ readelf -Ws glibc-2.12-1.212.el6_10.3.i686_2.so | head La table de symboles « .dynsym » contient 2330 entrées : Num: Valeur Tail Type Lien Vis Ndx Nom 0: 00000000 0 NOTYPE LOCAL DEFAULT UND 1: 00000000 0 NOTYPE WEAK DEFAULT UND _IO_stdin_used 2: 00000000 0 NOTYPE WEAK DEFAULT UND _dl_starting_up 3: 00000000 0 OBJECT GLOBAL DEFAULT UND __libc_stack_end@GLIBC_2.1 (28) 4: 00000000 0 FUNC GLOBAL DEFAULT UND ___tls_get_addr@GLIBC_2.3 (29) 5: 00000000 0 OBJECT GLOBAL DEFAULT UND _rtld_global_ro@GLIBC_PRIVATE (30) 6: 00000000 0 OBJECT GLOBAL DEFAULT UND __libc_enable_secure@GLIBC_PRIVATE (30)

To be more specific, here are symbols for the printf function:

$ readelf -Ws musl-1.2.1-r0.so | grep ' printf' 578: 0000000000049934 161 FUNC GLOBAL DEFAULT 8 printf $ readelf -Ws dietlibc_0.26-3_i386.so | grep ' printf' 802: 00013b60 37 FUNC GLOBAL DEFAULT 8 printf 283: 00000000 0 FILE LOCAL DEFAULT ABS printf.c 648: 00013b60 37 FUNC GLOBAL DEFAULT 8 printf $ readelf -Ws glibc-2.12-1.212.el6_10.3.i686_2.so | grep ' printf' 628: 0004b890 54 FUNC GLOBAL DEFAULT 12 printf@@GLIBC_2.0 1556: 0004ad80 43 FUNC GLOBAL DEFAULT 12 printf_size_info@@GLIBC_2.1 1979: 0004adb0 2730 FUNC GLOBAL DEFAULT 12 printf_size@@GLIBC_2.1 611: 00000000 0 FILE LOCAL DEFAULT ABS printf_fp.c 618: 00000000 0 FILE LOCAL DEFAULT ABS printf-prs.c 619: 00000000 0 FILE LOCAL DEFAULT ABS printf_fphex.c 632: 00000000 0 FILE LOCAL DEFAULT ABS printf_size.c 635: 00000000 0 FILE LOCAL DEFAULT ABS printf.c 784: 00000000 0 FILE LOCAL DEFAULT ABS printf-parsemb.c 808: 00000000 0 FILE LOCAL DEFAULT ABS printf-parsewc.c 3446: 00000000 0 FILE LOCAL DEFAULT ABS printf_chk.c 6655: 0004adb0 2730 FUNC GLOBAL DEFAULT 12 printf_size 7508: 0004ad80 43 FUNC GLOBAL DEFAULT 12 printf_size_info 8607: 0004b890 54 FUNC GLOBAL DEFAULT 12 printf

As shown by the previous code, glibc' symbols printf and printf@@GLIBC_2.0 are at the same addresses:

628: 0004b890 54 FUNC GLOBAL DEFAULT 12 printf@@GLIBC_2.0 8607: 0004b890 54 FUNC GLOBAL DEFAULT 12 printf

can you please make the regex more readable: /: (\w+)\s+\w+\s+(?:FUNC|OBJECT)\s+(?:\w+\s+){3}(\w+)/

also, shouldn't we then exclude the GLIBC version of the symbols?

/: (\w+)\s+\w+\s+(?:FUNC|OBJECT)\s+(?:\w+\s+){3}(\w+)\b(?!@@GLIBC)/

thanks a lot for the detailed explanation by the way, that was very helpful for me to understand better

You got right ^^
It's updated.

Changlog of the regex

Some glibc symbols has only symbols with @ chars. Moreover, symbols don't have a version without @

$ readelf -Ws db/libc6_2.10.1-0ubuntu15_amd64.so | grep -v @ La table de symboles « .dynsym » contient 2148 entrées : Num: Valeur Tail Type Lien Vis Ndx Nom 0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND 1: 000000000001e730 0 SECTION LOCAL DEFAULT 12 2: 0000000000365730 0 SECTION LOCAL DEFAULT 21 3: 0000000000000000 0 NOTYPE WEAK DEFAULT UND _dl_starting_up 644: 0000000000000000 0 OBJECT GLOBAL DEFAULT ABS GLIBC_PRIVATE 1009: 0000000000000000 0 OBJECT GLOBAL DEFAULT ABS GLIBC_2.3 1011: 0000000000000000 0 OBJECT GLOBAL DEFAULT ABS GLIBC_2.4 1012: 0000000000000000 0 OBJECT GLOBAL DEFAULT ABS GLIBC_2.5 1014: 0000000000000000 0 OBJECT GLOBAL DEFAULT ABS GLIBC_2.6 1020: 0000000000000000 0 OBJECT GLOBAL DEFAULT ABS GLIBC_2.7 1024: 0000000000000000 0 OBJECT GLOBAL DEFAULT ABS GLIBC_2.8 1026: 0000000000000000 0 OBJECT GLOBAL DEFAULT ABS GLIBC_2.9 1073: 0000000000000000 0 OBJECT GLOBAL DEFAULT ABS GLIBC_2.10 1373: 0000000000000000 0 OBJECT GLOBAL DEFAULT ABS GLIBC_2.2.5 1376: 0000000000000000 0 OBJECT GLOBAL DEFAULT ABS GLIBC_2.2.6 1541: 0000000000000000 0 OBJECT GLOBAL DEFAULT ABS GLIBC_2.3.2 1546: 0000000000000000 0 OBJECT GLOBAL DEFAULT ABS GLIBC_2.3.3 1549: 0000000000000000 0 OBJECT GLOBAL DEFAULT ABS GLIBC_2.3.4 $ readelf -Ws db/libc6_2.10.1-0ubuntu15_amd64.so | grep '\bprintf\b' 581: 000000000004fb50 162 FUNC GLOBAL DEFAULT 12 printf@@GLIBC_2.2.5

Actually (tested with printf only):

dietlibc and musl never have @

glibc-2.* files always have symbols with 2 @ and usually (but not always) symbols without @.

libc* files always have symbols with 2 @, and, possibly have symbols without @

Some libs can have duplicated symbols

$ # magik long quick-n-dirty oneliner giving a grepable answer: $ (for i in *.so ; do ; a="$(readelf -Ws $i | grep -E 'FUNC|OBJECT' | grep -o '\bprintf\b.*' | sort -u)"; echo -e "$i\t$(echo $a | wc -l)\t($(echo $a | tr "\n" ", "))"; ; done) | grep WhatYouWant dietlibc_0.26-3_i386.so 1 (printf,) musl-1.1.24-1-x86_64.so 1 (printf,) glibc-2.12-1.212.el6.x86_64.so 2 (printf,printf@@GLIBC_2.2.5,) glibc-2.12-4.el7.centos.x86_64.so 1 (printf@@GLIBC_2.2.5,) libc-2.29-20.mga7.i586.so 2 (printf,printf@@GLIBC_2.0,) libc6-x32_2.31-3_i386.so 1 (printf@@GLIBC_2.16,) [...] $ readelf -Ws dietlibc_0.26-3_i386.so | grep '\bprintf\b' 85: 000104d4 30 FUNC GLOBAL DEFAULT 7 printf 1855: 00000000 0 FILE LOCAL DEFAULT ABS printf.c 2226: 000104d4 30 FUNC GLOBAL DEFAULT 7 printf

Thus:

I've changed the capture group of @@GLIBC into a non-capturing and optional one.

I've added a sort -u filter to keep only unique symbols.

Tell me if it's ok for you :)

thanks a lot for the detailed explanation by the way, that was very helpful for me to understand better

You're welcome, with pleasure ;-)

Ah, moreover if you compare an old db with a new, to see if the regex do the work, you will possibly face to #33 ;-)

ajabep force-pushed the otherLibs branch 2 times, most recently from 655d9e7 to 935b808 Compare August 16, 2020 20:46

niklasb force-pushed the master branch 2 times, most recently from ef4699c to 7b00212 Compare August 19, 2020 07:53

ajabep force-pushed the otherLibs branch from 935b808 to 3d00cbc Compare August 20, 2020 10:15

ajabep added 4 commits August 20, 2020 17:04

Adapt API to other libc + Add Musl

f080a0e

Adapt regex to extract symbols (thank's @blukat29)

a386297

Regex from @blukat29 (blukat29@287ca62#diff-6f1488814a51063192c9aabb59112ef1R11)

Add Alpine

b885db4

Add DietLibc

27e30c3

ajabep force-pushed the otherLibs branch from 3d00cbc to 27e30c3 Compare August 20, 2020 15:07

niklasb approved these changes Aug 31, 2020

View reviewed changes

Fix symbols extracting regex

2852823

ajabep mentioned this pull request Sep 8, 2020

Name collision #33

Open

niklasb merged commit 4c77106 into niklasb:master Oct 19, 2020

ajabep deleted the otherLibs branch November 17, 2020 21:56

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add other libs #25

Add other libs #25

ajabep commented Aug 15, 2020 •

edited

Loading

niklasb commented Aug 16, 2020

ajabep commented Aug 16, 2020

ajabep commented Aug 18, 2020

niklasb commented Aug 20, 2020

niklasb Aug 31, 2020

ajabep Sep 5, 2020

niklasb Sep 6, 2020 •

edited

Loading

niklasb Sep 6, 2020 •

edited

Loading

niklasb Sep 6, 2020

ajabep Sep 8, 2020 •

edited

Loading

ajabep Sep 8, 2020

Add other libs #25

Add other libs #25

Conversation

ajabep commented Aug 15, 2020 • edited Loading

niklasb commented Aug 16, 2020

ajabep commented Aug 16, 2020

ajabep commented Aug 18, 2020

niklasb commented Aug 20, 2020

niklasb Aug 31, 2020

Choose a reason for hiding this comment

ajabep Sep 5, 2020

Choose a reason for hiding this comment

niklasb Sep 6, 2020 • edited Loading

Choose a reason for hiding this comment

niklasb Sep 6, 2020 • edited Loading

Choose a reason for hiding this comment

niklasb Sep 6, 2020

Choose a reason for hiding this comment

ajabep Sep 8, 2020 • edited Loading

Choose a reason for hiding this comment

Changlog of the regex

ajabep Sep 8, 2020

Choose a reason for hiding this comment

ajabep commented Aug 15, 2020 •

edited

Loading

niklasb Sep 6, 2020 •

edited

Loading

niklasb Sep 6, 2020 •

edited

Loading

ajabep Sep 8, 2020 •

edited

Loading