Add syslog's ident and facility parameters to config #2701

pprindeville · 2017-08-22T22:37:59Z

You might want to be able to set the ident (syslog.ident) and facility (syslog.facility) before a single line of scripting gets executed... especially if you're debugging code and it might crash before it has a chance to call openlog(), etc.

pprindeville · 2017-08-22T22:40:09Z

@bukka Should the existing support for ident and facility in FPM be refactored to share code?

bukka · 2017-08-23T09:10:53Z

@pprindeville This is nice but this won't give us an ability for setting an external server without going through the system logger. As I said I really like what nginx does as it re-implements the syslog client completely and allows direct remote logging. It has got many advantages and it's more flexible solution that could be especially useful for future improvements (especially in FPM where we could make things work in more non-blocking way...)

bukka · 2017-08-23T09:18:49Z

That being said I think it would be good to at least have this if we don't have an alternative solution before 7.3 feature deadline (this is a new feature so it's just for master which means 7.3).

pprindeville · 2017-08-23T16:21:55Z

This is nice but this won't give us an ability for setting an external server without going through the system logger.

@bukka Going directly to an external logger is, in some circles, considered a liability. One of the requirements of the Red Book section "2.2.2.2 Audit" is for centralized logging and collection, as it makes it easier to enforce access control, log rotation, etc. if it goes through a centralized mechanism.

Allowing individual apps or subsystems to defeat that would be counter to the the requirements of Red Book.

I would hope that this mechanism is configurable at build-time so that one could build PHP without logging bypass.

pprindeville · 2017-08-23T16:23:38Z

That being said I think it would be good to at least have this if we don't have an alternative solution before 7.3 feature deadline (this is a new feature so it's just for master which means 7.3).

@bukka These are really orthogonal issues. My fix is to make the usage of the syslog API more complete. Yours is an entire alternative to it.

It's not an "either/or" but rather "and".

pprindeville · 2017-08-23T16:30:57Z

Amended: if we call openlog() explicitly from the script, we don't want the first call to syslog() calling it again.

bukka · 2017-08-24T16:48:02Z

@pprindeville There are many use cases when direct remote logging is useful especially when it comes to containers. For example you might want to run Docker containers for rsyslog and another for PHP and log into the rsyslog container. Of course I'm sure that there are many more examples and I also believe that it wouldn't be in nginx if it was pointless :). The fact is that Red Book was released in 1987 and lots of things have changed since then.

Personally I see the future of PHP (namely FPM) running as a single app inside the containers and that's why I'm looking to the solutions how to improve logging in there because currently it there are lots of issues and limitations.

pprindeville · 2017-08-25T19:04:52Z

@bukka Can you please explain to me the order of .ini and .conf files read by php-cli vs. php-fpm? Or point me at where the differences are explained? Because with this change, it seems like php-fpm is picking up the syslog.ident and syslog.facility and the values from php7-fpm.ini or php7-fpm.d/*.conf are being ignored (or overwritten).

bukka · 2017-08-28T16:57:12Z

@pprindeville The main ini file (e.g. php.ini) is loaded during the startup in fpm which happens when you start master. You can find it in fpm_main.c and the call flow is like this:

main -> cgi_sapi_module.startup -> php_module_startup -> php_init_config

FPM allows to redefine the ini values which is described in here

php-src/sapi/fpm/www.conf.in

Lines 394 to 417 in ebb3396

    
           ; Additional php.ini defines, specific to this pool of workers. These settings 
        
           ; overwrite the values previously defined in the php.ini. The directives are the 
        
           ; same as the PHP SAPI: 
        
           ;   php_value/php_flag             - you can set classic ini defines which can 
        
           ;                                    be overwritten from PHP call 'ini_set'. 
        
           ;   php_admin_value/php_admin_flag - these directives won't be overwritten by 
        
           ;                                     PHP call 'ini_set' 
        
           ; For php_*flag, valid values are on, off, 1, 0, true, false, yes or no. 
        
           ; Defining 'extension' will load the corresponding shared extension from 
        
           ; extension_dir. Defining 'disable_functions' or 'disable_classes' will not 
        
           ; overwrite previously defined php.ini values, but will append the new value 
        
           ; instead. 
        
           ; Note: path INI options can be relative and will be expanded with the prefix 
        
           ; (pool, global or @prefix@) 
        
           ; Default Value: nothing is defined by default except the values in php.ini and 
        
           ;                specified at startup with the -d argument 
        
           ;php_admin_value[sendmail_path] = /usr/sbin/sendmail -t -i -f www@my.domain.com 
        
           ;php_flag[display_errors] = off 
        
           ;php_admin_value[error_log] = /var/log/fpm-php.www.log 
        
           ;php_admin_flag[log_errors] = on 
        
           ;php_admin_value[memory_limit] = 32M

and it happens during the child init (search for fpm_php_init_child in fpm_php.c where you can see the logic - fpmi_php_apply_defines is the main function for that) which happens when the new child process is created.

GDB should help you with the rest. The redefinition is after fork so it's useful to set follow-fork-mode to child and detach-on-fork to off ;)

pprindeville · 2017-08-28T23:42:31Z

@bukka Thanks, that helped. I had forgotten to set the have_called_openlog flag from inside fpm_stdio_open_error_log() but that's fixed now.

pprindeville · 2017-08-28T23:46:24Z

@nikic Can you please assign a reviewer?

weltling · 2017-08-31T09:13:06Z

main/php_syslog.c

@@ -37,6 +38,8 @@
 #define syslog std_syslog
 #endif

+zend_bool have_called_openlog = 0;


This is not thread safe, thus it should go into globals.

This is not thread safe, thus it should go into globals.

Not sure I understand. What's an example of a global variable that I can use as an example? Are you talking about SG() and sapi_globals or something else?

Yeah, something in this direction. Perhaps core globals were more suitable, as it's not something SAPI specific. Another option in PHP 7 is using ZEND_TLS, but should check whether its usage were suitable in this case.

Isn't core globals for configuration state?

This is not thread safe, thus it should go into globals.

@weltling Done

weltling · 2017-08-31T09:21:04Z

@pprindeville new features for feature frozen branches are handled on case by case base. As @bukka works on a patch for this for quite some time already, it probably would make sense you to coordinate with him. IMO the remote logging solution he suggests is promising for both functionality and maintenance.

Thanks.

pprindeville · 2017-08-31T21:53:56Z

Not sure what to do about the Travis failure. Looks like the build is taking too long and that master has some unrelated test case failures going on.

pprindeville · 2017-09-01T20:23:39Z

Is anything else needed at this point?

laruence · 2017-09-02T04:14:03Z

Zend/zend_globals.h

@@ -127,6 +127,8 @@ struct _zend_compiler_globals {
 	zval **static_members_table;
 	int last_static_member;
 #endif
+
+	zend_bool have_called_openlog;


I don't understand why this need to in CG? PG maybe better? thanks

Well, I asked and no one answered...

Isn't core_globals for configurable state?

CG is for compiler globals, PG is more suitable in this case IMO.

@pprindeville This really should be in PG...

@pprindeville This really should be in PG...

Where/how do you initialize a core_globals variable that's not user settable? What's the appropriate modifiable value?

I think I've got the correct STD_PHP_INI_BOOLEAN(...) for have_called_openlog but would appreciate someone else giving it a quick look. Thanks

KalleZ · 2017-09-02T10:48:57Z

Hi

CG is compiler globals
EG is engine and executor globals
PG is PHP globals like configuration and inis

pprindeville · 2017-09-03T03:08:37Z

@KalleZ: okay, so where should I put have_called_openlog given that it's not a configuration option? Are we good with CG()?

KalleZ · 2017-09-03T12:36:39Z

It's a PHP specific thing so leave it in PG, Note that only engine and core language is Zend, whereas PHP is sapis, standard libraries and extensions

pprindeville · 2017-09-07T00:00:25Z

Okay. No further rework required then? When might it merge?

Thanks

pprindeville · 2018-06-21T16:23:08Z

Any other issues?

bukka · 2018-06-22T14:31:52Z

ext/standard/syslog.c

@@ -152,6 +152,7 @@ PHP_FUNCTION(openlog)
 		RETURN_FALSE;
 	}
 	openlog(BG(syslog_device), option, facility);
+	PG(have_called_openlog) = 1;


I would suggest one small improvement. Maybe we could introduce php_openlog with the same args as openlog that would just call openlog and set PG(have_called_openlog) so it's done internally in that function. Then it would be used in here and fpm.

Then... syslog() would need to call php_openlog()... What would that change?

What I'm saying is to just replace:

openlog(BG(syslog_device), option, facility); PG(have_called_openlog) = 1;

with

php_openlog(BG(syslog_device), option, facility);

and implement php_openlog as

void php_openlog(const char *ident, int option, int facility) { openlog(BG(syslog_device), option, facility); PG(have_called_openlog) = 1; }

Then you also replace it in other place. It's just a wrapper as you use the same pattern in 2 other places - no change in your logic...

bukka · 2018-06-22T14:34:48Z

@pprindeville Just commented about two things and then it should be good if all tests are fine (I mean the manual testing that I plan to do). Sorry for the late requests, it's just because I finally found time to look into this and think about it... :)

pprindeville · 2018-06-22T18:07:30Z

win32/wsyslog.c

@@ -95,10 +95,6 @@ void vsyslog(int priority, const char *message, va_list args)
 	DWORD evid;
 	wchar_t *strsw[2];

-	/* default event source */


There are no calls to vsyslog() (directly), it all flows through php_syslog() so there's no way to reach here without openlog() having been called first (and thereby setting log_source).

pprindeville · 2018-06-22T18:08:25Z

BTW, what's the difference between PW32G() and PG()?

KalleZ · 2018-06-22T18:56:10Z

@pprindeville PW32G is "PHP Win32 Globals", so only globals that do stuff specifically for Windows, where as PG is simply "PHP Globals", as in globals that interact within the PHP layer of the PHP stack (like ini settings)

pprindeville · 2018-06-22T21:06:28Z

PW32G is "PHP Win32 Globals", so only globals that do stuff specifically for Windows, where as PG is simply "PHP Globals", as in globals that interact within the PHP layer of the PHP stack (like ini settings)

Got it. Well, hopefully everyone is okay with the latest version?

pprindeville · 2018-06-22T21:08:01Z

Uh...

========DIFF========
3753 + Wrong result for Africa/Addis_Ababa!
3754 - ===DONE===
3755 + 
3756 + Wrong result for Africa/Algiers!
3757 + 
3758 + Wrong result for Africa/Asmara!
3759 + 
3760 + Wrong result for Africa/Bangui!
3761 + 
3762 + Wrong result for Africa/Blantyre!
3763 + 
3764 + Wrong result for Africa/Brazzaville!
3765 + 
3766 + Wrong result for Africa/Bujumbura!
...

Pretty sure this has nothing to do with these changes. Also reasonably sure this was working yesterday.

pprindeville · 2018-06-26T01:19:16Z

If anyone is waiting on something from me to move this forward, please call it out.

pprindeville · 2018-06-29T20:37:26Z

So... has this fix missed 7.3.0 or am I mistaken?

nikic · 2018-06-29T20:41:14Z

@pprindeville Nope, hasn't missed 7.3 (yet...) Projected release schedule can be found here: https://wiki.php.net/todo/php73#timetable

pprindeville · 2018-06-29T20:46:12Z

Nope, hasn't missed 7.3 (yet...)

Well, it's getting close. If we don't make Alpha 3 (in 3 weeks) it looks like we'll have missed the feature freeze.

pprindeville · 2018-06-29T20:50:30Z

The CI builds are failing due to unrelated errors, btw.

pprindeville · 2018-06-29T21:07:35Z

@bukka Are there any blocking review comments still? You're showing (currently) as requesting changes.

bukka · 2018-06-30T17:43:37Z

php.ini-development

@@ -577,6 +577,8 @@ html_errors = On
 ;error_log = php_errors.log
 ; Log errors to syslog (Event Log on Windows).
 ;error_log = syslog
+;syslog.ident = php


Could you add a comment line (just short description and that it applies only if error_log is syslog) for each new ini and also update php.ini-production with the same setting?

Good idea.

Done!

bukka · 2018-06-30T17:45:36Z

I did a little bit of testing and all seems good. Could you just update ini files and I will try to merge it tomorrow or sometimes next week.

Signed-off-by: Philip Prindeville <philipp@redfish-solutions.com>

php-pulls · 2018-07-01T17:31:01Z

Comment on behalf of bukka at php.net:

Marge in 2475337

pprindeville · 2018-07-01T17:36:35Z

Awesome, thanks!

pprindeville mentioned this pull request Aug 22, 2017

handle non-NVT ASCII messages per the RFC's #2689

Closed

pprindeville force-pushed the syslog-add-facility-and-ident branch 3 times, most recently from 93d2765 to 393e350 Compare August 22, 2017 23:19

pprindeville force-pushed the syslog-add-facility-and-ident branch from 393e350 to a56e2d9 Compare August 23, 2017 16:10

pprindeville force-pushed the syslog-add-facility-and-ident branch from a56e2d9 to d1a74f4 Compare August 27, 2017 23:58

weltling reviewed Aug 31, 2017

View reviewed changes

pprindeville force-pushed the syslog-add-facility-and-ident branch 3 times, most recently from 0cf970b to a34f3c1 Compare August 31, 2017 19:33

laruence reviewed Sep 2, 2017

View reviewed changes

krakjoe added the Feature label Sep 6, 2017

pprindeville force-pushed the syslog-add-facility-and-ident branch 2 times, most recently from 9d3931e to 59d7164 Compare June 20, 2018 19:51

bukka reviewed Jun 22, 2018

View reviewed changes

pprindeville force-pushed the syslog-add-facility-and-ident branch 3 times, most recently from e07c1d8 to fe1163e Compare June 22, 2018 18:02

pprindeville commented Jun 22, 2018

View reviewed changes

pprindeville force-pushed the syslog-add-facility-and-ident branch from fe1163e to 69b46df Compare June 29, 2018 18:58

bukka reviewed Jun 30, 2018

View reviewed changes

pprindeville force-pushed the syslog-add-facility-and-ident branch from 69b46df to a490552 Compare June 30, 2018 18:22

Add syslog's ident and facility parameters to config

a667e38

Signed-off-by: Philip Prindeville <philipp@redfish-solutions.com>

pprindeville force-pushed the syslog-add-facility-and-ident branch from a490552 to a667e38 Compare June 30, 2018 18:37

php-pulls closed this Jul 1, 2018

pprindeville deleted the syslog-add-facility-and-ident branch July 1, 2018 17:39

cmb69 mentioned this pull request Jun 27, 2019

Fix #78212: Segfault in built-in webserver #4320

Closed

Add syslog's ident and facility parameters to config #2701

Add syslog's ident and facility parameters to config #2701

Uh oh!

Conversation

pprindeville commented Aug 22, 2017

Uh oh!

pprindeville commented Aug 22, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bukka commented Aug 23, 2017

Uh oh!

bukka commented Aug 23, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pprindeville commented Aug 23, 2017

Uh oh!

pprindeville commented Aug 23, 2017

Uh oh!

pprindeville commented Aug 23, 2017

Uh oh!

bukka commented Aug 24, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pprindeville commented Aug 25, 2017

Uh oh!

bukka commented Aug 28, 2017

Uh oh!

pprindeville commented Aug 28, 2017

Uh oh!

pprindeville commented Aug 28, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pprindeville Aug 31, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

weltling commented Aug 31, 2017

Uh oh!

pprindeville commented Aug 31, 2017

Uh oh!

pprindeville commented Sep 1, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pprindeville Dec 18, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

KalleZ commented Sep 2, 2017

Uh oh!

pprindeville commented Sep 3, 2017

Uh oh!

KalleZ commented Sep 3, 2017

Uh oh!

pprindeville commented Sep 7, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pprindeville commented Jun 21, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pprindeville Jun 22, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pprindeville commented Aug 22, 2017 •

edited

Loading

bukka commented Aug 23, 2017 •

edited

Loading

bukka commented Aug 24, 2017 •

edited

Loading

pprindeville Aug 31, 2017 •

edited

Loading

pprindeville Dec 18, 2017 •

edited

Loading

pprindeville commented Sep 7, 2017 •

edited

Loading

pprindeville Jun 22, 2018 •

edited

Loading

pprindeville commented Jun 29, 2018 •

edited

Loading