New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problems with type conversions #78

Closed
bigio opened this Issue Jan 13, 2017 · 38 comments

Comments

Projects
None yet
7 participants
@bigio

bigio commented Jan 13, 2017

Hi,
after upgrading to DBD::mysql 4.041 a problem appeared in amavisd-new:
https://lists.amavis.org/pipermail/amavis-users/2016-December/004674.html
The problem could be related to this commit:
caea0b7
Cheers
Giovanni

@bigio

This comment has been minimized.

Show comment
Hide comment
@bigio

bigio Jan 13, 2017

If it could be useful I am running MariaDB 10.0.28.

bigio commented Jan 13, 2017

If it could be useful I am running MariaDB 10.0.28.

@pali

This comment has been minimized.

Show comment
Hide comment
@pali

pali Jan 21, 2017

Contributor

Can you test last engineering version 4.041_1? Or version from git master?

Contributor

pali commented Jan 21, 2017

Can you test last engineering version 4.041_1? Or version from git master?

@bigio

This comment has been minimized.

Show comment
Hide comment
@bigio

bigio Jan 21, 2017

bigio commented Jan 21, 2017

@pali

This comment has been minimized.

Show comment
Hide comment
@pali

pali Jan 21, 2017

Contributor

Can you describe problem in DBD::mysql? Sorry, but from your report I just understand that amavisd-new does not work...

Contributor

pali commented Jan 21, 2017

Can you describe problem in DBD::mysql? Sorry, but from your report I just understand that amavisd-new does not work...

@bigio

This comment has been minimized.

Show comment
Hide comment
@bigio

bigio Jan 27, 2017

bigio commented Jan 27, 2017

@pali

This comment has been minimized.

Show comment
Hide comment
@pali

pali Jan 27, 2017

Contributor

Ideally if you can provide perl code which outputs that "wrong result".

Contributor

pali commented Jan 27, 2017

Ideally if you can provide perl code which outputs that "wrong result".

@brianmay

This comment has been minimized.

Show comment
Hide comment
@brianmay

brianmay Feb 11, 2017

I believe this is the relevant code in amavisd-new (note I haven't been able to reproduce this myself):

  eval {                       
    snmp_count('OpsSqlSelect');                      
    $conn_h->execute($sel,@pos_args);  # do the query
    # fetch query results                                       
    while ( defined($a_ref=$conn_h->fetchrow_arrayref($sel)) ) {
      my(@names) = @{$conn_h->sth($sel)->{NAME_lc}};
      $match = {}; @$match{@names} = @$a_ref;                         
      if ($clause_name eq 'sel_policy' && !exists $match->{'local'} &&
          defined $match->{'email'} && $match->{'email'} eq '@.') {    
          [ ... AFAIK not executed ...]
      }                                     
      push(@result, {%$match});  # copy hash                   
      push(@matchingkey, join(", ", map { sprintf("%s=>%s", $_,                 
                                !defined($match->{$_})?'-':'"'.$match->{$_}.'"'
                                ) } @names));
      last  if !$get_all;
    }

   [ ... ]
   do_log(4,'lookup_sql(%s) matches, result=(%s)', $addr,$_) for @matchingkey;

This results in output like:

(36754-01) lookup_sql(username at mydomain.tld) matches, result=(id=>"51", priority=>"3", policy_id=>"2", email=>"username at mydomain.tld", fullname=>-, local=>"Y", id=>"51", policy_name=>"Normal", virus_lover=>"N", spam_lover=>"N", banned_files_lover=>"N", bad_header_lover=>"N", bypass_virus_checks=>"N", bypass_spam_checks=>"N", bypass_banned_checks=>"N", bypass_header_checks=>"N", spam_modifies_subj=>"Y", virus_quarantine_to=>"", spam_quarantine_to=>-, banned_quarantine_to=>-, bad_header_quarantine_to=>-, clean_quarantine_to=>"", other_quarantine_to=>-, spam_tag_level=>"0", 
spam_tag2_level=>"0", spam_kill_level=>"0", spam_dsn_cutoff_level=>"0", spam_quarantine_cutoff_level=>"0", addr_extension_virus=>"", addr_extension_spam=>"", addr_extension_banned=>"", addr_extension_bad_header=>"", warnvirusrecip=>"N", warnbannedrecip=>"N", warnbadhrecip=>"Y", newvirus_admin=>"", virus_admin=>"", banned_admin=>"", bad_header_admin=>"", spam_admin=>"", spam_subject_tag=>"", spam_subject_tag2=>"[SPAMMY]", message_size_limit=>"0", banned_rulenames=>"", id=>"51")

Where the "0" values are wrong. These aren't just rounding errors, the values in the database were very different, e.g.:

spam_tag_level: -999
spam_tag2_level: 2.5
spam_kill_level: 4.7
spam_dsn_cutoff_level: 7
spam_quarantine_cutoff_level: 20

For more details see:

To me it looks like the relevant part is sprintf using "%s", I don't see how this could get the wrong value, even if it is string, float, or double. So I suspect DBD-mysql might be returning bad data.

Note that that I haven't reproduced this myself, just assembling data from existing reports.

brianmay commented Feb 11, 2017

I believe this is the relevant code in amavisd-new (note I haven't been able to reproduce this myself):

  eval {                       
    snmp_count('OpsSqlSelect');                      
    $conn_h->execute($sel,@pos_args);  # do the query
    # fetch query results                                       
    while ( defined($a_ref=$conn_h->fetchrow_arrayref($sel)) ) {
      my(@names) = @{$conn_h->sth($sel)->{NAME_lc}};
      $match = {}; @$match{@names} = @$a_ref;                         
      if ($clause_name eq 'sel_policy' && !exists $match->{'local'} &&
          defined $match->{'email'} && $match->{'email'} eq '@.') {    
          [ ... AFAIK not executed ...]
      }                                     
      push(@result, {%$match});  # copy hash                   
      push(@matchingkey, join(", ", map { sprintf("%s=>%s", $_,                 
                                !defined($match->{$_})?'-':'"'.$match->{$_}.'"'
                                ) } @names));
      last  if !$get_all;
    }

   [ ... ]
   do_log(4,'lookup_sql(%s) matches, result=(%s)', $addr,$_) for @matchingkey;

This results in output like:

(36754-01) lookup_sql(username at mydomain.tld) matches, result=(id=>"51", priority=>"3", policy_id=>"2", email=>"username at mydomain.tld", fullname=>-, local=>"Y", id=>"51", policy_name=>"Normal", virus_lover=>"N", spam_lover=>"N", banned_files_lover=>"N", bad_header_lover=>"N", bypass_virus_checks=>"N", bypass_spam_checks=>"N", bypass_banned_checks=>"N", bypass_header_checks=>"N", spam_modifies_subj=>"Y", virus_quarantine_to=>"", spam_quarantine_to=>-, banned_quarantine_to=>-, bad_header_quarantine_to=>-, clean_quarantine_to=>"", other_quarantine_to=>-, spam_tag_level=>"0", 
spam_tag2_level=>"0", spam_kill_level=>"0", spam_dsn_cutoff_level=>"0", spam_quarantine_cutoff_level=>"0", addr_extension_virus=>"", addr_extension_spam=>"", addr_extension_banned=>"", addr_extension_bad_header=>"", warnvirusrecip=>"N", warnbannedrecip=>"N", warnbadhrecip=>"Y", newvirus_admin=>"", virus_admin=>"", banned_admin=>"", bad_header_admin=>"", spam_admin=>"", spam_subject_tag=>"", spam_subject_tag2=>"[SPAMMY]", message_size_limit=>"0", banned_rulenames=>"", id=>"51")

Where the "0" values are wrong. These aren't just rounding errors, the values in the database were very different, e.g.:

spam_tag_level: -999
spam_tag2_level: 2.5
spam_kill_level: 4.7
spam_dsn_cutoff_level: 7
spam_quarantine_cutoff_level: 20

For more details see:

To me it looks like the relevant part is sprintf using "%s", I don't see how this could get the wrong value, even if it is string, float, or double. So I suspect DBD-mysql might be returning bad data.

Note that that I haven't reproduced this myself, just assembling data from existing reports.

@pali

This comment has been minimized.

Show comment
Hide comment
@pali

pali Feb 12, 2017

Contributor

That is strange :-( I'm still not able to reproduce this problem... I tried:

use strict;
use warnings;

use DBI;
use Data::Dumper;

my $dbh = DBI->connect("dbi:mysql:test", "user", "pass");
my $sth = $dbh->prepare("select * from t");
$sth->execute();

while ( my $ref = $sth->fetchrow_arrayref() ) {

  my @names = @{$sth->{NAME_lc}};
  my $match = {}; @$match{@names} = @$ref;

  my $str = join(", ", map { sprintf("%s=>%s", $_,                 
                                  !defined($match->{$_})?'-':'"'.$match->{$_}.'"'
                                  ) } @names);

  print Dumper [\@names, $ref, $match, $str];
  my $t = sprintf "%s", '"' . $ref->[0] . '"';
  print "$t\n";

}

and its output is correct:

$VAR1 = [
          [
            'id'
          ],
          [
            '-999'
          ],
          {
            'id' => '-999'
          },
          'id=>"-999"'
        ];
"-999"
$VAR1 = [
          [
            'id'
          ],
          [
            '2.5'
          ],
          {
            'id' => '2.5'
          },
          'id=>"2.5"'
        ];
"2.5"
$VAR1 = [
          [
            'id'
          ],
          [
            '4.7'
          ],
          {
            'id' => '4.7'
          },
          'id=>"4.7"'
        ];
"4.7"
$VAR1 = [
          [
            'id'
          ],
          [
            '7'
          ],
          {
            'id' => '7'
          },
          'id=>"7"'
        ];
"7"

Data stored in mysql are:

mysql> describe t;
+-------+-------+------+-----+---------+-------+
| Field | Type  | Null | Key | Default | Extra |
+-------+-------+------+-----+---------+-------+
| id    | float | YES  |     | NULL    |       |
+-------+-------+------+-----+---------+-------+
1 row in set (0.00 sec)

mysql> select * from t;
+------+
| id   |
+------+
| -999 |
|  2.5 |
|  4.7 |
|    7 |
+------+
4 rows in set (0.00 sec)

So I see only option now, enabling DBI tracing in that problematic amavis code and sending stderr trace output for later inspection. Tracing can be enabled by DBI->trace(2); and disabled by DBI->trace(0);.

Contributor

pali commented Feb 12, 2017

That is strange :-( I'm still not able to reproduce this problem... I tried:

use strict;
use warnings;

use DBI;
use Data::Dumper;

my $dbh = DBI->connect("dbi:mysql:test", "user", "pass");
my $sth = $dbh->prepare("select * from t");
$sth->execute();

while ( my $ref = $sth->fetchrow_arrayref() ) {

  my @names = @{$sth->{NAME_lc}};
  my $match = {}; @$match{@names} = @$ref;

  my $str = join(", ", map { sprintf("%s=>%s", $_,                 
                                  !defined($match->{$_})?'-':'"'.$match->{$_}.'"'
                                  ) } @names);

  print Dumper [\@names, $ref, $match, $str];
  my $t = sprintf "%s", '"' . $ref->[0] . '"';
  print "$t\n";

}

and its output is correct:

$VAR1 = [
          [
            'id'
          ],
          [
            '-999'
          ],
          {
            'id' => '-999'
          },
          'id=>"-999"'
        ];
"-999"
$VAR1 = [
          [
            'id'
          ],
          [
            '2.5'
          ],
          {
            'id' => '2.5'
          },
          'id=>"2.5"'
        ];
"2.5"
$VAR1 = [
          [
            'id'
          ],
          [
            '4.7'
          ],
          {
            'id' => '4.7'
          },
          'id=>"4.7"'
        ];
"4.7"
$VAR1 = [
          [
            'id'
          ],
          [
            '7'
          ],
          {
            'id' => '7'
          },
          'id=>"7"'
        ];
"7"

Data stored in mysql are:

mysql> describe t;
+-------+-------+------+-----+---------+-------+
| Field | Type  | Null | Key | Default | Extra |
+-------+-------+------+-----+---------+-------+
| id    | float | YES  |     | NULL    |       |
+-------+-------+------+-----+---------+-------+
1 row in set (0.00 sec)

mysql> select * from t;
+------+
| id   |
+------+
| -999 |
|  2.5 |
|  4.7 |
|    7 |
+------+
4 rows in set (0.00 sec)

So I see only option now, enabling DBI tracing in that problematic amavis code and sending stderr trace output for later inspection. Tracing can be enabled by DBI->trace(2); and disabled by DBI->trace(0);.

@brianmay

This comment has been minimized.

Show comment
Hide comment
@brianmay

brianmay Feb 13, 2017

Full debug output is available here: https://bugs.debian.org/cgi-bin/bugreport.cgi?att=2;bug=847311;filename=amavisd-debug.txt;msg=54

I believe these are the most relevant lines:

T  <- fetchrow_arrayref= ( [ 4 1 1 'riud' 'riud' '' 1 5 1 'RECEIVER@DOMAIN2.COM' 'RECEIVER@DOMAIN2.COM' 'Y' 1 1 0 'riud' 'riud' 'r' 'Normalni' 'N' 'N' 'N' 'N' 'N' 'N' 'N' 'N' 'N' '' '' '' '' '' '' 0 0 0 0 0 '' '' '' '' 'N' 'N' 'N' '' '' '' '' '' '' '' 0 '' -1 24 -1 24 'N' 4 ] ) [1 items] row1 at (eval 110) line 140
    -> FETCH for DBD::mysql::st (DBI::st=HASH(0x559a620afe90)~INNER 'NAME_lc') thr#559a5c331010
    -> dbd_st_FETCH_attrib for 559a5f004e98, key NAME_lc
    -> dbd_st_FETCH_attrib for 559a5f004e98, key NAME
 T  <- FETCH= ( [ 'id' 'sys_userid' 'sys_groupid' 'sys_perm_user' 'sys_perm_group' 'sys_perm_other' 'server_id' 'priority' 'policy_id' 'email' 'fullname' 'local' 'id' 'sys_userid' 'sys_groupid' 'sys_perm_user' 'sys_perm_group' 'sys_perm_other' 'policy_name' 'virus_lover' 'spam_lover' 'banned_files_lover' 'bad_header_lover' 'bypass_virus_checks' 'bypass_spam_checks' 'bypass_banned_checks' 'bypass_header_checks' 'spam_modifies_subj' 'virus_quarantine_to' 'spam_quarantine_to' 'banned_quarantine_to' 'bad_header_quarantine_to' 'clean_quarantine_to' 'other_quarantine_to' 'spam_tag_level' 'spam_tag2_level' 'spam_kill_level' 'spam_dsn_cutoff_level' 'spam_quarantine_cutoff_level' 'addr_extension_virus' 'addr_extension_spam' 'addr_extension_banned' 'addr_extension_bad_header' 'warnvirusrecip' 'warnbannedrecip' 'warnbadhrecip' 'newvirus_admin' 'virus_admin' 'banned_admin' 'bad_header_admin' 'spam_admin' 'spam_subject_tag' 'spam_subject_tag2' 'message_size_limit' 'banned_rulenames' 'policyd_quota_in' 'policyd_quota_in_period' 'policyd_quota_out' 'policyd_quota_out_period' 'policyd_greylist' 'id' ] ) [1 items] at (eval 112) line 308

(side note: what result would you expect if the database has NULL values?)

I believe (but not confirmed) this is the database schema used:
https://git.ispconfig.org/ispconfig/ispconfig3/blob/0f9fa27a6c7026f77c7d857c971b1db1b709f583/install/sql/ispconfig3.sql

I have requested the submitter provide the following information:

desc spamfilter_users;
desc spamfilter_policy;
SELECT *,spamfilter_users.id FROM spamfilter_users LEFT JOIN spamfilter_policy ON spamfilter_users.policy_id=spamfilter_policy.id WHERE spamfilter_users.email IN ('RECEIVER@DOMAIN2.COM','@domain2.com','@.domain2.com','@.com','@.') ORDER BY spamfilter_users.priority DESC;

brianmay commented Feb 13, 2017

Full debug output is available here: https://bugs.debian.org/cgi-bin/bugreport.cgi?att=2;bug=847311;filename=amavisd-debug.txt;msg=54

I believe these are the most relevant lines:

T  <- fetchrow_arrayref= ( [ 4 1 1 'riud' 'riud' '' 1 5 1 'RECEIVER@DOMAIN2.COM' 'RECEIVER@DOMAIN2.COM' 'Y' 1 1 0 'riud' 'riud' 'r' 'Normalni' 'N' 'N' 'N' 'N' 'N' 'N' 'N' 'N' 'N' '' '' '' '' '' '' 0 0 0 0 0 '' '' '' '' 'N' 'N' 'N' '' '' '' '' '' '' '' 0 '' -1 24 -1 24 'N' 4 ] ) [1 items] row1 at (eval 110) line 140
    -> FETCH for DBD::mysql::st (DBI::st=HASH(0x559a620afe90)~INNER 'NAME_lc') thr#559a5c331010
    -> dbd_st_FETCH_attrib for 559a5f004e98, key NAME_lc
    -> dbd_st_FETCH_attrib for 559a5f004e98, key NAME
 T  <- FETCH= ( [ 'id' 'sys_userid' 'sys_groupid' 'sys_perm_user' 'sys_perm_group' 'sys_perm_other' 'server_id' 'priority' 'policy_id' 'email' 'fullname' 'local' 'id' 'sys_userid' 'sys_groupid' 'sys_perm_user' 'sys_perm_group' 'sys_perm_other' 'policy_name' 'virus_lover' 'spam_lover' 'banned_files_lover' 'bad_header_lover' 'bypass_virus_checks' 'bypass_spam_checks' 'bypass_banned_checks' 'bypass_header_checks' 'spam_modifies_subj' 'virus_quarantine_to' 'spam_quarantine_to' 'banned_quarantine_to' 'bad_header_quarantine_to' 'clean_quarantine_to' 'other_quarantine_to' 'spam_tag_level' 'spam_tag2_level' 'spam_kill_level' 'spam_dsn_cutoff_level' 'spam_quarantine_cutoff_level' 'addr_extension_virus' 'addr_extension_spam' 'addr_extension_banned' 'addr_extension_bad_header' 'warnvirusrecip' 'warnbannedrecip' 'warnbadhrecip' 'newvirus_admin' 'virus_admin' 'banned_admin' 'bad_header_admin' 'spam_admin' 'spam_subject_tag' 'spam_subject_tag2' 'message_size_limit' 'banned_rulenames' 'policyd_quota_in' 'policyd_quota_in_period' 'policyd_quota_out' 'policyd_quota_out_period' 'policyd_greylist' 'id' ] ) [1 items] at (eval 112) line 308

(side note: what result would you expect if the database has NULL values?)

I believe (but not confirmed) this is the database schema used:
https://git.ispconfig.org/ispconfig/ispconfig3/blob/0f9fa27a6c7026f77c7d857c971b1db1b709f583/install/sql/ispconfig3.sql

I have requested the submitter provide the following information:

desc spamfilter_users;
desc spamfilter_policy;
SELECT *,spamfilter_users.id FROM spamfilter_users LEFT JOIN spamfilter_policy ON spamfilter_users.policy_id=spamfilter_policy.id WHERE spamfilter_users.email IN ('RECEIVER@DOMAIN2.COM','@domain2.com','@.domain2.com','@.com','@.') ORDER BY spamfilter_users.priority DESC;
@pali

This comment has been minimized.

Show comment
Hide comment
@pali

pali Feb 13, 2017

Contributor

Ok, so DBI trace show that DBI itself got 0 for spam_tag_level column. Strange.

Please provide that SELECT and schemas from mysql console client to verify that there is not problem on server.

I would like to see output from that SELECT by simple perl script (e.g. reuse my above example). And also please provide affected perl version (perl -V).

Also I would like to see what is internally stored in perl scalar represening that spam_tag_level. You can do that by calling function Dump from Devel::Peek:

use Devel::Peek;
Dump($ref->[3]);

(in check which column in $ref is spam_tag_level). Dump outputs scalar structure to stderr.

Also if you can it would be good to provide also TCP dump of mysql protocol communication. That could verify what is really sent from MySQL server. It should be possible e.g. via wireshark or tcpdump and setting mysql to connect via TCP on 127.0.0.1 port 3306 (beware that specifying "localhost" means to connect via UNIX filesystem socket! so IP address needs to be used).

Contributor

pali commented Feb 13, 2017

Ok, so DBI trace show that DBI itself got 0 for spam_tag_level column. Strange.

Please provide that SELECT and schemas from mysql console client to verify that there is not problem on server.

I would like to see output from that SELECT by simple perl script (e.g. reuse my above example). And also please provide affected perl version (perl -V).

Also I would like to see what is internally stored in perl scalar represening that spam_tag_level. You can do that by calling function Dump from Devel::Peek:

use Devel::Peek;
Dump($ref->[3]);

(in check which column in $ref is spam_tag_level). Dump outputs scalar structure to stderr.

Also if you can it would be good to provide also TCP dump of mysql protocol communication. That could verify what is really sent from MySQL server. It should be possible e.g. via wireshark or tcpdump and setting mysql to connect via TCP on 127.0.0.1 port 3306 (beware that specifying "localhost" means to connect via UNIX filesystem socket! so IP address needs to be used).

@brianmay

This comment has been minimized.

Show comment
Hide comment
@brianmay

brianmay Feb 14, 2017

Some additional information you asked for (but not all of it):

  desc spamfilter_users

    Field  Type  Null  Key  Default  Extra                          
    id             int(11) unsigned NO  PRI NULL auto_increment     
    sys_userid     int(11) unsigned NO      0                       
    sys_groupid    int(11) unsigned NO      0                       
    sys_perm_user  varchar(5)       NO                              
    sys_perm_group varchar(5)       NO                              
    sys_perm_other varchar(5)       NO                              
    server_id      int(11) unsigned NO      0                       
    priority       tinyint(3)       NO      7                       
                   unsigned                                         
    policy_id      int(11) unsigned NO      1                       
    email          varchar(255)     NO  UNI                         
    fullname       varchar(64)      YES     NULL                    
    local          varchar(1)       YES     NULL                    

  #############################################################################################

  desc spamfilter_policy

    Field  Type  Null  Key  Default  Extra                              
    id               int(11) unsigned NO  PRI NULL auto_increment       
    sys_userid       int(11) unsigned NO      0                         
    sys_groupid      int(11) unsigned NO      0                         
    sys_perm_user    varchar(5)       NO                                
    sys_perm_group   varchar(5)       NO                                
    sys_perm_other   varchar(5)       NO                                
    policy_name      varchar(64)      YES     NULL                      
    virus_lover      enum('N','Y')    YES     NULL                      
    spam_lover       enum('N','Y')    YES     NULL                      
    banned_files_lov enum('N','Y')    YES     NULL                      
    er                                                                  
    bad_header_lover enum('N','Y')    YES     NULL                      
    bypass_virus_che enum('N','Y')    YES     NULL                      
    cks                                                                 
    bypass_spam_chec enum('N','Y')    YES     NULL                      
    ks                                                                  
    bypass_banned_ch enum('N','Y')    YES     NULL                      
    ecks                                                                
    bypass_header_ch enum('N','Y')    YES     NULL                      
    ecks                                                                
    spam_modifies_su enum('N','Y')    YES     NULL                      
    bj                                                                  
    virus_quarantine varchar(255)     YES     NULL                      
    _to                                                                 
    spam_quarantine_ varchar(255)     YES     NULL                      
    to                                                                  
    banned_quarantin varchar(255)     YES     NULL                      
    e_to                                                                
    bad_header_quara varchar(255)     YES     NULL                      
    ntine_to                                                            
    clean_quarantine varchar(255)     YES     NULL                      
    _to                                                                 
    other_quarantine varchar(255)     YES     NULL                      
    _to                                                                 
    spam_tag_level   float            YES     NULL                      
    spam_tag2_level  float            YES     NULL                      
    spam_kill_level  float            YES     NULL                      
    spam_dsn_cutoff_ float            YES     NULL                      
    level                                                               
    spam_quarantine_ float            YES     NULL                      
    cutoff_level                                                        
    addr_extension_v varchar(64)      YES     NULL                      
    irus                                                                
    addr_extension_s varchar(64)      YES     NULL                      
    pam                                                                 
    addr_extension_b varchar(64)      YES     NULL                      
    anned                                                               
    addr_extension_b varchar(64)      YES     NULL                      
    ad_header                                                           
    warnvirusrecip   enum('N','Y')    YES     NULL                      
    warnbannedrecip  enum('N','Y')    YES     NULL                      
    warnbadhrecip    enum('N','Y')    YES     NULL                      
    newvirus_admin   varchar(64)      YES     NULL                      
    virus_admin      varchar(64)      YES     NULL                      
    banned_admin     varchar(64)      YES     NULL                      
    bad_header_admin varchar(64)      YES     NULL                      
    spam_admin       varchar(64)      YES     NULL                      
    spam_subject_tag varchar(64)      YES     NULL                      
    spam_subject_tag varchar(64)      YES     NULL                      
    2                                                                   
    message_size_lim int(11) unsigned YES     NULL                      
    it                                                                  
    banned_rulenames varchar(64)      YES     NULL                      
    policyd_quota_in int(11)          NO      -1                        
    policyd_quota_in int(11)          NO      24                        
    _period                                                             
    policyd_quota_ou int(11)          NO      -1                        
    t                                                                   
    policyd_quota_ou int(11)          NO      24                        
    t_period                                                            
    policyd_greylist enum('Y','N')    NO      N                         

  #############################################################################################

  SELECT *,spamfilter_users.id FROM spamfilter_users LEFT JOIN
  spamfilter_policy ON
  spamfilter_users.policy_id=spamfilter_policy.id WHERE
  spamfilter_users.email IN
  ('RECEIVER@DOMAIN2.COM','@domain2.com','@.domain2.com','@.com','@.')
  ORDER BY spamfilter_users.priority DESC;

    id  sys_userid  sys_groupi sys_perm_u sys_perm_g sys_perm_o server_id  priority  policy_id  email  fullname  loca l  id  sys_userid  sys_groupi sys_perm_u sys_perm_g sys_perm_o policy_nam virus_love spam_lover  banned_fil bad_header bypass_vir bypass_spa bypass_bann bypass_head spam_modif virus_quar spam_quara banned_quar bad_header clean_quar other_quar spam_tag_l spam_tag2_ spam_kill_ spam_dsn_c spam_quara addr_extens addr_exten addr_exten addr_exten warnvirusr warnbanned warnbadhre newvirus_a virus_admi banned_adm bad_header spam_admin  spam_subje spam_subje message_si banned_rul policyd_qu policyd_qu policyd_qu policyd_qu policyd_gr id    
                    d          ser        roup       ther                                                                                d          ser        roup       ther       e          r                      es_lover   _lover     us_checks  m_checks   ed_checks   er_checks   ies_subj   antine_to  ntine_to   antine_to   _quarantin antine_to  antine_to  evel       level      level      utoff_leve ntine_cuto ion_virus   sion_spam  sion_banne sion_bad_h ecip       recip      cip        dmin       n          in         _admin                 ct_tag     ct_tag2    ze_limit   enames     ota_in     ota_in_per ota_out    ota_out_pe eylist           
                                                                                                                                                                                                                                                                                                                                        e_to                                                              l          ff_level                          d          eader                                                                                                                                                      iod                   riod                        
    4 1 1 riud riud  1 5 1 receiver@domain2.com receiver@domain2.com Y 1 1 0 riud riud r Normalni N N N N N N N N N       -50 4.5 15 4.5 15     N N N        0  -1 24 -1 24 N 4                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                

  #############################################################################################

brianmay commented Feb 14, 2017

Some additional information you asked for (but not all of it):

  desc spamfilter_users

    Field  Type  Null  Key  Default  Extra                          
    id             int(11) unsigned NO  PRI NULL auto_increment     
    sys_userid     int(11) unsigned NO      0                       
    sys_groupid    int(11) unsigned NO      0                       
    sys_perm_user  varchar(5)       NO                              
    sys_perm_group varchar(5)       NO                              
    sys_perm_other varchar(5)       NO                              
    server_id      int(11) unsigned NO      0                       
    priority       tinyint(3)       NO      7                       
                   unsigned                                         
    policy_id      int(11) unsigned NO      1                       
    email          varchar(255)     NO  UNI                         
    fullname       varchar(64)      YES     NULL                    
    local          varchar(1)       YES     NULL                    

  #############################################################################################

  desc spamfilter_policy

    Field  Type  Null  Key  Default  Extra                              
    id               int(11) unsigned NO  PRI NULL auto_increment       
    sys_userid       int(11) unsigned NO      0                         
    sys_groupid      int(11) unsigned NO      0                         
    sys_perm_user    varchar(5)       NO                                
    sys_perm_group   varchar(5)       NO                                
    sys_perm_other   varchar(5)       NO                                
    policy_name      varchar(64)      YES     NULL                      
    virus_lover      enum('N','Y')    YES     NULL                      
    spam_lover       enum('N','Y')    YES     NULL                      
    banned_files_lov enum('N','Y')    YES     NULL                      
    er                                                                  
    bad_header_lover enum('N','Y')    YES     NULL                      
    bypass_virus_che enum('N','Y')    YES     NULL                      
    cks                                                                 
    bypass_spam_chec enum('N','Y')    YES     NULL                      
    ks                                                                  
    bypass_banned_ch enum('N','Y')    YES     NULL                      
    ecks                                                                
    bypass_header_ch enum('N','Y')    YES     NULL                      
    ecks                                                                
    spam_modifies_su enum('N','Y')    YES     NULL                      
    bj                                                                  
    virus_quarantine varchar(255)     YES     NULL                      
    _to                                                                 
    spam_quarantine_ varchar(255)     YES     NULL                      
    to                                                                  
    banned_quarantin varchar(255)     YES     NULL                      
    e_to                                                                
    bad_header_quara varchar(255)     YES     NULL                      
    ntine_to                                                            
    clean_quarantine varchar(255)     YES     NULL                      
    _to                                                                 
    other_quarantine varchar(255)     YES     NULL                      
    _to                                                                 
    spam_tag_level   float            YES     NULL                      
    spam_tag2_level  float            YES     NULL                      
    spam_kill_level  float            YES     NULL                      
    spam_dsn_cutoff_ float            YES     NULL                      
    level                                                               
    spam_quarantine_ float            YES     NULL                      
    cutoff_level                                                        
    addr_extension_v varchar(64)      YES     NULL                      
    irus                                                                
    addr_extension_s varchar(64)      YES     NULL                      
    pam                                                                 
    addr_extension_b varchar(64)      YES     NULL                      
    anned                                                               
    addr_extension_b varchar(64)      YES     NULL                      
    ad_header                                                           
    warnvirusrecip   enum('N','Y')    YES     NULL                      
    warnbannedrecip  enum('N','Y')    YES     NULL                      
    warnbadhrecip    enum('N','Y')    YES     NULL                      
    newvirus_admin   varchar(64)      YES     NULL                      
    virus_admin      varchar(64)      YES     NULL                      
    banned_admin     varchar(64)      YES     NULL                      
    bad_header_admin varchar(64)      YES     NULL                      
    spam_admin       varchar(64)      YES     NULL                      
    spam_subject_tag varchar(64)      YES     NULL                      
    spam_subject_tag varchar(64)      YES     NULL                      
    2                                                                   
    message_size_lim int(11) unsigned YES     NULL                      
    it                                                                  
    banned_rulenames varchar(64)      YES     NULL                      
    policyd_quota_in int(11)          NO      -1                        
    policyd_quota_in int(11)          NO      24                        
    _period                                                             
    policyd_quota_ou int(11)          NO      -1                        
    t                                                                   
    policyd_quota_ou int(11)          NO      24                        
    t_period                                                            
    policyd_greylist enum('Y','N')    NO      N                         

  #############################################################################################

  SELECT *,spamfilter_users.id FROM spamfilter_users LEFT JOIN
  spamfilter_policy ON
  spamfilter_users.policy_id=spamfilter_policy.id WHERE
  spamfilter_users.email IN
  ('RECEIVER@DOMAIN2.COM','@domain2.com','@.domain2.com','@.com','@.')
  ORDER BY spamfilter_users.priority DESC;

    id  sys_userid  sys_groupi sys_perm_u sys_perm_g sys_perm_o server_id  priority  policy_id  email  fullname  loca l  id  sys_userid  sys_groupi sys_perm_u sys_perm_g sys_perm_o policy_nam virus_love spam_lover  banned_fil bad_header bypass_vir bypass_spa bypass_bann bypass_head spam_modif virus_quar spam_quara banned_quar bad_header clean_quar other_quar spam_tag_l spam_tag2_ spam_kill_ spam_dsn_c spam_quara addr_extens addr_exten addr_exten addr_exten warnvirusr warnbanned warnbadhre newvirus_a virus_admi banned_adm bad_header spam_admin  spam_subje spam_subje message_si banned_rul policyd_qu policyd_qu policyd_qu policyd_qu policyd_gr id    
                    d          ser        roup       ther                                                                                d          ser        roup       ther       e          r                      es_lover   _lover     us_checks  m_checks   ed_checks   er_checks   ies_subj   antine_to  ntine_to   antine_to   _quarantin antine_to  antine_to  evel       level      level      utoff_leve ntine_cuto ion_virus   sion_spam  sion_banne sion_bad_h ecip       recip      cip        dmin       n          in         _admin                 ct_tag     ct_tag2    ze_limit   enames     ota_in     ota_in_per ota_out    ota_out_pe eylist           
                                                                                                                                                                                                                                                                                                                                        e_to                                                              l          ff_level                          d          eader                                                                                                                                                      iod                   riod                        
    4 1 1 riud riud  1 5 1 receiver@domain2.com receiver@domain2.com Y 1 1 0 riud riud r Normalni N N N N N N N N N       -50 4.5 15 4.5 15     N N N        0  -1 24 -1 24 N 4                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                

  #############################################################################################
@bigio

This comment has been minimized.

Show comment
Hide comment
@bigio

bigio Feb 14, 2017

bigio commented Feb 14, 2017

@pali

This comment has been minimized.

Show comment
Hide comment
@pali

pali Feb 14, 2017

Contributor

@brianmay so in your output is spam_tag_level=0, spam_tag2_level=-1, spam_kill_level=24 and spam_dsn_cutoff_level=-1. This is from mysql console client, right?

@bigio your output is from perl DBI code and is correct, right?

Contributor

pali commented Feb 14, 2017

@brianmay so in your output is spam_tag_level=0, spam_tag2_level=-1, spam_kill_level=24 and spam_dsn_cutoff_level=-1. This is from mysql console client, right?

@bigio your output is from perl DBI code and is correct, right?

@bigio

This comment has been minimized.

Show comment
Hide comment
@bigio

bigio Feb 14, 2017

This is from your perl code, some more info attached.
perl.txt
sql-output.txt

bigio commented Feb 14, 2017

This is from your perl code, some more info attached.
perl.txt
sql-output.txt

@pali

This comment has been minimized.

Show comment
Hide comment
@pali

pali Feb 14, 2017

Contributor

Ah... mysql (resp. mariadb) sees data correctly. Simple SELECT over perl DBI from amavisd-new code sees data correctly. And amavisd-new itself with DBI gets incorrect data, plus DBI sees incorrect data.

Has some else got similar problem with other application as amavisd-new? Otherwise conclusion is that just amavisd-new with DBD::mysql does not work and it is only one reproducer...

The last thing which can show some light into this problem is looking at Dump information from Devel::Peek.

Contributor

pali commented Feb 14, 2017

Ah... mysql (resp. mariadb) sees data correctly. Simple SELECT over perl DBI from amavisd-new code sees data correctly. And amavisd-new itself with DBI gets incorrect data, plus DBI sees incorrect data.

Has some else got similar problem with other application as amavisd-new? Otherwise conclusion is that just amavisd-new with DBD::mysql does not work and it is only one reproducer...

The last thing which can show some light into this problem is looking at Dump information from Devel::Peek.

@bigio

This comment has been minimized.

Show comment
Hide comment
@bigio

bigio Feb 15, 2017

Some more tests:
I tried with an OpenBSD vm and a remote MySQL (5.1.73) with same results (I think this is not MariaDB related).
Attached Devel::Peek infos.
amavisd.diff.txt
amavisd.log.txt

bigio commented Feb 15, 2017

Some more tests:
I tried with an OpenBSD vm and a remote MySQL (5.1.73) with same results (I think this is not MariaDB related).
Attached Devel::Peek infos.
amavisd.diff.txt
amavisd.log.txt

@pali

This comment has been minimized.

Show comment
Hide comment
@pali

pali Feb 15, 2017

Contributor

Please call Dump as early as possible, ideally before other functions (like that join) which read that scalar (as they can alter it). And you do not have to check if scalar is defined. Dump correctly handle also undefs.

From your Dump we can just see that variable is tainted. Are you running perl in taint (-T) mode? (Note that it should have no effect for DBI...)

Contributor

pali commented Feb 15, 2017

Please call Dump as early as possible, ideally before other functions (like that join) which read that scalar (as they can alter it). And you do not have to check if scalar is defined. Dump correctly handle also undefs.

From your Dump we can just see that variable is tainted. Are you running perl in taint (-T) mode? (Note that it should have no effect for DBI...)

@bigio

This comment has been minimized.

Show comment
Hide comment
@bigio

bigio Feb 15, 2017

Now I called Dump just after "$match = {}; @$match{@NAMEs} = @$a_ref;".
amavisd-new runs all on tainted mode, I tried to run your test-script in tainted mode but it still works.
amavisd.log.txt

bigio commented Feb 15, 2017

Now I called Dump just after "$match = {}; @$match{@NAMEs} = @$a_ref;".
amavisd-new runs all on tainted mode, I tried to run your test-script in tainted mode but it still works.
amavisd.log.txt

@bigio

This comment has been minimized.

Show comment
Hide comment
@bigio

bigio Feb 15, 2017

remove "perl -T" from amavisd-new first line, amavisd gives a lot of errors because it is not running in tainted mode but the query give correct results.
amavisd.log.txt

bigio commented Feb 15, 2017

remove "perl -T" from amavisd-new first line, amavisd gives a lot of errors because it is not running in tainted mode but the query give correct results.
amavisd.log.txt

@pali

This comment has been minimized.

Show comment
Hide comment
@pali

pali Feb 15, 2017

Contributor

Thank you for info! Now we know that problem is related to taint mode. I'm still not able to reproduce this problem, but from your Dump it looks like perl refused to assign value from mysql to perl scalar.

Can you try to apply this patch to DBD-mysql and recompile it?

diff --git a/dbdimp.c b/dbdimp.c
index 91cc1a8..ad4f1b7 100644
--- a/dbdimp.c
+++ b/dbdimp.c
@@ -4567,6 +4567,8 @@ process:
       char *col= cols[i];
       SV *sv= AvARRAY(av)[i]; /* Note: we (re)use the SV in the AV	*/
 
+      fprintf(stderr, "i=%u len=%lu flags=%u col='%s'\n", i, lengths[i], fields[i].flags, cols[i]);
+
       if (col)
       {
         STRLEN len= lengths[i];
@@ -4584,8 +4586,11 @@ process:
           if (!(fields[i].flags & ZEROFILL_FLAG))
           {
             /* Coerce to dobule and set scalar as NV */
+            sv_dump(sv);
             (void) SvNV(sv);
+            sv_dump(sv);
             SvNOK_only(sv);
+            sv_dump(sv);
           }
           break;
 

It should print additional info to stderr for all fetch commands. sv_dump is same as Dump.

Contributor

pali commented Feb 15, 2017

Thank you for info! Now we know that problem is related to taint mode. I'm still not able to reproduce this problem, but from your Dump it looks like perl refused to assign value from mysql to perl scalar.

Can you try to apply this patch to DBD-mysql and recompile it?

diff --git a/dbdimp.c b/dbdimp.c
index 91cc1a8..ad4f1b7 100644
--- a/dbdimp.c
+++ b/dbdimp.c
@@ -4567,6 +4567,8 @@ process:
       char *col= cols[i];
       SV *sv= AvARRAY(av)[i]; /* Note: we (re)use the SV in the AV	*/
 
+      fprintf(stderr, "i=%u len=%lu flags=%u col='%s'\n", i, lengths[i], fields[i].flags, cols[i]);
+
       if (col)
       {
         STRLEN len= lengths[i];
@@ -4584,8 +4586,11 @@ process:
           if (!(fields[i].flags & ZEROFILL_FLAG))
           {
             /* Coerce to dobule and set scalar as NV */
+            sv_dump(sv);
             (void) SvNV(sv);
+            sv_dump(sv);
             SvNOK_only(sv);
+            sv_dump(sv);
           }
           break;
 

It should print additional info to stderr for all fetch commands. sv_dump is same as Dump.

@bigio

This comment has been minimized.

Show comment
Hide comment
@bigio

bigio commented Feb 15, 2017

Data attached.
amavisd.log.txt

@pali

This comment has been minimized.

Show comment
Hide comment
@pali

pali Feb 15, 2017

Contributor

Thank you! From your output we can see that NV value (float) is not filled in second dump when it should be. So problem is in SvNV() call. Going to look into perl source code when and why should it happen. Which perl version are you using? And can you recheck that NV value in second dump is filled correctly?

Contributor

pali commented Feb 15, 2017

Thank you! From your output we can see that NV value (float) is not filled in second dump when it should be. So problem is in SvNV() call. Going to look into perl source code when and why should it happen. Which perl version are you using? And can you recheck that NV value in second dump is filled correctly?

@bigio

This comment has been minimized.

Show comment
Hide comment
@bigio

bigio Feb 15, 2017

Will double check tomorrow morning CEST,
latest test has been done with Perl 5.24.
perl.log.txt

bigio commented Feb 15, 2017

Will double check tomorrow morning CEST,
latest test has been done with Perl 5.24.
perl.log.txt

@pali

This comment has been minimized.

Show comment
Hide comment
@pali

pali Feb 15, 2017

Contributor

I suspect there is bug (or maybe it is feature?) in perl itself. Function sv_2nv_flags() in perl (called by SvNV()) does not upgrade scalar to NV (float) in specific conditions (and float value is lost). This is just observation from reading perl source code. First I need to be able to reproduce this bug and then I can say if problem is in amavis, DBD::mysql, DBI or perl.

Contributor

pali commented Feb 15, 2017

I suspect there is bug (or maybe it is feature?) in perl itself. Function sv_2nv_flags() in perl (called by SvNV()) does not upgrade scalar to NV (float) in specific conditions (and float value is lost). This is just observation from reading perl source code. First I need to be able to reproduce this bug and then I can say if problem is in amavis, DBD::mysql, DBI or perl.

@bigio

This comment has been minimized.

Show comment
Hide comment
@bigio

bigio Feb 16, 2017

Some more infos (maybe) with a new diff.
debug.log.txt
patch-dbdimp_c.diff.txt

bigio commented Feb 16, 2017

Some more infos (maybe) with a new diff.
debug.log.txt
patch-dbdimp_c.diff.txt

@pali

This comment has been minimized.

Show comment
Hide comment
@pali

pali Feb 16, 2017

Contributor

Yes, I expected that those dumps are from problematic float columns. As I wrote yesterday I will try to reproduce that scenario when SvNV() does not upgrade scalar to NV if it is really truth. And then decide next steps...

Contributor

pali commented Feb 16, 2017

Yes, I expected that those dumps are from problematic float columns. As I wrote yesterday I will try to reproduce that scenario when SvNV() does not upgrade scalar to NV if it is really truth. And then decide next steps...

@pali

This comment has been minimized.

Show comment
Hide comment
@pali

pali Feb 17, 2017

Contributor

I opened tiket in perl bug tracker for this problem: https://rt.perl.org/Public/Bug/Display.html?id=130801

Contributor

pali commented Feb 17, 2017

I opened tiket in perl bug tracker for this problem: https://rt.perl.org/Public/Bug/Display.html?id=130801

@brianmay

This comment has been minimized.

Show comment
Hide comment
@brianmay

brianmay commented Feb 17, 2017

Thanks!

@pali

This comment has been minimized.

Show comment
Hide comment
@pali

pali Feb 23, 2017

Contributor

So it is probably problem with understanding perlapi documentation and different behavior for floating point magic scalar seems to be OK... It is just (for me) strange that non-magic integer, magic integer and non-magic float is working, just magic float is problematic...

Can you try to apply this patch? It explicitly set float (NV) value via sv_setnv(). And in same case it set also integer values.

diff --git a/dbdimp.c b/dbdimp.c
index 91cc1a8..d484d80 100644
--- a/dbdimp.c
+++ b/dbdimp.c
@@ -4584,8 +4584,7 @@ process:
           if (!(fields[i].flags & ZEROFILL_FLAG))
           {
             /* Coerce to dobule and set scalar as NV */
-            (void) SvNV(sv);
-            SvNOK_only(sv);
+            sv_setnv(sv, SvNV(sv));
           }
           break;
 
@@ -4594,15 +4593,9 @@ process:
           {
             /* Coerce to integer and set scalar as UV resp. IV */
             if (fields[i].flags & UNSIGNED_FLAG)
-            {
-              (void) SvUV(sv);
-              SvIOK_only_UV(sv);
-            }
+              sv_setuv(sv, SvUV(sv));
             else
-            {
-              (void) SvIV(sv);
-              SvIOK_only(sv);
-            }
+              sv_setiv(sv, SvIV(sv));
           }
           break;
 
Contributor

pali commented Feb 23, 2017

So it is probably problem with understanding perlapi documentation and different behavior for floating point magic scalar seems to be OK... It is just (for me) strange that non-magic integer, magic integer and non-magic float is working, just magic float is problematic...

Can you try to apply this patch? It explicitly set float (NV) value via sv_setnv(). And in same case it set also integer values.

diff --git a/dbdimp.c b/dbdimp.c
index 91cc1a8..d484d80 100644
--- a/dbdimp.c
+++ b/dbdimp.c
@@ -4584,8 +4584,7 @@ process:
           if (!(fields[i].flags & ZEROFILL_FLAG))
           {
             /* Coerce to dobule and set scalar as NV */
-            (void) SvNV(sv);
-            SvNOK_only(sv);
+            sv_setnv(sv, SvNV(sv));
           }
           break;
 
@@ -4594,15 +4593,9 @@ process:
           {
             /* Coerce to integer and set scalar as UV resp. IV */
             if (fields[i].flags & UNSIGNED_FLAG)
-            {
-              (void) SvUV(sv);
-              SvIOK_only_UV(sv);
-            }
+              sv_setuv(sv, SvUV(sv));
             else
-            {
-              (void) SvIV(sv);
-              SvIOK_only(sv);
-            }
+              sv_setiv(sv, SvIV(sv));
           }
           break;
 
@bigio

This comment has been minimized.

Show comment
Hide comment
@bigio

bigio Feb 24, 2017

bigio commented Feb 24, 2017

pali added a commit to pali/DBD-mysql that referenced this issue Feb 24, 2017

Fix type conversions
Calling SvNV() for magical scalar is not enough for float type conversion.
It caused problem for Amavis in tainted mode -- all float values were zero.
On the other hand SvIV() and SvUV() seems to work fine. To be sure that
correct value of float is in scalar use sv_setnv() with explicit NV float
value. Similar code is changed also for integers IV/UV.

This patch should fix reported Amavis bug:
perl5-dbi#78

See also reported perl bug about SvNV():
https://rt.perl.org/Public/Bug/Display.html?id=130801

pali added a commit to pali/DBD-mysql that referenced this issue Feb 24, 2017

Fix type conversions
Calling SvNV() for magical scalar is not enough for float type conversion.
It caused problem for Amavis in tainted mode -- all float values were zero.
On the other hand SvIV() and SvUV() seems to work fine. To be sure that
correct value of float is in scalar use sv_setnv() with explicit NV float
value. Similar code is changed also for integers IV/UV.

This patch should fix reported Amavis bug:
perl5-dbi#78

See also reported perl bug about SvNV():
https://rt.perl.org/Public/Bug/Display.html?id=130801
@brianmay

This comment has been minimized.

Show comment
Hide comment
@brianmay

brianmay Feb 24, 2017

Hello.

@pali Thanks for you help with this.

Can I please confirm: Does this only cause problems when Perl is in tainted mode?

Thanks

brianmay commented Feb 24, 2017

Hello.

@pali Thanks for you help with this.

Can I please confirm: Does this only cause problems when Perl is in tainted mode?

Thanks

@pali

This comment has been minimized.

Show comment
Hide comment
@pali

pali Feb 24, 2017

Contributor

@brianmay Problem with floats seems to be only if DBD::mysql is filling values into magical scalars. Tainted scalars are magical. Scalars are created by DBI (not DBD::mysql) so this probably depends on DBI version...

I was not fully able to reproduce this problem even in tainted Perl mode. I just found code path in perl sources which could trigger it. So... in case DBI does not pass magical scalars to DBD::mysql then this problem could not happen. Note that tainted scalars are not the only type of magical scalars. I have no idea if DBI could not pass another magical scalars... So I cannot answer to your question.

What I can say is that @bigio already tested my change and confirmed that it fixes this problem. Function sv_setnv() really should store float value into variable. I re-checked not only documentation, but also implementation perl.

This is probably the most suspicious problem which I ever seen and I was not able to reproduce it.

Now I do not thing that SvNV() behavior would change to guarantee filling NV slot. Maybe just documentation would be extended or fixed for better understanding. For sure current and older perl versions would not be changed, so fix in DBD::mysql is needed.

Contributor

pali commented Feb 24, 2017

@brianmay Problem with floats seems to be only if DBD::mysql is filling values into magical scalars. Tainted scalars are magical. Scalars are created by DBI (not DBD::mysql) so this probably depends on DBI version...

I was not fully able to reproduce this problem even in tainted Perl mode. I just found code path in perl sources which could trigger it. So... in case DBI does not pass magical scalars to DBD::mysql then this problem could not happen. Note that tainted scalars are not the only type of magical scalars. I have no idea if DBI could not pass another magical scalars... So I cannot answer to your question.

What I can say is that @bigio already tested my change and confirmed that it fixes this problem. Function sv_setnv() really should store float value into variable. I re-checked not only documentation, but also implementation perl.

This is probably the most suspicious problem which I ever seen and I was not able to reproduce it.

Now I do not thing that SvNV() behavior would change to guarantee filling NV slot. Maybe just documentation would be extended or fixed for better understanding. For sure current and older perl versions would not be changed, so fix in DBD::mysql is needed.

bob-beck pushed a commit to openbsd/ports that referenced this issue Feb 27, 2017

giovanni
Temporarily switch to a Github snapshot to unbreak sql_lookups in mai…
…l/amavisd-new and other software that uses float fields and perl in tainted mode.

More info on the issue fixed: perl5-dbi/DBD-mysql#78
ok sthen@
@mbeijen

This comment has been minimized.

Show comment
Hide comment
@mbeijen

mbeijen Mar 20, 2017

Contributor

@bigio You can update OpenBSD to 4.042 now and the AmavisD problem should be fixed.

Contributor

mbeijen commented Mar 20, 2017

@bigio You can update OpenBSD to 4.042 now and the AmavisD problem should be fixed.

@mbeijen mbeijen closed this Mar 20, 2017

@pali

This comment has been minimized.

Show comment
Hide comment
@pali

pali Jul 1, 2017

Contributor

This fix was reverted in 4.043.

Contributor

pali commented Jul 1, 2017

This fix was reverted in 4.043.

netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this issue Jul 10, 2017

wen
Update to 4.043
Upstream changes:
2017-06-29 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.043)

YOUR ATTENTION PLEASE, THIS IS A REVERT TO 4.041
This version is the same as 4.041 with all its bugs and
limitations. In version 4.042 there were some changes to Unicode handling
that turned out to be causing issues with existing implementations.
While it is possible to argue that the old behaviour was wrong and buggy,
lots of applications and scripts were depending on this behaviour so it
is NOT a good idea to change this.

There were lots of commits since 4.041, we'll add those back bit by bit
in a future release, excluding the ones which cause problems.


2017-??-?? Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.042_01)
* Use Devel::CheckLib 1.09 or newer, fixes
   perl5-dbi/DBD-mysql#109
* Improve CI testing on AppVeyor: caching, path to cpan, configure deps (pali)
* Specify bigint as test dependency.

2017-03-08 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.042)
* Full release to include development releases 4.041_2 and 4.041_1.

2017-02-28 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.041_2)
* Statement handle now also has mysql_sock attribute, just as database handle.
  (by Pali)
* Fix type conversions for magic types. Issue reported by Dmitriy Yatsenko and
  Giovanni Bechis, fix by Pali.
    https://lists.amavis.org/pipermail/amavis-users/2016-December/004674.html
    perl5-dbi/DBD-mysql#78
* Fix UTF8-encoding of table names, field names, warnings and error messages.
  Reported by Tanabe Yoshinori, fix by Pali.
    https://rt.cpan.org/Public/Bug/Display.html?id=120141
* Fix mysql_auto_reconnect when using mysql_server_prepare (pali). Reported by
  Vladimir Marek.
    perl5-dbi/DBD-mysql#95
* Improve regex for removing database from dsn (pali)
    https://rt.cpan.org/Public/Bug/Display.html?id=118837
* Locate MySQL libs using Devel::CheckLib (pali)
* Support async on Windows (pali)

* Fix test suite on range of older and newer MySQL and MariaDB versions
   (perl5-dbi/DBD-mysql#87)
* Fix compilation on MySQL 4.1 (pali)
* Do not leak dangling pointer to mysql result (pali)
* Fix logic when assigning to variable bind_comment_placeholders (pali)
* mysql_fd() still returned file descriptor after closing connection.
  Reported by Larry Leszczynski, fixed by Pali Roh獺r.
   (https://rt.cpan.org/Public/Bug/Display.html?id=110983)
* Fix parsing configure libs from mysql_config --libs output in Makefile.PL
  Libraries in mysql_config --libs output can be specified by library name
  with the -l prefix or by absolute path to library name without any prefix.
  Parameters must start with a hyphen, so treat all options without leading
  hyphen in mysql_config --libs output as libraries with full path.
  Partially fixes bug https://rt.cpan.org/Public/Bug/Display.html?id=100898
  Fix by Pali Roh獺r.
* Fix support for magic scalars (pali)
   (perl5-dbi/DBD-mysql#76)

2016-12-12 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.041_1)
* Unicode fixes: when using mysql_enable_utf8 or mysql_enable_utf8mb4,
  previous versions of DBD::mysql did not properly encode input statements
  to UTF-8 and retrieved columns were always UTF-8 decoded regardless of the
  column charset.
  Fix by Pali Roh獺r.
  Reported and feedback on fix by Marc Lehmann
  (https://rt.cpan.org/Public/Bug/Display.html?id=87428)
  Also, the UTF-8 flag was not set for decoded data:
  (https://rt.cpan.org/Public/Bug/Display.html?id=53130)
* Return INTs with ZEROFILL as strings. Reported by Knarf, fix by Pali Roh獺r.
  (https://rt.cpan.org/Public/Bug/Display.html?id=118977)
@Fneufneu

This comment has been minimized.

Show comment
Hide comment
@Fneufneu

Fneufneu Sep 11, 2017

can we re open this issue plz ? 2 month and the last version of DBD-mysql is still broken

Fneufneu commented Sep 11, 2017

can we re open this issue plz ? 2 month and the last version of DBD-mysql is still broken

kentfredric added a commit to kentnl-gentoo/DBD-mysql that referenced this issue Oct 13, 2017

Fix type conversions
Calling SvNV() for magical scalar is not enough for float type conversion.
It caused problem for Amavis in tainted mode -- all float values were zero.
On the other hand SvIV() and SvUV() seems to work fine. To be sure that
correct value of float is in scalar use sv_setnv() with explicit NV float
value. Similar code is changed also for integers IV/UV.

This patch should fix reported Amavis bug:
perl5-dbi#78

See also reported perl bug about SvNV():
https://rt.perl.org/Public/Bug/Display.html?id=130801

Bugs: perl5-dbi#78
Bugs-Debian: https://bugs.debian.org/856064
@LsPjvVzszDt

This comment has been minimized.

Show comment
Hide comment
@LsPjvVzszDt

LsPjvVzszDt Jan 5, 2018

Fedora 27, for which the current shipped version is perl-DBD-MySQL-4.043-6, also still has this bug.

amavisd-new SQL lookups of float fields still return 0.

LsPjvVzszDt commented Jan 5, 2018

Fedora 27, for which the current shipped version is perl-DBD-MySQL-4.043-6, also still has this bug.

amavisd-new SQL lookups of float fields still return 0.

hakrdinesh pushed a commit to hakrtech/openbsd-ports0-test that referenced this issue Jan 16, 2018

giovanni
Temporarily switch to a Github snapshot to unbreak sql_lookups in mai…
…l/amavisd-new and other software that uses float fields and perl in tainted mode.

More info on the issue fixed: perl5-dbi/DBD-mysql#78
ok sthen@

kentfredric added a commit to kentnl-gentoo/DBD-mysql that referenced this issue Jan 24, 2018

Fix type conversions
Calling SvNV() for magical scalar is not enough for float type conversion.
It caused problem for Amavis in tainted mode -- all float values were zero.
On the other hand SvIV() and SvUV() seems to work fine. To be sure that
correct value of float is in scalar use sv_setnv() with explicit NV float
value. Similar code is changed also for integers IV/UV.

This patch should fix reported Amavis bug:
perl5-dbi#78

See also reported perl bug about SvNV():
https://rt.perl.org/Public/Bug/Display.html?id=130801

Bugs: perl5-dbi#78
Bugs-Debian: https://bugs.debian.org/856064

derekstraka pushed a commit to derekstraka/meta-openembedded that referenced this issue Jan 26, 2018

libdbd-mysql-perl: upgrade 4.036 -> 4.043
Changes:

2017-06-29 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.043)

YOUR ATTENTION PLEASE, THIS IS A REVERT TO 4.041
This version is the same as 4.041 with all its bugs and
limitations. In version 4.042 there were some changes to Unicode handling
that turned out to be causing issues with existing implementations.
While it is possible to argue that the old behaviour was wrong and buggy,
lots of applications and scripts were depending on this behaviour so it
is NOT a good idea to change this.

There were lots of commits since 4.041, we'll add those back bit by bit
in a future release, excluding the ones which cause problems.

2017-??-?? Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.042_01)
* Use Devel::CheckLib 1.09 or newer, fixes
   perl5-dbi/DBD-mysql#109
* Improve CI testing on AppVeyor: caching, path to cpan, configure deps (pali)
* Specify bigint as test dependency.

2017-03-08 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.042)
* Full release to include development releases 4.041_2 and 4.041_1.

2017-02-28 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.041_2)
* Statement handle now also has mysql_sock attribute, just as database handle.
  (by Pali)
* Fix type conversions for magic types. Issue reported by Dmitriy Yatsenko and
  Giovanni Bechis, fix by Pali.
    https://lists.amavis.org/pipermail/amavis-users/2016-December/004674.html
    perl5-dbi/DBD-mysql#78
* Fix UTF8-encoding of table names, field names, warnings and error messages.
  Reported by Tanabe Yoshinori, fix by Pali.
    https://rt.cpan.org/Public/Bug/Display.html?id=120141
* Fix mysql_auto_reconnect when using mysql_server_prepare (pali). Reported by
  Vladimir Marek.
    perl5-dbi/DBD-mysql#95
* Improve regex for removing database from dsn (pali)
    https://rt.cpan.org/Public/Bug/Display.html?id=118837
* Locate MySQL libs using Devel::CheckLib (pali)
* Support async on Windows (pali)

* Fix test suite on range of older and newer MySQL and MariaDB versions
   (perl5-dbi/DBD-mysql#87)
* Fix compilation on MySQL 4.1 (pali)
* Do not leak dangling pointer to mysql result (pali)
* Fix logic when assigning to variable bind_comment_placeholders (pali)
* mysql_fd() still returned file descriptor after closing connection.
  Reported by Larry Leszczynski, fixed by Pali Rohár.
   (https://rt.cpan.org/Public/Bug/Display.html?id=110983)
* Fix parsing configure libs from mysql_config --libs output in Makefile.PL
  Libraries in mysql_config --libs output can be specified by library name
  with the -l prefix or by absolute path to library name without any prefix.
  Parameters must start with a hyphen, so treat all options without leading
  hyphen in mysql_config --libs output as libraries with full path.
  Partially fixes bug https://rt.cpan.org/Public/Bug/Display.html?id=100898
  Fix by Pali Rohár.
* Fix support for magic scalars (pali)
   (perl5-dbi/DBD-mysql#76)

2016-12-12 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.041_1)
* Unicode fixes: when using mysql_enable_utf8 or mysql_enable_utf8mb4,
  previous versions of DBD::mysql did not properly encode input statements
  to UTF-8 and retrieved columns were always UTF-8 decoded regardless of the
  column charset.
  Fix by Pali Rohár.
  Reported and feedback on fix by Marc Lehmann
  (https://rt.cpan.org/Public/Bug/Display.html?id=87428)
  Also, the UTF-8 flag was not set for decoded data:
  (https://rt.cpan.org/Public/Bug/Display.html?id=53130)
* Return INTs with ZEROFILL as strings. Reported by Knarf, fix by Pali Rohár.
  (https://rt.cpan.org/Public/Bug/Display.html?id=118977)

2016-11-28 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.041)
* Fix use-after-free for repeated fetchrow_arrayref calls when
  mysql_server_prepare=1

  Function dbd_st_fetch() via Renew() can reallocate output buffer for
  mysql_stmt_fetch() call. But it does not update pointer to that buffer in
  imp_sth->stmt structure initialized by mysql_stmt_bind_result() function.
  That leads to use-after-free in any mysql function which access
  imp_sth->stmt structure (e.g. mysql_stmt_fetch()).

  This patch fix this problem and properly updates pointer in imp_sth->stmt
  structure after Renew() call.
  This is a medium level security issue to which the Debian security team
  assigned identifier CVE-2016-1251. Discovered and fixed by Pali Rohár.

* auto_reconnect now also matches  CR_SERVER_LOST, previously this only
  matched CR_SERVER_GONE.
  Fixes http://bugs.mysql.com/bug.php?id=27613
  Fix suggested by Wouter de Jong.
* Fix compilation fixes (Pali Rohár).

2016-11-19 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.040)
* Since 4.038 we had problems compiling on big-endian architectures, such
  as MIPS, s390 and Sparc. Thanks to Salvatore Bonaccorso @ Debian project
  (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=844538)
  and Vladimir Marek (https://rt.cpan.org/Public/Bug/Display.html?id=118835)
  for reporting the issues. Fix by Pali Rohár.

  Fix integer types when server side prepare statements are enabled
  Fixed problems:
  * SQL_BIGINT was incorrectly handled as 32bit MYSQL_TYPE_LONG type instead
    64bit MYSQL_TYPE_LONGLONG which led to integer overflow/underflow
  * 32bit MYSQL_TYPE_LONG was used for perl's IV storage when IV was 64bit
    and 64bit MYSQL_TYPE_LONGLONG was used when IV was 32bit
  * All unsigned types were handled as signed, so all high positive values
    were treated as negative
  * Numeric conversions in perl which led to overflow/underflow was ignored
    even when mysql strict mode was enabled
  * Test t/41int_min_max.t was running only for normal non-prepared statements
  * Test t/40server_prepare.t used incorrect SQL type for big (64bit) integers

2016-11-15 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.039)
* Fix for security issue Out-of-bounds read by DBD::mysql CVE-2016-1249 (pali)

2016-10-30 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.038_01)
* Fix compilation of embedded server (pali)
  (perl5-dbi/DBD-mysql#68)
* Fix compilation against libmariadbclient. First version by
  H.Merijn Brand, improved by Bernt Johnsen @ Oracle.
* For efficiency use newSVpvn() instead newSVpv() where possible (pali)
* Correctly coerce fetched scalar values when mysql_server_prepare is
  not used (pali)
* Add support for fetching columns of BIT type with
  mysql_server_prepare = 1 (pali)
  Fixes https://rt.cpan.org/Public/Bug/Display.html?id=88006
* Use correct format in printf instead of casting variable types (pali)
* Include errno.h for MYSQL_ASYNC because it uses errno variable (pali)
* Travis: also test on perl 5.22 and 5.24.

2016-10-19 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.038)
* Version 4.037_1 had fixes for MySQL 8.0 provided
  Bernt Johnsen @ Oracle that were not in the Changelogs
  (perl5-dbi/DBD-mysql#56)
* Fixes for compiling against newer libmysqlclient on Windows (kmx)
* Fix unit test for 40server_prepare_crash on Windows (pali)
* Perl's IV in scalar can store 64bit integer when perl was compiled
  with 64 bit support (default on 64bit linux with gcc). Use this
  feature and stores MYSQL_TYPE_LONGLONG as integers instead of strings
  when possible. (pali, perl5-dbi/DBD-mysql#57)

2016-10-14 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.037_01)
* Newest versions of libmysqlclient and the MariaDB C connector no longer
  export the __WIN__ macro. If this macro is not present we would not
  compile in the poll.h-based async-support. Changed to use the _WIN32
  macro instead. Thanks to Sergei Golubchik for suggesting the fix.
* Fix from Pali Rohár for use-after-free in prepared statements,
  changes to bind logic, and added test 40server_prepare_crash.

2016-10-03 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.037)
* Security release to patch possible buffer overflow in unsafe sprintf with
  variable length. Reported and fixed by Pali Rohár. This vulnerability
  is present in all releases at least back to versions 3.0 of the
  driver, which were released in 2005.
  The CVE identifier for this vulnerability is CVE-2016-1246.

Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
Signed-off-by: Armin Kuster <akuster808@gmail.com>

halstead pushed a commit to openembedded/meta-openembedded that referenced this issue Jan 27, 2018

libdbd-mysql-perl: upgrade 4.036 -> 4.043
Changes:

2017-06-29 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.043)

YOUR ATTENTION PLEASE, THIS IS A REVERT TO 4.041
This version is the same as 4.041 with all its bugs and
limitations. In version 4.042 there were some changes to Unicode handling
that turned out to be causing issues with existing implementations.
While it is possible to argue that the old behaviour was wrong and buggy,
lots of applications and scripts were depending on this behaviour so it
is NOT a good idea to change this.

There were lots of commits since 4.041, we'll add those back bit by bit
in a future release, excluding the ones which cause problems.

2017-??-?? Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.042_01)
* Use Devel::CheckLib 1.09 or newer, fixes
   perl5-dbi/DBD-mysql#109
* Improve CI testing on AppVeyor: caching, path to cpan, configure deps (pali)
* Specify bigint as test dependency.

2017-03-08 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.042)
* Full release to include development releases 4.041_2 and 4.041_1.

2017-02-28 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.041_2)
* Statement handle now also has mysql_sock attribute, just as database handle.
  (by Pali)
* Fix type conversions for magic types. Issue reported by Dmitriy Yatsenko and
  Giovanni Bechis, fix by Pali.
    https://lists.amavis.org/pipermail/amavis-users/2016-December/004674.html
    perl5-dbi/DBD-mysql#78
* Fix UTF8-encoding of table names, field names, warnings and error messages.
  Reported by Tanabe Yoshinori, fix by Pali.
    https://rt.cpan.org/Public/Bug/Display.html?id=120141
* Fix mysql_auto_reconnect when using mysql_server_prepare (pali). Reported by
  Vladimir Marek.
    perl5-dbi/DBD-mysql#95
* Improve regex for removing database from dsn (pali)
    https://rt.cpan.org/Public/Bug/Display.html?id=118837
* Locate MySQL libs using Devel::CheckLib (pali)
* Support async on Windows (pali)

* Fix test suite on range of older and newer MySQL and MariaDB versions
   (perl5-dbi/DBD-mysql#87)
* Fix compilation on MySQL 4.1 (pali)
* Do not leak dangling pointer to mysql result (pali)
* Fix logic when assigning to variable bind_comment_placeholders (pali)
* mysql_fd() still returned file descriptor after closing connection.
  Reported by Larry Leszczynski, fixed by Pali Rohár.
   (https://rt.cpan.org/Public/Bug/Display.html?id=110983)
* Fix parsing configure libs from mysql_config --libs output in Makefile.PL
  Libraries in mysql_config --libs output can be specified by library name
  with the -l prefix or by absolute path to library name without any prefix.
  Parameters must start with a hyphen, so treat all options without leading
  hyphen in mysql_config --libs output as libraries with full path.
  Partially fixes bug https://rt.cpan.org/Public/Bug/Display.html?id=100898
  Fix by Pali Rohár.
* Fix support for magic scalars (pali)
   (perl5-dbi/DBD-mysql#76)

2016-12-12 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.041_1)
* Unicode fixes: when using mysql_enable_utf8 or mysql_enable_utf8mb4,
  previous versions of DBD::mysql did not properly encode input statements
  to UTF-8 and retrieved columns were always UTF-8 decoded regardless of the
  column charset.
  Fix by Pali Rohár.
  Reported and feedback on fix by Marc Lehmann
  (https://rt.cpan.org/Public/Bug/Display.html?id=87428)
  Also, the UTF-8 flag was not set for decoded data:
  (https://rt.cpan.org/Public/Bug/Display.html?id=53130)
* Return INTs with ZEROFILL as strings. Reported by Knarf, fix by Pali Rohár.
  (https://rt.cpan.org/Public/Bug/Display.html?id=118977)

2016-11-28 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.041)
* Fix use-after-free for repeated fetchrow_arrayref calls when
  mysql_server_prepare=1

  Function dbd_st_fetch() via Renew() can reallocate output buffer for
  mysql_stmt_fetch() call. But it does not update pointer to that buffer in
  imp_sth->stmt structure initialized by mysql_stmt_bind_result() function.
  That leads to use-after-free in any mysql function which access
  imp_sth->stmt structure (e.g. mysql_stmt_fetch()).

  This patch fix this problem and properly updates pointer in imp_sth->stmt
  structure after Renew() call.
  This is a medium level security issue to which the Debian security team
  assigned identifier CVE-2016-1251. Discovered and fixed by Pali Rohár.

* auto_reconnect now also matches  CR_SERVER_LOST, previously this only
  matched CR_SERVER_GONE.
  Fixes http://bugs.mysql.com/bug.php?id=27613
  Fix suggested by Wouter de Jong.
* Fix compilation fixes (Pali Rohár).

2016-11-19 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.040)
* Since 4.038 we had problems compiling on big-endian architectures, such
  as MIPS, s390 and Sparc. Thanks to Salvatore Bonaccorso @ Debian project
  (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=844538)
  and Vladimir Marek (https://rt.cpan.org/Public/Bug/Display.html?id=118835)
  for reporting the issues. Fix by Pali Rohár.

  Fix integer types when server side prepare statements are enabled
  Fixed problems:
  * SQL_BIGINT was incorrectly handled as 32bit MYSQL_TYPE_LONG type instead
    64bit MYSQL_TYPE_LONGLONG which led to integer overflow/underflow
  * 32bit MYSQL_TYPE_LONG was used for perl's IV storage when IV was 64bit
    and 64bit MYSQL_TYPE_LONGLONG was used when IV was 32bit
  * All unsigned types were handled as signed, so all high positive values
    were treated as negative
  * Numeric conversions in perl which led to overflow/underflow was ignored
    even when mysql strict mode was enabled
  * Test t/41int_min_max.t was running only for normal non-prepared statements
  * Test t/40server_prepare.t used incorrect SQL type for big (64bit) integers

2016-11-15 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.039)
* Fix for security issue Out-of-bounds read by DBD::mysql CVE-2016-1249 (pali)

2016-10-30 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.038_01)
* Fix compilation of embedded server (pali)
  (perl5-dbi/DBD-mysql#68)
* Fix compilation against libmariadbclient. First version by
  H.Merijn Brand, improved by Bernt Johnsen @ Oracle.
* For efficiency use newSVpvn() instead newSVpv() where possible (pali)
* Correctly coerce fetched scalar values when mysql_server_prepare is
  not used (pali)
* Add support for fetching columns of BIT type with
  mysql_server_prepare = 1 (pali)
  Fixes https://rt.cpan.org/Public/Bug/Display.html?id=88006
* Use correct format in printf instead of casting variable types (pali)
* Include errno.h for MYSQL_ASYNC because it uses errno variable (pali)
* Travis: also test on perl 5.22 and 5.24.

2016-10-19 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.038)
* Version 4.037_1 had fixes for MySQL 8.0 provided
  Bernt Johnsen @ Oracle that were not in the Changelogs
  (perl5-dbi/DBD-mysql#56)
* Fixes for compiling against newer libmysqlclient on Windows (kmx)
* Fix unit test for 40server_prepare_crash on Windows (pali)
* Perl's IV in scalar can store 64bit integer when perl was compiled
  with 64 bit support (default on 64bit linux with gcc). Use this
  feature and stores MYSQL_TYPE_LONGLONG as integers instead of strings
  when possible. (pali, perl5-dbi/DBD-mysql#57)

2016-10-14 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.037_01)
* Newest versions of libmysqlclient and the MariaDB C connector no longer
  export the __WIN__ macro. If this macro is not present we would not
  compile in the poll.h-based async-support. Changed to use the _WIN32
  macro instead. Thanks to Sergei Golubchik for suggesting the fix.
* Fix from Pali Rohár for use-after-free in prepared statements,
  changes to bind logic, and added test 40server_prepare_crash.

2016-10-03 Patrick Galbraith, Michiel Beijen, DBI/DBD community (4.037)
* Security release to patch possible buffer overflow in unsafe sprintf with
  variable length. Reported and fixed by Pali Rohár. This vulnerability
  is present in all releases at least back to versions 3.0 of the
  driver, which were released in 2005.
  The CVE identifier for this vulnerability is CVE-2016-1246.

Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
@dbielefeldt

This comment has been minimized.

Show comment
Hide comment
@dbielefeldt

dbielefeldt May 9, 2018

I know this is an old thread, but the problem with amavis and DBD-MySQL still persist - At least on FreeBSD. Is there any conclusion ? The perl bug report is marked as solved -
https://rt.perl.org/Public/Bug/Display.html?id=130801

dbielefeldt commented May 9, 2018

I know this is an old thread, but the problem with amavis and DBD-MySQL still persist - At least on FreeBSD. Is there any conclusion ? The perl bug report is marked as solved -
https://rt.perl.org/Public/Bug/Display.html?id=130801

@pali

This comment has been minimized.

Show comment
Hide comment
@pali

pali May 9, 2018

Contributor

@dbielefeldt see my #78 (comment) fix was reverted due to issue #117

Contributor

pali commented May 9, 2018

@dbielefeldt see my #78 (comment) fix was reverted due to issue #117

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment