Fixed bug #81349

The ascii to wchar was reporting errors using conv_illegal_output,
while it should have been using WCSGROUP_THROUGH. Effectively that
replaced illegal characters with '?' for the purpose of
identification.

Author: nikic

NEWS

@@ -19,6 +19,9 @@ PHP                                                                        NEWS
   . Fixed bug #81106 (Regression in 8.1: add() now truncate ->f). (Derick)
   . Fixed bug #81273 (Date interval calculation not correct). (Derick)
 
+- Mbstring:
+  . Fixed bug #81349 (mb_detect_encoding misdetcts ASCII in some cases).
+    (Nikita)
 
 05 Aug 2021, PHP 8.1.0beta2
 

ext/mbstring/libmbfl/filters/mbfilter_singlebyte.c

@@ -99,7 +99,7 @@ static int mbfl_filt_conv_ascii_wchar(int c, mbfl_convert_filter *filter)
 	if (c < 0x80) {
 		CK((*filter->output_function)(c, filter->data));
 	} else {
-		CK(mbfl_filt_conv_illegal_output(c, filter));
+		CK((*filter->output_function)(c | MBFL_WCSGROUP_THROUGH, filter->data));
 	}
 	return c;
 }

ext/mbstring/tests/bug81349.phpt

@@ -0,0 +1,12 @@
+--TEST--
+Bug #81349: mb_detect_encoding misdetcts ASCII in some cases
+--FILE--
+<?php
+
+echo(mb_detect_encoding("\xe4,a", ['ASCII', 'UTF-8', 'ISO-8859-1'])."\n");
+echo(mb_detect_encoding("\xe4 a", ['ASCII', 'UTF-8', 'ISO-8859-1'])."\n");
+
+?>
+--EXPECT--
+ISO-8859-1
+ISO-8859-1