xp core w/o iconv

[kiesel]

After having installed xp runners - Mono version - when running xp -v I see:

# ad @ thinkpad in ~/bin [19:12:02]
$ xp -v
Uncaught exception: Error (Call to undefined function xp\iconv())
  at <source> [line 347 of ./class-main.php]
  at <main>('xp.runtime.Version') [line 0 of class-main.php]

Seems my PHP does not have libiconv:

# ad @ thinkpad in ~/bin [19:12:05] C:255
$ php -r 'iconv_set_encoding("utf-8");'
PHP Fatal error:  Uncaught Error: Call to undefined function iconv_set_encoding() in Command line code:1
Stack trace:
#0 {main}
  thrown in Command line code on line 1

# ad @ thinkpad in ~/bin [19:14:47] C:255
$ php -r 'iconv_set_encoding();'
PHP Fatal error:  Uncaught Error: Call to undefined function iconv_set_encoding() in Command line code:1
Stack trace:
#0 {main}
  thrown in Command line code on line 1

# ad @ thinkpad in ~/bin [19:14:50] C:255
$ php -m
[PHP Modules]
Core
ctype
curl
date
dom
fileinfo
filter
hash
json
libxml
mbstring
mysqlnd
openssl
pcntl
pcre
PDO
Phar
posix
readline
Reflection
session
SimpleXML
SPL
standard
tokenizer
xml
xmlreader
xmlwriter
zip
zlib

[Zend Modules]

After tinkering with the package system a bit, I guess it is not possible to have a distro-packaged PHP w/ iconv on ArchLinux.

[kiesel]

For investigation, there's a Docker image archlinux:latest that can be equipped w/ PHP:

# ad @ thinkpad in ~/bin [19:15:01]
$ docker run -it archlinux:latest
[...]
[root@24043c705fd2 /]# pacman -Syu
[...answer questions w/ 'y'...]
[root@24043c705fd2 /]# pacman -S php
[...]

# No signs of iconv
[root@24043c705fd2 /]# pacman -Ss php | grep iconv
[root@24043c705fd2 /]#

[thekid]

Interesting @kiesel - do you know why? The PHP documentation states:

This extension is enabled by default, although it may be disabled by compiling with --without-iconv.

...so the ArchLinux maintainers must've chosen explicitely to disable it; however, there's no mention of this decision here: https://wiki.archlinux.org/index.php/PHP

[thekid]

there's a Docker image archlinux:lates

Seems to be broken:

$ docker run --rm -it archlinux:latest /bin/bash
[root@a7c225b04c94 /]# pacman -Syu
error: failed to initialize alpm library
(could not find or read directory: /var/lib/pacman/)
[root@a7c225b04c94 /]# ls -al  /var/lib/pacman/
total 16
drwxr-xr-x   4 root root 4096 Apr 19 18:33 .
drwxr-xr-x   7 root root 4096 Apr 19 18:33 ..
drwxr-xr-x 110 root root 4096 Apr 19 18:33 local
drwxr-xr-x   2 root root 4096 Apr 18 00:04 sync

I have no idea what that's trying to tell me, but a quick search tells me I'm not the only one with this problem. I'll just try with a PHP build w/o iconv and see if we can create drop-in replacements.

[thekid]

I released new XP runners with a fix for this using the mbstring extension. To verify the entry points now work as expected, I compiled a PHP from scrath with --without-iconv --enable-mbstring, giving me:

thekid@Surface:~/bin/php$ ./sapi/cli/php -m | grep -E 'iconv|mbstring'
mbstring

Running XP with this gives me:

thekid@Surface:~/bin/php$ XP_RT=./sapi/cli/php xp -v | head -1
XP 10.9.1-dev { PHP/8.1.0-dev & Zend/4.1.0-dev } @ Linux Surface 4.19.104-microsoft-standard #1 SMP ... x86_64

The new release can be grabbed via this installer:

$ curl -sSL https://baltocdn.com/xp-framework/xp-runners/distribution/downloads/i/installer/setup-8.5.1.sh | sh

There are parts of the framework relying on iconv being available, I'll address these separately.

[thekid]

There are parts of the framework relying on iconv being available, I'll address these separately.

$ grep -Hrn iconv src/main/php
src/main/php/io/streams/TextReader.class.php:10: * @ext   iconv
src/main/php/io/streams/TextReader.class.php:86:    // an incomplete multi-byte sequence. In this case, iconv_strlen() will raise
src/main/php/io/streams/TextReader.class.php:94:    } while (($l= iconv_strlen($bytes, $this->charset)) < $size);
src/main/php/io/streams/TextReader.class.php:107:      return iconv($this->charset, \xp::ENCODING, $bytes);
src/main/php/io/streams/TextReader.class.php:144:    $line= iconv($this->charset, \xp::ENCODING, $bytes);
src/main/php/io/streams/TextWriter.class.php:8: * @ext   iconv
src/main/php/io/streams/TextWriter.class.php:76:    return $this->stream->write(iconv(\xp::ENCODING, $this->charset, $text));
src/main/php/lang/Runtime.class.php:325:    $pass= array_map(function($arg) { return iconv(\xp::ENCODING, 'utf-7', $arg); }, $arguments);
src/main/php/util/UUID.class.php:178:    $bytes= md5($namespace->getBytes().iconv(\xp::ENCODING, 'utf-8', $name));
src/main/php/util/UUID.class.php:198:    $bytes= sha1($namespace->getBytes().iconv(\xp::ENCODING, 'utf-8', $name))

[thekid]

I released a bugfix version of the XP Framework. It includes a fallback for when the iconv extension is not available and uses mbstring. However, this results in degraded performance and is not 100% compatible - e.g., no exception is raised when encountering malformed bytes in input strings.

[thekid]

@kiesel can you upgrade both XP runners and XP framework to their respective newest versions and verify they now work as expected?

[kiesel]

Looks good now:

# ad @ thinkpad in ~/bin [14:05:16]
$ xp -v
XP 10.9.1-dev { PHP/8.0.3 & Zend/4.0.3 } @ Linux thinkpad 5.11.15-arch1-2 #1 SMP PREEMPT Sat, 17 Apr 2021 00:22:30 +0000 x86_64
Copyright (c) 2001-2021 the XP group
FileSystemCL<~/dev/xp-framework/core/src/main/php>
FileSystemCL<~/dev/xp-framework/core/src/test/php>
FileSystemCL<~/dev/xp-framework/core/src/main/resources>
FileSystemCL<~/dev/xp-framework/core/src/test/resources>
FileSystemCL<.>