Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: ddeboer/transcoder
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: master
Choose a base ref
...
head repository: fossar/transcoder
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: master
Choose a head ref
Able to merge. These branches can be automatically merged.

Commits on Jan 11, 2021

  1. tests: Add suite name

    Modern PHPUnit does not run suites without names.
    jtojnar committed Jan 11, 2021

    Verified

    This commit was signed with the committer’s verified signature.
    jtojnar Jan Tojnar
    Copy the full SHA
    bc03ab8 View commit details

Commits on Jan 13, 2021

  1. tests: Use symfony/phpunit-bridge

    PHPUnit 7 is not compatible with PHP 8 and PHPUnit 8 adds return annotations to setUp method,
    requiring us to add those as well to pass return covariance checks.
    We can bypass that by renaming the setUp methods and using @before annotations
    but then there are other issues as well. For example, recent PHPUnit no longer supports
    @expectexception annotations an other issues.
    
    Let’s use Symfony’s PHPUnit bridge, which will allow us the widest range of supported PHP versions.
    
    We will need to bump to PHP 5.6+ since that is what php-unit bridge requires
    but PHP 5.4 is not even supported by Debian Jessie (oldoldstable).
    
    We also need to set locale appropriately for the null passed to iconv to work,
    since phpunit-bridge uses C locale (ASCII) for consistency.
    jtojnar committed Jan 13, 2021

    Verified

    This commit was signed with the committer’s verified signature.
    jtojnar Jan Tojnar
    Copy the full SHA
    37b3afb View commit details
  2. Bump PHP versions on Travis

    These are relevant now.
    
    Also had to explicitly set xdebug mode on versions using XDebug 3.
    jtojnar committed Jan 13, 2021

    Verified

    This commit was signed with the committer’s verified signature.
    jtojnar Jan Tojnar
    Copy the full SHA
    d3581bf View commit details
  3. composer: Add test script

    For easier running of scripts. Now we can just execute `composer test`.
    jtojnar committed Jan 13, 2021

    Verified

    This commit was signed with the committer’s verified signature.
    jtojnar Jan Tojnar
    Copy the full SHA
    b0ca55d View commit details

Commits on May 22, 2021

  1. Fork ddeboer/transcoder

    jtojnar committed May 22, 2021

    Verified

    This commit was signed with the committer’s verified signature.
    jtojnar Jan Tojnar
    Copy the full SHA
    c69a559 View commit details
  2. ci: Add Nix files

    This will allow people to easily enter a shell with dependencies like PHP and composer pre-installed.
    
    Just install *Nix package manager* and run `nix-shell`.
    You can also get *direnv* to be placed in the environment automatically
    whenever you `cd` into the project directory in a terminal emulator.
    
    You can also switch the used PHP version by changing `matrix.phpPackage` value
    in the `flake.nix` file and re-running `nix-shell`.
    
    For more information, check out https://github.com/fossar/selfoss/blob/e0a0ee2a88fc1be8563a3eb361c7b1142b239232/docs/content/docs/development/using-nix.md
    jtojnar committed May 22, 2021

    Verified

    This commit was signed with the committer’s verified signature.
    jtojnar Jan Tojnar
    Copy the full SHA
    5a5e094 View commit details
  3. ci: Add GitHub Actions workflow

    This will run tests on all supported PHP version 5.6 through 8.0.
    jtojnar committed May 22, 2021

    Verified

    This commit was signed with the committer’s verified signature.
    jtojnar Jan Tojnar
    Copy the full SHA
    5c2d66a View commit details
  4. iconv: Fix matching warning on PHP 8

    PHP 8 says encoding instead of charset: php/php-src@a4331a6
    It is also a warning instead of notice: php/php-src@9e3f352
    jtojnar committed May 22, 2021

    Verified

    This commit was signed with the committer’s verified signature.
    jtojnar Jan Tojnar
    Copy the full SHA
    c1a9af4 View commit details

Commits on May 23, 2021

  1. ci: Drop Travis

    jtojnar committed May 23, 2021

    Verified

    This commit was signed with the committer’s verified signature.
    jtojnar Jan Tojnar
    Copy the full SHA
    f710bfd View commit details
  2. Pair error handler restoration properly

    Previously, the error handlers in IconvTranscoder and MbTranscoder
    were not restored on error because exception was raised.
    And what is worse, MbTranscoder popped the error handler stack
    even when it did not push into it.
    
    Let’s fix the former by restoring in a finally block (not catching
    the raised exception), and the latter by restoring only conditionally.
    jtojnar committed May 23, 2021

    Verified

    This commit was signed with the committer’s verified signature.
    jtojnar Jan Tojnar
    Copy the full SHA
    09d89e2 View commit details
  3. mbstring: Restore the ability to pass “auto” as input language

    Since PHP 7.3, the array returned by `mb_list_encodings` [no longer contains “auto”].
    
    [no longer contains “auto”]: php/php-src@1151554
    
    `MbTranscoder::isSupported` therefore rejected “auto” and it could not be used as source encoding.
    
    To restore previous behaviour, let’s whitelist “auto” ourselves.
    
    We still reject “auto” for target encoding since PHP would reject it itself if passed.
    
    Note that PHP 8 started [throwing ValueError] when unknown encoding is passed so in the future, we might want to just rethrow that:
    
    [throwing ValueError]: php/php-src@229dff9
    
    Also note that the interpretation of “[auto]” depends on the value of `mbstring.language` setting:
    
    [auto]: https://github.com/php/php-src/blob/d61d21ad57d04b91a1155153811d16ea982fa106/ext/mbstring/mbstring.c#L296-L306
    
    > If "auto" is set, it is expanded to the list of encodings defined per the NLS. For instance, if the NLS is set to Japanese, the value is assumed to be "ASCII,JIS,UTF-8,EUC-JP,SJIS".
    > — https://www.php.net/manual/en/mbstring.supported-encodings.php
    
    Interestingly, you can include [auto] along with other encodings in `from_encodings` and the expanded encodings will get added to the list.
    
    But with the default “neutral” NLS settings, it actually just expands to [ASCII and UTF-8] and the available [identity lists] are useless for Western languages. I would therefore recommend avoiding “auto” and passing the source encoding(s) explicitly.
    
    [ASCII and UTF-8]: https://github.com/php/php-src/blob/d61d21ad57d04b91a1155153811d16ea982fa106/ext/mbstring/mbstring.c#L144-L147
    [identity lists]: https://github.com/php/php-src/blob/d61d21ad57d04b91a1155153811d16ea982fa106/ext/mbstring/mbstring.c#L88-L142
    
    Finally, although not really relevant for us, since we just use our default input encoding, PHP 8 added support for passing `null` to `from_encodings`, in which case the current internal encoding will be used: https://github.com/php/php-src/blob/d61d21ad57d04b91a1155153811d16ea982fa106/ext/mbstring/mbstring.c#L2529
    jtojnar committed May 23, 2021

    Verified

    This commit was signed with the committer’s verified signature.
    jtojnar Jan Tojnar
    Copy the full SHA
    a9ceb91 View commit details
  4. Clarify that “auto-detection” does not actually work

    for the most part.
    
    Change the tests to actually test the case where it works.
    
    This also fixes the warnings that the tests do not contain assertions.
    jtojnar committed May 23, 2021

    Verified

    This commit was signed with the committer’s verified signature.
    jtojnar Jan Tojnar
    Copy the full SHA
    7533db0 View commit details
  5. Transcoder: Fix instantiation without iconv

    If we do not have iconv, it should still work with just mbstring.
    jtojnar committed May 23, 2021

    Verified

    This commit was signed with the committer’s verified signature.
    jtojnar Jan Tojnar
    Copy the full SHA
    5df19bb View commit details
  6. Verified

    This commit was signed with the committer’s verified signature.
    jtojnar Jan Tojnar
    Copy the full SHA
    de61ecd View commit details
  7. Clean up coding style

    Mostly trailing spaces.
    jtojnar committed May 23, 2021

    Verified

    This commit was signed with the committer’s verified signature.
    jtojnar Jan Tojnar
    Copy the full SHA
    c1ee207 View commit details
  8. ci: Lint coding style

    jtojnar committed May 23, 2021

    Verified

    This commit was signed with the committer’s verified signature.
    jtojnar Jan Tojnar
    Copy the full SHA
    2d2d361 View commit details
  9. Add changelog

    jtojnar committed May 23, 2021

    Verified

    This commit was signed with the committer’s verified signature.
    jtojnar Jan Tojnar
    Copy the full SHA
    4ca3044 View commit details
  10. Verified

    This commit was signed with the committer’s verified signature.
    jtojnar Jan Tojnar
    Copy the full SHA
    3bcca53 View commit details
  11. flake: update

    This fixes php iconv extension.
    jtojnar committed May 23, 2021

    Verified

    This commit was signed with the committer’s verified signature.
    jtojnar Jan Tojnar
    Copy the full SHA
    3774c76 View commit details
  12. ci: Install php-cs-fixer through Nix

    Since this is library, we do not have a lock file so let’s just use the package from Nix.
    jtojnar committed May 23, 2021

    Verified

    This commit was signed with the committer’s verified signature.
    jtojnar Jan Tojnar
    Copy the full SHA
    0b8e82c View commit details
  13. Release 1.0.1

    jtojnar committed May 23, 2021

    Verified

    This commit was signed with the committer’s verified signature.
    jtojnar Jan Tojnar
    Copy the full SHA
    dd4012b View commit details

Commits on Mar 7, 2023

  1. flake.lock: Update

    Flake lock file updates:
    
    • Updated input 'phps':
        'github:fossar/nix-phps/da4f25e4a230e42b349df2d8c9c66c1717191413' (2021-05-23)
      → 'github:fossar/nix-phps/181dee32f3d47312674462cff30fcca661c6c643' (2023-03-05)
    • Updated input 'phps/flake-compat':
        'github:edolstra/flake-compat/99f1c2157fba4bfe6211a321fd0ee43199025dbf' (2020-11-26)
      → 'github:edolstra/flake-compat/35bb57c0c8d8b62bbfd284272c928ceb64ddbde9' (2023-01-17)
    • Updated input 'phps/nixpkgs':
        'github:NixOS/nixpkgs/efee454783c5c14ae78687439077c1d3f0544d97' (2021-05-22)
      → 'github:NixOS/nixpkgs/f5ffd5787786dde3a8bf648c7a1b5f78c4e01abb' (2023-03-03)
    • Updated input 'phps/utils':
        'github:numtide/flake-utils/b543720b25df6ffdfcf9227afafc5b8c1fabfae8' (2021-05-11)
      → 'github:numtide/flake-utils/3db36a8b464d0c4532ba1c7dda728f4576d6d073' (2023-02-13)
    jtojnar committed Mar 7, 2023
    Copy the full SHA
    e0cd0e4 View commit details
  2. Copy the full SHA
    2d9265f View commit details
  3. ci: Add newer PHP versions

    jtojnar committed Mar 7, 2023
    Copy the full SHA
    446f0b1 View commit details
  4. iconv: Fix warning on PHP 8.2

        Deprecated: iconv(): Passing null to parameter #1 ($from_encoding) of type string is deprecated in src/IconvTranscoder.php on line 49
    jtojnar committed Mar 7, 2023
    Copy the full SHA
    2e1d8fd View commit details
  5. Raise minimum PHP version to 7.2.5

    This will allow to use type hints plus clean-up a bit.
    jtojnar committed Mar 7, 2023
    Copy the full SHA
    2a6b03d View commit details
  6. Copy the full SHA
    6f7ac8b View commit details
  7. ci: Enable PHPStan

    And fix some issues.
    
    Install PHPStan using Nix so that it is pinned and we do not have to also manage `composer.lock`.
    jtojnar committed Mar 7, 2023
    Copy the full SHA
    0a7c68c View commit details
  8. Release 2.0.0

    jtojnar committed Mar 7, 2023
    Copy the full SHA
    1864d5f View commit details

Commits on Mar 8, 2023

  1. README: Update version badge

    jtojnar committed Mar 8, 2023
    Copy the full SHA
    941ab65 View commit details

Commits on Mar 15, 2025

  1. mbstring: Make encoding detection stricter

    PHP 8.3 changed how source encoding detection works:
    https://www.php.net/manual/en/migration83.other-changes.php#migration83.other-changes.functions.mbstring
    Most locales only consider `ASCII` and `UTF-8` (see `mb_detect_order()`),
    and when a byte sequence invalid in both tested encodings (such as 0x91 for ‘ in Windows-1252) is encountered,
    one of them might now be chosen as the most fitting encoding.
    (This is done using the heuristics introduced in PHP 8.1:
    php/php-src@28b346b)
    
    Compare the output of the following script across PHP versions:
    
        <?php
        $result = hex2bin("91");
        var_dump(mb_detect_encoding($result));
        var_dump(mb_detect_encoding($result, 'auto', true));
        var_dump(mb_convert_encoding($result, 'UTF-8', 'auto'));
    
    Let’s run the `mb_detect_encoding()` ourselves with `$strict` argument set to `true`, to ensure consistent behaviour across all PHP versions.
    This might potentially cause a regression is some cases. Not sure.
    
    Additionally, since we are now ensuring all encodings are valid, we can drop the warning capture mechanism.
    It does not work on PHP ≥ 8.0 anyway, since that raises a `ValueError` instead of a warning when an invalid encoding is provided.
    https://www.php.net/manual/en/function.mb-convert-encoding.php#refsect1-function.mb-convert-encoding-errors
    
    Also adjust the confusing string in tests.
    
    https://www.php.net/manual/en/function.mb-convert-encoding.php
    https://www.php.net/manual/en/function.mb-detect-encoding.php
    https://www.php.net/manual/en/function.mb-detect-order.php
    jtojnar committed Mar 15, 2025
    Copy the full SHA
    5d40f6b View commit details
  2. devenv: Make direnv reload on flake changes

    We do not use `flake` direnv feature since flakes currently copy the whole repo into Nix store, so we need to watch the files manually.
    jtojnar committed Mar 15, 2025
    Copy the full SHA
    f4eef1f View commit details
  3. ci: Update actions

    jtojnar committed Mar 15, 2025
    Copy the full SHA
    b0ea6bc View commit details
  4. Copy the full SHA
    2db54e0 View commit details
  5. phpstan: Rename config to use dist suffix

    This allows developer to create their own own config file, e.g. for setting `editorUrl`:
    https://phpstan.org/user-guide/output-format#opening-file-in-an-editor
    jtojnar committed Mar 15, 2025
    Copy the full SHA
    8094799 View commit details
  6. Install PHPStan through composer

    It has isolated dependencies nowadays and it will be helpful for other developers, who probably expect it to be installed that way and do not want to use Nix.
    
    Do the same with PHP-CS-Fixer.
    jtojnar committed Mar 15, 2025
    Copy the full SHA
    fabb6a0 View commit details
  7. flake.lock: Update

    Flake lock file updates:
    
    • Updated input 'phps':
        'github:fossar/nix-phps/181dee32f3d47312674462cff30fcca661c6c643' (2023-03-05)
      → 'github:fossar/nix-phps/e48727edd2dbbbd7b9181bd284569554f6e4e5a6' (2025-03-10)
    • Updated input 'phps/flake-compat':
        'github:edolstra/flake-compat/35bb57c0c8d8b62bbfd284272c928ceb64ddbde9' (2023-01-17)
      → 'github:edolstra/flake-compat/ff81ac966bb2cae68946d5ed5fc4994f96d0ffec' (2024-12-04)
    • Updated input 'phps/nixpkgs':
        'github:NixOS/nixpkgs/f5ffd5787786dde3a8bf648c7a1b5f78c4e01abb' (2023-03-03)
      → 'github:NixOS/nixpkgs/de0fe301211c267807afd11b12613f5511ff7433' (2025-03-07)
    • Updated input 'phps/utils':
        'github:numtide/flake-utils/3db36a8b464d0c4532ba1c7dda728f4576d6d073' (2023-02-13)
      → 'github:numtide/flake-utils/11707dc2f618dd54ca8739b309ec4fc024de578b' (2024-11-13)
    • Added input 'phps/utils/systems':
        'github:nix-systems/default/da67096a3b9bf56a91d16901293e51ba5b49a27e' (2023-04-09)
    jtojnar committed Mar 15, 2025
    Copy the full SHA
    0bf5a8f View commit details
  8. Copy the full SHA
    52c84e2 View commit details
  9. ci: Check PHP 8.4 and PHP 8.3

    jtojnar committed Mar 15, 2025
    Copy the full SHA
    8d309bd View commit details
  10. devenv: Install Phpactor

    This is a Language Server Provider for PHP.
    jtojnar committed Mar 15, 2025
    Copy the full SHA
    976d0f4 View commit details
  11. devenv: Provide locale archive

    Without this, the Iconv tests on CI started to fail in `setLocale` invocation:
    
        PHPUnit\Framework\Exception: The locale functionality is not implemented on your platform, the specified locale does not exist or the category name is invalid.
    
        /home/runner/work/transcoder/transcoder/tests/IconvTranscoderTest.php:25
    jtojnar committed Mar 15, 2025
    Copy the full SHA
    2a7809a View commit details
  12. Raise minimum PHP version to 7.4

    https://www.php.net/releases/7_3_0.php
    https://www.php.net/releases/7_4_0.php
    
    This installed a newer version of CS Fixer
    
     - friendsofphp/php-cs-fixer updated from v3.4.0 to v3.72.0 minor
       See changes: PHP-CS-Fixer/PHP-CS-Fixer@FriendsOfPHP:v3.4.0...PHP-CS-Fixer:v3.72.0
       Release notes: https://github.com/PHP-CS-Fixer/PHP-CS-Fixer/releases/tag/v3.72.0
    
       - `no_superfluous_phpdoc_tags` removes `inheritdoc`.
       - `phpdoc_to_param_type` did its thing in one place.
    
    As a result, I had to fix some rules:
    
    - Disable `phpdoc_to_param_type` since it now tries to use union type in a param type hint, which is only supported on PHP 8.0.
    
    Also enable PHP 7.3 migration in PHP-CS-Fixer, although there was nothing to do.
    jtojnar committed Mar 15, 2025
    Copy the full SHA
    ac7fc2c View commit details
  13. Use property type hints

    Not a BC break since all of them are private.
    
    Had to change an initialization check in `MbTranscoder::__construct`, since PHPStan figured out it is not nullable.
    jtojnar committed Mar 15, 2025
    Copy the full SHA
    f396153 View commit details
  14. Upgrade PHPStan to version 2

     - phpstan/phpstan updated from 1.12.21 to 2.1.8 major
       See changes: phpstan/phpstan@1.12.21...2.1.8
       Release notes
    
       Had to remove redundant assertion. Not clear when false can actually occur on PHP ≥ 8.0 – on PHP 7.4, it can happen when passing an invalid target encoding but on PHP 8.0, it just throws a `ValueError`.
    jtojnar committed Mar 15, 2025
    Copy the full SHA
    b4f8126 View commit details
  15. Release 3.0.0

    jtojnar committed Mar 15, 2025
    Copy the full SHA
    3afd573 View commit details
2 changes: 2 additions & 0 deletions .envrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
use nix
watch_file flake.nix flake.lock
68 changes: 68 additions & 0 deletions .github/workflows/main.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# This file lints the code and runs tests.
name: CI

on:
pull_request:
push:

env:
COMPOSER_FLAGS: "--ansi --no-interaction --no-progress --no-suggest --prefer-dist"

jobs:
tests:
runs-on: ubuntu-latest
strategy:
matrix:
php:
- '8.4'
- '8.3'
- '8.2'
- '8.1'
- '8.0'
- '7.4'
include:
- php: '7.4'
cs_fixer: true
- php: '8.0'
phpstan: true
name: 'Check with PHP ${{ matrix.php }}'
steps:
- uses: actions/checkout@v4

- name: Install Nix
uses: cachix/install-nix-action@v31

- name: Set up Nix cache
uses: cachix/cachix-action@v16
with:
# Use cache from https://github.com/fossar/nix-phps
name: fossar

- name: Update flake.nix to match the current CI job from matrix
run: sed -i 's/matrix.phpPackage = "php";/matrix.phpPackage = builtins.replaceStrings ["."] [""] "php${{ matrix.php }}";/' flake.nix

- name: Get Composer Cache Directory
id: composer-cache
run: |
echo "dir=$(composer config cache-files-dir)" >> "$GITHUB_OUTPUT"
- uses: actions/cache@v4
with:
path: ${{ steps.composer-cache.outputs.dir }}
key: ${{ runner.os }}-composer-${{ hashFiles('**/composer.lock') }}
restore-keys: |
${{ runner.os }}-composer-
- name: Install dependencies
run: nix-shell --run 'composer install'

- name: Lint source code
if: matrix.cs_fixer
run: nix-shell --run 'composer cs-check'

- name: Run unit tests
run: nix-shell --run 'composer test'

- name: Statically analyze source code
if: matrix.phpstan
run: nix-shell --run 'composer phpstan'
6 changes: 5 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,6 @@
composer.lock
vendor
vendor/
.direnv/
.php-cs-fixer.cache
.phpunit.result.cache
phpstan.neon
24 changes: 24 additions & 0 deletions .php-cs-fixer.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
<?php

$finder = PhpCsFixer\Finder::create()
->in(__DIR__);

$rules = [
'@Symfony' => true,
'@PHP74Migration' => true,
'@PHP74Migration:risky' => true,
'phpdoc_to_property_type' => true,
// 'phpdoc_to_param_type' => true, // requires PHP 8.0
'phpdoc_to_return_type' => true,
// overwrite some Symfony rules
'concat_space' => ['spacing' => 'one'],
'phpdoc_align' => false,
'yoda_style' => false,
];

$config = new PhpCsFixer\Config();

return $config
->setRules($rules)
->setRiskyAllowed(true)
->setFinder($finder);
16 changes: 0 additions & 16 deletions .travis.yml

This file was deleted.

34 changes: 34 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Transcoder Changelog

This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [3.0.0] – 2025-03-15

- Mb: Make encoding detection stricter.
- Raise minimum PHP version to 7.4.0.


## [2.0.0] – 2023-03-07

- Iconv: Fix warning on PHP 8.2 when passing `null` as source encoding.
- Raise minimum PHP version to 7.2.5.
- Add parameter and return type hints.


## [1.0.1] – 2021-05-23

The project has been revived and is now available under the name [`fossar/transcoder`](https://packagist.org/packages/fossar/transcoder). This is a first release since the fork.

- Iconv: Fixed detection of unsupported encoding on PHP 8.
- Mb: Restored the ability to pass “auto” as input language on PHP ≥ 7.3.
- Fixed the pairing of setting and restoring error handlers in the transcoders so that they do not clear handlers that they did not create nor do they leave any around when something unexpected happens.
- Fixed creating `Transcoder` with just `mbstring` (without `iconv`).
- Clarified that “auto-detection” does not actually work.
- Fixed various minor test issues.
- Cleaned up coding style.
- Added Nix expression for easier development and sharing the environment with CI.
- Switched to GitHub Actions for CI and added more PHP versions.

[3.0.0]: https://github.com/fossar/transcoder/compare/v2.0.0...v3.0.0
[2.0.0]: https://github.com/fossar/transcoder/compare/v1.0.1...v2.0.0
[1.0.1]: https://github.com/fossar/transcoder/compare/1.0.0...v1.0.1
94 changes: 39 additions & 55 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,44 +1,35 @@
Transcoder
==========
# Transcoder

[![Build Status](https://travis-ci.org/ddeboer/transcoder.svg?branch=master)](https://travis-ci.org/ddeboer/transcoder)
[![Release](https://img.shields.io/github/release/ddeboer/transcoder.svg?style=flat)](https://packagist.org/packages/ddeboer/transcoder)
[![Packagist Version](https://img.shields.io/packagist/v/fossar/transcoder)](https://packagist.org/packages/fossar/transcoder)

Introduction
------------
## Introduction

This is a wrapper around PHP’s `mb_convert_encoding` and `iconv` functions.
This library adds:
This is a wrapper around PHP’s `mb_convert_encoding` and `iconv` functions. This library adds:

* fallback from `mb` to `iconv` for unknown encodings
* conversion of warnings to proper exceptions.
- fallback from `mb` to `iconv` for encodings it does not support
- conversion of warnings to proper exceptions.

Installation
------------
## Installation

The recommended way to install the Transcoder library is through
[Composer](http://getcomposer.org):
The recommended way to install the Transcoder library is through [Composer](http://getcomposer.org):

```bash
$ composer require ddeboer/transcoder
$ composer require fossar/transcoder
```

This command requires you to have Composer installed globally, as explained
in the [installation chapter](https://getcomposer.org/doc/00-intro.md)
of the Composer documentation.
This command requires you to have Composer installed globally, as explained in the [installation chapter](https://getcomposer.org/doc/00-intro.md) of the Composer documentation.

Usage
-----
## Usage

### Basics

Create the right transcoder for your platform and translate some strings:
Create the right transcoder for your platform and translate a string to ISO-8859-1 encoding:

```php
use Ddeboer\Transcoder\Transcoder;

$transcoder = Transcoder::create();
$result = $transcoder->transcode('España');
$result = $transcoder->transcode('España', 'iso-8859-1');
```

You can also manually instantiate a transcoder of your liking:
@@ -47,7 +38,6 @@ You can also manually instantiate a transcoder of your liking:
use Ddeboer\Transcoder\MbTranscoder;

$transcoder = new MbTranscoder();

```

Or:
@@ -60,79 +50,73 @@ $transcoder = new IconvTranscoder();

### Source encoding

By default, the source encoding is detected automatically. However, you get
much more reliable results when you specify it explicitly:
The second argument accepts source encoding and can actually be omitted or passed `null`.

```php
$transcoder->transcode('España', 'iso-8859-1');
$transcoder->transcode('España');
```

In that case, however, the behaviour is backend-specific:

- `IconvTranscoder` will use the encoding of the current [locale](https://www.php.net/manual/en/function.setlocale.php) of the process.
- `MbTranscoder` will try to detect encoding from a list based on the value of [`mbstring.language`](https://www.php.net/manual/en/mbstring.configuration.php#ini.mbstring.language) setting. By default, this tries ASCII, followed by UTF-8. The number of [supported languages](https://github.com/php/php-src/blob/d61d21ad57d04b91a1155153811d16ea982fa106/ext/mbstring/mbstring.c#L88-L147) is limited though and the encoding tables often overlap so the detection might be unreliable.

As you can see, this is mostly useless for western languages. You will get much more reliable results when you specify the source encoding explicitly.

### Target encoding

Specify a default target encoding as the first argument to `create()`:


```php
use Ddeboer\Transcoder\Transcoder;

$isoTranscoder = Transcoder::create('iso-8859-1');
```

Alternatively, specify a target encoding as the third argument in a
`transcode()` call:

Alternatively, specify a target encoding as the third argument in a `transcode()` call:

```php
use Ddeboer\Transcoder\Transcoder;

$transcoder->transcode('España', null, 'UTF-8');
$transcoder->transcode('España', 'iso-8859-1', 'UTF-8');
```

### Error handling

PHP’s `mv_convert_encoding` and `iconv` are inconvenient to use because they
generate notices and warnings instead of proper exceptions. This library fixes
that:

PHP’s `mv_convert_encoding` and `iconv` are inconvenient to use because they generate notices and warnings instead of proper exceptions. This library fixes that:

```php
use Ddeboer\Transcoder\Exception\UndetectableEncodingException;
use Ddeboer\Transcoder\Exception\UnsupportedEncodingException;
use Ddeboer\Transcoder\Exception\IllegalCharacterException;

$input = 'España';

try {
$transcoder->transcode($input);
} catch (UndetectableEncodingException $e) {
// Failed to automatically detect $input’s encoding
}

try {
$transcoder->transcode($input, null, 'not-a-real-encoding');
$transcoder->transcode($input, 'utf-8', 'not-a-real-encoding');
} catch (UnsupportedEncodingException $e) {
// ‘not-a-real-encoding’ is an unsupported encoding
// ‘not-a-real-encoding’ is an unsupported encoding
}

try {
$transcoder->transcode('Illegal quotes: ‘ ’', null, 'iso-8859-1');
$transcoder->transcode('Illegal quotes: ‘ ’', 'utf-8', 'iso-8859-1');
} catch (IllegalCharacterException $e) {
// Curly quotes ‘ ’ are illegal in ISO-8859-1
}

try {
$transcoder->transcode($input);
} catch (UndetectableEncodingException $e) {
// Failed to automatically detect $input’s encoding (mb) or not a valid string in current locale locale (iconv)
}
```

### Transcoder fallback

In general, `mb_convert_encoding` is faster than `iconv`. However, as `iconv`
supports more encodings than `mb_convert_encoding`, it makes sense to combine
the two.
In general, `mb_convert_encoding` is faster than `iconv`. However, as `iconv` supports more encodings than `mb_convert_encoding`, it makes sense to combine the two.

So, the Transcoder returned from `create()`:

* uses `mb_convert_encoding` if the
[mbstring](http://php.net/manual/en/book.mbstring.php) PHP extension is
installed;
* if not, it uses `iconv` instead if the
[iconv](http://php.net/manual/en/book.iconv.php) extension is installed;
* if both the mbstring and iconv extension are available, the Transcoder will
first try `mb_convert_encoding` and fall back to `iconv`.

- uses `mb_convert_encoding` if the [mbstring](http://php.net/manual/en/book.mbstring.php) PHP extension is installed;
- if not, it uses `iconv` instead if the [iconv](http://php.net/manual/en/book.iconv.php) extension is installed;
- if both the mbstring and iconv extension are available, the Transcoder will first try `mb_convert_encoding` and fall back to `iconv`.
33 changes: 25 additions & 8 deletions composer.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"name": "ddeboer/transcoder",
"name": "fossar/transcoder",
"description": "Better encoding conversion for PHP",
"keywords": [ "encoding", "utf-8", "iso", "mb", "iconv", "multibyte", "charset", "mb_convert_encoding" ],
"license": "MIT",
@@ -10,14 +10,19 @@
},
{
"name": "Community contributors",
"homepage": "https://github.com/ddeboer/transcoder/graphs/contributors"
}
"homepage": "https://github.com/fossar/transcoder/graphs/contributors"
}
],
"replace": {
"ddeboer/transcoder": "self.version"
},
"require": {
"php": ">=5.4.0"
"php": ">=7.4.0"
},
"require-dev": {
"phpunit/phpunit": "~4.0"
"symfony/phpunit-bridge": "^6.2 || ^7.0",
"phpstan/phpstan": "^2.1",
"friendsofphp/php-cs-fixer": "^3.4"
},
"suggest": {
"ext-mbstring": "For using the MbTranscoder",
@@ -29,9 +34,21 @@
"Ddeboer\\Transcoder\\Tests\\": "tests/"
}
},
"extra": {
"branch-alias": {
"dev-master": "1.0.x-dev"
"config": {
"platform": {
"php": "7.4.0"
}
},
"scripts": {
"cs-check": "php-cs-fixer fix --verbose --dry-run --diff",
"cs-fix": "php-cs-fixer fix --verbose --diff",
"phpstan": "phpstan analyze",
"test": "simple-phpunit"
},
"scripts-descriptions": {
"cs-check": "Check coding style",
"cs-fix": "Fix coding style",
"phpstan": "Fix coding style",
"test": "Run all tests"
}
}
96 changes: 96 additions & 0 deletions flake.lock
59 changes: 59 additions & 0 deletions flake.nix
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
{
description = "Better encoding conversion for PHP";

inputs = {
# For old PHP versions.
phps.url = "github:fossar/nix-phps";
};

outputs = { self, phps }:
let
# nixpkgs is a repository with software packages and some utilities.
# From simplicity, we inherit it from the phps flake.
inherit (phps.inputs) nixpkgs;

# Configure the development shell here (e.g. for CI).

# By default, we use the default PHP version from Nixpkgs.
matrix.phpPackage = "php";
# Allow easily removing some extensions for testing.
matrix.forbiddenExtensionNames = [
# "iconv"
# "mbstring"
];
in
let
# We only support a single platform at the moment,
# since our binary cache only contains PHP packages for that.
system = "x86_64-linux";

# Get Nixpkgs packages for current platform.
pkgs = nixpkgs.legacyPackages.${system};

# Create a PHP package from the selected PHP package.
phpBase = phps.packages.${system}.${matrix.phpPackage};
php =
phpBase.withExtensions
({ all, enabled }:
let
# Get the forbidden extensions by name.
forbiddenExtensions = pkgs.lib.attrVals matrix.forbiddenExtensionNames all;
in
builtins.filter (ext: !builtins.elem ext forbiddenExtensions) enabled
);
in {
# Expose shell environment for development.
devShell.${system} = pkgs.mkShell {
nativeBuildInputs = [
# Composer and PHP.
php
phpBase.packages.composer
pkgs.phpactor
];

env = pkgs.lib.optionalAttrs pkgs.stdenv.hostPlatform.isLinux {
LOCALE_ARCHIVE = "${pkgs.glibcLocales}/lib/locale/locale-archive";
};
};
};
}
6 changes: 6 additions & 0 deletions phpstan.dist.neon
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
parameters:
level: max

paths:
- src/
- tests/
2 changes: 1 addition & 1 deletion phpunit.xml.dist
Original file line number Diff line number Diff line change
@@ -4,7 +4,7 @@
convertErrorsToExceptions="false"
>
<testsuites>
<testsuite>
<testsuite name="Tests">
<directory suffix="Test.php">./tests/</directory>
</testsuite>
</testsuites>
11 changes: 11 additions & 0 deletions shell.nix
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
(import (
let
lock = builtins.fromJSON (builtins.readFile ./flake.lock);
in
fetchTarball {
url = "https://github.com/edolstra/flake-compat/archive/${lock.nodes.flake-compat.locked.rev}.tar.gz";
sha256 = lock.nodes.flake-compat.locked.narHash;
}
) {
src = ./.;
}).shellNix
3 changes: 2 additions & 1 deletion src/Exception/ExtensionMissingException.php
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
<?php

declare(strict_types=1);

namespace Ddeboer\Transcoder\Exception;

class ExtensionMissingException extends \RuntimeException
{

}
4 changes: 3 additions & 1 deletion src/Exception/IllegalCharacterException.php
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
<?php

declare(strict_types=1);

namespace Ddeboer\Transcoder\Exception;

class IllegalCharacterException extends \RuntimeException
{
public function __construct($string, $warning)
public function __construct(string $string, string $warning)
{
parent::__construct(
sprintf(
4 changes: 3 additions & 1 deletion src/Exception/UndetectableEncodingException.php
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
<?php

declare(strict_types=1);

namespace Ddeboer\Transcoder\Exception;

class UndetectableEncodingException extends \RuntimeException
{
public function __construct($string, $error)
public function __construct(string $string, string $error)
{
parent::__construct(sprintf('Encoding for %s is undetectable: %s', $string, $error));
}
10 changes: 6 additions & 4 deletions src/Exception/UnsupportedEncodingException.php
Original file line number Diff line number Diff line change
@@ -1,16 +1,18 @@
<?php

declare(strict_types=1);

namespace Ddeboer\Transcoder\Exception;

class UnsupportedEncodingException extends \RuntimeException
{
public function __construct($encoding, $message = null)
public function __construct(string $encoding, ?string $message = null)
{
$error = sprintf('Encoding %s is unsupported on this platform', $encoding);
if ($message) {
if ($message !== null) {
$error .= ': ' . $message;
}
return parent::__construct($error);

parent::__construct($error);
}
}
38 changes: 23 additions & 15 deletions src/IconvTranscoder.php
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
<?php

declare(strict_types=1);

namespace Ddeboer\Transcoder;

use Ddeboer\Transcoder\Exception\ExtensionMissingException;
@@ -8,36 +10,42 @@

class IconvTranscoder implements TranscoderInterface
{
private $defaultEncoding;

public function __construct($defaultEncoding = 'UTF-8')
private string $defaultEncoding;

/**
* Create an Iconv-based transcoder.
*
* @throws ExtensionMissingException
*/
public function __construct(string $defaultEncoding = 'UTF-8')
{
if (!function_exists('iconv')) {
throw new ExtensionMissingException('iconv');
}

$this->defaultEncoding = $defaultEncoding;
}

/**
* {@inheritdoc}
*/
public function transcode($string, $from = null, $to = null)
public function transcode(string $string, ?string $from = null, ?string $to = null): string
{
set_error_handler(
function ($no, $message) use ($string) {
if (1 === preg_match('/Wrong charset, conversion (.+) is/', $message, $matches)) {
function (int $no, string $message) use ($string): void {
if (1 === preg_match('/Wrong (charset|encoding), conversion (.+) is/', $message, $matches)) {
throw new UnsupportedEncodingException($matches[1], $message);
} else {
throw new IllegalCharacterException($string, $message);
}
},
E_NOTICE | E_USER_NOTICE
E_NOTICE | E_USER_NOTICE | E_WARNING
);

$result = iconv($from, $to ?: $this->defaultEncoding, $string);
restore_error_handler();


try {
$result = iconv($from ?? '', $to ?? $this->defaultEncoding, $string);
assert($result !== false); // For PHPStan, ensured by error handler.
} finally {
restore_error_handler();
}

return $result;
}
}
76 changes: 44 additions & 32 deletions src/MbTranscoder.php
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
<?php

declare(strict_types=1);

namespace Ddeboer\Transcoder;

use Ddeboer\Transcoder\Exception\ExtensionMissingException;
@@ -8,73 +10,83 @@

class MbTranscoder implements TranscoderInterface
{
private static $encodings;
private $defaultEncoding;

public function __construct($defaultEncoding = 'UTF-8')
/**
* @var array<string, int>
*/
private static array $encodings;

private string $defaultEncoding;

/**
* Create a Mb-based transcoder.
*
* @throws ExtensionMissingException
*/
public function __construct(string $defaultEncoding = 'UTF-8')
{
if (!function_exists('mb_convert_encoding')) {
throw new ExtensionMissingException('mb');
}
if (null === self::$encodings) {

if (!isset(self::$encodings)) {
self::$encodings = array_change_key_case(
array_flip(mb_list_encodings()),
CASE_LOWER
);
}
$this->assertSupported($defaultEncoding);

$this->assertSupported($defaultEncoding, false);
$this->defaultEncoding = $defaultEncoding;
}

/**
* {@inheritdoc}
* @param array<string>|string|null $from
*/
public function transcode($string, $from = null, $to = null)
public function transcode(string $string, $from = null, ?string $to = null): string
{
if ($from) {
if (is_array($from)) {
array_map(array($this, 'assertSupported'), $from);
array_map([$this, 'assertSupported'], $from);
} else {
$this->assertSupported($from);
}
} else {
$from = 'auto';
}

if (!$from || 'auto' === $from) {
set_error_handler(
function ($no, $warning) use ($string) {
throw new UndetectableEncodingException($string, $warning);
},
E_WARNING
);
if ($to) {
$this->assertSupported($to, false);
}


if ($to) {
$this->assertSupported($to);
if ($from === 'auto') {
$from = mb_detect_encoding($string, 'auto', true);
}


if ($from === false) {
throw new UndetectableEncodingException($string, 'Unable to detect character encoding');
}

$result = mb_convert_encoding(
$string,
$to ?: $this->defaultEncoding,
$from ?: 'auto'
$from
);

restore_error_handler();


return $result;
}

private function assertSupported($encoding)

/**
* @throws UnsupportedEncodingException
*/
private function assertSupported(string $encoding, bool $allowAuto = true): void
{
if (!$this->isSupported($encoding)) {
if (!$this->isSupported($encoding, $allowAuto)) {
throw new UnsupportedEncodingException($encoding);
}
}
private function isSupported($encoding)

private function isSupported(string $encoding, bool $allowAuto): bool
{
return isset(self::$encodings[strtolower($encoding)]);
return ($allowAuto && $encoding === 'auto') || isset(self::$encodings[strtolower($encoding)]);
}
}
49 changes: 29 additions & 20 deletions src/Transcoder.php
Original file line number Diff line number Diff line change
@@ -1,28 +1,33 @@
<?php

declare(strict_types=1);

namespace Ddeboer\Transcoder;

use Ddeboer\Transcoder\Exception\ExtensionMissingException;
use Ddeboer\Transcoder\Exception\UnsupportedEncodingException;

class Transcoder implements TranscoderInterface
{
private static $chain;

/**
* @var array<string, TranscoderInterface>
*/
private static array $chain;

/**
* @var TranscoderInterface[]
*/
private $transcoders = [];

private array $transcoders = [];

/**
* @param TranscoderInterface[] $transcoders
*/
public function __construct(array $transcoders)
{
$this->transcoders = $transcoders;
}

/**
* {@inheritdoc}
*/
public function transcode($string, $from = null, $to = null)
public function transcode(string $string, ?string $from = null, ?string $to = null): string
{
foreach ($this->transcoders as $transcoder) {
try {
@@ -31,27 +36,27 @@ public function transcode($string, $from = null, $to = null)
// Ignore as long as the fallback transcoder is all right
}
}


if (!isset($e)) {
throw new UnsupportedEncodingException('of any kind', 'No transcoder provided.');
}

throw $e;
}

/**
* Create a transcoder
*
* @param string $defaultEncoding
*
* @return TranscoderInterface
* Create a transcoder.
*
* @throws ExtensionMissingException
*/
public static function create($defaultEncoding = 'UTF-8')
public static function create(string $defaultEncoding = 'UTF-8'): TranscoderInterface
{
if (isset(self::$chain[$defaultEncoding])) {
return self::$chain[$defaultEncoding];
}

$transcoders = [];

try {
$transcoders[] = new MbTranscoder($defaultEncoding);
} catch (ExtensionMissingException $mb) {
@@ -61,10 +66,14 @@ public static function create($defaultEncoding = 'UTF-8')
try {
$transcoders[] = new IconvTranscoder($defaultEncoding);
} catch (ExtensionMissingException $iconv) {
// Neither mbstring nor iconv
throw $iconv;
// Ignore missing iconv extension; hopefully, we have mbstring.
}

// Neither mbstring nor iconv
if (count($transcoders) === 0) {
throw new ExtensionMissingException('mbstring or iconv');
}

self::$chain[$defaultEncoding] = new self($transcoders);

return self::$chain[$defaultEncoding];
13 changes: 6 additions & 7 deletions src/TranscoderInterface.php
Original file line number Diff line number Diff line change
@@ -1,21 +1,20 @@
<?php

declare(strict_types=1);

namespace Ddeboer\Transcoder;

use Ddeboer\Transcoder\Exception\UnsupportedEncodingException;

interface TranscoderInterface
{
/**
* Transcode a string from one into another encoding
* Transcode a string from one encoding into another.
*
* @param string $string String
* @param string $from From encoding (optional)
* @param string $to To encoding (optional)
* @param string $from Source encoding (optional)
* @param string $to Target encoding (optional)
*
* @return string
*
* @throws UnsupportedEncodingException
*/
public function transcode($string, $from = null, $to = null);
public function transcode(string $string, ?string $from = null, ?string $to = null): string;
}
54 changes: 28 additions & 26 deletions tests/IconvTranscoderTest.php
Original file line number Diff line number Diff line change
@@ -1,56 +1,58 @@
<?php

declare(strict_types=1);

namespace Ddeboer\Transcoder\Tests;

use Ddeboer\Transcoder\IconvTranscoder;

class IconvTranscoderTest extends \PHPUnit_Framework_TestCase
class IconvTranscoderTest extends \PHPUnit\Framework\TestCase
{
private IconvTranscoder $transcoder;

/**
* @var IconvTranscoder
* @before
*/
private $transcoder;

protected function setUp()
protected function doSetUp(): void
{
$this->transcoder = new IconvTranscoder();
// Passing null (empty encoding name) to iconv makes it detect encoding from locale.
// The phpunit-bridge sets locale to C for consistency but that implies ASCII.
// This file uses UTF-8 so we have to set the locale accordingly.
$this->setLocale(\LC_ALL, 'C.UTF-8');
}

/**
* @expectedException \Ddeboer\Transcoder\Exception\UnsupportedEncodingException
* @expectedExceptionMessage bad-encoding
*/
public function testTranscodeUnsupportedFromEncoding()

public function testTranscodeUnsupportedFromEncoding(): void
{
$this->expectException(\Ddeboer\Transcoder\Exception\UnsupportedEncodingException::class);
$this->expectExceptionMessage('bad-encoding');
$this->transcoder->transcode('bla', 'bad-encoding');
}
public function testDetectEncoding()

public function testDetectEncoding(): void
{
$this->transcoder->transcode('España', null, 'iso-8859-1');
$isoLatin1 = $this->transcoder->transcode('España', null, 'iso-8859-1');
$this->assertEquals($isoLatin1, "Espa\xf1a");
}

/**
* @expectedException \Ddeboer\Transcoder\Exception\IllegalCharacterException
*/
public function testTranscodeIllegalCharacter()
public function testTranscodeIllegalCharacter(): void
{
$this->transcoder->transcode('', null, 'iso-8859-1');
$this->expectException(\Ddeboer\Transcoder\Exception\IllegalCharacterException::class);
$this->transcoder->transcode('', 'utf-8', 'iso-8859-1');
}

/**
* @dataProvider getStrings
*/
public function testTranscode($string, $encoding)
public function testTranscode(string $string, string $encoding): void
{
$result = $this->transcoder->transcode($string, null, $encoding);
$result = $this->transcoder->transcode($string, 'utf-8', $encoding);
$this->assertEquals($string, $this->transcoder->transcode($result, $encoding));
}

public function getStrings()

/** @return iterable<array{string, string}> */
public function getStrings(): iterable
{
return [
['España', 'iso-8859-1']
];
yield ['España', 'iso-8859-1'];
}
}
72 changes: 38 additions & 34 deletions tests/MbTranscoderTest.php
Original file line number Diff line number Diff line change
@@ -1,53 +1,58 @@
<?php

declare(strict_types=1);

namespace Ddeboer\Transcoder\Tests;

use Ddeboer\Transcoder\MbTranscoder;

class MbTranscoderTest extends \PHPUnit_Framework_TestCase
class MbTranscoderTest extends \PHPUnit\Framework\TestCase
{
private MbTranscoder $transcoder;

/**
* @var MbTranscoder
* @before
*/
private $transcoder;

protected function setUp()
protected function doSetUp(): void
{
$this->transcoder = new MbTranscoder();
}

/**
* @expectedException \Ddeboer\Transcoder\Exception\UnsupportedEncodingException
* @expectedExceptionMessage bad-encoding
*/
public function testTranscodeUnsupportedFromEncoding()
public function testTranscodeUnsupportedFromEncoding(): void
{
$this->expectException(\Ddeboer\Transcoder\Exception\UnsupportedEncodingException::class);
$this->expectExceptionMessage('bad-encoding');
$this->transcoder->transcode('bla', 'bad-encoding');
}

/**
* @expectedException \Ddeboer\Transcoder\Exception\UnsupportedEncodingException
* @expectedExceptionMessage bad-encoding
*/
public function testTranscodeUnsupportedToEncoding()
public function testTranscodeUnsupportedToEncoding(): void
{
$this->transcoder->transcode('bla', null, 'bad-encoding');
$this->expectException(\Ddeboer\Transcoder\Exception\UnsupportedEncodingException::class);
$this->expectExceptionMessage('bad-encoding');
$this->transcoder->transcode('bla', 'utf-8', 'bad-encoding');
}
public function testDetectEncoding()

public function testDetectEncoding(): void
{
$result = $this->transcoder->transcode('España', null, 'iso-8859-1');
$this->transcoder->transcode($result);
$oldLanguage = ini_get('mbstring.language');
assert($oldLanguage !== false);
ini_set('mbstring.language', 'ru');

$utf8 = 'пирожки';
$this->assertEquals($utf8, $this->transcoder->transcode($utf8));

$koi8r = $this->transcoder->transcode($utf8, 'utf-8', 'koi8-r');
$this->assertEquals($utf8, $this->transcoder->transcode($koi8r));

ini_set('mbstring.language', $oldLanguage);
}

/**
* @expectedException \Ddeboer\Transcoder\Exception\UndetectableEncodingException
* @expectedExceptionMessage is undetectable
*/
public function testUndetectableEncoding()

public function testUndetectableEncoding(): void
{
$this->expectException(\Ddeboer\Transcoder\Exception\UndetectableEncodingException::class);
$this->expectExceptionMessage('is undetectable');
$result = $this->transcoder->transcode(
'‘curly quotes make this incompatible with 1252',
'Windows-1252 encodes curly quotes as 0x91 and 0x92, which are indistinguishable from any other single-byte encoding',
null,
'windows-1252'
);
@@ -57,17 +62,16 @@ public function testUndetectableEncoding()
/**
* @dataProvider getStrings
*/
public function testTranscode($string, $encoding)
public function testTranscode(string $string, string $encoding): void
{
$result = $this->transcoder->transcode($string, null, $encoding);
$this->assertEquals($string, $this->transcoder->transcode($result, $encoding));
}

public function getStrings()

/** @return iterable<array{string, string}> */
public function getStrings(): iterable
{
return [
['‘España’', 'windows-1252'],
['España', 'iso-8859-1']
];
yield ['‘España’', 'windows-1252'];
yield ['España', 'iso-8859-1'];
}
}
26 changes: 14 additions & 12 deletions tests/TranscoderTest.php
Original file line number Diff line number Diff line change
@@ -1,35 +1,37 @@
<?php

declare(strict_types=1);

namespace Ddeboer\Transcoder\Tests;

use Ddeboer\Transcoder\Transcoder;
use Ddeboer\Transcoder\TranscoderInterface;

class TranscoderTest extends \PHPUnit_Framework_TestCase
class TranscoderTest extends \PHPUnit\Framework\TestCase
{
private TranscoderInterface $transcoder;

/**
* @var Transcoder
* @before
*/
private $transcoder;

protected function setUp()
protected function doSetUp(): void
{
$this->transcoder = Transcoder::create();
}

/**
* @dataProvider getStrings
*/
public function testTranscode($string, $encoding)
public function testTranscode(string $string, string $encoding): void
{
$result = $this->transcoder->transcode($string, 'UTF-8', $encoding);
$this->assertEquals($string, $this->transcoder->transcode($result, $encoding));
}

public function getStrings()

/** @return iterable<array{string, string}> */
public function getStrings(): iterable
{
return [
['España', 'UTF-8'],
['bla', 'windows-1257'] // Encoding only supported by iconv
];
yield ['España', 'UTF-8'];
yield ['bla', 'windows-1257']; // Encoding only supported by iconv
}
}