Add PHP client #10

stevenay · 2018-05-29T18:37:11Z

I've been implementing PHP client for myanmar-tools. It is fully tested and already built on travis.

PHP client is targeted to PHP ver 7.0 as minimum because we want type declarations to method arguments and return type. (http://php.net/manual/en/migration70.new-features.php)

As one notice, PHP natural system of logarithms calculation result is a little bit different from the Standard one (e.g. Java version that I took referenced). That's why I've to add very very small delta value (0.00000001) in testing compatibility when implementing test cases.

For the demo code, I haven't added to the samples folder because the composer package for php-client is yet uploaded to packagist. I'm now planning to upload to the packagist after the code has been approved and merged.

Thanks.

googlebot · 2018-05-29T18:37:15Z

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here (e.g. I signed it!) and we'll verify it.

What to do if you already signed the CLA

Individual signers

It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
The email used to register you as an authorized contributor must also be attached to your GitHub account.

stevenay · 2018-05-29T18:42:27Z

I signed it CLA now.

googlebot · 2018-05-29T18:42:32Z

CLAs look good, thanks!

sffc

Thank you for the contribution! Please see my suggestions. Let me know when you have updated your branch.

sffc · 2018-05-30T00:43:00Z

.gitignore

+.idea
+.idea/workspace.xml
+.idea/php.xml
+clients/php/vendor


Please create a new .gitignore file in clients/php/.gitignore and add these two files there, vendor and composer.lock.

For the .idea stuff, you should be able to exclude the whole directory via ".idea/", correct?

sffc · 2018-05-30T00:44:58Z

.travis.yml

@@ -60,3 +60,15 @@ matrix:
        - bundle install
      script:
        - rake test
+
+    # PHP Client


Thanks for updating travis.yml. May as well keep it in alphabetical order, though: move this up between JavaScript and Ruby.

sffc · 2018-05-30T01:08:03Z

clients/php/src/BinaryMarkov.php

+
+        for ($i1 = 0; $i1 < $size; $i1++) {
+            $entries  = unpack('n*', fread($stream, 2))[1];
+            $fallback = ($entries == 0) ? 0.0 : round(unpack("Gnum", fread($stream, 4))['num'], 8);


How about unpack("G*", $readData)[1] like the other call sites? Also, I don't think you need the round. Could you change this line to:

$fallback = ($entries == 0) ? 0.0 : unpack("G*", fread($stream, 4))[1];

sffc · 2018-05-30T01:17:17Z

clients/php/src/BinaryMarkov.php

+
+                if ($next == $i2) {
+                    $readData = fread($stream, 4);
+                    $logProbabilityDifferences[$i1][$i2] = round(unpack("Gnum", $readData)['num'], 8);


Same here: please simplify the unpack and remove the round.

sffc · 2018-05-30T01:17:50Z

clients/php/src/BinaryMarkov.php

+    }
+
+    // Deprecated
+    public static function hex2Float($strHex) {


Delete this?

sffc · 2018-05-30T01:36:18Z

clients/php/src/ZawgyiDetector.php

+            fclose($inStream);
+
+        } catch (Exception $exception) {
+            throw new Exception("Could not load Markov model from resource file");


Please change this to

throw new Exception("Could not load Markov model from resource file", 0, $exception);

sffc · 2018-05-30T01:56:47Z

clients/php/src/ZawgyiUnicodeMarkovModel.php

+    }
+
+    public static function ord_utf8($c)
+    {


Please do this instead:

public static function ord_utf8($c) { if (empty($c)) return; $u32 = mb_convert_encoding($c, "UTF-32BE"); return unpack("N", $u32)[1]; }

Or, if you are okay adding a dependency on the Intl extension, you can do this:

use IntlChar; // ... public static function ord_utf8($c) { return IntlChar::ord($c); }

sffc · 2018-05-30T02:01:37Z

clients/php/src/ZawgyiUnicodeMarkovModel.php

+                $seenTransition = true;
+            }
+
+            $offset    += self::charCount($cp);


Since you are using mb_substr, this should be just

$offset += 1;

sffc · 2018-05-30T02:03:02Z

clients/php/src/ZawgyiUnicodeMarkovModel.php

+    # Determine the number of char values needed to represent the specified character (Unicode code point)
+    # if the code point is equal to or greater than 0x10000, then the method returns 2.
+    # otherwise, the method returns 1.
+    public static function charCount($codePoint)


This is for UTF-16 handling in Java and JavaScript. It is irrelevant in PHP, which is UTF-8. Please delete this function.

sffc · 2018-05-30T02:06:30Z

clients/php/tests/ZawgyiDetectorTest.php

+                $actual = $this->detector->getZawgyiProbability($data[1]);
+                // Since php natural logarithmic calculation (exp()) is a little bit far from exact value
+                // we have to put very very small delta point
+                $delta = 0.00000001;


After you remove the round on line 71 of BinaryMarkov.php, try removing the delta here. That might be the problem. Otherwise, it is fine if you need to keep the delta here.

…f removing round function

stevenay · 2018-05-30T05:23:36Z

Thanks you so much for your insightful reviews, Shane @sffc . Fixing the code is finished. Let me know if there is something that I should fix.

Thanks.

…file

stevenay · 2018-05-31T15:38:32Z

I would like to add a side note about Packagist and Composer.

To upload to Packagist and be downloadable by composer, we have some limitations. Packagist does not support to link to subdirectory of the github repo. Other projects have that problem too. We can see the discussion here.

To solve this problem, I can think of 3 ways according to my knowledge.
The first one is to create another separate repo for PHP version like this Google project does. It only keep Readme file under Php directory in the main repo. It is the best way and easy to maintain in the long run.

The second approach is to move composer.json to the root of the project like this Google project does. But there is one awful downside that the composer will also download all the unnecessary files to the user project. In our case, Java, Ruby, Javascript and other language folders will also be downloaded into the user's PHP project. I think it is a dirty way.

The third approach is to keep another branch for PHP client. In this approach, we will not upload to Packagist. Instead of it, the users have to add repositories definition in their composer.json to link to our github repo like the following.

"repositories": [
        {
            "type": "package",
            "package": {
                "name": "googlei18n/myanmar-tools",
                "version": "v0.0.4",
                "source": {
                    "url": "https://github.com/stevenay/myanmar-tools.git",
                    "type": "git",
                    "reference": "php-client"
                },
                "autoload": {
                    "psr-4": {
                        "Googlei18n\\MyanmarTools\\": "clients/php/src"
                    }
                }
            }
        }
    ],

But in this approach, the composer will not read the composer.json file of the library. As a result, it will not generate autoload for the library. So we have to explicitly add autoload clause in our repositories definition. People also think it is harder to maintain.

Would like to say sorry that to add PHP-Client is a little bothersome for the library maintainers.

sffc · 2018-06-03T00:03:10Z

It is unfortunate that Composer has that limitation. This is Composer's problem, and we should not make it our problem.

I lean toward your option 2. Some extra junk will get bundled in the Composer package, but again, that's their problem, not ours, and it might give Composer an incentive to fix their system to allow subdirectory projects. Also, the project is less than 500 KB right now, which really isn't that much overhead. I do not want the extra maintenance cost associated with your options 1 and 3.

You can find much more discussion about the advantages of monorepos here:

www.google.com/search?q=monorepo

stevenay · 2018-06-04T13:51:47Z

It's ok, @sffc. I agree with your suggestion. So I've updated the repo for this approach:

Move composer.json to the root of the project.
Update travis.yml and Makefile by removing cd statement and add phpunit.xml test config file explicitly in the test command.
Update .gitignore file to reflect updated paths.

- Fixing make test for PHP - composer.lock is intended to be in source control - Platform-agnostic file separators

sffc · 2018-06-04T22:58:37Z

Thanks for the contribution! I merged it and made a few more minor changes in 7624e17 and 0e061f1.

Adds PHP client. Includes a composer.json file at the top level, required for limitations in Composer's package management system.

- Fixing make test for PHP - composer.lock is intended to be in source control - Platform-agnostic file separators

stevenay added 9 commits May 29, 2018 22:48

[add] php client and configurations

78b938f

[update] remove vendor package and update gitingore

676c936

[update] travis script

592c457

[update] travis reduce PHP version

b9e3d0f

[update] directory separator updated for linux env

92f8f92

[update] delete testing php files and clean the comments

fbd64f6

[update] readme to describe pending composer install

aa42e33

[update] travis.yml comment

869e61c

[update] ignore .idea IDE metadata folder

1423505

sffc requested changes May 30, 2018

View reviewed changes

stevenay added 10 commits May 30, 2018 10:18

[update] specific gitIngore for PHP client

24c288f

[update] order travis in alphabetical order

9966d2c

[update] Gnum to G* to be consistent and remove round

781224c

[remove] remove delta value and testCompatibility works now because o…

d0c8264

…f removing round function

[remove] remove deprecated function

485651f

[update] substitue bin2hex with unpack and make consistent

0231f68

[update] remove mb_internal_encoding and move to test

0a11a6e

[update] throw nested excpetion

b75ed2f

[update] modify ord_utf8 function to be compact

7d19f62

[remove] delete invalid charCount function

3922e35

[update] added DIR to make more precise directory open training data …

e58d814

…file

stevenay added 3 commits June 4, 2018 13:13

[update] composer.json and other build files to the project root

7e928c4

[add] add sample file

b33c631

[update] changes to reflect composer vendor folder at project root

0180f3a

[update] add "myanmar-tools" keyword to composer

21768d4

sffc approved these changes Jun 4, 2018

View reviewed changes

sffc merged commit d7d9266 into google:master Jun 4, 2018

sffc added a commit that referenced this pull request Jun 4, 2018

Minor changes after merge of #10:

7624e17

- Fixing make test for PHP - composer.lock is intended to be in source control - Platform-agnostic file separators

sffc added a commit that referenced this pull request Jul 9, 2018

Merge pull request #10 from stevenay/master

0cad406

Adds PHP client. Includes a composer.json file at the top level, required for limitations in Composer's package management system.

sffc added a commit that referenced this pull request Jul 9, 2018

Minor changes after merge of #10:

bfad4bc

- Fixing make test for PHP - composer.lock is intended to be in source control - Platform-agnostic file separators

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add PHP client #10

Add PHP client #10

stevenay commented May 29, 2018 •

edited

Loading

googlebot commented May 29, 2018

stevenay commented May 29, 2018

googlebot commented May 29, 2018

sffc left a comment

sffc May 30, 2018

sffc May 30, 2018

sffc May 30, 2018

sffc May 30, 2018

sffc May 30, 2018

sffc May 30, 2018

sffc May 30, 2018

sffc May 30, 2018

sffc May 30, 2018

sffc May 30, 2018

stevenay commented May 30, 2018 •

edited

Loading

stevenay commented May 31, 2018 •

edited

Loading

sffc commented Jun 3, 2018

stevenay commented Jun 4, 2018

sffc commented Jun 4, 2018 •

edited

Loading

Add PHP client #10

Add PHP client #10

Conversation

stevenay commented May 29, 2018 • edited Loading

googlebot commented May 29, 2018

What to do if you already signed the CLA

Individual signers

Corporate signers

stevenay commented May 29, 2018

googlebot commented May 29, 2018

sffc left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stevenay commented May 30, 2018 • edited Loading

stevenay commented May 31, 2018 • edited Loading

sffc commented Jun 3, 2018

stevenay commented Jun 4, 2018

sffc commented Jun 4, 2018 • edited Loading

stevenay commented May 29, 2018 •

edited

Loading

stevenay commented May 30, 2018 •

edited

Loading

stevenay commented May 31, 2018 •

edited

Loading

sffc commented Jun 4, 2018 •

edited

Loading