Use zf1s packages to gain php 7.2 compatibility

[hairmare]

This PR replaces the zend/zendframework1 dependency with composer packages from the zf1s orga. The zf1s packages already contain loads of fixes to allow zend framework 1 to run on php 7.2 and the maintainer is willing to merge more PHP 7.2 compat changes as needed. The switch to modularized zf1 packages will also make replacing zf1 piece by piece easier if we decide to go down that route.

This will allow us to run LibreTime on Ubuntu Bionic as well as the upcoming CentOS 8 which both bump PHP to 7.2.

As mentioned in #580 (comment) I've tested this on Ubuntu 18.04 (Bionic Beaver) as well as Ubuntu 14.04 (Trusty Thar). I checked that the program grid, uploading tracks, basic playout and, podcast downloading works.

This is a rather large change and as such I'd like to ask everyone to do some serious testing with this change to ensure that all or your use cases still work. If you find any issues, please post the output of php --version as well as the full error message and stacktrace from either Apaches error.log or the zendphp.log and I'll fix them in this PR. To be sure we don't break any existing use cases there also needs to be some testing on the old still supported PHP 5.x versions.

Fixes #580

[frecuencialibre]

modern distro support is going to be huge. 👍 👍 👍

This is a rather large change and as such I'd like to ask everyone to do some serious testing with this change to ensure that all or your use cases still work.

does a list of use cases to test exist anywhere?

[Robbt]

I don't believe there is a list of test cases. In theory we would have tests written with Selenium that would automatically test various functionalities but the existing selenium scripts don't appear to be working. Other than just trying to click around and see if everything works I don't think anyone has developed a specific plan. I'll try to play around with it and maybe I can play around with selenium testing but my last attempt didn't make it very far.

[paddatrapper]

Do we have a dev setup guide lying around somewhere? Preferably for someone who has never used Vagrant before...

[Robbt]

No guide in particular but sudo apt install vagrant and then pull down this PR and vagrant up ubuntu-bionic and I think it'll work. I've ran into times when I needed to install a newer version of vagrant than my distro supported out of the box but it's pretty straight forward. Writing some docs would be a good idea as well.

[paddatrapper]

Thanks. I'll try that and add some docs to the wiki

[Robbt]

So I just tried this via vagrant up ubuntu-bionic and I got the Rest error with z1fs
2019-01-03T20:50:47+00:00 ERR (3): localhost [Logging.php:166 - loggingShutdownCallback()] - Declaration of Zend_Rest_Route::assemble($data = Array, $reset = false, $encode = true) must be compatible with Zend_Controller_Router_Route_Module::assemble($data = Array, $reset = false, $encode = true, $partial = false) at /vagrant/vendor/zf1s/zend-rest/library/Zend/Rest/Route.php:52 array ( 'type' => 64, 'message' => 'Declaration of Zend_Rest_Route::assemble($data = Array, $reset = false, $encode = true) must be compatible with Zend_Controller_Router_Route_Module::assemble($data = Array, $reset = false, $encode = true, $partial = false)', 'file' => '/vagrant/vendor/zf1s/zend-rest/library/Zend/Rest/Route.php',

I think that is related to the change you merged with zf1s but for some reason it isn't applied upon installation. I'll look in further. Using PHP 7.2.10-0ubuntu0.18.04.1 (cli) (built: Sep 13 2018 13:45:02) ( NTS )

So it looks like line in composer.json
"zf1s/zend-rest": "dev-master@dev",
isn't pulling the code with the PR merged.

Update:
after going into /vagrant/composer and running sudo composer update it pulled in the latest zf1s rest commit and the site is working.

Not sure why it is requiring this step.

[hairmare]

OK, I'll rerun the composer update and repush the results... I should find some time tomorrow, for then please run it to test, that reflects what we should be comitting anyway.

[Robbt]

Yeah so far the UI seems pretty functional after the update. I've been able to add podcasts and create smartblocks.

[hairmare]

@paddatrapper There are some docs on the vagrant setup at http://libretime.org/vagrant/ maybe we should merge those with your efforts.

[paddatrapper]

@hairmare I will create a MR with any changes I have to that instruction list then

[hairmare]

Looking at your wiki page adding something about the --provider arg might make sense. I've been wanting to start switching to base boxes that also support --provider=libvirt for a while now but never got around to pushing the change.

[paddatrapper]

@hairmare - tracking it in #674 so we don't pollute this PR.

On that note - I haven't found any issues with PHP 7.2 testing this PR after applying the same fix @Robbt applied.

[hairmare]

I just pushed the composer update fix. I had to run composer update 'zf1s/*' to ensure that phpunit wasn't bumped to a version that breaks travis.

[frecuencialibre]

i just checked out radiorabe:dev/zf1s and ran vagrant up with no args from the working directory. setup failed with this error:

    ubuntu-bionic: Do not run Composer as root/super user! See https://getcomposer.org/root fordetails
    ubuntu-bionic: Loading composer repositories with package information
    ubuntu-bionic: Installing dependencies from lock file
    ubuntu-bionic: Package operations: 31 installs, 0 updates, 1 removal
    ubuntu-bionic:   - Removing zendframework/zendframework1 (1.12.20)
    ubuntu-bionic:
    ubuntu-bionic:   [RuntimeException]
    ubuntu-bionic:   Could not delete /vagrant/vendor/zendframework/zendframework1/library/Zend/OpenId/Consumer/Storage/File.php:
    ubuntu-bionic:
    ubuntu-bionic:
    ubuntu-bionic: install [--prefer-source] [--prefer-dist] [--dry-run] [--dev] [--no-dev] [--no-custom-installers] [--no-autoloader] [--no-scripts] [--no-progress] [--no-suggest] [-v|vv|vvv|--verbose] [-o|--optimize-autoloader] [-a|--classmap-authoritative] [--apcu-autoloader] [--ignore-platform-reqs] [--] [<packages>]...
The SSH command responded with a non-zero exit status. Vagrant
assumes that this means the command failed. The output for this command
should be in the log above. Please read the output to determine what
went wrong.

[frecuencialibre]

update: i got past this one with

sudo chown -R myuser:mygroup python_apps
sudo vagrant provision

had some files in there that were owned by root. not sure why. now continuing troubleshoot:

    ubuntu-bionic: -----------------------------------------------------
    ubuntu-bionic:                 * Basic Setup DONE! *                
    ubuntu-bionic:                                                      
    ubuntu-bionic:  To get started with Airtime, visit             
    ubuntu-bionic:  or, if you've set up your own web configuration,    
    ubuntu-bionic:  the Airtime webroot on your webserver               
    ubuntu-bionic: -----------------------------------------------------
==> ubuntu-bionic: Running provisioner: install-mkdocs (shell)...
    ubuntu-bionic: Running: /tmp/vagrant-shell20190110-14886-1o0md0b.sh
    ubuntu-bionic: Updating Apt.
    ubuntu-bionic: Ensuring Pip is installed.
    ubuntu-bionic: Updating Pip.
    ubuntu-bionic: Ensuring Mkdocs is installed.
    ubuntu-bionic: Traceback (most recent call last):
    ubuntu-bionic:   File "/usr/bin/pip", line 9, in <module>
    ubuntu-bionic:     
    ubuntu-bionic: from pip import main
    ubuntu-bionic: ImportError
    ubuntu-bionic: : 
    ubuntu-bionic: cannot import name main

[paddatrapper]

LibreTime should be up and running now on localhost:8080. I also can't get the docs build to work on anything, but it isn't required to test this

[frecuencialibre]

oh, sure enough!!

http://localhost:8080

$ vagrant ssh ubuntu-bionic
vagrant@vagrant:/var/log$ php --version
PHP 7.2.10-0ubuntu0.18.04.1 (cli) (built: Sep 13 2018 13:45:02) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.2.0, Copyright (c) 1998-2018 Zend Technologies
    with Zend OPcache v7.2.10-0ubuntu0.18.04.1, Copyright (c) 1999-2018, by Zend Technologies

thanks! checking use cases now

[frecuencialibre]

works great!!!

i checked: track import, playlist creation/editing, webstream creation, podcast creation/edit, calendar manipulation, master source connection, stream settigns changes, user creation/edit, user assign to show, login/logout, user settings edit, set autoloading playlist.

i obviously haven't clicked every single thing, but the only functionality areas i haven't tested are those that are scheduled, like podcast download, and autoloading playlists.

[frecuencialibre]

update: an autoloading playlist i scheduled worked, as did automatic download of a podcast. i think we're ready to go!

[Robbt]

What do we want to do before we push this ?

[paddatrapper]

I don't think there is anything else required to merge this. Are there any doc related things that need updating to reflect new php support? The support matrix in the wiki at least will need changes to Debian Buster and Ubuntu Bionic

[hairmare]

I'd like to do the 3.0.0-alpha.7 release pretty soon after we merge this as it will expose it to a larger audience. I think the milestone covers what's still open for that to happen.

When updating in-situ with a vagrant install from master you might need to remove the vendor for everything to work out. Releases shouldn't be affected by this issue.

If we want to I can update this to use

    locale-gen "en_US.UTF-8"
    echo -e "LC_ALL=en_US.UTF-8\nLANG=en_US.UTF-8" >> /etc/default/locale

as the dirty hack for ensuring a proper locale on Ubuntu Bionic. I'm assuming we would want to push some fixes for the locale issue before we announce official Bionic and Buster support. Maybe we can figure out how to get rid of the hack completely. Checking Buster so see if it's also affected by the PostgreSQL issue/hack could also be of interest.

[paddatrapper]

I'm happy to test the Buster postgreSQL locale hack on Monday.

The rest of the milestone is pretty much dependant on this PR though. I can't really test the install without the new zf1s dependencies fixing the Vagrant build

[paddatrapper]

It seems that Buster install works fine without the hack, but locales are generated in the installer later on here

Running this on Buster using PHP 7.3, I get the following error:

compact(): Undefined variable: extras#0 [internal function]: exception_error_handler(8, 'compact(): Unde...', '/vagrant/vendor...', 404, Array) #1 /vagrant/vendor/zf1s/zend-view/library/Zend/View/Helper/HeadLink.php(404): compact('rel', 'type', 'href', 'media', 'conditionalStyl...', 'extras') #2 /vagrant/vendor/zf1s/zend-view/library/Zend/View/Helper/HeadLink.php(172): Zend_View_Helper_HeadLink->createDataStylesheet(Array) #3 /vagrant/airtime_mvc/application/controllers/ErrorController.php(68): Zend_View_Helper_HeadLink->__call('appendStyleshee...', Array) #4 /vagrant/airtime_mvc/application/controllers/ErrorController.php(9): ErrorController->setupCSS() #5 /vagrant/vendor/zf1s/zend-controller/library/Zend/Controller/Action.php(133): ErrorController->init() #6 /vagrant/vendor/zf1s/zend-controller/library/Zend/Controller/Dispatcher/Standard.php(281): Zend_Controller_Action->__construct(Object(Zend_Controller_Request_Http), Object(Zend_Controller_Response_Http), Array) #7 /vagrant/vendor/zf1s/zend-controller/library/Zend/Controller/Front.php(954): Zend_Controller_Dispatcher_Standard->dispatch(Object(Zend_Controller_Request_Http), Object(Zend_Controller_Response_Http)) #8 /vagrant/vendor/zf1s/zend-application/library/Zend/Application/Bootstrap/Bootstrap.php(105): Zend_Controller_Front->dispatch() #9 /vagrant/vendor/zf1s/zend-application/library/Zend/Application.php(384): Zend_Application_Bootstrap_Bootstrap->run() #10 /vagrant/airtime_mvc/application/airtime-boot.php(84): Zend_Application->run() #11 /vagrant/airtime_mvc/public/index.php(68): require_once('/vagrant/airtim...') #12 {main}

#0 /vagrant/vendor/zf1s/zend-controller/library/Zend/Controller/Front.php(965): Zend_Controller_Plugin_Broker->postDispatch(Object(Zend_Controller_Request_Http))
#1 /vagrant/vendor/zf1s/zend-application/library/Zend/Application/Bootstrap/Bootstrap.php(105): Zend_Controller_Front->dispatch()
#2 /vagrant/vendor/zf1s/zend-application/library/Zend/Application.php(384): Zend_Application_Bootstrap_Bootstrap->run()
#3 /vagrant/airtime_mvc/application/airtime-boot.php(84): Zend_Application->run()
#4 /vagrant/airtime_mvc/public/index.php(68): require_once('/vagrant/airtim...')
#5 {main}

[hairmare]

PHP has this in their docs

Before PHP 7.3, any strings that are not set will silently be skipped.

So it looks like we'll need to fix some more stuff before we're ready for 7.3...

Most likely initializing $extras to array() before the following if should be enough...

https://github.com/zf1s/zend-view/blob/61363f26f90ac3b92a11b40d5085130b878a3000/library/Zend/View/Helper/HeadLink.php#L399-L404

I'll look into taking care of this later this week.

[hairmare]

I rebased this onto master, removed the locale hack and, force pushed it.

My rationale for removing the locale hack is that we need to look into it and it shouldn't be introduced as part of this pr.

This only supports up to php 7.2 for now but I think we should merge it soonish rather than wait for php 7.3 support. 7.3 is in the pipeline and I'll make more update PRs when the needed fixes are here.

Let me know if I should update anything else so we can get this merged.

[Robbt]

Sounds good to me. I'm willing to go ahead with the merge.

[frecuencialibre]

i already tested functionality following this change as systematically as i reasonably could and it worked great. i won't be able to test the most recent vagrant changes until later this week, and know Robbt is deep in another problem today. @paddatrapper to the rescue? happy rainy sunday folks, take care

[Robbt]

It started up fine with vagrant so I'm going to merge it.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please chat to us on discourse or ask for help on our chat if you have any questions or need further assistance with this issue.