Merge cc844fd into 5dabd47

.travis.yml

@@ -20,10 +20,10 @@ env:
   - TEST_PHP_SQL_PWD=Password123
 
 before_install:
-  - docker pull microsoft/mssql-server-linux:2017-latest
+  - docker pull mcr.microsoft.com/mssql/server:2019-CTP2.4-ubuntu
 
 install:
-  - docker run -e 'ACCEPT_EULA=Y' -e 'SA_PASSWORD=Password123' -p 1433:1433 --name=$TEST_PHP_SQL_SERVER -d microsoft/mssql-server-linux:2017-latest
+  - docker run -e 'ACCEPT_EULA=Y' -e 'SA_PASSWORD=Password123' -p 1433:1433 --name=$TEST_PHP_SQL_SERVER -d mcr.microsoft.com/mssql/server:2019-CTP2.4-ubuntu
   - docker build --build-arg PHPSQLDIR=$PHPSQLDIR -t msphpsql-dev -f Dockerfile-msphpsql .
 
 before_script:
@@ -43,7 +43,7 @@ script:
   - docker exec client bash -c 'for f in ./test/functional/pdo_sqlsrv/*.out; do ls $f 2>/dev/null; cat $f 2>/dev/null; done || true' 
   - docker exec client python ./test/functional/setup/cleanup_dbs.py -dbname $SQLSRV_DBNAME
   - docker exec client python ./test/functional/setup/cleanup_dbs.py -dbname $PDOSQLSRV_DBNAME
-  - docker exec client coveralls -e ./source/shared/ --gcov-options '\-lp'
+  - docker exec client coveralls -i ./source/ -e ./source/shared/ -e ./test/ --gcov-options '\-lp'
   - docker stop client
   - docker ps -a 
 

CHANGELOG.md

@@ -3,6 +3,50 @@ All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](http://keepachangelog.com/)
 
+## 5.7.0-preview - 2019-09-05
+Updated PECL release packages. Here is the list of updates:
+
+### Added
+- Support for PHP 7.4 RC 1
+- Support for Linux Ubuntu 19.04 and Debian 10
+- Feature Request [#929](https://github.com/microsoft/msphpsql/issues/929) - new [Language option](https://github.com/microsoft/msphpsql/wiki/Features#language) - Pull Request [#930](https://github.com/microsoft/msphpsql/pull/930)
+- [Data Classification Sensitivity Metadata Retrieval](https://github.com/microsoft/msphpsql/wiki/Features#data-classification-sensitivity-metadata), which requires [MS ODBC Driver 17.2+](https://docs.microsoft.com/sql/connect/odbc/linux-mac/installing-the-microsoft-odbc-driver-for-sql-server) and [SQL Server 2019 release candidate](https://docs.microsoft.com/sql/sql-server/sql-server-ver15-release-notes?view=sqlallproducts-allversions#-release-candidate-rc)
+
+### Removed
+- Dropped support for Ubuntu 18.10
+
+### Fixed
+- Issue [#570](https://github.com/microsoft/msphpsql/issues/570) - Fixed fetching varbinary data using client buffer with sqlsrv
+- Pull Request [#972](https://github.com/microsoft/msphpsql/pull/972) - Removed redundant calls to retrieve the number of columns or rows in the current query result set
+- Pull Request [#978](https://github.com/microsoft/msphpsql/pull/978) - PDO_SQLSRV implementation of PDO::getColumnMeta now references cached metadata rather than making an ODBC call every time
+- Pull Request [#979](https://github.com/microsoft/msphpsql/pull/979) - Added support for data classification Sensitivity metadata retrieval
+- Pull Request [#985](https://github.com/microsoft/msphpsql/pull/985) - Fixed memory issues with data classification data structures
+- Issue [#432](https://github.com/microsoft/msphpsql/issues/432) - Having any invalid UTF-8 name in the connection string will no longer invoke misleading error messages
+- Issue [#909](https://github.com/microsoft/msphpsql/issues/909) - Fixed potential exception with locale issues in macOS 
+- Pull Request [#992](https://github.com/microsoft/msphpsql/pull/992) - Produced the correct error when requesting Data Classification metadata with ODBC drivers prior to 17
+- Pull Request [#1001](https://github.com/microsoft/msphpsql/pull/1001) - Fixed compilation issue with PHP 7.4 alpha
+- Pull Request [#1004](https://github.com/microsoft/msphpsql/pull/1004) - Fixed another compilation issue with PHP 7.4 alpha
+- Pull Request [#1008](https://github.com/microsoft/msphpsql/pull/1008) - Improved data caching when fetching datetime objects
+- Pull Request [#1011](https://github.com/microsoft/msphpsql/pull/1011) - Fixed a potential buffer overflow when parsing for escaped braces in the connection string
+- Pull Request [#1015](https://github.com/microsoft/msphpsql/pull/1015) - Fixed compilation issues and addressed various memory leaks detected by PHP 7.4 beta 1 
+
+### Limitations
+- No support for inout / output params when using sql_variant type
+- No support for inout / output params when formatting decimal values
+- In Linux and macOS, setlocale() only takes effect if it is invoked before the first connection. Attempting to set the locale after connecting will not work
+- Always Encrypted requires [MS ODBC Driver 17+](https://docs.microsoft.com/sql/connect/odbc/linux-mac/installing-the-microsoft-odbc-driver-for-sql-server)
+  - Only Windows Certificate Store and Azure Key Vault are supported. Custom Keystores are not yet supported
+  - Issue [#716](https://github.com/Microsoft/msphpsql/issues/716) - With Always Encrypted enabled, named parameters in subqueries are not supported
+  - [Always Encrypted limitations](https://docs.microsoft.com/sql/connect/php/using-always-encrypted-php-drivers#limitations-of-the-php-drivers-when-using-always-encrypted)
+
+### Known Issues
+- Data Classification metadata retrieval is not compatible with ODBC Driver 17.4.1
+- Connection pooling on Linux or macOS is not recommended with [unixODBC](http://www.unixodbc.org/) < 2.3.7
+- When pooling is enabled in Linux or macOS
+  - unixODBC <= 2.3.4 (Linux and macOS) might not return proper diagnostic information, such as error messages, warnings and informative messages
+  - due to this unixODBC bug, fetch large data (such as xml, binary) as streams as a workaround. See the examples [here](https://github.com/Microsoft/msphpsql/wiki/Features#pooling)
+- With ColumnEncryption enabled, calling stored procedures with XML parameters does not work (Issue [#674](https://github.com/Microsoft/msphpsql/issues/674))
+
 ## 5.6.1 - 2019-03-19
 Updated PECL release packages. Here is the list of updates:
 

Dockerfile-msphpsql

@@ -1,34 +1,44 @@
-#Download base image ubuntu 16.04
+# Download base image ubuntu 18.04
 
-FROM ubuntu:16.04
+FROM ubuntu:18.04
 
 # Update Ubuntu Software repository
 RUN export DEBIAN_FRONTEND=noninteractive && apt-get update && \
+    apt-get install -y software-properties-common && \
+    add-apt-repository ppa:ondrej/php -y && \
     apt-get -y install \
-    apt-transport-https     \
+    apt-transport-https \
     apt-utils \
     autoconf \
     curl \
-    libcurl3 \
+    libcurl4 \
     g++ \
     gcc    \
     git \
     lcov \
     libxml2-dev \
     locales \
     make \
-    php7.0 \
-    php7.0-dev \
+    php7.3 \
+    php7.3-dev \
     python-pip \
     re2c \
     unixodbc-dev \
-    unzip && apt-get clean
-
+    unzip && apt-get clean && \
+    curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add - && \
+    curl https://packages.microsoft.com/config/ubuntu/18.04/prod.list > /etc/apt/sources.list.d/mssql-release.list && \
+    apt-get -y update && \
+    export ACCEPT_EULA=Y && apt-get -y install msodbcsql17 mssql-tools && \
+    update-alternatives --set php /usr/bin/php7.3
+
 ARG PHPSQLDIR=/REPO/msphpsql-dev
 ENV TEST_PHP_SQL_SERVER sql
 ENV TEST_PHP_SQL_UID sa
 ENV TEST_PHP_SQL_PWD Password123
 
+# update PATH after ODBC driver and tools are installed
+ENV PATH="/opt/mssql-tools/bin:${PATH}"	
+
 # add locale iso-8859-1
 RUN sed -i 's/# en_US ISO-8859-1/en_US ISO-8859-1/g' /etc/locale.gen
 RUN locale-gen en_US
@@ -37,19 +47,12 @@ RUN locale-gen en_US
 RUN locale-gen en_US.UTF-8
 ENV LANG='en_US.UTF-8' LANGUAGE='en_US:en' LC_ALL='en_US.UTF-8'
 
-#install ODBC driver	
-RUN curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add -	
-RUN curl https://packages.microsoft.com/config/ubuntu/16.04/prod.list > /etc/apt/sources.list.d/mssql-release.list	
-
-RUN export DEBIAN_FRONTEND=noninteractive && apt-get update && ACCEPT_EULA=Y apt-get install -y msodbcsql17 mssql-tools
-ENV PATH="/opt/mssql-tools/bin:${PATH}"	
-
-#install coveralls (upgrade both pip and requests first)
+# install coveralls (upgrade both pip and requests first)
 RUN python -m pip install --upgrade pip
 RUN python -m pip install --upgrade requests
 RUN python -m pip install cpp-coveralls
 
-#Either Install git / download zip (One can see other strategies : https://ryanfb.github.io/etc/2015/07/29/git_strategies_for_docker.html )
+# Either Install git / download zip (One can see other strategies : https://ryanfb.github.io/etc/2015/07/29/git_strategies_for_docker.html )
 #One option is to get source from zip file of repository.
 #another option is to copy source to build directory on image
 RUN mkdir -p $PHPSQLDIR
@@ -59,14 +62,17 @@ WORKDIR $PHPSQLDIR/source/
 RUN chmod +x ./packagize.sh
 RUN /bin/bash -c "./packagize.sh"
 
-RUN echo "extension = pdo_sqlsrv.so" >> /etc/php/7.0/cli/conf.d/20-pdo_sqlsrv.ini
-RUN echo "extension = sqlsrv.so" >> `php --ini | grep "Loaded Configuration" | sed -e "s|.*:\s*||"`
+RUN echo "; priority=20\nextension=sqlsrv.so\n" > /etc/php/7.3/mods-available/sqlsrv.ini
+RUN echo "; priority=30\nextension=pdo_sqlsrv.so\n" > /etc/php/7.3/mods-available/pdo_sqlsrv.ini
 
 WORKDIR $PHPSQLDIR/source/sqlsrv
-RUN phpize && ./configure LDFLAGS="-lgcov" CXXFLAGS="-O0 --coverage" && make && make install
+RUN /usr/bin/phpize && ./configure LDFLAGS="-lgcov" CXXFLAGS="-O0 --coverage" && make && make install
 
 WORKDIR $PHPSQLDIR/source/pdo_sqlsrv
-RUN phpize && ./configure LDFLAGS="-lgcov" CXXFLAGS="-O0 --coverage" && make && make install
+RUN /usr/bin/phpize && ./configure LDFLAGS="-lgcov" CXXFLAGS="-O0 --coverage" && make && make install
+
+RUN phpenmod sqlsrv pdo_sqlsrv
+RUN php --ri sqlsrv && php --ri pdo_sqlsrv
 
 # set name of sql server host to use
 WORKDIR $PHPSQLDIR/test/functional/pdo_sqlsrv

Linux-mac-install.md

@@ -5,13 +5,13 @@ These instructions install PHP 7.3 by default. Note that some supported Linux di
 
 ## Contents of this page:
 
-- [Installing the drivers on Ubuntu 16.04, 18.04, and 18.10](#installing-the-drivers-on-ubuntu-1604-1804-and-1810)
+- [Installing the drivers on Ubuntu 16.04, 18.04, and 19.04](#installing-the-drivers-on-ubuntu-1604-1804-and-1904)
 - [Installing the drivers on Red Hat 7](#installing-the-drivers-on-red-hat-7)
-- [Installing the drivers on Debian 8 and 9](#installing-the-drivers-on-debian-8-and-9)
+- [Installing the drivers on Debian 8, 9 and 10](#installing-the-drivers-on-debian-8-9-and-10)
 - [Installing the drivers on Suse 12 and 15](#installing-the-drivers-on-suse-12-and-15)
 - [Installing the drivers on macOS Sierra, High Sierra, and Mojave](#installing-the-drivers-on-macos-sierra-high-sierra-and-mojave)
 
-## Installing the drivers on Ubuntu 16.04, 18.04, and 18.10
+## Installing the drivers on Ubuntu 16.04, 18.04, and 19.04
 
 > [!NOTE]
 > To install PHP 7.1 or 7.2, replace 7.3 with 7.1 or 7.2 in the following commands.
@@ -31,19 +31,20 @@ Install the ODBC driver for Ubuntu by following the instructions on the [Linux a
 sudo pecl install sqlsrv
 sudo pecl install pdo_sqlsrv
 sudo su
-echo extension=pdo_sqlsrv.so >> `php --ini | grep "Scan for additional .ini files" | sed -e "s|.*:\s*||"`/30-pdo_sqlsrv.ini
-echo extension=sqlsrv.so >> `php --ini | grep "Scan for additional .ini files" | sed -e "s|.*:\s*||"`/20-sqlsrv.ini
+printf "; priority=20\nextension=sqlsrv.so\n" > /etc/php/7.3/mods-available/sqlsrv.ini
+printf "; priority=30\nextension=pdo_sqlsrv.so\n" > /etc/php/7.3/mods-available/pdo_sqlsrv.ini
 exit
+sudo phpenmod -v 7.3 sqlsrv pdo_sqlsrv
 ```
+If there is only one PHP version in the system then the last step can be simplified to `phpenmod sqlsrv pdo_sqlsrv`.
+
 ### Step 4. Install Apache and configure driver loading
 ```
 sudo su
 apt-get install libapache2-mod-php7.3 apache2
 a2dismod mpm_event
 a2enmod mpm_prefork
 a2enmod php7.3
-echo "extension=pdo_sqlsrv.so" >> /etc/php/7.3/apache2/conf.d/30-pdo_sqlsrv.ini
-echo "extension=sqlsrv.so" >> /etc/php/7.3/apache2/conf.d/20-sqlsrv.ini
 exit
 ```
 ### Step 5. Restart Apache and test the sample script
@@ -91,8 +92,8 @@ exit
 An issue in PECL may prevent correct installation of the latest version of the drivers even if you have upgraded GCC. To install, download the packages and compile manually (similar steps for pdo_sqlsrv):
 ```
 pecl download sqlsrv
-tar xvzf sqlsrv-5.6.1.tgz
-cd sqlsrv-5.6.1/
+tar xvzf sqlsrv-5.7.0.tgz
+cd sqlsrv-5.7.0/
 phpize
 ./configure --with-php-config=/usr/bin/php-config
 make
@@ -116,7 +117,7 @@ sudo apachectl restart
 ```
 To test your installation, see [Testing your installation](#testing-your-installation) at the end of this document.
 
-## Installing the drivers on Debian 8 and 9
+## Installing the drivers on Debian 8, 9 and 10
 
 > [!NOTE]
 > To install PHP 7.1 or 7.2, replace 7.3 in the following commands with 7.1 or 7.2.
@@ -145,19 +146,20 @@ locale-gen
 sudo pecl install sqlsrv
 sudo pecl install pdo_sqlsrv
 sudo su
-echo extension=pdo_sqlsrv.so >> `php --ini | grep "Scan for additional .ini files" | sed -e "s|.*:\s*||"`/30-pdo_sqlsrv.ini
-echo extension=sqlsrv.so >> `php --ini | grep "Scan for additional .ini files" | sed -e "s|.*:\s*||"`/20-sqlsrv.ini
+printf "; priority=20\nextension=sqlsrv.so\n" > /etc/php/7.3/mods-available/sqlsrv.ini
+printf "; priority=30\nextension=pdo_sqlsrv.so\n" > /etc/php/7.3/mods-available/pdo_sqlsrv.ini
 exit
+sudo phpenmod -v 7.3 sqlsrv pdo_sqlsrv
 ```
+If there is only one PHP version in the system then the last step can be simplified to `phpenmod sqlsrv pdo_sqlsrv`.
+
 ### Step 4. Install Apache and configure driver loading
 ```
 sudo su
 apt-get install libapache2-mod-php7.3 apache2
 a2dismod mpm_event
 a2enmod mpm_prefork
 a2enmod php7.3
-echo "extension=pdo_sqlsrv.so" >> /etc/php/7.3/apache2/conf.d/30-pdo_sqlsrv.ini
-echo "extension=sqlsrv.so" >> /etc/php/7.3/apache2/conf.d/20-sqlsrv.ini
 ```
 ### Step 5. Restart Apache and test the sample script
 ```
@@ -168,7 +170,7 @@ To test your installation, see [Testing your installation](#testing-your-install
 ## Installing the drivers on Suse 12 and 15
 
 > [!NOTE]
-> In the following instructions, replace <SuseVersion> with your version of Suse - if you are using Suse Enterprise Linux 15, it will be SLE_15 or SLE_15_SP1, and similarly for other versions. Not all versions of PHP are available for all versions of Suse Linux - please refer to `http://download.opensuse.org/repositories/devel:/languages:/php` to see which versions of Suse have the default version PHP available, or to `http://download.opensuse.org/repositories/devel:/languages:/php:/` to see which other versions of PHP are available for which versions of Suse.
+> In the following instructions, replace <SuseVersion> with your version of Suse - if you are using Suse Enterprise Linux 15, it will be SLE_15 or SLE_15_SP1. For Suse 12, use SLE_12_SP4 (or above if applicable). Not all versions of PHP are available for all versions of Suse Linux - please refer to `http://download.opensuse.org/repositories/devel:/languages:/php` to see which versions of Suse have the default version PHP available, or to `http://download.opensuse.org/repositories/devel:/languages:/php:/` to see which other versions of PHP are available for which versions of Suse.
 
 > [!NOTE]
 > Packages for PHP 7.3 are not available for Suse 12. 

README.md

@@ -13,14 +13,16 @@ Thank you for taking the time to participate in our last survey. You can continu
 <a href="https://aka.ms/mssqlphpsurvey"><img style="float: right;"  height="67" width="156" src="https://sqlchoice.blob.core.windows.net/sqlchoice/static/images/survey.png"></a>
 
 ### Status of Most Recent Builds
-| AppVeyor (Windows)       | Travis CI (Linux)        | Coverage (Windows)                    | Coverage (Linux)                          |
-|--------------------------|--------------------------|---------------------------------------|-------------------------------------------|
-| [![av-image][]][av-site] | [![tv-image][]][tv-site] | [![Coverage Codecov][]][codecov-site] | [![Coverage Coveralls][]][coveralls-site] |
+Azure Pipelines       | AppVeyor (Windows)       | Travis CI (Linux)        | Coverage (Windows)                    | Coverage (Linux)                          |
+|---------------------|--------------------------|--------------------------|---------------------------------------|-------------------------------------------|
+| [![az-image][]][az-site] | [![av-image][]][av-site] | [![tv-image][]][tv-site] | [![Coverage Codecov][]][codecov-site] | [![Coverage Coveralls][]][coveralls-site] |
 
 [av-image]: https://ci.appveyor.com/api/projects/status/vo4rfei6lxlamrnc?svg=true
 [av-site]: https://ci.appveyor.com/project/msphpsql/msphpsql/branch/dev
 [tv-image]:  https://travis-ci.org/microsoft/msphpsql.svg?branch=dev
 [tv-site]: https://travis-ci.org/microsoft/msphpsql/
+[az-site]: https://dev.azure.com/sqlclientdrivers-ci/msphpsql/_build/latest?definitionId=6&branchName=dev
+[az-image]: https://dev.azure.com/sqlclientdrivers-ci/msphpsql/_apis/build/status/Microsoft.msphpsql?branchName=dev
 [Coverage Coveralls]: https://coveralls.io/repos/github/microsoft/msphpsql/badge.svg?branch=dev
 [coveralls-site]: https://coveralls.io/github/microsoft/msphpsql?branch=dev
 [Coverage Codecov]: https://codecov.io/gh/microsoft/msphpsql/branch/dev/graph/badge.svg

appveyor.yml

@@ -81,10 +81,10 @@ install:
         } Else {
             $env:PHP_VERSION=$env:PHP_MAJOR_VER + '.' + $env:PHP_MINOR_VER;
         }
-    - echo Downloading MSODBCSQL 17.2
+    - echo Downloading MSODBCSQL 17
     # AppVeyor build works are x64 VMs and 32-bit ODBC driver cannot be installed on it
-    - ps: (new-object net.webclient).DownloadFile('https://download.microsoft.com/download/E/6/B/E6BFDC7A-5BCD-4C51-9912-635646DA801E/en-US/msodbcsql_17.2.0.1_x64.msi', 'c:\projects\msodbcsql_17.2.0.1_x64.msi')
-    - cmd /c start /wait msiexec /i "c:\projects\msodbcsql_17.2.0.1_x64.msi" /q IACCEPTMSODBCSQLLICENSETERMS=YES ADDLOCAL=ALL
+    - ps: (new-object net.webclient).DownloadFile('https://download.microsoft.com/download/E/6/B/E6BFDC7A-5BCD-4C51-9912-635646DA801E/en-US/msodbcsql_17.4.1.1_x64.msi', 'c:\projects\msodbcsql_17.4.1.1_x64.msi')
+    - cmd /c start /wait msiexec /i "c:\projects\msodbcsql_17.4.1.1_x64.msi" /q IACCEPTMSODBCSQLLICENSETERMS=YES ADDLOCAL=ALL
     - echo Checking the version of MSODBCSQL
     - reg query "HKLM\SOFTWARE\ODBC\odbcinst.ini\ODBC Driver 17 for SQL Server"
     - dir %WINDIR%\System32\msodbcsql*.dll