Install instructions for remote MySQL

[chris-hailstorm]

My goal is to configure Airflow to run on my local machine, but using a MySQL instance running on Amazon RDS -- I'm using the install steps outlined on http://pythonhosted.org/airflow/installation.html.

The step for installing MySQL support is pip install airflow[mysql]. On my OSX box, the pip install fails with this error: EnvironmentError: mysql_config not found. Reading a few StackOverflow articles, it appears the mysql_config executable needs to be on the path for this install to succeed. You wouldn't expect to always find that utility on the local machine if MySQL will be running remotely.

I created a work-around by installing MySQL locally and putting this utility on my path. I suspect that may cause a problem with Airframe since the version and configuration of local vs. remote MySQL can differ.

Two suggested changes to the documentation:

(1) To run MySQL locally -- state that MySQL must be installed, and mysql_config must be on the path.

(2) To run MySQL remotely -- say what needs to be done. Maybe my workaround is the right approach, but you would be a better judge of that. For the general case, assume MySQL is not installed locally.

Thanks

[mistercrunch]

We use the mysqlclient python lib which has its own set of dependencies. Originally I didn't want to get in OS-specific, db backend-specific instructions in the docs.

Maybe at the bottom of the installation docs page there could be a section for OS specific and/or backend specific instructions.

[d-lee]

You don't need a full MySQL running locally if you want remote metadata store. All you need is development libs/headers. On linux based systems you can get them by doing
sudo apt-get install libmysqlclient-dev -y . That's all you need to make pip install airflow[mysql] work.

If you use OS X for local development then the easiest way to get these libraries is via homebrew. Install homebrew then run brew install mysql-connector-c then pip install airflow[mysql]

May be it can be added to FAQ (instead of installation docs) as I suspect many folks run into this mysql_config not found issue when trying to setup remote MySQL based metadata store

[adadle]

I met this problem in CentOS when add new workers
yum install mysql-devel
sovled my problem.

On Mon, Feb 22, 2016 at 11:02 PM, Dmitriy Lee notifications@github.com
wrote:

You don't need a full MySQL running locally if you use remote metadata
store. All you need is development libs/headers. On linux based systems you
can get them by doing
sudo apt-get install libmysqlclient-dev -y . That's all you need to make pip
install airflow[mysql] work.

If you use Macs for local development then the easiest way to get these
libraries is by using homebrew. Install homebrew then tun brew install
mysql-connector-c then pip install airflow[mysql]

May be it can be added to FAQ (instead of installation docs) as I suspect
many folks run into this mysql_config not found issue when trying to
setup remote MySQL based metadata store

—
Reply to this email directly or view it on GitHub
#1047 (comment).

Best Wishes from TonyLee.

[adadle]

after that, install python mysql connector
pip install "mysqlclient>=1.3.6"

On Mon, Feb 29, 2016 at 2:46 PM, Lee Tony tonylee0329@gmail.com wrote:

I met this problem in CentOS when add new workers
yum install mysql-devel
sovled my problem.

On Mon, Feb 22, 2016 at 11:02 PM, Dmitriy Lee notifications@github.com
wrote:

You don't need a full MySQL running locally if you use remote metadata
store. All you need is development libs/headers. On linux based systems you
can get them by doing
sudo apt-get install libmysqlclient-dev -y . That's all you need to make pip
install airflow[mysql] work.

If you use Macs for local development then the easiest way to get these
libraries is by using homebrew. Install homebrew then tun brew install
mysql-connector-c then pip install airflow[mysql]

May be it can be added to FAQ (instead of installation docs) as I suspect
many folks run into this mysql_config not found issue when trying to
setup remote MySQL based metadata store

—
Reply to this email directly or view it on GitHub
#1047 (comment).

Best Wishes from TonyLee.

Best Wishes from TonyLee.