Ruby 2.2 & 0.6.3 Release #196

jalmargyyk · 2015-05-08T07:46:40Z

It seems that ruby 2.2 compatibility is only in master. Any chance for a new release?

metaskills · 2015-05-08T11:54:10Z

It is not yet easy for me to do the precompiled Windows binaries. If you are on a POSIX system, you can just bundle to master.

jalmargyyk · 2015-05-08T20:36:33Z

Ah, sorry I didn't think other platforms then linux. I prefer to use released gems as this is used in production. Well, I'll just use previous ruby version for the time being. Thanks.

aharpervc · 2015-05-09T17:29:40Z

Referencing master is kind of a hassle when writing a gem (cant reference the repo in a gemspec). And it would be great to have an official release for 2.2. Is there any technical hold up?

metaskills · 2015-05-09T19:42:06Z

Referencing master is kind of a hassle when writing a gem

Great points!

Is there any technical hold up?

Getting the latest master branch to work with rake-compiler-dev-box for the Windows binaries. Anyone got some free time to debug? I suspect I could make a release for just any system that can compile natively too.

metaskills · 2015-05-14T12:29:10Z

I have pushed 0.6.3.rc1 which is POSIX only. No pre-build win binaries now. HTH!

aharpervc · 2015-05-14T18:37:27Z

A windows build is also necessary. I have attempted to get rake-compiler working for ruby 2.2, not yet successful: rake-compiler/rake-compiler#112

aharpervc · 2015-05-18T17:05:33Z

Good news, I have successfully cross compiled tiny_tds 0.6.3.rc1 for ruby 2.2, for windows. And it works! So getting 0.6.3 released for all platforms for all supported ruby versions for real should totally be possible.

I believe there are only two major blocking issues to make this work. These might be rake-compiler issues or things that can be patched from this repo, not sure. I'm posting my progress here as I work on solutions in case anyone has some quick fixes.

In order to cross compile a gem for ruby 2.2, rake-compiler needs to cross build ruby 2.2. System ruby >= 2.0 is needed to build 2.2, but that's nbd since rake-compiler already acquires ruby 2.0 and 2.1. Rake-compile-dev-box has a prep script to cross build target ruby versions. Here's the patch to add ruby 2.2.

This repo already has a patch for rake-compiler-dev-box to remove 1.8.7 and add 2.1 to the cross-compile version list for win32 fat binaries. We still want to do that basic logic but the patch file is out of date and partially superceded by my patch file. The other part is to update package_win32_fat_binary to target 2.2 as a cc version. For testing I removed the other ruby versions and set RUBY_CC_VERSION to 2.2.1. Also for testing, I commented out the other ruby versions in prepare_xrubies just to save time & focus on 2.2, but they should still work.

Now the first major problem: rake-compiler can't successfully cross build ruby 2.2, as described in the linked issue. The situation is that ruby 2.2's securerandom.rb references windows libraries in a way that causes the cross build to fail (it's works differently in 2.1). I don't know the "right" way to solve this problem, but hacking securerandom.rb to wrap the two windows library references in begin/rescue allow the build to succeed (this may not even actually be the worst wrong way to do it, either). Here's the patch with the hack. Don't forget that rake-compiler creates ruby build artifacts in ~/.rake-compiler/build and ~/.rake-compiler/ruby that you'll need to remove before re-compiling changes to ruby source files.

So, with these patches, you should be able to cd into rake-compile-dev-box and vagrant up and be ready to cross-compile gems for 2.2.

That's all pretty straightforward. The second major issue is a bit more murky and I'm hoping someone will be able to more clearly see what's wrong.

The rake cross-compile task fails, because the package_win32_fat_binary tiny_tds command fails. The error is due to gemspec validation missing tiny_tds.so. I believe that the compiled tiny_tds.so file which is present in /vagrant/tiny_tds/lib/tiny_tds/ isn't making it into the pkg directory. This is causing package_win32_fat_binary to fail, which causes rake cross-compile to fail. I don't know what part of the process is responsible for copying those files, OR, if something somewhere is looking for tiny_tds.so in the wrong place. I'm not sure. To hack my way through and get something working, I modified the system ruby rubygems gemspec validation class to manually copy tiny_tds.so into the right places before failing validation, and it worked:

# the worst hack ever
# line ~2940  /home/vagrant/.rvm/rubies/ruby-2.0.0-p643/lib/ruby/site_ruby/2.0.0/rubygems/specification.rb
`cp /vagrant/tiny_tds/lib/tiny_tds/tiny_tds.so /vagrant/tiny_tds/pkg/tiny_tds-0.6.3.rc1/lib/tiny_tds/tiny_tds.so`
`cp /vagrant/tiny_tds/lib/tiny_tds/tiny_tds.so /vagrant/tiny_tds/pkg/tiny_tds-0.6.3.rc1-x64-mingw32/lib/tiny_tds/tiny_tds.so`
`cp /vagrant/tiny_tds/lib/tiny_tds/tiny_tds.so /vagrant/tiny_tds/pkg/tiny_tds-0.6.3.rc1-x86-mingw32/lib/tiny_tds/tiny_tds.so`

Weirdly, I removed these lines later and the rake task worked.... so even worse, I don't know if I caused the problem myself or what the heck is going on. Re-running the rake task, even after clobbering pkg, seems to work. Installing the output x86 gem seems to work and using it in a project in ruby 2.2 seems to work.

Any thoughts, @metaskills ?

metaskills · 2015-05-18T17:10:30Z

INITIAL THOUGHS... OMG! 🎉

Give me a few hours or so and I will reply in detail.

aharpervc · 2015-05-19T15:14:06Z

Looks like the windows library loading thing was possibly fixed in ruby 2.2.2... so perhaps that patch is not necessary. Giving it a try now...

E: Yep, updating rcdb scripts to 2.2.2 obviates the need for the 2.2.1 hackery. And there's an open pr on that repo to update the cross build ruby versions: tjschuck/rake-compiler-dev-box#20

aharpervc · 2015-05-19T15:52:03Z

Looks like rake cross-compile is still blowing up for some reason (possibly different than the 2.2.1 issues). But I manually ssh'd into the rcdb vm and ran package_win32_fat_binary tiny_tds which worked successfully (& tiny_tds installs & works on windows w/ 2.2 with a quick test). I did notice that bundler was missing, and actually I can't find where rcdb is installing bundler for its system rubies. Surely that cannot be the last blocker?

aharpervc · 2015-05-20T13:22:18Z

I created a fork of rake-compiler-dev-box which includes all out standing pull requests required to cross compile tiny tds 0.6.3 for windows including the outdated patch from this repo to remove 187. rake cross-compile seems to work fine. I guess my second issue there was also related to 2.2.1? Not sure, but this should mean everything is good to go. cc @metaskills

aharpervc · 2015-05-26T15:06:53Z

@metaskills any chance you'll be able to look at this this week? It'd be great if we could get a full 0.6.3 asap.

metaskills · 2015-05-29T01:23:44Z

Looking now!!!

metaskills · 2015-05-29T01:31:40Z

@aharpervc Thanks for your patience... I was out of town last week :)

I have forked your fork to the rails-sqlserver account, see here https://github.com/rails-sqlserver/rake-compiler-dev-box and you have been added as a collaborator both for that repo and this TinyTDS repo.

Can you do the updates to this repo to 1) remove the outdated patch and 2) write up some docs on the process in the [Compiling Gems for Windows] section of the README? If so, I would like to test and then we can get these gems released. Afterward, I can even get you added to rubygems.org list too. Cant wait!

dicom · 2015-06-17T08:05:53Z

Any news on this? I would love to see a windows gem release, so that I can upgrade my Ruby environment to 2.2!

aharpervc · 2015-06-17T12:33:03Z

I will be revisiting this in the next couple weeks if @metaskills doesn't get to it first. We want to see a full cross platform 0.6.3 release soon. If that's not quick enough for you @dicom you should be able to build your own copy using the https://github.com/rails-sqlserver/rake-compiler-dev-box

larskanis · 2015-07-02T20:15:18Z

tiny_tds Windows binary gems already build great with rake-compiler-dock, at least on Linux:

git clone https://github.com/rails-sqlserver/tiny_tds
cd tiny_tds
gem install rake-compiler-dock
rake-compiler-dock bash -c "bundle && rake cross native gem RUBY_CC_VERSION=1.9.3:2.0.0:2.1.6:2.2.2"

I don't need rake-compiler-dev-box any more 😋

extconf.rb now checks for availability of freetds, with a fallback to the bundled mini_portile recipes. As a further extension, it might be useful to include the stable freetds sources into the gem. This allows native build on Windows with download and compilation of all dependent libraries. This also adds a appveyor config for ci-testing of the native build. However not all tests pass, so far. For building windows binary gems this makes the switch to the new rake-compiler-dock. The mini_portile recipes are changed, now, to build shared libraries. This decreases the fat binary Windows gem from around 7 MB to 2.5 MB, compared to a static build. To avoid any version mismatchs with OpenSSL, I used a more specific naming scheme for OpenSSL dlls. This should avoid any contact with the DLL hell. This fixes issues rails-sqlserver#200, rails-sqlserver#75 and hopefully helps for rails-sqlserver#196.

aharpervc · 2015-07-21T14:08:22Z

Alright, with c2abdb7 (which points to the updated fork of rcdb) I'm able to rake cross-compile:setup followed by rake cross-compile, which results in a successful compilation of 32 & 64 bit fat win32 gems which install & work in ruby 2.2.2 in windows. Mission accomplished? @metaskills

metaskills · 2015-07-23T12:01:28Z

Hey @aharpervc !!! I am trying this out today and catching up on #207 The cross-compile failed for me, can you give me any hints as to why? Here is my output https://gist.github.com/metaskills/98076c179f0de82b6c3d and TIA!!!

larskanis · 2015-07-23T12:35:20Z

@metaskills You should be able to cross compile per rake-compiler-dock on OS-X even without #207 . See comment above.

metaskills · 2015-07-23T12:47:21Z

Thanks @larskanis made a comment there... will try now too...

extconf.rb now checks for availability of freetds, with a fallback to the bundled mini_portile recipes. As a further extension, it might be useful to include the stable freetds sources into the gem. This allows native build on Windows with download and compilation of all dependent libraries. This also adds a appveyor config for ci-testing of the native build. However not all tests pass, so far. For building windows binary gems this makes the switch to the new rake-compiler-dock. The mini_portile recipes are changed, now, to build shared libraries. This decreases the fat binary Windows gem from around 7 MB to 2.5 MB, compared to a static build. To avoid any version mismatchs with OpenSSL, I used a more specific naming scheme for OpenSSL dlls. This should avoid any contact with the DLL hell. This fixes issues rails-sqlserver#200, rails-sqlserver#75 and hopefully helps for rails-sqlserver#196.

aharpervc · 2015-07-23T13:50:50Z

@metaskills from a quick glance at your gist, it appears that maybe the bootstrap.sh isn't running. This could possibly happen if the vagrant box already exists, since it won't run provisioner scripts more than once. Can you nuke the old box (or move/rename it) so that it boots fresh?

larskanis · 2015-08-02T07:31:17Z

I tried tiny-tds-0.6.3-rc2 on free versions of Hyper-V Server 2012 R2 with SQL Server Express 2014 with both ruby-2.2.2-x86 and ruby-2.2.2-x64 and the Devkit installed. Per default it installed the fat binary gems, as expected. The test suite raises several failures, similar to the appveyor results. Appart from that everything worked fine with both 32 and 64 bits.

Installation with

gem inst tiny_tds --pre --platform=ruby

took some time, but it worked equally well for both 32 and 64 bit environment.

metaskills · 2015-08-03T11:33:12Z

Going to close this out as the 0.7.0 release with Ruby 2.2 will happen by end of the week.

extconf.rb now checks for availability of freetds, with a fallback to the bundled mini_portile recipes. As a further extension, it might be useful to include the stable freetds sources into the gem. This allows native build on Windows with download and compilation of all dependent libraries. This also adds a appveyor config for ci-testing of the native build. However not all tests pass, so far. For building windows binary gems this makes the switch to the new rake-compiler-dock. The mini_portile recipes are changed, now, to build shared libraries. This decreases the fat binary Windows gem from around 7 MB to 2.5 MB, compared to a static build. To avoid any version mismatchs with OpenSSL, I used a more specific naming scheme for OpenSSL dlls. This should avoid any contact with the DLL hell. This fixes issues rails-sqlserver#200, rails-sqlserver#75 and hopefully helps for rails-sqlserver#196.

jalmargyyk closed this as completed May 8, 2015

aharpervc mentioned this issue May 14, 2015

Problem building ruby 2.2.1 rake-compiler/rake-compiler#112

Closed

metaskills mentioned this issue May 29, 2015

Using TinyTDS for SQL Azure on Windows #200

Closed

metaskills reopened this Jul 8, 2015

metaskills changed the title ~~ruby 2.2 compatibility~~ Ruby 2.2 & 0.6.3 Release Jul 8, 2015

This was referenced Jul 8, 2015

When are you planning to release version 0.6.3? #203

Closed

Assertion failed: tds->state < TDS_VECTOR_SIZE(state_names), file util.c, line 95 #170

Closed

larskanis mentioned this issue Jul 16, 2015

Refactor build of freetds and iconv recipes and add OpenSSL. #207

Merged

metaskills closed this as completed Aug 3, 2015

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ruby 2.2 & 0.6.3 Release #196

Ruby 2.2 & 0.6.3 Release #196

jalmargyyk commented May 8, 2015

metaskills commented May 8, 2015

jalmargyyk commented May 8, 2015

aharpervc commented May 9, 2015

metaskills commented May 9, 2015

metaskills commented May 14, 2015

aharpervc commented May 14, 2015

aharpervc commented May 18, 2015

metaskills commented May 18, 2015

aharpervc commented May 19, 2015

aharpervc commented May 19, 2015

aharpervc commented May 20, 2015

aharpervc commented May 26, 2015

metaskills commented May 29, 2015

metaskills commented May 29, 2015

dicom commented Jun 17, 2015

aharpervc commented Jun 17, 2015

larskanis commented Jul 2, 2015

aharpervc commented Jul 21, 2015

metaskills commented Jul 23, 2015

larskanis commented Jul 23, 2015

metaskills commented Jul 23, 2015

aharpervc commented Jul 23, 2015

larskanis commented Aug 2, 2015

metaskills commented Aug 3, 2015

Ruby 2.2 & 0.6.3 Release #196

Ruby 2.2 & 0.6.3 Release #196

Comments

jalmargyyk commented May 8, 2015

metaskills commented May 8, 2015

jalmargyyk commented May 8, 2015

aharpervc commented May 9, 2015

metaskills commented May 9, 2015

metaskills commented May 14, 2015

aharpervc commented May 14, 2015

aharpervc commented May 18, 2015

metaskills commented May 18, 2015

aharpervc commented May 19, 2015

aharpervc commented May 19, 2015

aharpervc commented May 20, 2015

aharpervc commented May 26, 2015

metaskills commented May 29, 2015

metaskills commented May 29, 2015

dicom commented Jun 17, 2015

aharpervc commented Jun 17, 2015

larskanis commented Jul 2, 2015

aharpervc commented Jul 21, 2015

metaskills commented Jul 23, 2015

larskanis commented Jul 23, 2015

metaskills commented Jul 23, 2015

aharpervc commented Jul 23, 2015

larskanis commented Aug 2, 2015

metaskills commented Aug 3, 2015