Gempacks

Cecil Coupe edited this page Jan 23, 2016 · 3 revisions

Gempacks is a new "confusing feature" with Shoes 3.2.23 and only for those who are willing to install their platform development tools if needed - Rubyinstaller and devkit on windows, or rvm, Xcode, command line tools for OSX. Or the linux equivalents. Not that hard to do. You can't create gempacks if you can't compile the gem somewhere and copy it to Shoes.

Shoes has built in gems They appear to be part of the included Ruby. Shoes also can install gems in HOME/+gems - see Cobbler-> Manage Gems. Shoes scripts or apps can also request a gem be installed with Shoes.setup {}. If that gem needs to be compiled with C or C++ then you need the development tools. You can find gems that have precompiled binaries (usually for Windows). That leaves OSX and Linux out in the pre-compiled cold. Yes, there are library version issues on Linux and OSX but that doesn't mean Shoes shouldn't allow app developers to try - it might work after all, you know?.

Use case

Imagine you have a Raspberry Pi and some special hardware attached to it (perhaps the 'hell in a hand basket' temperature monitor at your kickstarter page) that needs the i2c gem and an Arudino gem to communicate the level of doom ahead. No big deal. Just install the development tools and libraries and compile the gems. Most of the tools are on the pi already or just an apt-get install away. Well... Try staying awake watching Nokogiri built on a pi (even on a pi2). Imagine the love you'll receive from the users of your "Pi Doom Monitor" app when they too fall asleep waiting for your doom monitor to finish installing. This might be a bad example since the Doomsday folk love adversity - it makes them stronger, but you get the drift.

You could build your own Shoes from source with the i2c gem built in (like sqlite3 is) and package your script up in a 11MB download from your website. Not that hard to do unless you make mistakes or want special things. But you're kind of on you're lonesome if something goes wrong in Shoes. We might fix the problem but your copy of Shoes doesn't track the home base. Nor does your user's copy of Shoes have that fix.

Would it not be better to just package up your Doom Monitor script using the existing Shoes 3.2 infrastructure and include the gems you need with your script/app (pre-compiled gems if needed).

Perhaps your app only needs the USB serial port gem on Windows and you don't care about OSX or Linux but you don't want to force users to spend a long afternoon with dos command line tools and devkit installs. Or maybe you want to write an app to use usb serial on Linux to re-program some remote control device that is locked into Windows and OSX apps. Don't ask.

Gempacks

A gempack is a gzipped tar ball of gems to put in end users HOME/+gems. Cobbler (Maintain->Shoes) can do it in Shoes 3.2.23. It's just barely gems since they are stripped of anything but the most essential files. No doc, no tests, no fluff. Pre-compiled if needed. At the moment [May 2015], you can't include gempacks with your app, they have to be a separate download from "somewhere (aka your website)" that your user must install. You might suspect I'll be working on that restriction in the future.

If you can build it (or copy it correctly from a good gem for the platform - not that hard if you know what is going on)

Creating a gempack

It's Shoes script - surprised? If you git clone https://github.com/Shoes3/shoes3.git you'll have the CopyGem.rb script in the gemutils/ directory.

You need to select the folder just above specifications. On Windows that might be C:\Ruby22\lib\ruby\gems\2.2.0 or something close to that. On Windows when dealing with pre-built native gems you may get large ruby version-ed libraries that Shoes doesn't need. For instance, if your Shoes is using Ruby 2.2.x, then you need to winnow those out your self. You might note that the code does not actually create a tar - it only claims to. Consider that a good thing for now.

# copy a gem (and all dependent gems to a a specified location
# for Shoes purposes (only lib, spec, and ext..../gem.build_complete
require 'rubygems/dependency_installer'
# Don't use gems ?
Shoes.app do
  stack do
    flow do
      para "Load gems from:"
      @srcfld = edit_line ENV['GEM_HOME']
      button "select ..." do
        @srcfld.text = ask_open_folder
        Gem.use_paths(@srcfld.text, [GEM_DIR, GEM_CENTRAL_DIR])
        Gem.refresh
      end
    end
    flow do 
      para "Copy gems to here: "
      @dirfld = edit_line
      button "select..." do
        @dirfld.text = ask_open_folder 
      end
    end
    flow do
      button "List" do
        @panel.clear 
        @panel.append do
          @gemlist = stack
        end
        gem_refresh_local
      end
      @tgzfld = check; para "tgz"
      @cpbtn = button "Copy" do
        copy_gem_files
      end
      button "Quit" do
        exit
      end
    end
    @panel = stack do
        @gemlist = stack {}
    end
  end
  
  def gem_refresh_local
    @gemlist.clear
    @gemlist.background white
    @cpbtn.state ='disabled'
    rbpath = @srcfld.text+'/specifications/*.gemspec'
    if RUBY_PLATFORM =~ /mingw/
      rbpath.gsub!(/\\/, '/')  # Dir.glob is special here
    end
    Dir.glob(rbpath).sort.each do |path|
      fnm = File.basename(path)
      fnm.gsub!('.gemspec','')
      @gemlist.append do
        flow margin: 5 do
          check do
            spec = eval(File.read(path))
            show_deps spec
            @cpbtn.state = nil
          end
          para "#{fnm}"
        end
      end
    end
   end
   
   def show_deps spec
     @panel.clear
     @panel.append do
       # get a hash of ALL the dependent gems - order does not matter
       # they've been installed already and we only need to copy. 
       @deplist = {}
       @deplist[spec.name] = spec
       gem_build_deps spec
       # now filter out the built in gems in default (minitest, rake, rdoc)
       # HACK ALERT - HARD CODED
       inRuby = ['bigdecimal', 'io-console', 'json', 'psych' , 'rake', 'test-unit']
       inRuby.each do |key| 
         @deplist.delete key
       end
       @deplist.each do |nm, dep|
         para "#{nm} => #{dep.full_name} #{Gem::Platform.local}"
       end
     end
   end
   
   def gem_build_deps spec
     #puts "spec #{spec.name}"
     return if spec == nil
     speclist = spec.dependent_specs # returns spec [] or element
     if speclist.instance_of? Array
       speclist.each do |spec| 
         @deplist[spec.name] = spec
         gem_build_deps spec
       end
     end
   end
  
  
  def copy_gem_files
    alert "empty destination" if @dirfld.text == ""
    srcpath = @srcfld.text
    destpath =  @dirfld.text
    if RUBY_PLATFORM =~ /mingw/
      srcpath.gsub!(/\\/, '/')  
      destpath.gsub!(/\\/, '/')  
    end

    @deplist.each do |name, spec|
      puts "copy #{spec.full_name} to #{destpath}"
      mkdir_p(File.join(destpath, spec.full_name))
      cp File.join(srcpath, 'specifications', "#{spec.full_name}.gemspec") , 
         File.join(destpath, spec.full_name,'gemspec')
      # check for binary code
      rubyv = RUBY_VERSION[/\d.\d/]+'.0'
      gemcompl = File.join(srcpath, 'extensions', "#{Gem::Platform.local}",
         rubyv, spec.full_name, 'gem.build_complete')
      if File.exist? gemcompl
        puts "binary #{gemcompl}"
        cp gemcompl, File.join(destpath, spec.full_name)
      end 
      # copy lib/ or ext/ or whatever the spec says.
      # caution: spec is modified by the eval() so it's filled in with stuff
      puts "Require paths #{spec.require_paths}"
      skip1 = spec.require_paths
      if (skip1.length > 1) &&  (skip1.include? 'ext')
       # A confused gem! Me too. Perhaps Any binary gem?
       src = File.join(skip1)
       dest = File.join(destpath, spec.full_name, skip1[1])
       mkdir_p dest
       puts "weird ext copy #{src} -> #{dest}"
       cp_r src, dest
      elsif (skip1.length > 1) &&  (skip1.include? 'lib')
        puts "weird lib copy  #{}"
        cp_r File.join(srcpath,'gems', spec.full_name, 'lib'), File.join(destpath, spec.full_name)
      else
        skip1.each do |rqp| 
          puts "copy this  #{rqp}"
          cp_r File.join(srcpath,'gems', spec.full_name, rqp), File.join(destpath, spec.full_name)
        end
      end
    end 
    if @tgzfld.checked? 
      create_tar if confirm "Tar up #{destpath}"
    end
  end
  
  def create_tar
    puts "create_tar called"
  end
  
end

Gemspec

We copy this file. It has the License terms and author credits. It also tells us about which directories are really important and provides names and versions numbers.

lib and ext

Most gems put their code, both ruby and compiled binaries in lib. Unless it's only an .so in ext/ Grrr. Rather than send scornful emails to those authors, we just deal with it.

gem.build_complete

This file is created in when the .so binary was built (somewhere, that we copied from). In a gempack load, we copy it to the correct place. Without it your end user would need to build the gem from source, which we probably stripped out because we didn't want to impose that on users.

Problems?

Too many to count. There are so many ways you, your users or I can screw this up. But, for some folks, it's a damn good idea even it's clumsy and imperfect.

Clone this wiki locally
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.