Rubygems should continue when a gemspec is missing ("gem generate_index")

[slyphon]

As discussed in #ruby-lang

rubygems@mb1-rubygems01:~/rubygems$ gem generate_index -V --update -d .
Generating Marshal quick index gemspecs for 21 gems
1/21: activesupport-3.0.10
2/21: builder-3.0.0
3/21: builder-2.1.2
4/21: cloudfiles-1.4.17
5/21: cloudgems-0.3.9
6/21: cloudgems-0.3.8
7/21: cloudgems-0.3.7
8/21: i18n-0.6.0
9/21: little-plugger-1.1.2
10/21: logging-1.5.2
11/21: mime-types-1.16
12/21: rake-0.8.7
13/21: rake-0.8.7
14/21: rubygems-update-1.8.15
15/21: rubygems-update-1.8.14
16/21: rubygems-update-1.8.13
17/21: rubygems-update-1.8.12
18/21: rubygems-update-1.8.11
19/21: rubygems-update-1.8.10
20/21: slyphon-zookeeper-0.2.4
21/21: zk-0.8.5
Complete
Generated Marshal quick index gemspecs: 0.013s
Updated indexes: 0.031s
Compressing indicies
Compressed indicies: 0.002s
Updating production dir /opt/rubygems
mv -f /tmp/gem_generate_index_26462/quick/Marshal.4.8/activesupport-3.0.10.gemspec.rz /opt/rubygems/quick/Marshal.4.8/activesupport-3.0.10.gemspec.rz
mv -f /tmp/gem_generate_index_26462/quick/Marshal.4.8/builder-3.0.0.gemspec.rz /opt/rubygems/quick/Marshal.4.8/builder-3.0.0.gemspec.rz
mv -f /tmp/gem_generate_index_26462/quick/Marshal.4.8/builder-2.1.2.gemspec.rz /opt/rubygems/quick/Marshal.4.8/builder-2.1.2.gemspec.rz
mv -f /tmp/gem_generate_index_26462/quick/Marshal.4.8/cloudfiles-1.4.17.gemspec.rz /opt/rubygems/quick/Marshal.4.8/cloudfiles-1.4.17.gemspec.rz
mv -f /tmp/gem_generate_index_26462/quick/Marshal.4.8/cloudgems-0.3.9.gemspec.rz /opt/rubygems/quick/Marshal.4.8/cloudgems-0.3.9.gemspec.rz
mv -f /tmp/gem_generate_index_26462/quick/Marshal.4.8/cloudgems-0.3.8.gemspec.rz /opt/rubygems/quick/Marshal.4.8/cloudgems-0.3.8.gemspec.rz
mv -f /tmp/gem_generate_index_26462/quick/Marshal.4.8/cloudgems-0.3.7.gemspec.rz /opt/rubygems/quick/Marshal.4.8/cloudgems-0.3.7.gemspec.rz
mv -f /tmp/gem_generate_index_26462/quick/Marshal.4.8/i18n-0.6.0.gemspec.rz /opt/rubygems/quick/Marshal.4.8/i18n-0.6.0.gemspec.rz
mv -f /tmp/gem_generate_index_26462/quick/Marshal.4.8/little-plugger-1.1.2.gemspec.rz /opt/rubygems/quick/Marshal.4.8/little-plugger-1.1.2.gemspec.rz
mv -f /tmp/gem_generate_index_26462/quick/Marshal.4.8/logging-1.5.2.gemspec.rz /opt/rubygems/quick/Marshal.4.8/logging-1.5.2.gemspec.rz
mv -f /tmp/gem_generate_index_26462/quick/Marshal.4.8/mime-types-1.16.gemspec.rz /opt/rubygems/quick/Marshal.4.8/mime-types-1.16.gemspec.rz
mv -f /tmp/gem_generate_index_26462/quick/Marshal.4.8/rake-0.8.7.gemspec.rz /opt/rubygems/quick/Marshal.4.8/rake-0.8.7.gemspec.rz
mv -f /tmp/gem_generate_index_26462/quick/Marshal.4.8/rake-0.8.7.gemspec.rz /opt/rubygems/quick/Marshal.4.8/rake-0.8.7.gemspec.rz
ERROR:  While executing gem ... (Errno::ENOENT)
    No such file or directory - /tmp/gem_generate_index_26462/quick/Marshal.4.8/rake-0.8.7.gemspec.rz

the output of ls gems/rake*.gem:

gems/rake-0.8.1.gem  gems/rake-0.8.4.gem  gems/rake-0.9.1.gem  gems/rake-remote_task-2.0.1.gem  gems/rake-remote_task-2.0.4.gem
gems/rake-0.8.3.gem  gems/rake-0.8.7.gem  gems/rake-0.9.2.gem  gems/rake-remote_task-2.0.2.gem

the output of for f in gems/*.gem; do echo -n $f ' '; gem spec $f name ; done (filtered with |grep rake)

gems/rake-0.8.1.gem  --- rake
gems/rake-0.8.3.gem  --- rake
gems/rake-0.8.4.gem  --- rake
gems/rake-0.8.7.gem  --- rake
gems/rake-0.9.1.gem  --- rake
gems/rake-0.9.2.gem  --- rake
gems/rake-remote_task-2.0.1.gem  --- rake-remote_task
gems/rake-remote_task-2.0.2.gem  --- rake-remote_task
gems/rake-remote_task-2.0.4.gem  --- rake-remote_task

[slyphon]

patch for this issue is: https://gist.github.com/115d23122e5a6bb32ff6

[slyphon]

Actually, this is a better patch, the one above breaks --update https://gist.github.com/76b401c35dd951253f8e

[drbrain]

@evanphx can you pull this into the next 1.8 release?

[evanphx]

@drbrain I was waiting on some tests before pulling this in. Should I wait or pull it in now, sans tests?

[drbrain]

I think it's safe without a patch, but I've bugged @slyphon in IRC

[drbrain]

I have been informed that I will write tests for this…

[evanphx]

@drbrain I shall await your tests then.

[duckinator]

Is this issue still valid? If it is, has there been any movement on it?

[copiousfreetime]

I don't think this is a bug anymore. I attempted to reproduce and was unable to.

From the bug report it looks like it generate_index was including gems that were installed as part of the gems to put into the index instead of just those gems in the ./gems directory.

In Gem::Indexer the list of gems to index is gathered via #gem_file_list which looks only at the files in the destination directory.