Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fails to detect binary format STL file #2

Open
ChocoNutDancer opened this issue Jun 7, 2015 · 8 comments

Comments

@ChocoNutDancer
Copy link

@ChocoNutDancer ChocoNutDancer commented Jun 7, 2015

Go to http://www.eng.nus.edu.sg/LCEL/RP/u21/wwwroot/stl_library.htm#Download and download the porsche STL file. The STL file is in binary format but the converter thinks it's ascii and so fails to convert it.

@toreyheinz

This comment has been minimized.

Copy link

@toreyheinz toreyheinz commented Jun 7, 2015

I have used that porsche file for other purposes, and believe there is some bad data in it.

@toreyheinz

This comment has been minimized.

Copy link

@toreyheinz toreyheinz commented Jun 7, 2015

Opening and saving the file as UTF-8, allows convetSTL to recoginze it as binary. I had another version of the file that I had "resized" through SOLIDWORKS, and that one converted just fine.

I believe the "blame" lies with the porsche file. Let me know if you actually need the porsche in ascii format, and I can get it to you.

@ChocoNutDancer

This comment has been minimized.

Copy link
Author

@ChocoNutDancer ChocoNutDancer commented Jun 7, 2015

Zeroing out the 80 byte header caused convertSTL to recognize it as a binary file. But that shouldn't be necessary. If the header doesn't start with ascii "solid" (optionally prefixed by spaces) then the file should be treated as a binary format file.

The format of the binary file seems fine. It has the correct number of bytes based on the number of facets. There is nothing in the file that should make convertSTL think it is an ascii formatted STL file.

@toreyheinz

This comment has been minimized.

Copy link

@toreyheinz toreyheinz commented Jun 8, 2015

Currently the script checks if the header has "solid" in it, and if it doesn't, it treats the file as a binary file. I was curious how/why it would think the porsche file was ascii, so I examined the file directly, and the first line DOES have "solid" in it!

File.new('porsche.stl', 'r').gets
#=> "104-9.stl   tol 0.01   ac 0.02   solid  layer 209                          3DV*\x904O\u0002\u0000\xD7\xFD\u0018\xBD\u0000\u0000\u0000\...

Same thing for the brain-gear.stl

File.new('brain-gear.stl', 'r').gets
#=> "solid brain_gear\u0000STL Output                                                     \x8A#\u0001\u0000\u0000\u0000\u0000\u0000\u0000\u0000

So I'm thinking simply checking for "solid" is not a reliable way to detect binary vs ascii.

@ChocoNutDancer

This comment has been minimized.

Copy link
Author

@ChocoNutDancer ChocoNutDancer commented Jun 8, 2015

Ah, that explains what's happening.
You could skip any leading spaces and then see if the next word is "solid".

@toreyheinz

This comment has been minimized.

Copy link

@toreyheinz toreyheinz commented Jun 8, 2015

OK, so this is interesting. A number of other tools/technics fail to detect these as binary files because the first "line" has non-binary data.

http://stackoverflow.com/questions/2355866/ruby-how-to-determine-if-file-being-read-is-binary-or-text
https://github.com/djberg96/ptools

require 'ptools'
File.binary?('porsche.stl') #=> false
File.binary?('brain-gear.stl') #=> false

https://github.com/djberg96/ptools/blob/master/lib/ptools.rb#L90

def self.binary?(file)
  return false if image?(file)
  bytes = File.stat(file).blksize
  bytes = 4096 if bytes > 4096
  s = (File.read(file, bytes) || "")
  s = s.encode('US-ASCII', :undef => :replace).split(//)
  ((s.size - s.grep(" ".."~").size) / s.size.to_f) > 0.30
end

https://github.com/brianmario/charlock_holmes

require 'charlock_holmes'
CharlockHolmes::EncodingDetector.detect('porsche.stl')
#=> {:type=>:text, :encoding=>"ISO-8859-1", :ruby_encoding=>"ISO-8859-1", :confidence=>75, :language=>"en"}

CharlockHolmes::EncodingDetector.detect('brain-gear.stl')
#=> {:type=>:text, :encoding=>"ISO-8859-1", :ruby_encoding=>"ISO-8859-1", :confidence=>60, :language=>"da"}

Using the unix file command does detect the file correctly.

file porsche.stl
#=> porsche.stl: data

file brain-gear.stl
#=> brain-gear.stl: data

file cylinder.stl
#=> cylinder.stl: ASCII text

And this works as well:
https://www.ruby-forum.com/topic/122170#544577

class File
  def self.binary?(name)
    ascii = control = binary = 0

    File.open(name, "rb") {|io| io.read(1024)}.each_byte do |bt|
      case bt
        when 0...32
          control += 1
        when 32...128
          ascii += 1
        else
          binary += 1
      end
    end

    control.to_f / ascii > 0.1 || binary.to_f / ascii > 0.05
  end
end

File.binary?('porsche.stl') #=> true

Given what we've found I think we can come up with a reliable solution, but I need to set this aside for now.

@sancelot

This comment has been minimized.

Copy link

@sancelot sancelot commented Apr 10, 2018

Hi,
I had same problem, it does not recognize binary stl file.

A binary stl file is defined as follow :
BINARY STL file format is accessed by byte. The format is as follows: the first 80 bytes are used for description, and the next 4 bytes represents the total number of the facets(Long Int), followed by the facet information (normal and 3 vertices), the normal and vertices are stored in floating point format, each occupying 4 bytes. At the end of each facet information section, there are two bytes spaces, then the next facet is repeated till the end of the file. When BINARY format is used to describe STL file, the data size is much smaller than ASCII format, so most STL files available now use BINARY format.

::=
::=<80 bytes entity name, spaces are used to fill the blank>
::=<4 bytes long integer>
::=<2 bytes spaces><2 bytes spaces> ... ...
::=<lx,ly,lz, float, 12 bytes>
::=<x1,y1,z1,x2,y2,z2,x3,y3,z3, float, 36 bytes>

@hadisfr

This comment has been minimized.

Copy link

@hadisfr hadisfr commented Dec 24, 2018

@cmpolis I've got an AutoCAD-generated binary STL which is recognized as ASCII STL by the script,because it started by AutoCAD solid (containing solid).

I think

if tempLine.include? "solid"

should be changed to the following:

if tempLine.start_with? "solid"

based on the documentations on (wikipedia)[https://en.wikipedia.org/wiki/STL_(file_format)]

A binary STL file has an 80-character header (which is generally ignored, but should never begin with "solid" because that may lead some software to assume that this is an ASCII STL file).
and the aforementioned AutoCAD-generated file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.