Unetbootin does not recognize FAT32 USB drives unless the name is "FAT32" #86

Open
randomInteger opened this Issue Sep 21, 2016 · 29 comments

Comments

Projects
None yet

randomInteger commented Sep 21, 2016

As per comment 46 here:

#46 (comment)

This is an issue that should be fixed as soon as possible. I encountered this when trying to copy the ESXi 5.0 installer onto a usb drive for install on a blade with no cd/dvd drive.

The ONLY way to make Unetbootin recognize the usb drive in /dev/disk2 was to format it as FAT32, with GUID Partitioning, and then manually set the NAME of the drive to "FAT32". I tried about a dozen other permutations of fat/exfat and partitioning schemes + various names before finding that comment. Worked like a charm...only if the name was "FAT32". I suspect some kind of regex matching is looking for a hit on ['FAT32|'fat32'] and that may be why the name is the key to getting the device recognized.

This was on the very latest Unetbootin (unetbootin-mac-625.dmg) as of 9/16/16, OSX El Capitan 10.11.6.

I formatted the drive via OS X Disk Utility.

The drive is a Samsung 32GB BAR USB 3.0 Flash Drive (MUF-32BA/AM).

Contributor

michaelqyee commented Sep 21, 2016

If you still have the drive that doesn't work, could you please plug it in, open Terminal and paste the results of the command "diskutil list"? I also am using El Capitan and unetbootin successfully recognized a USB drive named "USB20FD".

Looking at line 619 of unetbootin.cpp, it should be filtering correctly (and actually should actually falsely list disks that have names that contain "FAT" or "Microsoft", even if they're not FAT32):
#ifdef Q_OS_MAC QString diskutilList = callexternapp("diskutil", "list"); QStringList usbdevsL = diskutilList.split("\n").filter(QRegExp("(FAT|Microsoft)")).join(" ").split(" ").filter("disk"); for (int i = 0; i < usbdevsL.size(); ++i) { fulldrivelist.append("/dev/"+usbdevsL.at(i)); } #endif

Hello Michael.

Yes I just reformatted the drive back to FAT32, GUID, with the name "UNTITLED" and it immediately reproduces the issue. Please see the attached screenshot, which captures the output from "disktuil list" as well as the unetbootin GUI and OS X Disk Utility.
screen shot 2016-09-21 at 3 38 30 pm

And here is another screenshot, showing the same drive, same formatting, but with the name set to "FAT32" and you can see that unetbootin now recognizes the drive:

screen shot 2016-09-21 at 3 41 48 pm

Hope that is helpful. I am an SDET working in the Network Security space, so if you want/need more detail, just let me know what else to dig into and I will happily provide more data.

I love this tool and I am happy to help with this issue.

Contributor

michaelqyee commented Sep 21, 2016

I'm not exactly sure why, but it seems like you only have the volume and not an underlying disk in your screenshots. The person in this thread seems to have had a similar issue, and the presumed fix was to create a partition on the disk. For comparison, here's what happens when I execute "diskutil list":
image

You'll notice that in my output, I also have an entry under "Type"; the code currently depends on Type being set to something that contains "FAT" or "Microsoft", but the code doesn't specifically check the "Type" column, so those words could really appear anywhere in the line.

I tried several permutations of erasing the disk or volume (erasing the volume using Disk Utility, erasing the disk using Disk Utility, using the command "diskutil eraseDisk FAT32 UNTITLED /dev/disk2"), but I wasn't able to get rid of line 0 denoting the partition scheme on the USB drive. Could you provide the steps for how you're reformatting the disk?

Yes the steps are simply to use the OS X Disk Utility.

  1. Choose the USB flash drive.
  2. Click "erase" and when prompted, enter a name, choose FAT, and GUID partitioning.

If you use "FAT32" as the name, unetbootin sees it...other names cause it to not be seen.

Its strange right?

It could be simply an artifact of how the Disk Utility formats in FAT, but the fact that the same FAT/GUID with a different name will cause the drive to not be seen is very strange.

If you can't resolve this, I think it may be worth noting in the README or the release notes/help so that others are not confused by this as I was.

Contributor

michaelqyee commented Sep 22, 2016

The strange thing is that I am able to reformat the drive in a way that unetbootin successfully recognizes the drive also using Disk Utility. Therefore I believe it cannot be an artifact of how Disk Utility formats in FAT.

Your approach is identical to mine, so I believe that there is some sort of intrinsic difference in our USB drives. For your disk, can you run the command "diskutil partitionDisk /dev/disk2 GPT JHFS+ UNTITLED 0b"? This will erase the disk completely. It should also hopefully create a volume where it seems it only has a disk today.

After you do that, could you please

  1. Run "diskutil list" and show the results,
  2. Run Disk Utility and take a screen shot of the disk (and hopefully volume),
  3. Reformat the drive in Disk Utility to use FAT and MBR,
  4. Attempt to run unetbootin to see the disk

Sure!

Here is "diskutil list" from before the command:

#diskutil list
/dev/disk0 (internal, physical):
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *500.3 GB disk0
1: EFI EFI 209.7 MB disk0s1
2: Apple_CoreStorage Macintosh HD 499.4 GB disk0s2
3: Apple_Boot Recovery HD 650.0 MB disk0s3
/dev/disk1 (internal, virtual):
#: TYPE NAME SIZE IDENTIFIER
0: Apple_HFS Macintosh HD +499.1 GB disk1
Logical Volume on disk0s2
980B1CDB-68D0-48C0-AB7D-DFBC1AC44420
Unencrypted
/dev/disk2 (external, physical):
#: TYPE NAME SIZE IDENTIFIER
0: FAT32 *32.1 GB disk2
/dev/disk5 (disk image):
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme +29.2 MB disk5
1: Apple_HFS UNetbootin 29.1 MB disk5s1

Your command:

#diskutil partitionDisk /dev/disk2 GPT JHFS+ UNTITLED 0b
Started partitioning on disk2 FAT32
Unmounting disk
Creating the partition map
Waiting for the disks to reappear
Formatting disk2s2 as Mac OS Extended (Journaled) with name UNTITLED
Initialized /dev/rdisk2s2 as a 30 GB case-insensitive HFS Plus volume with a 8192k journal
Mounting disk
Finished partitioning on disk2 FAT32
/dev/disk2 (external, physical):
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *32.1 GB disk2
1: EFI EFI 209.7 MB disk2s1
2: Apple_HFS UNTITLED 31.7 GB disk2s2

And the resulting "diskutil list":

#diskutil list
/dev/disk0 (internal, physical):
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *500.3 GB disk0
1: EFI EFI 209.7 MB disk0s1
2: Apple_CoreStorage Macintosh HD 499.4 GB disk0s2
3: Apple_Boot Recovery HD 650.0 MB disk0s3
/dev/disk1 (internal, virtual):
#: TYPE NAME SIZE IDENTIFIER
0: Apple_HFS Macintosh HD +499.1 GB disk1
Logical Volume on disk0s2
980B1CDB-68D0-48C0-AB7D-DFBC1AC44420
Unencrypted
/dev/disk2 (external, physical):
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *32.1 GB disk2
1: EFI EFI 209.7 MB disk2s1
2: Apple_HFS UNTITLED 31.7 GB disk2s2
/dev/disk5 (disk image):
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme +29.2 MB disk5
1: Apple_HFS UNetbootin 29.1 MB disk5s1

The state of the disk in Disk Utility:

screen shot 2016-09-22 at 5 34 19 pm

Next, the reformat. I chose to reformat the entire drive, not just the "untitled" partition because you requested MBR, which is only an option if I format the entire drive...I do not get that option if I only reformat the untitled partition.

Here it is post reformat as you requested:

screen shot 2016-09-22 at 5 38 32 pm

And unetbootin DOES see the drive, even though the name is "UNTITLED"!

screen shot 2016-09-22 at 5 39 24 pm

Also now this is the output from disktuil:

/dev/disk2 (external, physical):
#: TYPE NAME SIZE IDENTIFIER
0: FDisk_partition_scheme *32.1 GB disk2
1: DOS_FAT_32 UNTITLED 32.1 GB disk2s1

So then...what does this tell us?

It seems like my drive (and others based on that other issue thread) can get into a state where a basic fat/MBR format doesn't produce the correct disk state for unetbootin to see the drive as a valid candidate...and somehow changing the name to "FAT32" is a workaround.

I think I see the issue.

In the "bad" state, there is NO type listed by diskutil:

/dev/disk2 (external, physical):
#: TYPE NAME SIZE IDENTIFIER
0: UNTITLED *32.1 GB disk2

In that state, you will not have a match on "FAT" as a pattern because it would not be there at all to be matched...unless you set the name!

/dev/disk2 (external, physical):
#: TYPE NAME SIZE IDENTIFIER
0: FAT32 *32.1 GB disk2

But when the drive is formatted a different way, the TYPE field is correctly written and WILL contain the "FAT" string pattern so that the name is no longer needed to match.

If you cannot think of a way to improve the logic to work around this issue, my suggestion is that you document in your release notes or in the readme, that your code is looking for "FAT" in the output of diskutil, and so the user needs to verify that FAT is in the TYPE field, and format the drive in such a way that the TYPE is FAT/FAT32.

Contributor

michaelqyee commented Sep 23, 2016

Okay, so that's good to know that the hunch for partitioning was correct.

Now the question is how to get a drive back INTO the state where there was no partitioning at all. I've been looking into this, as there's no way to verify that the issue is fixed unless we have an unpartitioned drive that has a blank "Type" as the output of "diskutil list". Once we have that, my current hunch is to use the information found in "system_profiler SPStorageDataType" to distinguish the formatting type for the mounted drive as the criteria to display it in the dropdown list. Unsuccessful things I've tried so far to reproduce the no partitioning scheme:

  • Using the "Erase" function of Disk Utility
  • Using the command "gpt destroy /dev/disk2" then use the "Erase" function of Disk Utility to reformat
  • Using the command "dd if=/dev/zero of=/dev/disk2 bs=1 count=(size of disk)" then use the "Erase" function of Disk Utility to reformat

The last two of these do not hold up after using "Erase" again in Disk Utility (i.e., they create a partition), so I don't think that it's a reproduction of what you were doing. I only have access to a Mac and don't own Windows, so I can't try any of the formatting tools available there. Did you previously format the disk you had through some other means, or was it straight from the store?

Also, for clarification, I'm not the owner of unetbootin - I just happened to be using it on the day you filed the issue (I was looking through the code for why it required elevated privileges to open besides being unsigned), and I happened upon your issue, which I was curious about because I hadn't run into it.

It's worth noting that the owner, @gkovacs, provided a possible workaround for this in #46 (comment), which is already documented in the wiki.

Sure Michael, thanks.

I will experiment on my side with trying to get the drive back into that state. I believe that at one point, I took it over to a windows machine and formatted it with the Windows formatting utility...so maybe that is a good place for me to start looking. I am super busy today, but I think I can put a few cycles on this early next week.

Thank you for your interest.

@gkovacs gkovacs added the bug label Apr 23, 2017

dpgailey commented May 5, 2017

I can't even get this working with a FAT32 + GUID + renaming it "FAT32" on OSX 10.10.5

endeebee commented May 6, 2017

Same problem here. My setup:
Samsung T3 SSD 500GB, OS X 10.10.5, latest UnetBootin. SSD is formatted with a single FAT32 partition called "FAT32", however no drive appears in the dropdown menu to select as a target for the .iso.

The SSD probably reports itself differently than a regular USB flash drive.

endeebee commented May 6, 2017

Is that intended behavior or can I provide some information to help add this type of drive to Unetbootin?

The SSD probably shows itself as a hard drive, rather than a USB drive, does it show up if you set "Type:" to "Hard Drive"?

endeebee commented May 6, 2017

For the Type, I have only a single option in the dropdown saying "USB Drive".

artur-pub commented May 15, 2017

I can't even get this working with a FAT32 + GUID + renaming it "FAT32" on OSX 10.10.5

I'm experiencing it with current 647 build. Build 638 (couldn't find .dmg in older releases) recognized my usb stick. 10.10.5 (Yosemite) + FAT32 + GUID + "FAT" + build 638; this combination works for me.

artur-pub commented May 15, 2017

These lines in build 647, src/unetbootin/unetbootin.cpp:

#ifdef Q_OS_MAC
QString diskutilList = callexternapp("diskutil", "list");
QStringList usbdevsL = diskutilList.split("\n").filter(QRegExp("(FAT|Microsoft)")).join(" ").split(" ").filter("disk");
for (int i = 0; i < usbdevsL.size(); ++i)
{
    if (is_external_drive_macos(usbdevsL.at(i))) {
        fulldrivelist.append("/dev/"+usbdevsL.at(i));
    }
}
#endif

... execute diskutil list and search for Microsoft of FAT. My experiments show that these strings appear there in any way, but the problem is in is_external_drive_macos call:

bool unetbootin::is_external_drive_macos(const QString &drivename)
{
    // drivename: disk3s1
    QString device_info = callexternapp("diskutil", "info " + drivename);
    if (device_info.contains("External"))
    {
        return true;
    }
    return false;
}

The output of diskutil info disk1s1 contains no External string. I'm providing full outputs of both commands here. The stick is fresh-formatted as UNTITLED and FAT in Disk Utility.

$ diskutil list
/dev/disk0
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *500.1 GB   disk0
   1:                        EFI EFI                     209.7 MB   disk0s1
   2:                  Apple_HFS Macintosh HD            499.2 GB   disk0s2
   3:                 Apple_Boot Recovery HD             650.0 MB   disk0s3
/dev/disk1
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:     FDisk_partition_scheme                        *31.6 GB    disk1
   1:                 DOS_FAT_32 UNTITLED                31.6 GB    disk1s1
/dev/disk2
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:     Apple_partition_scheme                        *35.1 MB    disk2
   1:        Apple_partition_map                         32.3 KB    disk2s1
   2:                  Apple_HFS UNetbootin              35.1 MB    disk2s2

$ diskutil info disk1s1
   Device Identifier:        disk1s1
   Device Node:              /dev/disk1s1
   Part of Whole:            disk1
   Device / Media Name:      Untitled 1

   Volume Name:              UNTITLED

   Mounted:                  Yes
   Mount Point:              /Volumes/UNTITLED

   File System Personality:  MS-DOS FAT32
   Type (Bundle):            msdos
   Name (User Visible):      MS-DOS (FAT32)

   Partition Type:           DOS_FAT_32
   OS Can Be Installed:      No
   Media Type:               Generic
   Protocol:                 USB
   SMART Status:             Not Supported
   Volume UUID:              75785FE3-FFFF-396F-BA83-CFB9A882BBA1

   Total Size:               31.6 GB (31591496704 Bytes) (exactly 61702142 512-Byte-Units)
   Volume Free Space:        31.6 GB (31574507520 Bytes) (exactly 61668960 512-Byte-Units)
   Device Block Size:        512 Bytes
   Allocation Block Size:    16384 Bytes

   Read-Only Media:          No
   Read-Only Volume:         No
   Ejectable:                Yes

   Whole:                    No
   Internal:                 No

My guess is that the last line Internal: No is brand new way to tell that it is "External".

artur-pub commented May 15, 2017

I quickly checked release 638 and it doesn't check for is_external_drive_macos:

#ifdef Q_OS_MAC
QString diskutilList = callexternapp("diskutil", "list");
QStringList usbdevsL = diskutilList.split("\n").filter(QRegExp("(FAT|Microsoft)")).join(" ").split(" ").filter("disk");
for (int i = 0; i < usbdevsL.size(); ++i)
{
    fulldrivelist.append("/dev/"+usbdevsL.at(i));
}
#endif

That's very probably why it worked for me.

UPD: this untested patch for release 647 should in theory fix the issue:

 bool unetbootin::is_external_drive_macos(const QString &drivename)
 {
     // drivename: disk3s1
     QString device_info = callexternapp("diskutil", "info " + drivename);
-    if (device_info.contains("External"))
+    QStringList internal_is_no = device_info.split("\n").filter(QRegExp("Internal:\\s*No"));
+    if (device_info.contains("External") || internal_is_no.size() > 0)
     {
         return true;
     }
     return false;
 }
Contributor

michaelqyee commented May 16, 2017

So I haven't been able to build unetbootin because of QT4 not being well-supported for newer Mac OS versions, but I put some code up on https://github.com/michaelqyee/unetbootin if anyone wants to give it a whirl. At a high level, it uses "system_profiler SPStorageDataType" instead of "diskutil list" to detect FAT32. @artur-pub, I combined in the changes you suggested as well.

If anyone has any tips for building QT4 on El Capitan or Sierra, I'd be willing to give this a whirl.

Owner

gkovacs commented May 16, 2017

@michaelqyee thank you for the patch! I will test it. for build instructions, try https://github.com/unetbootin/unetbootin/wiki/compile#instructions-mac-os-x - I have a prebuilt version that works on sierra there ( qt4.8.7-mac-x86.tar.xz )

Contributor

michaelqyee commented May 16, 2017

Sorry, I messed up a really dumb thing on the patch, but I think I fixed it now and it's on my repo (although I can't test). I think there's an issue with the prebuilt version where it's not the right architecture. Or the build file is trying to build for the wrong architecture. I'm not sure. http://stackoverflow.com/questions/30948807/static-library-link-issue-with-mac-os-x-symbols-not-found-for-architecture-x8 suggests the library may be to blame. I tried changing to libstdc++ per http://stackoverflow.com/questions/19637164/c-linking-error-after-upgrading-to-mac-os-x-10-9-xcode-5-0-1, but just ended up getting "libstdc++ is deprecated" warning. I think this is the issue:

ld: warning: ignoring file /Users/michaelyee/Documents/Development/unetbootin/src/unetbootin/qt4-mac/lib/QtGui.framework/QtGui, file was built for i386 which is not the architecture being linked (x86_64): /Users/michaelyee/Documents/Development/unetbootin/src/unetbootin/qt4-mac/lib/QtGui.framework/QtGui
ld: warning: ignoring file /Users/michaelyee/Documents/Development/unetbootin/src/unetbootin/qt4-mac/lib/QtCore.framework/QtCore, file was built for i386 which is not the architecture being linked (x86_64): /Users/michaelyee/Documents/Development/unetbootin/src/unetbootin/qt4-mac/lib/QtCore.framework/QtCore
ld: warning: ignoring file /Users/michaelyee/Documents/Development/unetbootin/src/unetbootin/qt4-mac/lib/QtNetwork.framework/QtNetwork, file was built for i386 which is not the architecture being linked (x86_64): /Users/michaelyee/Documents/Development/unetbootin/src/unetbootin/qt4-mac/lib/QtNetwork.framework/QtNetwork

Contributor

michaelqyee commented May 22, 2017

I was finally able to get the build to work well enough for me to test. My previously installed version of Qt4 in /usr/local/Trolltech really messed up the process, so I ended up deleting the version I had previously installed and replacing it with the static build Geza had uploaded.

I also replaced all instances of "/Users/geza/qt4-mac/" with my own user and path. I think it was mainly in the .prl files in the Qt framework folders (e.g., QtNetwork.prl). @gkovacs, you may want to look at this because it was leading to a warning on build. You might be able to recreate this if you build with another user on your computer besides "geza".

I did verify that the fix allows you to see FAT32-formatted drives without renaming them, but I didn't do much negative testing because I lost many of my USB drives and didn't get to reformat to another type.

craigmacgregor commented Jun 4, 2017

I just had this issue and it was resolved by manually targeting the USB drive via command line options:

/Volumes/UNetbootin/unetbootin.app/Contents/MacOS/unetbootin installtype=USB targetdrive=/dev/disk2s1

Replacing "/dev/disk2s1" with whichever partition is the correct one when running "diskutil list"

https://github.com/unetbootin/unetbootin/wiki/commands#targetdrive

Alternatively, just write the image to the USB using the diskutil directly:

https://help.ubuntu.com/community/How%20to%20install%20Ubuntu%20on%20MacBook%20using%20USB%20Stick

TipiT commented Jun 27, 2017

Thanks, craigmacgregor! Targeting the USB drive via command line work exactly as you described.
The "FAT32" name changing trick did not work. :D

gverm commented Jul 12, 2017

This was fixed, right @gkovacs ?

Contributor

michaelqyee commented Jul 14, 2017

Geza merged my pull request, which I believe should have fixed it. Have you tried installing again?

perojozo123 commented Aug 11, 2017

The soulution for Windows is to execute a command from cmd and it will popup the app's form with the selected USB. Still can't beliave it works...
C:\unetbootin-windows-655.exe installtype=USB targetdrive=G: isofile=C:\Downloads\someFileName.iso

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment