LibreCrypt doesn't work with LUKS partitions on Windows 8.1 Pro 64 bit

[VasoVV]

I've create two LUKS partitions (with defaults) 20Gb both under Ubuntu 15.04: first on internal SSD with GPT, second on external HDD with MBR.
#1. EE: Unable to open container.

Please check your keyphrase and settings, and try again.
This error appears when I try to open one any LUKS partitions via
File / Linux container / Open LUKS partition...
#2. EE: LUKS container could not be created

Got this error when trying to create new LUKS via New... / New LUKS ...
#3. EE: Overwrite of data FAILED

Got this if on Stage 8 in New Container Wizard the "Type of overwrite data" is set to "Secure pseudorandom data"

Creation summary:

Partition: \Device\Harddisk1\Partition2
Container size: 21472738816 + 512 (for CDB) = 21472739328 bytes
CDB stored: At start of container file
Hash algorithm: SHA-512
  [Hash driver: \Device\FreeOTFE\Hash\{00000000-0000-0000-0000-0000000D0001}]
  [Hash GUID: {00000000-0000-0000-0000-0000000D0006}]
Key iterations: 2048
Cypher has fixed, defined blocksize; sector IVs will be used
Sector IV generation method: Null IV
A per-container IV will not be used
Cypher algorithm: AES (256 bit XTS)
  [Cypher driver: \Device\FreeOTFE\Cypher\{00000000-0000-0000-0000-000000010001}]
  [Cypher GUID: {00000000-0000-0000-0000-000000010204}]
Master key length: 512 bits
RNG: Microsoft CryptoAPI, cryptlib
Password: <entered>
Salt length: 256 bits
Requested drive letter: Use default

#4. Dump LUKS details is broken

LIbreCrypt produces that dump for both partitions:

LUKS Dump
=========

Dump Created By
---------------
Platform              : PC
Application version   : v6.2.5613.42403
Driver ID             : v5.00.0000
ERROR: Unable to read LUKS header?!

#4. FreeOTFE

At this time FreeOTFE almost successfully dump LUKS details from both partitions (only Master Key couldn't be recovered from drive with GPT) and could mount one of them (from drive with MBR of course)

LUKS partition on drive with GPT

LUKS Dump
=========

Dump Created By
---------------
Platform              : PC
Application version   : v5.21.00.4058
Driver ID             : v5.00.0000


cryptsetup Style Dump
---------------------
LUKS header information for \Device\Harddisk0\Partition4

Version:        1
Cipher name:    aes
Cipher mode:    xts-plain64
Hash spec:      sha1
Payload offset: 4096
MK bits:        256
MK digest:      da 91 ec 91 36 17 1e 8a 57 0c 55 8d 1b 5d e5 8b ef dc 29 44 
MK salt:        ba 56 99 84 da 7b b9 00 cc ec e3 69 4e 4d 36 af 
                b5 74 7c be ff ef a6 51 1d e2 a7 38 46 cf b8 23 
MK iterations:  121125
UUID:           263e4aeb-48c1-48bb-8ded-47145eb62930

Key Slot 0: ENABLED
        Iterations:             484847
        Salt:                   ef 04 f6 6b 05 e5 9e ad ae 48 f6 14 87 ad b6 f9 
                                67 c0 85 11 1b d0 65 23 c1 13 68 d3 b3 51 5d 55 
        Key material offset:    8
        AF stripes:             4000
Key Slot 1: DISABLED
Key Slot 2: DISABLED
Key Slot 3: DISABLED
Key Slot 4: DISABLED
Key Slot 5: DISABLED
Key Slot 6: DISABLED
Key Slot 7: DISABLED


Mapped FreeOTFE Drivers
-----------------------
Hash pretty title         : SHA-1 (160/512)
Hash driver KM name       : \Device\FreeOTFE\Hash\{00000000-0000-0000-0000-0000000D0001}
Hash GUID                 : {00000000-0000-0000-0000-0000000D0002}
Cypher pretty title       : AES (XTS; 128/128)
Cypher driver KM name     : \Device\FreeOTFE\Cypher\{00000000-0000-0000-0000-000000010001}
Cypher GUID               : {00000000-0000-0000-0000-000000010202}
Sector IV generation      : 32 bit sector ID


Master Key
----------
User supplied password   : lollipop
No master key could be recovered with the specified password.

LUKS partition on drive with MBR

LUKS Dump
=========

Dump Created By
---------------
Platform              : PC
Application version   : v5.21.00.4058
Driver ID             : v5.00.0000


cryptsetup Style Dump
---------------------
LUKS header information for \Device\Harddisk1\Partition2

Version:        1
Cipher name:    aes
Cipher mode:    xts-plain64
Hash spec:      sha1
Payload offset: 4096
MK bits:        256
MK digest:      d2 ab 01 c1 9a f6 09 35 d3 a6 96 1c d6 cb 7e 40 87 c1 d8 b5 
MK salt:        3e b0 81 a5 a8 a4 0e d4 4c f8 3f f5 8d db 30 39 
                c7 97 e4 47 6f f8 1c dd 6a a4 a1 99 20 ea 3c 5e 
MK iterations:  119375
UUID:           6e97847a-a223-45d6-97b2-2d968f72148b

Key Slot 0: ENABLED
        Iterations:             488548
        Salt:                   f6 41 6f e1 41 4b 5a 2b 90 03 5f f1 05 8d 78 21 
                                0b 90 d6 38 48 03 63 32 35 16 56 fd e3 1a 0f c3 
        Key material offset:    8
        AF stripes:             4000
Key Slot 1: DISABLED
Key Slot 2: DISABLED
Key Slot 3: DISABLED
Key Slot 4: DISABLED
Key Slot 5: DISABLED
Key Slot 6: DISABLED
Key Slot 7: DISABLED


Mapped FreeOTFE Drivers
-----------------------
Hash pretty title         : SHA-1 (160/512)
Hash driver KM name       : \Device\FreeOTFE\Hash\{00000000-0000-0000-0000-0000000D0001}
Hash GUID                 : {00000000-0000-0000-0000-0000000D0002}
Cypher pretty title       : AES (XTS; 128/128)
Cypher driver KM name     : \Device\FreeOTFE\Cypher\{00000000-0000-0000-0000-000000010001}
Cypher GUID               : {00000000-0000-0000-0000-000000010202}
Sector IV generation      : 32 bit sector ID


Master Key
----------
User supplied password   : lollipop
Password unlocks key slot: 0
Recovered master key     :
00000000 | BB AB 5F 91 31 DC 93 1A | .._.1...
00000008 | 29 63 83 7B 2A C2 A1 C7 | )c.{*...
00000010 | F5 BB 10 68 9A 07 4F B1 | ...h..O.
00000018 | E1 33 87 E4 51 92 7D 3E | .3..Q.}>

[linux-modder]

@VasoVV, will be taking look this week, While I don't have a windows box handy I will use friends and (hopefully) your feedback to help close this issue... A very damning one at that..

@t-d-k mind assigning for me as i can't seem to do so myself

[t-d-k]

@VasoVV Thanks for reporting these with so much detail.

• Create LUKS
  This feature is work in progress, and I wouldn't expect it to work. The bug is that the menu item was left enabled in the release build. For now LibreCrypt can only open LUKS containers created on 'nix, or create FreeOTFE type containers.
• FreeOTFE drivers
  It looks like you have FreeOTFE and LibreCrypt installed on the same PC. This doesn't work the way you would expect.
  The drivers for both have the same names, meaning either will use the drivers for the other. If FreeOTFE was installed first, LibreCrypt will be using the FreeOTFE drivers.
  The drivers changed in version 6.2 to add GPT support, so this would explain LibreCrypt not opening the GPT partition (but not the MBR partition).
  Could you please uninstall FreeOTFE and LibreCrypt, then reboot and install LibreCrypt 6.2. After this if you start FreeOTFE it should detect and use the LC drivers (i.e. it won't prompt to start in portable mode).
  It would be interesting to see if FreeOTFE can still open the MBR partition after this - if it can't, it implies a problem in the drivers.
• Opening LUKS containers.
  When you browse for the LUKS partition in LC, does it show the LUKS partition as a "Windows data partition" and with the correct size?
  Are you using LVM on any partition?
  Could you please also try running LibreCrypt as administrator and dumping the LUKS data, and let me know how you get on.

@linux-modder I'll assign it to you.

thanks
tdk

[VasoVV]

@t-d-k

• Create LUKS
  This feature is work in progress […]

Please, temporary make such warning message instead of error, until this feature will be implemented. It will save time and nerves for new users.

FreeOTFE drivers
It looks like you have FreeOTFE and LibreCrypt installed on the same PC. This doesn't work the way you would expect.

I'd installed FreeOTFE only after LibreCrypt totally failed. So the behavior is described in my first post I discovered on clean Windows installation with a few tools installed (without FreeOTFE).

The drivers for both have the same names, meaning either will use the drivers for the other. If FreeOTFE was installed first, LibreCrypt will be using the FreeOTFE drivers.

Strange observation: After first installation FreeOTFE shows that it drivers are not loaded, but LibreCrypt does. I had to start drivers manually in FreeOTFE to make this one work. But now after numerous un/installing, I can't reproduce this.

The drivers changed in version 6.2 to add GPT support, so this would explain LibreCrypt not opening the GPT partition (but not the MBR partition).
Could you please uninstall FreeOTFE and LibreCrypt, then reboot and install LibreCrypt 6.2. After this if you start FreeOTFE it should detect and use the LC drivers (i.e. it won't prompt to start in portable mode).
It would be interesting to see if FreeOTFE can still open the MBR partition after this - if it can't, it implies a problem in the drivers.

Yes, as I mention above.

If FreeOTFE is installed before LibreCrypt, the behavior of both is the same.

Opening LUKS containers.
When you browse for the LUKS partition in LC, does it show the LUKS partition as a "Windows data partition" and with the correct size?
On GPT disk – "Windows data partition", 19,9GB
On MB – FAT16, 19,99GB

Are you using LVM on any partition?

No.

Here are some additional info:

$ sudo parted /dev/sda print
Model: ATA MTFDDAK128MAM-1J (scsi)
Disk /dev/sda: 128GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags: 

Number  Start   End     Size    File system  Name                          Flags
 1      1049kB  316MB   315MB   ext2                                       msftdata
 2      316MB   840MB   524MB   ext4
 3      840MB   52.4GB  51.5GB
 4      52.4GB  73.9GB  21.5GB               Basic data partition          msftdata #<---LUKS
 5      73.9GB  74.2GB  315MB   ntfs         Basic data partition          hidden, diag
 6      74.2GB  74.3GB  105MB   fat32        EFI system partition          boot, esp
 7      74.3GB  74.4GB  134MB                Microsoft reserved partition  msftres
 8      74.4GB  128GB   53.2GB               Basic data partition          msftdata
 9      128GB   128GB   477MB   ntfs                                       hidden, diag

$ sudo parted /dev/sdb print
Model: TOSHIBA MK3252GSX (scsi)
Disk /dev/sdb: 320GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags: 

Number  Start   End    Size    Type     File system  Flags
 1      1049kB  299GB  299GB   primary  ntfs
 2      299GB   320GB  21.5GB  primary #<---LUKS

After unLUKSing as "meta" and "meta2":

$ sudo ntfsinfo -m /dev/mapper/meta
Volume Information 
        Name of device: /dev/mapper/meta
        Device state: 11
        Volume Name: Meta
        Volume State: 91
        Volume Flags: 0x0000
        Volume Version: 3.1
        Sector Size: 512
        Cluster Size: 4096
        Index Block Size: 4096
        Volume Size in Clusters: 5242111
MFT Information 
        MFT Record Size: 1024
        MFT Zone Multiplier: 0
        MFT Data Position: 24
        MFT Zone Start: 0
        MFT Zone End: 655267
        MFT Zone Position: 4
        Current Position in First Data Zone: 655267
        Current Position in Second Data Zone: 0
        Allocated clusters 7 (0.0%)
        LCN of Data Attribute for FILE_MFT: 4
        FILE_MFTMirr Size: 4
        LCN of Data Attribute for File_MFTMirr: 2621055
        Size of Attribute Definition Table: 2560
        Number of Attached Extent Inodes: 0
FILE_Bitmap Information 
        FILE_Bitmap MFT Record Number: 6
        State of FILE_Bitmap Inode: 80
        Length of Attribute List: 0
        Number of Attached Extent Inodes: 0
FILE_Bitmap Data Attribute Information
        Decompressed Runlist: not done yet
        Base Inode: 6
        Attribute Types: not done yet
        Attribute Name Length: 0
        Attribute State: 3
        Attribute Allocated Size: 655360
        Attribute Data Size: 655264
        Attribute Initialized Size: 655264
        Attribute Compressed Size: 0
        Compression Block Size: 0
        Compression Block Size Bits: 0
        Compression Block Clusters: 0
        Free Clusters: 5225455 (99.7%)

$ sudo ntfsinfo -m /dev/mapper/meta2
Volume Information 
        Name of device: /dev/mapper/meta2
        Device state: 11
        Volume Name: Meta2
        Volume State: 91
        Volume Flags: 0x0000
        Volume Version: 3.1
        Sector Size: 512
        Cluster Size: 4096
        Index Block Size: 4096
        Volume Size in Clusters: 5241855
MFT Information 
        MFT Record Size: 1024
        MFT Zone Multiplier: 0
        MFT Data Position: 24
        MFT Zone Start: 0
        MFT Zone End: 655235
        MFT Zone Position: 4
        Current Position in First Data Zone: 655235
        Current Position in Second Data Zone: 0
        Allocated clusters 64 (0,0%)
        LCN of Data Attribute for FILE_MFT: 4
        FILE_MFTMirr Size: 4
        LCN of Data Attribute for File_MFTMirr: 2620927
        Size of Attribute Definition Table: 2560
        Number of Attached Extent Inodes: 0
FILE_Bitmap Information 
        FILE_Bitmap MFT Record Number: 6
        State of FILE_Bitmap Inode: 80
        Length of Attribute List: 0
        Number of Attached Extent Inodes: 0
FILE_Bitmap Data Attribute Information
        Decompressed Runlist: not done yet
        Base Inode: 6
        Attribute Types: not done yet
        Attribute Name Length: 0
        Attribute State: 3
        Attribute Allocated Size: 655360
        Attribute Data Size: 655232
        Attribute Initialized Size: 655232
        Attribute Compressed Size: 0
        Compression Block Size: 0
        Compression Block Size Bits: 0
        Compression Block Clusters: 0
        Free Clusters: 5217443 (99,5%)

Could you please also try running LibreCrypt as administrator and dumping the LUKS data, and let me know how you get on.

I did this also before opening the issuer.

[linux-modder]

@t-d-k what would lvm on the ntfs luks have to do with it?

[t-d-k]

@VasoVV

If FreeOTFE is installed before LibreCrypt, the behavior of both is the same

The same as you described in your first post, or the same as each other?
This is a puzzling bug - it looks like a permissions issue on Windows 8.1 - but running as admin should fix that. It is also puzzling that it works for FreeOTFE and not LibreCrypt - is FreeOTFE being run in 'compatability mode'? If so could you try the same for LC.

Can you please also test with the file luks.box under the test_vols folder https://github.com/t-d-k/librecrypt/tree/master/test_vols. The password is 'password' and it should open with all default options.

Unfortunately I can't try to reproduce this at the moment, I only have Windows 8 in a virtual machine, and this doesn't support direct access to devices. LC works fine on Windows 8 when accessing file-based containers for me.
I'll look into setting up a better Windows 8 test machine and get back to you.
tdk

@linux-modder
The other LUKS issue is using LVM, so I am trying to find out if these have the same cause.
tdk

[linux-modder]

@t-d-k , whats the lvm issue I use solely lvm can surely test that

[t-d-k]

@linux-modder
The possibly lvm-related issue is this one 22
I'll add a note there.

[VasoVV]

@t-d-k

If FreeOTFE is installed before LibreCrypt, the behavior of both is the same

The same as you described in your first post, or the same as each other?

As in first post.

This is a puzzling bug - it looks like a permissions issue on Windows 8.1 - but running as admin should fix that.

It doesn't. I always try run apps with administrator privileges if something doesn't work.
Observation: FreeOTFE without admin permissions can not see partitions' attributes, but LibreCrypt can.

It is also puzzling that it works for FreeOTFE and not LibreCrypt - is FreeOTFE being run in 'compatability mode'? If so could you try the same for LC.

No, FreeOTFE running in native mode. I ran both apps in compatibility mode before, but without any success.

Can you please also test with the file luks.box under the test_vols folder https://github.com/t-d-k/librecrypt/tree/master/test_vols. The password is 'password' and it should open with all default options.

EE: Unable to open container. Please check your keyphrase and settings, and try again.

Unfortunately I can't try to reproduce this at the moment, I only have Windows 8 in a virtual machine, and this doesn't support direct access to devices. LC works fine on Windows 8 when accessing file-based containers for me.
I'll look into setting up a better Windows 8 test machine and get back to you.
tdk

I was able to reproduce this bug on another Windows 8.1 Pro 64 bit PC.

[t-d-k]

HI @mmatuska , I have moved your responses to the new issue you have opened, as this may be a different issue to @VasoVV 's, which is specific to LUKs and Windows 8.1

[t-d-k]

I have found a bug relating to luks.box, a LUKS volume will only open if it is writeable, even if opening read-only. This is fixed for the next release.
Could you please retest with "luks.box", after setting the file to writeable?
This still doesn't explain the problems with partitions that may be another bug.
thanks
tdk

[t-d-k]

@VasoVV I have reproduced your point number 3: "Overwrite of data FAILED" and this is fixed for 6.3.