Skip to content
This repository has been archived by the owner on Apr 16, 2022. It is now read-only.

Exporting encrypted csv file using ODK Briefcase fails on Windows #65

Closed
yanokwa opened this issue Mar 4, 2017 · 12 comments
Closed

Exporting encrypted csv file using ODK Briefcase fails on Windows #65

yanokwa opened this issue Mar 4, 2017 · 12 comments
Labels
bug infrastructure

Comments

@yanokwa
Copy link
Member

yanokwa commented Mar 4, 2017 •
edited

I have been trying to export my encrypted test data using ODK Briefcase and my public-private key pair, but every time I export it, the csv file is blank except for the variable names in the header. Has anyone run into this problem before or does anyone know if I am doing anything wrong.

I never noticed the message in the corner but no it did not complete. I forgot to download the JCE but after I did so, it still failed. Here is my error:

Error decrypting submission uuid8d2c8480-d625-4058-86e2-57aab74f124a Cause: org.opendatakit.briefcase.model.CryptoException: Error decrypting base64EncryptedKey Cause: java.security.NoSuchAlgorithmException: Cannot find any provider supporting RSA/NONE/OAEPWithSHA256AndMGF1Padding
FAILED!

I am using java 8 on my computer but downloaded JCE for the Java 7 version since it said so in the manual. Should I downgrade my java to java 7?

– bendo28@gmail.com

Software versions

Briefcase v1.x.x, Java v1.x.x, operating system, Aggregate v1.x.x, Collect v1.x.x...

Problem description

Steps to reproduce the problem

Expected behavior

Other information

Things you tried, stack traces, related issues, suggestions on how to fix it...
@SteadyCadence
Copy link

What files are you loading up in ODK Briefcase? Do you have the form XML in the folder structure?

When I tried loading up a folder with all of the form instances, I received a similar blank file. When I loaded up the instances with the form schemas also in the folder structure then the export worked fine.

@mitchellsundt
Copy link
Contributor

You need the JCE that is for your Java installation. i.e., Java 8 JCE if you are using Java 8.

And if that doesn't work, downgrade to Java 7 and the Java 7 JCE.

@rbendo
Copy link

rbendo commented Mar 7, 2017

I've tried using the Java 8 JCE but still had no luck but it still wouldn't work.

As requested, I've attached my public and private keys as docx files.

IMPACT_PublicKey.docx
IMPACT_PrivateKey.docx

Sorry, but I cannot attach my form as well.

Here are the steps I took:

  1. I copied the public key into the public_key column of my form in the settings tab.
  2. I tested my form on a tablet and sent a couple finalized forms to our aggregate server.
  3. Using ODK Briefcase v1.4.10, I pulled the data from my server into my local folder and received the "SUCCESS!" message. All the instances and the xml file were in my folder after pulling.
  4. Afterwards, I tried to export the data into a csv file and then received the error shown above.

At first, I noticed that my private key and public key had line breaks in it and proceeded to remove them but still came into the same error. Then I thought that I created my public-private key pair incorrectly and recreated my keys a couple times but still ran into the same error.

Then I decided to try a previous version of ODK Briefcase and that was the only thing that worked.

@mitchellsundt
Copy link
Contributor

Keep in mind that exposing your private key completely compromises all security of your data.

Sounds like a regression in the build of 1.4.10.

There were a lot of changes in that build. Letting @lognaturel take over.

@yanokwa
Copy link
Member Author

yanokwa commented Mar 8, 2017 •
edited

I did a test on Windows and Mac with Briefcase 1.4.9 and 1.4.1. Both OSes are using Java 1.8.0 with the latest Java Cryptography Extension and here are the results.

Windows 7 SP1, Briefcase 1.4.9

Starting Export...
Processing instance: uuidfbf126e9-3ae8-4211-83d6-5e43a17252bf
SUCCEEDED!

Windows 7 SP1, Briefcase 1.4.10
Cause: java.security.NoSuchAlgorithmException: Cannot find any provider supporting RSA/NONE/OAEPWithSHA256AndMGF1Padding

Starting Export...
Processing instance: uuidfbf126e9-3ae8-4211-83d6-5e43a17252bf
Error decrypting submission uuidfbf126e9-3ae8-4211-83d6-5e43a17252bf Cause: org.opendatakit.briefcase.model.CryptoException: Error decrypting base64EncryptedKey Cause: java.security.NoSuchAlgorithmException: Cannot find any provider supporting RSA/NONE/OAEPWithSHA256AndMGF1Padding
FAILED!

macOS 10.12.3, Briefcase 1.4.9
Cause: java.security.InvalidKeyException: Illegal key size

Starting Export...
Processing instance: uuidfbf126e9-3ae8-4211-83d6-5e43a17252bf
Error decrypting submission uuidfbf126e9-3ae8-4211-83d6-5e43a17252bf Cause: org.opendatakit.briefcase.model.CryptoException: Error decrypting:submission.xml.enc Cause: java.security.InvalidKeyException: Illegal key size
FAILED!

macOS 10.12.3, Briefcase 1.4.10
Cause: java.security.NoSuchAlgorithmException: Cannot find any provider supporting RSA/NONE/OAEPWithSHA256AndMGF1Padding

Starting Export...
Processing instance: uuidfbf126e9-3ae8-4211-83d6-5e43a17252bf
Error decrypting submission uuidfbf126e9-3ae8-4211-83d6-5e43a17252bf Cause: org.opendatakit.briefcase.model.CryptoException: Error decrypting base64EncryptedKey Cause: java.security.NoSuchAlgorithmException: Cannot find any provider supporting RSA/NONE/OAEPWithSHA256AndMGF1Padding
FAILED!

To reproduce this bug, you can find the encrypted form and a submission at https://nafundi-test.appspot.com. No username or password is needed to pull forms.

Once you export, you'll need the MyPrivateKey.pem.txt. For completeness, this is the MyPublicKey.pem.txt. I've put both in encrypted-form.xml.txt as a comment (the first is the public, the second is the private key).

For the Mac, I can confirm that the Java Cryptography Extension is indeed installed because the following code returns true.

import javax.crypto.Cipher;

public class TestUCE {
  public static void main(String args[]) throws Exception {
    int length = Cipher.getMaxAllowedKeyLength("AES");
    boolean unlimited = (length == Integer.MAX_VALUE);
    System.out.println("Unlimited cryptography enabled: " + unlimited);
  }
}

I think 1.4.10 does indeed have a regression due to the gradle build. From https://github.com/opendatakit/opendatakit/wiki/Briefcase-Release-Notes

During the export process, be sure to select Package required libraries into generated JAR. The cryptography libraries make extensive use of dynamic class loading and may perform runtime verification of their jar signatures.

If you instead choose Extract required libraries into generated JAR, then the resulting runnable jar will fail with

I also think export is broken on macOS, but it's not clear why. I've tried Java 1.7 and Java 1.8. @mitchellsundt did you remember trying 1.4.9 on a Mac?

@yanokwa yanokwa changed the title Exporting encrypted encrypted csv file using ODK Briefcase fails Exporting encrypted csv file using ODK Briefcase fails Mar 8, 2017
@yanokwa
Copy link
Member Author

yanokwa commented Mar 8, 2017

I think the first thing for someone to try here is to change the build.gradle file so it does not expand the dependant jars when it makes the Briefcase jar. An easy way to see what is inside the jar is to use this task inside the gradle file.

task explodedJar(type: Copy) {
  into "$buildDir/libs/$jar.baseName"
  with jar
}

@yanokwa
Copy link
Member Author

yanokwa commented Mar 8, 2017

@meletis I vaguely remember SurveyCTO having this issue at some point. Do you use gradle to build your desktop client? How did you resolve it?

@meletis
Copy link
Contributor

meletis commented Mar 8, 2017

I don't remember having that issue in SurveyCTO. We use Maven and we don't expand the dependent jars.

@mitchellsundt
Copy link
Contributor

mitchellsundt commented Mar 9, 2017 via email

@mitchellsundt
Copy link
Contributor

mitchellsundt commented Mar 9, 2017 via email

@yanokwa
Copy link
Member Author

yanokwa commented Mar 9, 2017

My goal is to get Briefcase to run with a single click (e.g.,java -jar briefcase.jar). I spent a lot of time digging into this, and it's most certainly a packaging problem. Here's what I've learned...

Getting a single-click Java application requires an uber/fat/flattened jar or a nested jar class loader. Flattened jars won't work because as @mitchellsundt's points out, Bouncy Castle won't work when exploded. That's the problem we see in 1.4.10.

Nested jar class loading is one reasonable path forward. The catch is, there aren't many class loaders that I've gotten to work. 1.4.9 used Eclipse's classloader which works on Windows, but doesn't seem to work on my Mac.

What does work on my Mac is gradle run. This confirms my findings that if you are willing to write some script to run the briefcase jar with the classpath pointed to an external directory, it'll work great.

My next steps are to

  1. Try to find a modern class loader that works with macOS and gradle.
  2. See can package the jar as an .app for macOS and an .exe for Windows.

@mitchellsundt
Copy link
Contributor

mitchellsundt commented Mar 9, 2017 via email

@yanokwa yanokwa changed the title Exporting encrypted csv file using ODK Briefcase fails Exporting encrypted csv file using ODK Briefcase fails on Windows Mar 23, 2017
This was referenced Mar 23, 2017
Merged
Closed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
bug infrastructure
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@yanokwa @meletis @mitchellsundt @rbendo @SteadyCadence