Cannot authenticate users imported with SHA512 passwords

[ozydingo]

[READ] Step 1: Are you in the right place?

• For issues related to the code in this repository file a Github issue.
• If the issue pertains to Cloud Firestore, read the instructions in the "Firestore issue"
  template.
• For general technical questions, post a question on StackOverflow
  with the firebase tag.
• For general Firebase discussion, use the firebase-talk
  google group.
• For help troubleshooting your application that does not fall under one
  of the above categories, reach out to the personalized
  Firebase support channel.

[REQUIRED] Step 2: Describe your environment

• Operating System version: OS X 10.14.6
• Firebase SDK version: firebase-admin 8.9.2
• Firebase Product: auth
• Node.js version: 12.6.0
• NPM version: 6.9.0

[REQUIRED] Step 3: Describe the problem

Steps to reproduce:

I cannot authenticate with imported users using SHA512-encrypted passwords. The import is successful and the password hashes look correct using listUsers(), but authentication fails with auth/wrong-password.

What happened? How can we make the problem occur?
This could be a description, log/console output, etc.

See code below for demo. I started from the [firebase docs instructions on importing users] (https://firebase.google.com/docs/auth/admin/import-users#import_users_with_md5_sha_and_pbkdf_hashed_passwords). Users imported using (known) SHA512 encrypted passwords are not able to authenticate.

I have tried this using both existing encrypted passwords using the AuthLogic ruby gem and by generating the encrypted passwords myself, as below. I have confirmed that my method produces identical password hashes to AuthLogic. Neither method is producing valid logins.

Relevant Code:

package.json dependencies:

{
    "firebase": "^7.9.1",
    "firebase-admin": "^8.9.2",
    "js-base64": "^2.5.2",
    "js-sha512": "^0.8.0"
  }

Node script to reproduce the issue:

// ---- SETUP ---- //
const Base64 = require('js-base64').Base64;
const firebase = require('firebase');
const firebaseAdmin = require('firebase-admin');
const fs = require('fs')
const path = require('path');
const { sha512 } = require('js-sha512')
const util = require('util');

const adminCredsPath = path.join(process.env.HOME, 'etc', 'firebase', 'personal-admin.json');
const clientConfig = path.join(process.env.HOME, 'etc', 'firebase', 'personal-client.json');


// ---- HELPER FUNCTIONS ---- //
function encryptPassword({password, rounds, salt}) {
  let c = password;
  if (salt) { c += salt; }
  for (let ii = 0; ii < rounds; ii++) { c = sha512(c) }
  return c;
}

async function importUser({uid, email, encryptedPassword, salt, shaRounds}) {
  const user = {
    uid: uid,
    email: email,
    emailVerified: true,
    passwordHash: Buffer.from(encryptedPassword),
  };
  if (salt) { user.passwordSalt = Buffer.from(salt) }
  const importOptions = {
    hash: {
      algorithm: 'SHA512',
      rounds: shaRounds
    }
  };

  // allow repeatedly running the script without needing to use new uids
  await firebaseAdmin.auth().deleteUser(uid).catch(err => console.log("Did not delete user:", err.message));

  return firebaseAdmin.auth().importUsers(
    [user], importOptions
  );
}

function sleep(time) {
  return new Promise((res, rejj) => {setTimeout(res, time)})
}

async function setupApp() {
  firebaseAdmin.initializeApp({
    credential: firebaseAdmin.credential.cert(adminCredsPath)
  });

  const clientCreds = await util.promisify(fs.readFile)(clientConfig, 'utf-8').then(JSON.parse);
  await firebase.initializeApp(clientCreds);

  firebase.auth().onAuthStateChanged(user => {
    console.log("Auth change:", user)
  })
}

// ----- MAIN FUNCTIONS ---- //
async function main() {
  await setupApp();

  const email = 'test@foo.com';
  const password = 'test123';
  const rounds = 20;
  // const salt = 'abc';
  // const encryptedPassword = encryptPassword({password, rounds, salt});
  const encryptedPassword = encryptPassword({password, rounds});
  console.log({password, encryptedPassword, base64: Base64.encode(encryptedPassword)});

  const userDetails = {uid: '123', email, encryptedPassword, shaRounds: rounds};
  console.log("Importing", email);
  const result = await importUser(userDetails);
  console.log(result);
  console.log("Created user for", email);

  const {users} = await firebaseAdmin.auth().listUsers();
  console.log("USERS", users.map(({uid, email, passwordHash, passwordSalt}) => ({uid, email, passwordHash, passwordSalt})));

  await sleep(5000);
  console.log("Attempting to sign in")
  await firebase.auth().signInWithEmailAndPassword(email, password).catch(console.error)
}

main().then(() => { process.exit() })

OUTPUT:

{
  password: 'test123',
  encryptedPassword: 'ac888afb942d693a154323d180d708bc2b4c6d4902d3b90dfddc3e2e88d975a4c641addd492c67af24656cf4252c18fce8a45337d8c72e9a4f9b4e3782d90589',
  base64: 'YWM4ODhhZmI5NDJkNjkzYTE1NDMyM2QxODBkNzA4YmMyYjRjNmQ0OTAyZDNiOTBkZmRkYzNlMmU4OGQ5NzVhNGM2NDFhZGRkNDkyYzY3YWYyNDY1NmNmNDI1MmMxOGZjZThhNDUzMzdkOGM3MmU5YTRmOWI0ZTM3ODJkOTA1ODk='
}
Importing test@foo.com
Auth change: null
{ successCount: 1, failureCount: 0, errors: [] }
Created user for test@foo.com
USERS [
  {
    uid: '123',
    email: 'test@foo.com',
    passwordHash: 'YWM4ODhhZmI5NDJkNjkzYTE1NDMyM2QxODBkNzA4YmMyYjRjNmQ0OTAyZDNiOTBkZmRkYzNlMmU4OGQ5NzVhNGM2NDFhZGRkNDkyYzY3YWYyNDY1NmNmNDI1MmMxOGZjZThhNDUzMzdkOGM3MmU5YTRmOWI0ZTM3ODJkOTA1ODk=',
    passwordSalt: ''
  }
]
Attempting to sign in
[M [Error]: The password is invalid or the user does not have a password.] {
  code: 'auth/wrong-password',
  message: 'The password is invalid or the user does not have a password.'
}

To prove that password hashes match the third-party output:

My method in Node:

> encryptPassword({password: "test123", rounds: 20})
'ac888afb942d693a154323d180d708bc2b4c6d4902d3b90dfddc3e2e88d975a4c641addd492c67af24656cf4252c18fce8a45337d8c72e9a4f9b4e3782d90589'
> encryptPassword({password: "test123", rounds: 20, salt: "abc"})
'1e878fcdb010cb6d4013a2bd4aa24e5b28eb0f7749b6c2429aa30c8bdf53af24d9f0055e8277196b25dff06845f85e2b99c44cf705d4556436c2abd023bef6ed'

Using AuthLogic in ruby:

> Authlogic::CryptoProviders::Sha512.encrypt("test123")
 => "ac888afb942d693a154323d180d708bc2b4c6d4902d3b90dfddc3e2e88d975a4c641addd492c67af24656cf4252c18fce8a45337d8c72e9a4f9b4e3782d90589"
> Authlogic::CryptoProviders::Sha512.encrypt("test123", "abc")
 => "1e878fcdb010cb6d4013a2bd4aa24e5b28eb0f7749b6c2429aa30c8bdf53af24d9f0055e8277196b25dff06845f85e2b99c44cf705d4556436c2abd023bef6ed"

I have tried the above script with salted and unsalted passwords. Same result.

Any help is greatly appreciated!

[ozydingo]

One more public-api-method-only demonstration of correctness of my password hashing:

2.3.3 :014 > user = User.new; user.crypted_password
 => nil
2.3.3 :015 > user.password = "test123"; user.crypted_password
 => "ac888afb942d693a154323d180d708bc2b4c6d4902d3b90dfddc3e2e88d975a4c641addd492c67af24656cf4252c18fce8a45337d8c72e9a4f9b4e3782d90589"

where the User class in ruby (Rails) is configured according to the authlogic docs

[hiranya911]

SDK only supports salt-first password hashing, whereas your code is attempting a password-first hash:

if (salt) { c += salt; }

Here's a working example for SHA256 for comparison. This is from our own integration tests, and it is run regularly against actual Firebase projects. I expect SHA512 to work similarly.

firebase-admin-node/test/integration/auth.spec.ts

Lines 1375 to 1390 in 5abaab6

	{
	name: 'SHA256',
	importOptions: {
	hash: {
	algorithm: 'SHA256',
	rounds: 1,
	},
	} as any,
	computePasswordHash: (userImportTest: UserImportTest): Buffer => {
	const currentRawPassword = userImportTest.rawPassword;
	const currentRawSalt = userImportTest.rawSalt;
	return crypto.createHash('sha256').update(currentRawSalt + currentRawPassword).digest();
	},
	rawPassword,
	rawSalt,
	},

firebase-admin-node/test/integration/auth.spec.ts

Lines 1487 to 1500 in 5abaab6

	it(`successfully imports users with ${fixture.name} to Firebase Auth.`, () => {
	importUserRecord = {
	uid: randomUid,
	email: randomUid + '@example.com',
	};
	importUserRecord.passwordHash = fixture.computePasswordHash(fixture);
	if (typeof fixture.rawSalt !== 'undefined') {
	importUserRecord.passwordSalt = Buffer.from(fixture.rawSalt);
	}
	return testImportAndSignInUser(
	importUserRecord, fixture.importOptions, fixture.rawPassword)
	.should.eventually.be.fulfilled;
	});

firebase-admin-node/test/integration/auth.spec.ts

Lines 1593 to 1613 in 5abaab6

	function testImportAndSignInUser(
	importUserRecord: any, importOptions: any, rawPassword: string): Promise<void> {
	const users = [importUserRecord];
	// Import the user record.
	return admin.auth().importUsers(users, importOptions)
	.then((result) => {
	// Verify the import result.
	expect(result.failureCount).to.equal(0);
	expect(result.successCount).to.equal(1);
	expect(result.errors.length).to.equal(0);
	// Sign in with an email and password to the imported account.
	return clientAuth().signInWithEmailAndPassword(users[0].email, rawPassword);
	})
	.then(({user}) => {
	// Confirm successful sign-in.
	expect(user).to.exist;
	expect(user!.email).to.equal(users[0].email);
	expect(user!.providerData[0]).to.exist;
	expect(user!.providerData[0]!.providerId).to.equal('password');
	});
	}

[hiranya911]

I managed to get a successful sign in by modifying your function as follows:

function encryptPassword({password, rounds, salt}) {
  let c = password;
  if (salt) { c = salt + c; } // Salt-first
  for (let ii = 0; ii < rounds; ii++) {
      const hex = sha512(c)
      c = new Buffer(hex, 'hex');
  }
  return c;
}

In addition to using salt-first hashing, you should also account for the fact that the js-sha512 API you're using returns output as a hex string.

[ozydingo]

@hiranya911 Thank you for the explanation and the fix! Unfortunately, since it seems that Authlogic makes this same error of hashing the hex string instead of the hex output as bytes, this seems to imply that I will not be able to import passwords hashed using this library. Does that sound correct?

Authlogic is a pretty popular gem, is there any sense in which this method of hashing could be supported by firebase auth so that applications that use(d) authlogic can import users into Firebase?

[ozydingo]

For reference:

Here's the line from AuthLogic that shows how they use SHA512

stretches.times { digest = Digest::SHA512.hexdigest(digest) }

And demonstrating how this code works in ruby:

2.3.3 :187 > password = "test123"
 => "test123"
2.3.3 :188 > c = Digest::SHA512.hexdigest(password)
 => "daef4953b9783365cad6615223720506cc46c5167cd16ab500fa597aa08ff964eb24fb19687f34d7665f778fcb6c5358fc0a5b81e1662cf90f73a2671c53f991"
2.3.3 :189 > Digest::SHA512.hexdigest(c)
 => "113c466936018ad6d5ba5c565f7aa7f44006013f15c0793c020b799e12f1d3560accfd7b4b94caac7c3dbaf55eb8f22a37b9194df040fb59083a9fa7c07468a9"

vs representing the hexdigest method output as hex:

2.3.3 :184 > password = "test123"
 => "test123"
2.3.3 :185 > c = Digest::SHA512.hexdigest(password)
 => "daef4953b9783365cad6615223720506cc46c5167cd16ab500fa597aa08ff964eb24fb19687f34d7665f778fcb6c5358fc0a5b81e1662cf90f73a2671c53f991"
2.3.3 :186 > Digest::SHA512.hexdigest([c].pack("H*"))
 => "a9bdede324c91e7a390a70e641254cd03c17e879d50f96c2c7a3107de6a45596467deb2849d7aa3aa0fcac452b51bc5ab4a8e881d72e61ae6a07cdc299643dc2"

Now using your method in Node:

> password = "test123"
'test123'
> c = sha512(password)
'daef4953b9783365cad6615223720506cc46c5167cd16ab500fa597aa08ff964eb24fb19687f34d7665f778fcb6c5358fc0a5b81e1662cf90f73a2671c53f991'
> sha512(Buffer.from(c, "hex"))
'a9bdede324c91e7a390a70e641254cd03c17e879d50f96c2c7a3107de6a45596467deb2849d7aa3aa0fcac452b51bc5ab4a8e881d72e61ae6a07cdc299643dc2'

This matches the ruby method using the byte pack method.

The equivalent "incorrect" method that I was initially using that matches how Authlogic uses the hashing function:

> password = "test123"
'test123'
> c = sha512(password)
'daef4953b9783365cad6615223720506cc46c5167cd16ab500fa597aa08ff964eb24fb19687f34d7665f778fcb6c5358fc0a5b81e1662cf90f73a2671c53f991'
> sha512(c)
'113c466936018ad6d5ba5c565f7aa7f44006013f15c0793c020b799e12f1d3560accfd7b4b94caac7c3dbaf55eb8f22a37b9194df040fb59083a9fa7c07468a9'

So we can see that the method I used initially matches what Authlogic is doing, and if I'm getting this correct that means the only way to import passwords that were encrypted with the authglogic library would be if firebase-auth were to build support for this (incorrect) method of hashing, right?

[hiranya911]

Yes, this will require some fix from the Authlogic end if you wish to use that library. It seems you've already initiated the process for getting it fixed.

[AdamBCo]

I'm experiencing a similar issue where users are unable to authenticate after importing our user database into Firebase. An example of what we're doing can be seen below. Has anyone run into a similar issue? @hiranya911

const salt = crypto.randomBytes(16).toString('hex');  
const user = {  
  uid: "tbSL53PN7zVTMsWc9up24hev4CN2",  
  email: "test@test.com",  
  hash: crypto.pbkdf2Sync("test123", salt, 100, 64, 'sha512').toString('hex'),  
  salt  
}  

const importUser = {  
  uid: user.uid,  
  email: user.email,  
  passwordHash: Buffer.from(user.hash, 'hex'),  
  passwordSalt: Buffer.from(user.salt, 'hex')  
}  
  
const results = await auth().importUsers([importUser], {  
    hash: {  
        algorithm: "SHA512",  
        rounds: 100,  
  }  
})  

console.log(results)  
const c = await auth().listUsers()  
console.log(c)

I'm also noticing that passwordHash and passwordSalt are always undefined.

{ successCount: 1, failureCount: 0, errors: [] }
{
  users: [
    UserRecord {
      uid: 'tbSL53PN7zVTMsWc9up24hev4CN2',
      email: 'test@test.com',
      emailVerified: false,
      displayName: undefined,
      photoURL: undefined,
      phoneNumber: undefined,
      disabled: false,
      metadata: [UserMetadata],
      providerData: [Array],
      passwordHash: undefined,
      passwordSalt: undefined,
      tokensValidAfterTime: 'Wed, 07 Oct 2020 10:17:12 GMT',
      tenantId: undefined
    }
  ]
}

[hiranya911]

I might be wrong, but it seems you're setting the hash algorithm type to SHA512 while actually using a PBKDF2 hash. I think the hashing logic for SHA512 should look something like this:

crypto.createHash('sha512').update(rawSalt + rawPassword).digest();

That might explain why the login is not working as expected. As for why the password hash is not included in the listUsers() results, that may be due to your service account lacking some required IAM permission (I believe it needs to have the "Firebase Authentication Admin" role for those fields to be returned -- See #660).