Running cell for Analyze in Notebook fails when Knox endpoint in BDC has a different password

[corivera]

• Azure Data Studio Version: master

Steps to Reproduce:

1. Create a SQL Big Data Cluster that uses different passwords for the SQL and Knox endpoints.
2. Run Analyze in Notebook on a csv in the cluster's HDFS.
3. Run the generated cell. It throws the following 401 Unauthorized error.

Error:
The code failed because of a fatal error:
Invalid status code '401' from https:// X.X.X.X:30443/gateway/default/livy/v1/sessions with error payload: .

[adsbot]

Thanks for submitting this issue. Please also check if it is already covered by an existing one, like:

• Unified big data cluster login option... (#4083)

[alanrenmsft]

@kevcunnane , IIUC, this is by design, the passwords has to match. right?

[kevcunnane]

@alanrenmsft somewhat correct. We know this is a potential issue, so should handle it. In HDFS expansion scenario we prompt for password if authentication fails. That's the correct solution here, so let's keep it open.

[kevcunnane]

Moving into June. On connecting with Spark kernels, we should be able to verify password (by pinging HDFS) and error / show connection dialog if fails.

@YurongHe this would be a great one to pick up this week if you can.

[YurongHe]

Relate to #5753

[Guillaume-Fourrat]

Hi, confirming this is still active using ADS

Version: 1.10.0-insider (user setup)
Commit: 7f6839d38e24ebd637f2b7ae3b7a5995421c44b2
Date: 2019-08-18T14:37:30.357Z

Hitting following msg for any spark interactive payload as soon as SAPWD != KNOXPWD:

The code failed because of a fatal error:
	Invalid status code '401' from https://10.1.0.5:30443/gateway/default/livy/v1/sessions with error payload: .

Some things to try:
a) Make sure Spark has enough available resources for Jupyter to create a Spark context.
b) Contact your Jupyter administrator to make sure the Spark magics library is configured correctly.
c) Restart the kernel.

Could you confirm the following ?

@kevcunnane , IIUC, this is by design, the passwords has to match. right?

The "somewhat correct" answer from @kevcunnane is a bit confusing =)

As it stands, this conflicts with ongoing Doc as follows

1. No mention of requirement of aligning SA PWD with KNOX PWD
2. Worse, the "SA PWD discoverable and to be changed after deployment" is likely to lead to a different password set in most situations.

So not sure if we have a doc bug where we need to mention that the pwd must be aligned, or a ADS bug where there's some invalid reuse of credentials ?

[chlafreniere]

@kevcunnane does this still repro?

[kevcunnane]

@chlafreniere yes it does.

[Charles-Gagnon]

BDC support in ADS is being deprecated so closing all related issues.