connection string credentials

[mrasnake]

I am having trouble finding documentation on how to format the connection string to include username and password for the host. could someone please show me an example? or point me to a more complete documentation of this package? there are very few examples of how to use it.

[colinmarc]

@mrasnake hi! I'm a bit confused by your question. What do you mean by password? How do you usually use hdfs?

Full package documentation is here: https://godoc.org/github.com/colinmarc/hdfs

[mrasnake]

honestly still learning hdfs. I was to use your service to connect to the hdfs nodes remotely. So I was trying to format the connection string to pass the username and password, like you would for an ssh tunnel, to connect to one of the hadoop cluster machines. Is there a way to do this or is the package is just meant to run from one of the cluster machines?

[colinmarc]

@mrasnake you can definitely connect through an SSH tunnel, but the library doesn't know anything about that (neither does HDFS itself).

Use a guide like this one to bind a port locally for the namenode: https://www.revsys.com/writings/quicktips/ssh-tunnel.html

Then you should be able to connect to the namenode at localhost:<local port>. Note, though, that this won't let you connect to the datanodes - so you can do list operations and stat etc but not read the data in files.

Going to close this, but feel free to reach out if you still need help.

[mrasnake]

@colinmarc hey thanks for the help. that got everything sorted out. I do have another question. I am probably just missing something staring me in the face but how do I get hdfs FileInfo (not the os.Fileinfo) on a file? So that I can then get the owner of the file?

I am ultimately trying to check read/write permissions against the user(I am using pam for my web service auth so the user is actually the linux user) but I need to find the owner of the hdfs file, and I dont see any way to do that without using your FileInfo struct. But for the life of me cannot figure out how to get that from a file. Only the os.FileInfo which doesnt have that functionality.

[colinmarc]

Hi @mrasnake,

You want Stat.

[mrasnake]

@colinmarc I figured at least that much but I still cant seem to figure out how to get the owner from the return value. again I must be missing something obvious but I have tried things like the following:

fi, _ := client.Stat(path)

printlin(fi.Owner())
println(Owner(fi))

but the compiler always complains. what am I missing?

[colinmarc]

what's the error?

[mrasnake]

-undefined: Owner
-fi.Owner undefined (type os.FileInfo has no field or method Owner)

[colinmarc]

Oops, my mistake. Try this:

fi, _ := client.Stat("path")
fmt.Println(fi.(*hdfs.FileInfo).Owner())

It's a bit confusing if you're unused to this sort of thing in go, but Stat returns a *hdfs.FileInfo which is duck-typed to os.FileInfo. It should maybe be duck-typed later? I'd have to think about it, and the interface is frozen in any case.