Host Key getting changed #3

akshaykayande · 2019-07-10T09:33:12Z

Please provide us with the following information:

This issue is for a: (mark with an `x`)

- [ ] bug report -> please search issues before submitting
- [ ] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

We u.
(https://user-images.githubusercontent.com/48773217/60958313-c83af280-a323-11e9-80fc-eb18cd5a2bb4.png)
sed the template mentioned in article:
https://azure.microsoft.com/en-us/resources/samples/sftp-creation-template/

to create the SFTP. Some clients fail to connect since the finger print is getting changed.

I need to know if host key can be set to static do it does not change

Any log messages given by the failure

The server's host key doesnot match the key that has been cached. This means that either the administrator has changed the host key , or you are actually trying to connect to another computer pretending to be the server.

Expected/desired behavior

OS and Version?

Linux

Versions

Mention any other details that might be useful

Thanks! We'll be in touch soon.

The text was updated successfully, but these errors were encountered:

bhummerstone · 2019-07-11T09:26:18Z

Hi! In order to do this, you'll need to customise the container image that is being deployed, as the one from Docker Hub generates a new host key each time using this script

Specifically:

    # Generate unique ssh keys for this container, if needed
    if [ ! -f /etc/ssh/ssh_host_ed25519_key ]; then
        ssh-keygen -t ed25519 -f /etc/ssh/ssh_host_ed25519_key -N ''
    fi
    if [ ! -f /etc/ssh/ssh_host_rsa_key ]; then
        ssh-keygen -t rsa -b 4096 -f /etc/ssh/ssh_host_rsa_key -N ''
    fi

You'll also need to remove a line from the Dockerfile that cleans up the generated host keys.

bhummerstone · 2019-07-11T09:28:26Z

I've got a sample for deploying this with a custom container image here: https://github.com/bhummerstone/azure-templates/blob/master/compute/sftp/sftp-custom-image.json

akshaykayande · 2019-07-16T10:32:07Z

Hello Ben, Thank you for your response. If I understood the solution correctly, I need to deploy the solution again after modifying the template with the recommended changes and there is no way I can modify the template in existing solution. Is that correct? Regards, Akshay Kayande From: Ben Hummerstone <notifications@github.com> Sent: 11 July 2019 14:58 To: Azure-Samples/sftp-creation-template <sftp-creation-template@noreply.github.com> Cc: Akshay Kayande <akshay.kayande@outlook.com>; Author <author@noreply.github.com> Subject: Re: [Azure-Samples/sftp-creation-template] Host Key getting changed (#3) I've got a sample for deploying this with a custom container image here: https://github.com/bhummerstone/azure-templates/blob/master/compute/sftp/sftp-custom-image.json — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#3?email_source=notifications&email_token=ALUDQYMMLDJD5VLVHBXTPYLP634LZA5CNFSM4H7NMVUKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZWDO2Y#issuecomment-510408555>, or mute the thread<https://github.com/notifications/unsubscribe-auth/ALUDQYIAWLDSQ73C523FHBTP634LZANCNFSM4H7NMVUA>.

bhummerstone · 2019-07-18T11:19:53Z

You would need to modify the original container image that is being used for the solution: at the moment it is using the atmoz/sftp image on Docker Hub: https://hub.docker.com/r/atmoz/sftp

bhummerstone · 2019-07-23T21:15:38Z

Let me know if you need some more detailed steps and I can put something together :)

srusru · 2019-07-24T21:14:46Z

Hi @bhummerstone can you please update some steps regarding this? thanks in advance

akshaykayande · 2019-07-29T15:17:45Z

Thank You Ben!!! If you can list down the steps, that would be much appreciated!! Regards, Akshay Kayande From: Ben Hummerstone <notifications@github.com> Sent: 24 July 2019 02:46 To: Azure-Samples/sftp-creation-template <sftp-creation-template@noreply.github.com> Cc: Akshay Kayande <akshay.kayande@outlook.com>; Author <author@noreply.github.com> Subject: Re: [Azure-Samples/sftp-creation-template] Host Key getting changed (#3) Let me know if you need some more detailed steps and I can put something together :) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#3?email_source=notifications&email_token=ALUDQYMI7SHK6PNBV7EOGD3QA5YH3A5CNFSM4H7NMVUKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2UOWUA#issuecomment-514386768>, or mute the thread<https://github.com/notifications/unsubscribe-auth/ALUDQYMUJJTAIIFLVNXUAMLQA5YH3ANCNFSM4H7NMVUA>.

bhummerstone · 2019-08-02T11:32:25Z

Sure, some steps would be:

Clone the Github repo for the current container image: https://github.com/atmoz/sftp
Edit the files/entrypoint file to remove the section that generates the new host key each time
Edit the Dockerfile to remove the line that removes the host keys; specifically this: rm -f /etc/ssh/ssh_host_key
Build the new container image using docker build
Push the new image to a repository of your choice e.g. Docker Hub or Azure Container Registry
Modify the ARM template sample to deploy from a different repository (see the sample link I posted above)

Give that a try and see if it works for you :)

chingjason · 2020-08-09T21:55:51Z

I've got a sample for deploying this with a custom container image here: https://github.com/bhummerstone/azure-templates/blob/master/compute/sftp/sftp-custom-image.json

Hi Ben,

We created an Azure Container Instance and the corresponding file storage to be used as SFTP. We did this by the custom deployment template (uploaded).

We were able to connect and upload files not problem. However, after a few days we noticed the host key was changed (probably after Azure did some regular maintenance). See screenshot uploaded.

Can anyone guide us how to make the host key from changing? Can the custom deployment template be modified to specify a constant host key?

Thanks!

Azure SFTP Deployment Template.docx

bhummerstone · 2020-08-10T08:38:14Z

Heya! Yes, you would need to create a custom container image using the steps I outlined above: this would ensure that your container always had the same host key, otherwise it gets regenerated every time the ACI restarts:

Clone the Github repo for the current container image: https://github.com/atmoz/sftp

Edit the files/entrypoint file to remove the section that generates the new host key each time

Edit the Dockerfile to remove the line that removes the host keys; specifically this: rm -f /etc/ssh/ssh_host__key_

Build the new container image using docker build

Push the new image to a repository of your choice e.g. Docker Hub or Azure Container Registry

Modify the ARM template sample to deploy from a different repository (see the sample link I posted above)

chingjason · 2020-08-11T21:14:21Z

Hi Ben,
Thanks for the reply. First of all, I have to confess I am new to Azure. I only know how to create VM's from the Azure Portal using GUI. I downloaded the ARM template to create the ACI for SFTP is as far I could go.

Now, I am able to complete steps 1 to 3. However, I don't know how to perform steps 4 to 6. Could you give me more detailed instructions?

Also, let me explain more what I am trying to achieve. We want to create a SFTP server on Azure to:

Allow 10 clients to upload their files
Each client must only see their only files
Some clients may upload the files manually via a SFTP client and some clients may automate the upload from their systems therefore we need to keep the host key static
We need the SFTP server running long term 24/7, our system will have a background process to monitor and process the uploaded files periodically
All files uploaded must be kept in Canada (i.e. Azure Canada Central or Canada East region)
I did the research and learned that using an Azure VM will be expensive and hard to maintain, therefore I landed on Azure Container Instance (ACI) together with Azure File Share.
Q1. Is this the right workaround for our needs until SFTP on Azure as a service become available?
Q2. Before we ran into this host key problem, we tried to create a new ACI and file share for each client using the ARM template I uploaded, however, we created the 4th we run into this error

Now this makes me to question whether this is the right solution for us. What do you think?

If it's easier, I could give you a call to discuss if you don't mind. If need be, maybe I could compensate for your time.
Thanks. Much appreciated.
Jason

bhummerstone · 2020-08-12T08:41:08Z

Gotcha, thanks for the background. The message you are getting there is due to the default quota for ACI being 10 cores: you can raise a support ticket to increase that if you'd like.

That said, it sounds like this may not be the best solution for you as running 10 different ACIs 24/7 will definitely be more expensive than one or two VMs.

In your case, I'd probably recommend having a look at the Azure Marketplace and seeing if there is an SFTP solution there that might meet your requirements: https://azuremarketplace.microsoft.com/en-us/marketplace/apps?search=sftp&page=1

chingjason · 2020-08-12T22:21:04Z

Thanks Ben. I took your advise and looked for a SFTP solution in Azure marketplace and we've decide to go with FileMage.

dafunkphenomenon · 2020-09-09T08:24:01Z

Sure, some steps would be:

Clone the Github repo for the current container image: https://github.com/atmoz/sftp

Edit the files/entrypoint file to remove the section that generates the new host key each time

Edit the Dockerfile to remove the line that removes the host keys; specifically this: rm -f /etc/ssh/ssh_host__key_

Build the new container image using docker build

Push the new image to a repository of your choice e.g. Docker Hub or Azure Container Registry

Modify the ARM template sample to deploy from a different repository (see the sample link I posted above)

Give that a try and see if it works for you :)

Thanks for these steps @bhummerstone. I've followed them and created a new image. It's creating the the sftp server but I can't connect to it. If I check the logs on the container it says.

[/usr/local/bin/create-sftp-user] Parsing user data: "devmwsftpuser::1001" cat: '/home/devmwsftpuser/.ssh/keys/*': No such file or directory /usr/local/bin/create-sftp-user: Error on line 90: cat "$publickey" >> "$userKeysAllowedFileTmp" /entrypoint: Error on line 60: create-sftp-user "$user"

Is there anything I'm missing? Thanks Matt

bhummerstone · 2020-09-16T10:02:14Z

Could you share your Dockerfile, please?

voidsstr · 2020-09-20T16:47:22Z

Isn't there an option to use an existing key? Per Atmoz' documentation (see below). Is there a way to do this within Azure?

Providing your own SSH host key (recommended)
This container will generate new SSH host keys at first run. To avoid that your users get a MITM warning when you recreate your container (and the host keys changes), you can mount your own host keys.

docker run
-v /ssh_host_ed25519_key:/etc/ssh/ssh_host_ed25519_key
-v /ssh_host_rsa_key:/etc/ssh/ssh_host_rsa_key
-v /share:/home/foo/share
-p 2222:22 -d atmoz/sftp
foo::1001
Tip: you can generate your keys with these commands:

ssh-keygen -t ed25519 -f ssh_host_ed25519_key < /dev/null
ssh-keygen -t rsa -b 4096 -f ssh_host_rsa_key < /dev/null

varinder87singh · 2020-09-21T02:02:29Z

Could you share your Dockerfile, please?

Hello,
When you get that, can you please share that with me as well?
Need to do the same.

bhummerstone · 2020-09-21T11:06:19Z

@voidsstr in theory, this would work but unfortunately the mounting of Azure Files into the ACI resets the permissions to 777, whereas Linux needs it to be 600 (iirc); I've provided this feedback to the Azure Files and ACI teams to see if we can sort this out, and hopefully the introduction of NFS on Azure Files might be a step towards this: https://docs.microsoft.com/en-us/azure/storage/files/storage-files-how-to-mount-nfs-shares

bhummerstone · 2020-09-21T11:07:23Z

For ease, here is a sample Dockerfile that I've used:

FROM alpine:latest

# Steps done in one RUN layer:
# - Install packages
# - Fix default group (1000 does not exist)
# - OpenSSH needs /var/run/sshd to run
# - Remove generic host keys, entrypoint generates unique keys
RUN echo "@community http://dl-cdn.alpinelinux.org/alpine/edge/community" >> /etc/apk/repositories && \
    apk add --no-cache bash shadow@community openssh openssh-sftp-server && \
    sed -i 's/GROUP=1000/GROUP=100/' /etc/default/useradd && \
    mkdir -p /var/run/sshd

COPY files/sshd_config /etc/ssh/sshd_config
COPY files/create-sftp-user /usr/local/bin/
COPY files/entrypoint /

EXPOSE 22

ENTRYPOINT ["/entrypoint"]

You'll also need to edit the entrypoint script from the original image to remove lines 67 - 73, as this generates the new host key each time.

atmoz · 2020-09-30T16:18:11Z

You don't need to change the original Dockerfile or entrypoint script at all. You can just add another layer on top of the original!

Something like this (not tested, try it out yourself):

FROM atmoz/sftp:latest

# Assuming you have the files ssh_host_ed25519_key and ssh_host_rsa_key
COPY ssh_*_key /etc/ssh/
RUN chmod 600 /etc/ssh/ssh_*_key

Removed the section that generates the new host key each time in Entrypoint file Also removed this section in Dockerfile rm -f /etc/ssh/ssh_host_*key* (Azure-Samples/sftp-creation-template#3)

vasanthtt · 2020-10-26T14:19:29Z

Hi! In order to do this, you'll need to customise the container image that is being deployed, as the one from Docker Hub generates a new host key each time using this script

Specifically:
    # Generate unique ssh keys for this container, if needed
    if [ ! -f /etc/ssh/ssh_host_ed25519_key ]; then
        ssh-keygen -t ed25519 -f /etc/ssh/ssh_host_ed25519_key -N ''
    fi
    if [ ! -f /etc/ssh/ssh_host_rsa_key ]; then
        ssh-keygen -t rsa -b 4096 -f /etc/ssh/ssh_host_rsa_key -N ''
    fi
You'll also need to remove a line from the Dockerfile that cleans up the generated host keys.

Thanks @bhummerstone It worked well.

My repo alone with minor change to arm template

asif2017 · 2020-11-11T09:47:33Z

@bhummerstone I have tried the basic arm template, the link which you posted above. I'm able to login through private key. But the .ssh/keys folder is also visible in SFTP client. I want to hide the .ssh folders from users whosoever are going to access the SFTP. Is it possible to do? I have gone through secret volume for container group but not able to crack it. Is there anyway to hide or control the access of .ssh folder?

bhummerstone · 2020-11-11T14:00:35Z

The .ssh folder belongs to the user, so it will always be visible in their home directory as they need permission to read it; it is their public key after all!

Note that it is only visible to that particular user, not anyone else... although it's a public key, so worthless without the corresponding private key

asif2017 · 2020-11-11T14:10:05Z

@bhummerstone Thanks for your prompt response. One question here, Do we still need to delete the lines which you've mentioned in previous comment. Can't we just add the ssh keys in file share and point to that location in arm template and use the private key to login. Do you see any issue by going with this approach?

dafunkphenomenon · 2020-11-11T14:25:28Z

I got this working in the end by just removing this line from the docker file
rm -f /etc/ssh/ssh_host_key

I am also pushing the image into my own repository from Azure Devops and using that from my ARM template pointing it to a specific build tag. That has stopped the host keys changing

bhummerstone · 2020-11-12T10:26:42Z

@asif2017 unfortunately that approach currently doesn't work: when mounting an Azure Files share into an ACI it resets the permissions and so breaks the SSH process

bhummerstone · 2021-06-23T13:46:00Z

Closing this issue as it has been idle for a while

Remove fork as suggested from Azure-Samples/sftp-creation-template#3

sthakkar17 · 2022-01-10T14:16:00Z

You don't need to change the original Dockerfile or entrypoint script at all. You can just add another layer on top of the original!

Something like this (not tested, try it out yourself):
FROM atmoz/sftp:latest

# Assuming you have the files ssh_host_ed25519_key and ssh_host_rsa_key
COPY ssh_*_key /etc/ssh/
RUN chmod 600 /etc/ssh/ssh_*_key

Where do I add this code? And after changing the code, Do I need to build a docker file and host it on the container registry and modify ARM?
(I am fairly new to docker. So, looking to get a better understanding to solve this issue.)

Or if anyone can help to fix this issue, with the keeping host keys static.
I have also tried Azure Blobs SFTP. However, they also have the rotating keys in a month. This was mentioned in the known list of issue of Azure Blobs SFTP.

sthakkar17 · 2022-01-11T07:31:08Z

I've got a sample for deploying this with a custom container image here: https://github.com/bhummerstone/azure-templates/blob/master/compute/sftp/sftp-custom-image.json

This link was broken after the re-structuring the files. Here is the new link for the sample: https://github.com/bhummerstone/azure-templates/blob/master/arm/compute/sftp/sftp-custom-image.json

sthakkar17 · 2022-01-11T08:43:26Z

Sure, some steps would be:

Clone the Github repo for the current container image: https://github.com/atmoz/sftp

Edit the files/entrypoint file to remove the section that generates the new host key each time

Edit the Dockerfile to remove the line that removes the host keys; specifically this: rm -f /etc/ssh/ssh_host__key_

Build the new container image using docker build

Push the new image to a repository of your choice e.g. Docker Hub or Azure Container Registry

Modify the ARM template sample to deploy from a different repository (see the sample link I posted above)

Give that a try and see if it works for you :)

Based on this, step 6 is not pulling the docker image, I have created the issue below for it. Please see if anyone help me with the issue.
bhummerstone/azure-templates#6 (comment)

bhummerstone mentioned this issue Jan 27, 2020

Multiple users? #7

Closed

flawed4life mentioned this issue Sep 30, 2020

Issues with the SFTP resetting host key atmoz/sftp#229

Closed

bhummerstone closed this as completed Jun 23, 2021

bhummerstone mentioned this issue Oct 22, 2021

Cannot upload to root of SFTP Service #4

Closed

wooddani added a commit to wooddani/sftp that referenced this issue Nov 8, 2021

Update Dockerfile

90cfa43

Remove fork as suggested from Azure-Samples/sftp-creation-template#3

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Host Key getting changed #3

Host Key getting changed #3

akshaykayande commented Jul 10, 2019 •

edited

Please provide us with the following information:

bhummerstone commented Jul 11, 2019 •

edited

bhummerstone commented Jul 11, 2019

akshaykayande commented Jul 16, 2019 via email

bhummerstone commented Jul 18, 2019

bhummerstone commented Jul 23, 2019

srusru commented Jul 24, 2019

akshaykayande commented Jul 29, 2019 via email

bhummerstone commented Aug 2, 2019

chingjason commented Aug 9, 2020

bhummerstone commented Aug 10, 2020

chingjason commented Aug 11, 2020

bhummerstone commented Aug 12, 2020

chingjason commented Aug 12, 2020

dafunkphenomenon commented Sep 9, 2020

bhummerstone commented Sep 16, 2020

voidsstr commented Sep 20, 2020

varinder87singh commented Sep 21, 2020

bhummerstone commented Sep 21, 2020

bhummerstone commented Sep 21, 2020

atmoz commented Sep 30, 2020 •

edited

vasanthtt commented Oct 26, 2020

asif2017 commented Nov 11, 2020 •

edited

bhummerstone commented Nov 11, 2020 •

edited

asif2017 commented Nov 11, 2020

dafunkphenomenon commented Nov 11, 2020

bhummerstone commented Nov 12, 2020

bhummerstone commented Jun 23, 2021

sthakkar17 commented Jan 10, 2022

sthakkar17 commented Jan 11, 2022

sthakkar17 commented Jan 11, 2022

Host Key getting changed #3

Host Key getting changed #3

Comments

akshaykayande commented Jul 10, 2019 • edited

Please provide us with the following information:

This issue is for a: (mark with an x)

Minimal steps to reproduce

Any log messages given by the failure

Expected/desired behavior

OS and Version?

Versions

Mention any other details that might be useful

bhummerstone commented Jul 11, 2019 • edited

bhummerstone commented Jul 11, 2019

akshaykayande commented Jul 16, 2019 via email

bhummerstone commented Jul 18, 2019

bhummerstone commented Jul 23, 2019

srusru commented Jul 24, 2019

akshaykayande commented Jul 29, 2019 via email

bhummerstone commented Aug 2, 2019

chingjason commented Aug 9, 2020

bhummerstone commented Aug 10, 2020

chingjason commented Aug 11, 2020

bhummerstone commented Aug 12, 2020

chingjason commented Aug 12, 2020

dafunkphenomenon commented Sep 9, 2020

bhummerstone commented Sep 16, 2020

voidsstr commented Sep 20, 2020

varinder87singh commented Sep 21, 2020

bhummerstone commented Sep 21, 2020

bhummerstone commented Sep 21, 2020

atmoz commented Sep 30, 2020 • edited

vasanthtt commented Oct 26, 2020

asif2017 commented Nov 11, 2020 • edited

bhummerstone commented Nov 11, 2020 • edited

asif2017 commented Nov 11, 2020

dafunkphenomenon commented Nov 11, 2020

bhummerstone commented Nov 12, 2020

bhummerstone commented Jun 23, 2021

sthakkar17 commented Jan 10, 2022

sthakkar17 commented Jan 11, 2022

sthakkar17 commented Jan 11, 2022

akshaykayande commented Jul 10, 2019 •

edited

This issue is for a: (mark with an `x`)

bhummerstone commented Jul 11, 2019 •

edited

atmoz commented Sep 30, 2020 •

edited

asif2017 commented Nov 11, 2020 •

edited

bhummerstone commented Nov 11, 2020 •

edited