TCP SSH timeout while running remote_exec on AWS instance

[sdoan99]

Terraform Version

hashicorp/terraform:0.12.23

Terraform Configuration Files

resource "aws_instance" "jellyfin_server" {
  key_name = aws_key_pair.jellyfin_keys.key_name
  ami = data.aws_ami.amazon_linux_2.id
  instance_type = var.EC2_INSTANCE_TYPE
  iam_instance_profile = aws_iam_instance_profile.jellyfin_instance_profile.name
  associate_public_ip_address = true
  root_block_device {
    volume_size = var.EBS_ROOT_VOLUME_SIZE
  }
  ebs_block_device {
    device_name = "/dev/sdf"
    volume_size = var.EBS_MEDIA_VOLUME_SIZE
    volume_type = var.EBS_MEDIA_VOLUME_TYPE
  }

  vpc_security_group_ids = [aws_security_group.jellyfin_server_sg.id]
  subnet_id = aws_subnet.jellyfin_a.id
  connection {
    type     = "ssh"
    user     = "ec2-user"
    private_key = file(".ssh/jellyfin-key")
    host     = self.public_ip
  }

  provisioner "remote-exec" {
    inline = [
      "mkdir ~/jellyfin",
      "mkdir ~/jellyfin/scripts",
      "mkdir ~/jellyfin/media",
      "mkdir ~/jellyfin/cache",
      "mkdir ~/jellyfin/config",
      "sudo amazon-linux-extras enable nginx1",
      "sudo yum install -y docker nginx dos2unix",
      "sudo usermod -a -G docker ec2-user",
      "sudo systemctl enable docker",
      "sudo systemctl enable nginx"
    ]
  }

  provisioner "remote-exec" {
    inline = [
      "EBS_DEVICE_NAME=$(lsblk | grep ${var.EBS_MEDIA_VOLUME_SIZE}G | awk '{print $1}')",
      "sudo mkfs -t xfs /dev/$${EBS_DEVICE_NAME}",
      "EBS_DEVICE_UUID=$(sudo blkid | grep $${EBS_DEVICE_NAME} | awk -F'\"' '{print $2}')",
      "sudo echo -e \"UUID=$${EBS_DEVICE_UUID} /home/ec2-user/jellyfin/media  xfs  defaults,nofail  0  2\" | sudo tee -a /etc/fstab",
      "sudo mount -a",
      "sudo chown -R ec2-user /home/ec2-user/jellyfin/media"
    ]
  }

  // Creates nginx conf file for jellyfin server based on ALB or Domain name
  provisioner "file" {
    content = templatefile("templates/server.conf.tmpl", { server_name = local.server_name })
    destination = "/tmp/instant-jellyfin.conf"
  }

  // Lays down cron for syncing files form s3 bucket to local storage
  provisioner "file" {
    content = file("files/start-sync.sh") 
    destination = "/tmp/start-s3sync.sh"
  }

  // Creates cron job script with bucket variable
  provisioner "file" {
    content = templatefile("templates/s3sync.sh.tmpl", { BUCKET = aws_s3_bucket.jellyfin_media.id })
    destination = "/tmp/s3sync.sh"
  }

  // Lays down docker script
  provisioner "file" {
    content = file("files/start-jellyfin.sh")
    destination = "/tmp/start-jellyfin.sh"
  }

  // Starts Nginx with jellyfin conf
  provisioner "file" {
    content = file("files/start-nginx.sh")
    destination = "/tmp/start-nginx.sh"
  }

  // Starts jellyfin backups to S3
  provisioner "file" {
    content = file("files/start-backup.sh")
    destination = "/tmp/start-backup.sh"
  }


  // Creates backup script for jellyfin config
  provisioner "file" {
    content = templatefile("templates/backup.sh.tmpl", { BUCKET = aws_s3_bucket.jellyfin_backup.id })
    destination = "/tmp/backup.sh"
  }

  // Creates restore script for jellyfin config
  provisioner "file" {
    content = templatefile("templates/restore.sh.tmpl", { BUCKET = aws_s3_bucket.jellyfin_backup.id })
    destination = "/tmp/restore.sh"
  }


   provisioner "remote-exec" {
    inline = [
      "sudo mv /tmp/instant-jellyfin.conf /etc/nginx/conf.d/instant-jellyfin.conf",
      "mv /tmp/start-s3sync.sh ~/jellyfin/scripts/start-s3sync.sh",
      "mv /tmp/s3sync.sh ~/jellyfin/scripts/s3sync.sh",
      "mv /tmp/start-jellyfin.sh ~/jellyfin/scripts/start-jellyfin.sh",
      "mv /tmp/start-nginx.sh ~/jellyfin/scripts/start-nginx.sh",
      "mv /tmp/start-backup.sh ~/jellyfin/scripts/start-backup.sh",
      "mv /tmp/backup.sh ~/jellyfin/scripts/backup.sh",
      "mv /tmp/restore.sh ~/jellyfin/scripts/restore.sh",
      "sudo dos2unix /etc/nginx/conf.d/instant-jellyfin.conf",
      "sudo sed -i '/^[ ]*http[ ]*{.*/a     server_names_hash_bucket_size  128;' /etc/nginx/nginx.conf",
      "dos2unix ~/jellyfin/scripts/start-s3sync.sh",
      "dos2unix ~/jellyfin/scripts/s3sync.sh",
      "dos2unix ~/jellyfin/scripts/start-jellyfin.sh",
      "dos2unix ~/jellyfin/scripts/start-nginx.sh",
      "dos2unix ~/jellyfin/scripts/start-backup.sh",
      "dos2unix ~/jellyfin/scripts/backup.sh",
      "dos2unix ~/jellyfin/scripts/restore.sh",
      "/bin/bash ~/jellyfin/scripts/start-s3sync.sh",
      "/bin/bash ~/jellyfin/scripts/start-jellyfin.sh",
      "/bin/bash ~/jellyfin/scripts/start-nginx.sh",
      "/bin/bash ~/jellyfin/scripts/start-backup.sh"
    ]
  } 

  tags = {
    Name        = "${var.ENVIRONMENT}-jellyfin-server"
    Environment = var.ENVIRONMENT
  }
}

Debug Output

Crash Output

Error: timeout - last error: dial tcp 54.191.141.95:22: i/o timeout

Expected Behavior

Should create files on the newly created aws instance

Actual Behavior

terraform_1 | aws_instance.jellyfin_server (remote-exec): Connecting to remote host via SSH...
terraform_1 | aws_instance.jellyfin_server (remote-exec): Host: 54.191.141.95
terraform_1 | aws_instance.jellyfin_server (remote-exec): User: ec2-user
terraform_1 | aws_instance.jellyfin_server (remote-exec): Password: false
terraform_1 | aws_instance.jellyfin_server (remote-exec): Private key: true
terraform_1 | aws_instance.jellyfin_server (remote-exec): Certificate: false
terraform_1 | aws_instance.jellyfin_server (remote-exec): SSH Agent: false
terraform_1 | aws_instance.jellyfin_server (remote-exec): Checking Host Key: false
terraform_1 | aws_instance.jellyfin_server: Still creating... [5m20s elapsed]
terraform_1 | aws_instance.jellyfin_server: Still creating... [5m30s elapsed]
terraform_1 |
terraform_1 |
terraform_1 | Error: timeout - last error: dial tcp 54.191.141.95:22: i/o timeout
terraform_1 |
terraform_1 |
instant-jellyfin_terraform_1 exited with code 1

Steps to Reproduce

Additional Context

References

[jbardin]

Hi @sdoan99

The error indicates that the server did not respond. This could be because of a number of things, it was not online, the SSH daeomon was not running, or network rules were blocking the packets at the edge or the host itself. It's possible that something in a newer terraform release may help with the situation, but I would start by troubleshooting the host and network.

We use GitHub issues for tracking bugs and enhancements, rather than for questions. While we can sometimes help with certain simple problems here, it's better to use the community forum where there are more people ready to help.

Thanks!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.