Using bot as https destination without ngrok

[AlexGruPerm]

Hello @mukel
I try to use your library like this:
class telegBotWH(log :org.slf4j.Logger, config :Config) extends AkkaTelegramBot with Webhook with Commands[Future] { ...
and next want explisitely load certificate with:
val cfile :java.io.File= new File("C:\\tcert\\mtspredbot.pem") val inpCertFilePath :java.nio.file.Path = cfile.toPath override val certificate :Option[InputFile] = Option(InputFile(inpCertFilePath))
I use next WebHook address https://xxx.xxx.xxx.xxx
where xxx.xxx.xxx.xxx is my VDS

When I run bot there is next output
[trace, com.bot4s.telegram.clients.AkkaHttpClient] REQUEST 5ebe2b43-f7da-4fe9-80f3-a04448d3df8e SetWebhook(https://xxx.xxx.xxx.xxx,Some(Path(C:\tcert\mtspredbot.pem)),None,None) Press [ENTER] to shutdown the bot, it may take a few seconds... [trace, com.bot4s.telegram.clients.AkkaHttpClient] RESPONSE 5ebe2b43-f7da-4fe9-80f3-a04448d3df8e true [WARN] [06/28/2019 15:58:30.996] [default-akka.actor.default-dispatcher-16] [akka.actor.ActorSystemImpl(default)] Illegal request, responding with status '400 Bad Request': Unsupported HTTP method: The HTTP method started with 0x16 rather than any known HTTP method. Perhaps this was an HTTPS request sent to an HTTP endpoint?
My question. Is it possible to use bot wuth this configuration as destination for telegram with https.

I know about ngrok and it works fine but wants to use it directly, because it's vds.

[mukel]

The bot will spawn a tiny server to listen for updates using the default SSL context (no encryption). You have to define a custom SSL context (read below).

Here are the steps I followed:
First, generate a self-signed certificate (.pem), I followed the Java instructions. The certificate generation is too user-friendly, YOU MUST USE your static IP as first last name (Common Name) e.g.
What is your first and last name? 12.34.56.78

The generated .pem will also include the private key, you have to strip it and send only the public key to Telegram (setWebhook).

Create and set as default a custom SSL context: relevant Akka-HTTP instructions using the certificates you generated.

Beware of the allowed ports.
Forcing users to override the default SSL context is hacky and totally unexpected by users, but I didn't want to deal with certificates in the bot.

Let me know if you have any issue.

[mukel]

Here's the modified WebhookBot I used:

import java.io.{File, FileInputStream, InputStream}
import java.net.URLEncoder
import java.nio.file.Paths
import java.security.{KeyStore, SecureRandom}

import akka.http.scaladsl.{ConnectionContext, Http, HttpsConnectionContext}
import akka.http.scaladsl.model.{HttpRequest, Uri}
import akka.http.scaladsl.unmarshalling.Unmarshal
import com.bot4s.telegram.api.Webhook
import com.bot4s.telegram.methods._
import com.bot4s.telegram.models.{InputFile, Message}
import javax.net.ssl.{KeyManagerFactory, SSLContext, TrustManagerFactory}

import scala.concurrent.duration.Duration
import scala.concurrent.{Await, Future}

/**
  * Webhook-backed JS calculator.
  * To test Webhooks locally, use an SSH tunnel or ngrok.
  */
class WebhookBot(token: String) extends AkkaExampleBot(token) with Webhook {

  val port = 443
  val webhookUrl = "https://12.34.56.78"

  override val interfaceIp: String = "0.0.0.0"

  val baseUrl = "http://api.mathjs.org/v1/?expr="

  override def certificate: Option[InputFile] = Some(
    InputFile(new File("/root/bot/telegram/stripped.pem").toPath)
  )

  override def receiveMessage(msg: Message): Future[Unit] = {
    msg.text.fold(Future.successful(())) { text =>
      val url = baseUrl + URLEncoder.encode(text, "UTF-8")
      for {
        res <- Http().singleRequest(HttpRequest(uri = Uri(url)))
        if res.status.isSuccess()
        result <- Unmarshal(res).to[String]
        _ <- request(SendMessage(msg.source, result))
      } yield ()
    }
  }

  // Set custom context. 
  Http().setDefaultServerHttpContext(httpsContext())

  def httpsContext(): HttpsConnectionContext = {
    // Manual HTTPS configuration
    val password: Array[Char] = "password".toCharArray // do not store passwords in code, read them from somewhere safe!

    val ks: KeyStore = KeyStore.getInstance("PKCS12")
    val keystore: InputStream = new FileInputStream("/root/bot/telegram/keystore.p12") // getClass.getClassLoader.getResourceAsStream("server.p12")

    require(keystore != null, "Keystore required!")
    ks.load(keystore, password)

    val keyManagerFactory: KeyManagerFactory = KeyManagerFactory.getInstance("SunX509")
    keyManagerFactory.init(ks, password)

    val tmf: TrustManagerFactory = TrustManagerFactory.getInstance("SunX509")
    tmf.init(ks)

    val sslContext: SSLContext = SSLContext.getInstance("TLS")
    sslContext.init(keyManagerFactory.getKeyManagers, tmf.getTrustManagers, new SecureRandom)
    val https: HttpsConnectionContext = ConnectionContext.https(sslContext)

    https
  }
}

object Main {
  def main(args: Array[String]): Unit = {
    val bot = new WebhookBot(System.getenv("BOT_TOKEN"))
    val token = bot.run()
    println("Press [ENTER] to shutdown")
    readLine()
    bot.shutdown()
    Await.result(token, Duration.Inf)
  }
} 

[AlexGruPerm]

Hello @mukel.
Thanks so much for the wide explanation.
I try carefully go step by step.
I use Windows platform.
yyyyyy - my keystore password.
12.34.56.78 - my static (VDS) ip.

--#1
keytool -genkey -keyalg RSA -alias mtspredbot -keystore mtspredbot.jks -storepass yyyyy -validity 360 -keysize 2048
First and Last name = 12.34.56.78 

on each question, I type my IP : 12.34.56.78

Enter key password for <mtspredbot>
        (RETURN if same as keystore password):
Re-enter new password:

--#2
keytool -importkeystore -srckeystore mtspredbot.jks -destkeystore mtspredbot.p12 -srcstoretype jks -deststoretype pkcs12

Entry for alias mtspredbot successfully imported.
Import command completed:  1 entries successfully imported, 0 entries failed or cancelled

--#3
C:\tcert>C:\openssl\openssl pkcs12 -in mtspredbot.p12 -out mtspredbot.pem
Enter Import Password:
MAC verified OK
Enter PEM pass phrase:
Verifying - Enter PEM pass phrase:

--#4
C:\openssl\openssl rsa -in mtspredbot.pem -pubout > mtspredbot.pub

After this steps I try run bot and receive next message:

[trace, com.bot4s.telegram.clients.AkkaHttpClient] REQUEST fe6e6ef6-8008-4c57-80e1-6ce6b15e9e76 SetWebhook(https://12.34.56.78,Some(Path(C:\tcert\mtspredbot.pub)),None,None)
Press [ENTER] to shutdown the bot, it may take a few seconds...
[error, com.bot4s.telegram.clients.AkkaHttpClient] RESPONSE fe6e6ef6-8008-4c57-80e1-6ce6b15e9e76 com.bot4s.telegram.api.TelegramApiException: Bad Request: bad webhook: Failed to set custom certificate file

I have some doubts about step #4 openssl rsa ...

Next time I try with pem

[trace, com.bot4s.telegram.clients.AkkaHttpClient] REQUEST 5f8a5488-ebff-471e-aea3-63bb18a5f798 SetWebhook(https://12.34.56.78,Some(Path(C:\tcert\mtspredbot.pem)),None,None)
Press [ENTER] to shutdown the bot, it may take a few seconds...
[trace, com.bot4s.telegram.clients.AkkaHttpClient] RESPONSE 5f8a5488-ebff-471e-aea3-63bb18a5f798 true

No errors, but bot don't receive something.

[mukel]

I had the same issue (no error, but didn't work), I stripped the .pem file manually, just leave the public key e.g.
----BEGIN----
somehexstuffhereakdaskdjfhfafasf
-----END-----
and send the stripped .pem file in setWebhook.

[AlexGruPerm]

I stripped .pem manually, but when I stay part
-----BEGIN-----hexstaffhere-----END-----
there is error:

09:41:02.590 [main] INFO  mtspredbot.Main$ -  webhookUrl=https://12.34.56.78 port=8443
[trace, com.bot4s.telegram.clients.AkkaHttpClient] REQUEST 319a444e-9e4a-43a2-85a4-03af837971de SetWebhook(https://12.34.56.78,Some(Path(C:\tcert\mtspredbot.pem)),None,None)
Press [ENTER] to shutdown the bot, it may take a few seconds...
[error, com.bot4s.telegram.clients.AkkaHttpClient] RESPONSE 319a444e-9e4a-43a2-85a4-03af837971de com.bot4s.telegram.api.TelegramApiException: Bad Request: bad webhook: Failed to set custom certificate file

Then I try form:
-----BEGIN CERTIFICATE-----hexstaffhere-----END CERTIFICATE-----
And now no errors, but not working

09:44:21.683 [main] INFO  mtspredbot.Main$ -  webhookUrl=https://12.34.56.78 port=8443
[trace, com.bot4s.telegram.clients.AkkaHttpClient] REQUEST 21ceee1f-5578-477a-a0dd-341b93a858a2 SetWebhook(https://12.34.56.78,Some(Path(C:\tcert\mtspredbot.pem)),None,None)
Press [ENTER] to shutdown the bot, it may take a few seconds...
[trace, com.bot4s.telegram.clients.AkkaHttpClient] RESPONSE 21ceee1f-5578-477a-a0dd-341b93a858a2 true

I can connect to bot with telnet from remote machine:

telnet 12.34.56.78 8443

May be we need create public key instead of public certificate
https://superuser.com/questions/620121/what-is-the-difference-between-a-certificate-and-a-key-with-respect-to-ssl

Will continue my investigation.

[mukel]

Did you specified the 8443 port in the webhook url? Otherwise it's 443 by default.

[AlexGruPerm]

"Did you specified the 8443 port in the webhook url?", NO
I set it with

...
class telegBotWH(log :org.slf4j.Logger,
                 config :Config,
                 sessSrc :CassSessionSrc)
    extends AkkaTelegramBot
    with Webhook
    with CommonFuncs
    with Commands[Future]
{
  LoggerConfig.factory = PrintLoggerFactory()
  LoggerConfig.level = LogLevel.TRACE
...
  val port :Int = 8443
  val webhookUrl = 
...

I will try port in url, like this https://12.34.56.78:8443

[AlexGruPerm]

Oh, Thanks so much, we solve it.

url = https://12.34.56.78:8443
And pem File certificate contains next data

Bag Attributes
    friendlyName: mtspredbot
    localKeyID: 54 69.... ..
-----BEGIN CERTIFICATE----- hexstaffhere
-----END CERTIFICATE-----

Also want add that for output connection from bot to telegram it's useful 3proxy (SOCKS), because telegram locked in Russia.
So simple 3proxy config

nserver dns1ip
nserver dns2ip
nscache 65536
timeouts 1 5 30 60 180 1800 15 60
service
log c:\3proxy\logs\3proxy.log D
logformat "- +_L%t.%.  %N.%p %E %U %C:%c %R:%r %O %I %h %T"
archiver rar rar a -df -inul %A %F
rotate 30
auth iponly
allow * 127.0.0.1, 12.34.56.78 * *
external 0.0.0.0
internal 12.34.56.78
maxconn 20
socks

where 12.34.56.78 static ip of VDS

and in the project application.conf additionally

akka {
  http.client.proxy {
    https {
      host = "12.34.56.78"
      port = 1080
    }}}

of course open port 1080 in FW.

I am going use this Bot for full control of Information system inside VDS.
And it looks like next issue will be related to CPU consumption of bot when no communication state.

Thanks one more time.

[AlexGruPerm]

Do I need (may) to close this issue?

[mukel]

Great, about the high CPU load: When using polling, the bot retry immediately if GetUpdates fail. For such purpose I added pollingGetUpdates so you can implement your own (exponential back-off+retry strategy). For webhooks it shouldn't be an issue, the bot should be dormant while there's no requests/updates to process.