wrangler tail fails with 10013: workers.api.error.unknown

[jpwilliams]

🐛 Bug Report

Environment

• operating system: Windows
• output of rustc -V: not installed
• output of node -v: v12.13.0
• output of wrangler -V: wrangler 1.10.1
• contents of wrangler.toml

name = "REDACTED"
type = "webpack"
account_id = "REDACTED"
zone_id = "REDACTED"
webpack_config = "webpack.config.js"
vars = { REDACTED }
routes = [ REDACTED ]

[env.prod]
name = "REDACTED"
vars = { REDACTED }
routes = [ REDACTED ]

Steps to reproduce

1. Run wrangler tail --env prod

What did you expect to see?

Logging from the prod environment.

What did you see instead?

$ wrangler tail --env prod
 Setting up log streaming from Worker script "REDACTED". Using ports 8080 and 8081.
⠁   This may take a few seconds...
[INFO] Cannot determine default configuration path. No file [config.yml config.yaml] in [~/.cloudflared ~/.cloudflare-warp ~/cloudflare-warp /usr/local/etc/cloudflared /etc/cloudflared]
[INFO] Version 2020.6.5
[INFO] GOOS: windows, GOVersion: go1.12.9, GoArch: amd64
[INFO] Environment variables map[metrics:localhost:8081 proxy-dns-upstream:https://1.1.1.1/dns-query, https://1.0.0.1/dns-query url:localhost:8080]
[INFO] Starting metrics server on 127.0.0.1:8081/metrics
[INFO] cloudflared will not automatically update on Windows systems.
[INFO] cloudflared does not support loading the system root certificate pool on Windows. Please use the --origin-ca-pool to specify it
⠖   This may take a few seconds...
⠂   This may take a few seconds...
[INFO] Each HA connection's tunnel IDs: map[0:bnggl2bx3re26qxli1cbb3p92n8aue0ads361nhikd42al52xi40]
[INFO] +--------------------------------------------------+
[INFO] |  Your free tunnel has started! Visit it:         |
[INFO] |    https://owen-jets-hits-llp.trycloudflare.com  |
[INFO] +--------------------------------------------------+
⠄   This may take a few seconds...
⠠   This may take a few seconds...
⠤   This may take a few seconds...
[INFO] Each HA connection's tunnel IDs: map[0:bnggl2bx3re26qxli1cbb3p92n8aue0ads361nhikd42al52xi40 1:bnggl2bx3re26qxli1cbb3p92n8aue0ads361nhikd42al52xi40]
[INFO] +--------------------------------------------------+
[INFO] |  Your free tunnel has started! Visit it:         |
[INFO] |    https://owen-jets-hits-llp.trycloudflare.com  |
[INFO] +--------------------------------------------------+
⠤   This may take a few seconds...
[INFO] Connected to LHR
[INFO] Already connected to this server, selecting a different one
⠠   This may take a few seconds...
[INFO] Each HA connection's tunnel IDs: map[0:bnggl2bx3re26qxli1cbb3p92n8aue0ads361nhikd42al52xi40 1:bnggl2bx3re26qxli1cbb3p92n8aue0ads361nhikd42al52xi40 2:bnggl2bx3re26qxli1cbb3p92n8aue0ads361nhikd42al52xi40]
[INFO] +--------------------------------------------------+
[INFO] |  Your free tunnel has started! Visit it:         |
[INFO] |    https://owen-jets-hits-llp.trycloudflare.com  |
[INFO] +--------------------------------------------------+
⠓   This may take a few seconds...
[INFO] Connected to LHR
[INFO] Already connected to this server, selecting a different one
Error:  Code 10013: workers.api.error.unknown

Extra info

I had an attack on the zone that this worker is on, so had to turn on "Attack Mode". The prod environment logs were working previous to that, but even though "Attack Mode" is now off, I can't access them.

I'm assuming this has something to do with Attack Mode, but am unsure due to the generic error! What can I do to help debug on my side?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[jpwilliams]

Can I "bump" my own ticket? This is still an issue,

[ashleymichal]

i keep seeing this one and without spelunking in the api code it's hard to know what triggers this error code; probably best that a CF contributor does so and amends the error message to give the user better insight.

[cdloh]

Also ran into this today. Some work on it would be appreciated.

[nataliescottdavidson]

Hi @jpwilliams, could you share your webpack.config.js?

[jpwilliams]

@nataliescottdavidson Sure. It's in a private repo so can't link to it, but:

// core
const path = require("path");

// config
const mode = process.env.NODE_ENV || "production";

module.exports = {
  output: {
    filename: "worker.js",
    path: path.join(__dirname, "dist"),
  },
  mode,
  resolve: {
    extensions: [".ts", ".js"],
    plugins: [],
  },
  module: {
    rules: [
      {
        test: /\.ts$/,
        loader: "ts-loader",
        options: {
          transpileOnly: true,
        },
      },
    ],
  },
};

[cdloh]

I managed to get around this by fully deleting the worker in Cloudflare and then republishing it.

Thankfully I only hit this on a development worker and not a production one but this at least fixed it for the dev one.

Note I made no changes to webpack or the actual worker in between the delete and re-publish.

[jpwilliams]

I managed to get around this by fully deleting the worker in Cloudflare and then republishing it.

I experienced the same thing as @cdloh above, though it took me a little while longer to try this as mine was a production worker when it began displaying this issue. Even if I update the original worker with new code I continue to see the same issue, but publishing it under a new name gave me immediate access to the logs for that newly-published worker.

This would lead me to believe that it's not a configuration issue, but something internal on Cloudflare's side that applies to individual workers.

Sorry - I would've posted this with the initial issue posting but only did this myself a few weeks ago and didn't think to update it. @cdloh's answer is a wonderful workaround if anyone else is currently experiencing this.

[nataliescottdavidson]

Hey @jpwilliams, to diagnose the error I need to take a look at the logs. Could you email your account id, script name, and zone id (or just your wrangler.toml) to wrangler@cloudflare.com?

[jpwilliams]

Of course, @nataliescottdavidson. I've sent an email over with my wrangler.toml attached.

Shout if you need anything else!

[jpwilliams]

@nataliescottdavidson I'm afraid my mail to wrangler@cloudflare.com failed:

We're writing to let you know that the group you tried to contact (wrangler) may not exist, or you may not have permission to post messages to the group. A few more details on why you weren't able to post:

 * You might have spelled or formatted the group name incorrectly.
 * The owner of the group may have removed this group.
 * You may need to join the group before receiving permission to post.
 * This group may not be open to posting.

[nataliescottdavidson]

@jpwilliams gah, I'm sorry about that. I'm new to the team and still figuring out some processes. I've changed the permissions to allow non members to post to the group. Would you mind resending the email?

[jpwilliams]

No worries whatsoever, @nataliescottdavidson!

I've resent the email and it looks like it's gone through fine this time. 👍🏻

[Erisa]

In my case, I had this error by accidentally attempting to create a variable (vars =) that had its name conflict with an existing secret.

The line in question was

vars = {WORKERLINKS_SECRET  = "changeme" }

And it already existed as a secret:

Obviously this won't always be the case for this error and it was completely my fault but maybe error output could be improved for edge cases like this so we don't have to go on a debugging adventure trying to find the small thing that's wrong?

[spencercap]

@Erisa 🙏

[noah-seltzer]

@nataliescottdavidson I'm experiencing the same error. I'm very reluctant to try and delete + recreate the worker as its in production. Is there any way you could assist me?

[saul-data]

I just experienced the same issue - would be good to have this resolved...

[venom90]

Any update on this issue? I deleted the worker but still the error.

wrangler publish
✨  Built successfully, built project size is 13 KiB.
Error: Something went wrong! Status: 500 Internal Server Error, Details {
  "result": null,
  "success": false,
  "errors": [
    {
      "code": 10013,
      "message": "workers.api.error.unknown"
    }
  ],
  "messages": []
}

[nataliescottdavidson]

@venom90 , could you retry and let me know if error is persisting? We had an incident with another service today that caused 10013s and is now resolved.

[mrl5]

@venom90 , could you retry and let me know if error is persisting? We had an incident with another service today that caused 10013s and is now resolved.

I also had this issue today for wrangler publish:

Error: Something went wrong! Status: 500 Internal Server Error, Details {
  "result": null,
  "success": false,
  "errors": [
    {
      "code": 10013,
      "message": "workers.api.error.unknown"
    }
  ],
  "messages": []
}

however it's no longer present. @nataliescottdavidson at least for me it's solved now

[Electroid]

This was fixed in our API.

[developerdino]

This just happened for me, however, it was because I already had another worker using the same routes which then clashed. Once I removed the routes on the old worker it published fine.