Yomichan Anki card creation isn't working

[Ayokana]

I have anki running with AnkiConnect, and I go into Yomichan's settings and click "enable anki integration", and it just says "Error: Invalid Response". It's not allowing me to select a deck to make cards with. I'm not sure why that's happening, I tried re-installing Yomichan and re-installing Ankiconnect and it didn't fix it. It used to work, but I recently ran CCleaner and it stopped working after that.

[toasted-nutbread]

The Invalid response error comes from here, which seems to indicate that an invalid JSON response is being sent from Anki/AnkiConnect to Yomichan.

Few questions:

• Have you had Yomichan+AnkiConnect working in the past, or this is the first time trying to use it and its failing?
• What operating system are you using?
• What version of Anki are you using?

This is probably a long shot guess, but maybe it's a firewall thing, so make sure nothing's blocking Anki.

[atran14]

Late to the thread, but I was having the exact same issue as described above.

• I had Yomichan + AnkiConnect working before, however it has stopped working now.
• My OS is MacOS Catalina (10.15.2)
• My Anki version is 2.1.13 (3ba55990)

I have also followed the notes to disable the App Nap feature on Mac OS and disabled the firewall, but to no avail.

[FooSoft]

There have been some recent changes to AnkiConnect, but there should not be be anything breaking. Here are some things to try:

Can you access http://127.0.0.1:8765/ in your browser?
You should be able to see the current AnkiConnect version displayed.

From Anki, select Tools | Add-ons. Select AnkiConnect. Press the "Config" button on the right. Edit the configuration JSON to include a path for a log file. For example:

{
    "apiKey": null,
    "apiLogPath": "d:\\log.txt",
    "webBindAddress": "127.0.0.1",
    "webBindPort": 8765,
    "webCorsOrigin": "http://localhost"
}

You would obviously need to update the path to something that makes sense on your system, and use \\ instead of \ for paths on Windows. Note that you will probably not be able to write to C: drive directly on Windows, if you are using that. If you can post the log that is generated as a result of that, it would be helpful in figuring out what is going on with your guys' setup :)

[atran14]

I did have a log file show up in the directory I'm using, but can I get a clarification of what should I be doing in order to have the information logged into the file? Because so far, my log file is empty right now, despite me doing virtually everything that I think that would be related, like restarting Anki (and the whole computer), or restarting Chrome (which is has the Yomichan extension installed).

[FooSoft]

@atran14 you should try to use the AnkiConnect enabled features of Yomichan (for example, simply enabling AnkiConnect should cause some stuff to show up in the log). Can you also verify what is displayed when you access http://localhost:8765 when AnkiConnect is running?

[atran14]

http://localhost:8765 displays the message "AnkiConnect v.6" for me.
And while I have been trying to do what you've suggested (enabling/disabling Integration with Anki feature, and restarting Anki entirely), the log file created in the directory I specified still wasn't getting written with any data; the file still stays empty.

[toasted-nutbread]

Another option is to try and see what exactly AnkiConnect is sending to Yomichan. This can be simulated by running a script in the developer console on Yomichan's options page (press F12 to open). Copy and pasting the script below into the dev console and pressing enter to run it will log some information to the console.

The script is similar to how Yomichan makes some requests to AnkiConnect from the settings page. It will request some information from AnkiConnect (version, deck names, card model names) and log it to the console. (Disclaimer: only run dev console scripts you trust.)

(async () => {
  const results = [];
  for (const action of ['version', 'deckNames', 'modelNames']) {
    const request = await fetch('http://127.0.0.1:8765', {
      method: 'POST',
      headers: {'Content-Type': 'application/json'},
      body: JSON.stringify({action, version: 2})
    });
    results.push(await request.text());
  }
  console.log(JSON.stringify(results, null, 4));
})();

For example, my results look like this:

[
    "6",
    "[\"Default\", \"Core\", \"Test Deck\"]",
    "[\"Japanese-75658\", \"Basic\", \"Cloze\", \"Basic (and reversed card)\", \"Basic (optional reversed card)\"]"
]

Screenshot of how the console looks:

If there is a connection error, messages that look like the following will be displayed:

[atran14]

This is the result that I have back from executing that piece of code

[
    "6",
    "[\"Japanese - \\u5143\\u6c17\", \"Default\"]",
    "[\"Basic (and reversed card)\", \"Basic\", \"Cloze\", \"Core Vocab\", \"Basic (optional reversed card)\", \"Japanese\\uff08\\u8aad\\u307f\\u66f8\\u304d\\uff09\"]"
]

In addition, the log file has also been logged with the following by executing the above piece of code

[request]
{
    "action": "version",
    "version": 2
}

[reply]
6

[request]
{
    "action": "deckNames",
    "version": 2
}

[reply]
[
    "Japanese - \u5143\u6c17",
    "Default"
]

[request]
{
    "action": "modelNames",
    "version": 2
}

[reply]
[
    "Basic (and reversed card)",
    "Basic",
    "Cloze",
    "Core Vocab",
    "Basic (optional reversed card)",
    "Japanese\uff08\u8aad\u307f\u66f8\u304d\uff09"
]

However though, the Anki integration in Yomichan's settings is still displaying the "Error: Invalid response" message.

[toasted-nutbread]

Okay, from those values I don't see anything that looks immediately incorrect, so here is another script to try. This time, the code is taken directly from Yomichan's source code rather than trying to use a shortened version.

(async () => {
  function requestJson(url, action, params) {
    return new Promise((resolve, reject) => {
      const xhr = new XMLHttpRequest();
      xhr.overrideMimeType('application/json');
      xhr.addEventListener('load', () => resolve(xhr.responseText));
      xhr.addEventListener('error', () => reject(new Error('Failed to connect')));
      xhr.open(action, url);
      if (params) {
        xhr.send(JSON.stringify(params));
      } else {
        xhr.send();
      }
    });
  }

  const results = [];
  for (const action of ['version', 'deckNames', 'modelNames']) {
    const text = await requestJson('http://127.0.0.1:8765', 'POST', {action, version: 2});
    try { JSON.parse(text); } catch (e) { console.log(action, e); }
    results.push(text);
  }
  console.log(JSON.stringify(results, null, 4));
})();

[FooSoft]

@toasted-nutbread the version is being sent as "2", that is incorrect, it should be "6". This is causing AnkiConnect to send back data in a deprecated format.

[FooSoft]

Actually nevermind, Yomichan never needed to switch to the format which contained the additional response information... This is very strange.

[atran14]

Apologies for the late reply. Here's the image of the console's output from executing the script
https://ibb.co/7k1snW4

In addition, here's the log:

[request]
{
    "action": "version",
    "version": 2
}

[reply]
6

[request]
{
    "action": "deckNames",
    "version": 2
}

[reply]
[
    "Japanese - \u5143\u6c17",
    "Default"
]

[request]
{
    "action": "modelNames",
    "version": 2
}

[reply]
[
    "Basic (and reversed card)",
    "Basic",
    "Cloze",
    "Core Vocab",
    "Basic (optional reversed card)",
    "Japanese\uff08\u8aad\u307f\u66f8\u304d\uff09"
]

[toasted-nutbread]

This is quite strange indeed, everything appears to be working based on these results. The only other thing I can think to do is to try and edit the extension code to display more diagnostic information, but I don't know if you're comfortable doing that.

Regardless, I think thinks merits adding some additional info to the "Invalid response" message. I also tweaked the request method recently in 7686e56 which hasn't worked its way into a new build yet, but I wouldn't think that would fix the issue. Thank you for helping to debug the issue thus far.

[siikamiika]

@atran14
When did you update to Catalina? I've heard it cause random issues due to its security features. Wouldn't ask otherwise, but there's not much else to look at.

Another suspect is

yomichan/ext/bg/js/settings/anki.js

Line 66 in 1fd568a

const [deckNames, modelNames] = await Promise.all([utilAnkiGetDeckNames(), utilAnkiGetModelNames()]);

The AnkiConnect code looks like it should handle asynchronous requests without issues, but the previous code waited for each of those to complete before running the next query. 5339381

[toasted-nutbread]

I have added additional debugging information to the "Invalid response" message in 26ea278. The message on the settings page can be expanded to show extra data and a stack trace. With this, we should hopefully hopefully at least be able to see what response is triggering the issue.

[atran14]

@siikamiika If memory serves me correctly, it was late last year. Anki integration with AnkiConnect was still working properly until recently.
And @toasted-nutbread, no problem! And I'm fine with digging deeper into the source code, if you guys are OK with it.

[siikamiika]

@atran14
Ok, probably not the Mac update then. It can be something in Yomichan, AnkiConnect or Anki itself.

Would be awesome if you could checkout master and test it with Anki! Just go to chrome://extensions/, enable developer mode and "load unpacked" the yomichan/ext directory.

[toasted-nutbread]

Looks like this may have also previously happened in #142, or something similar.

[atran14]

Alright, I have been doing some messing around with the yomichan/ext extension in my Chrome browser. Here's what happened:

1. Regarding the version that I pulled directly from the Github repository, the error disappeared. Things seemed to be working fine. However, while the integration option did show the deck that I want to create the card into, it wasn't properly any sort of fields that I want it to at all, as shown in the picture below

2. However, when switching back to the stable version (the one that I installed from the Google Extension store), the error message showed up once again, in addition to not being able to load the deck and the field information

So, at this point, I'm not really sure what kind of issue I'm dealing with here.

[siikamiika]

@atran14 Are you able to export your settings from the stable version using "Backup" at the bottom of the settings and import them to the debugging version?

edit: never mind if it was a fresh install already

[toasted-nutbread]

it wasn't properly any sort of fields that I want it to at all, as shown in the picture below

You have to select a model also. Try changing that and then the fields should appear.

If that works, then it seems like the issue will be fixed with the next release, which is good. However, that doesn't really tell us what was broken in the first place. The only remaining thing that I can think of is the content-type override changing from 'application/json' to 'text/plain' in 7686e56#diff-3c423cf7eed4ab8a739f35db8c82bd3aL23-R23

[FooSoft]

I'll put out a testing version, and if that goes fine, I will promote it to official next weekend.

[atran14]

An update: Fresh uninstall and install of the extension has apparently solved my problem now, but there's something I just noticed. For the context, I did, prior to this, export the my old settings from the stable version that was getting the "Invalid response" error.
A fresh install of the Yomichan extension does indeed restore the functionality of the Anki integration and I am now be able to integrate with Anki and adjusting the model and changing the field with no issues at all. However, if I import back the settings that I mentioned above, the error showed up again.

(Supplement 1: Restoring Yomichan back to its default settings also appears to solve the error message too)

Do you guys mind if I upload the settings profile here?

[siikamiika]

@atran14
Just make sure you don't leave any personal info in the export and I think it's going to be helpful for debugging

[atran14]

Attached is the settings profile that was causing the "Invalid response" for me (You need to change the extension to .json).
I think the only data that pertains to personal information would be the environment, which I think wouldn't be a problem. I don't know whether this issue can be reproduce on your machine, but see if it helps.
yomichan-settings-2020-01-26-21-08-51.txt

[siikamiika]

It was missing anki.server "Interface server" (which can actually also be seen in your screenshot too). It's supposed to be http://127.0.0.1:8765. It's defined as that value in the schema so I'm not sure what's going on. An empty string is a valid value, though.

You have to enable advanced options to see and edit the server address.

[siikamiika]

So apparently an empty string requests a path relative to the background page, which is the background page itself. That's not valid JSON.

[toasted-nutbread]

That would explain the invalid response, since it's effectively requesting background.html. It may be a good idea for us to add a better message indicating why this might be happening.

[atran14]

That completely solves the problem now. No more issues regarding with the error message anymore.
Thanks and appreciated all the help so far.

[siikamiika]

@toasted-nutbread is it possible to ban an empty string in the schema like "minLength": 1 or something and default to http://127.0.0.1:8765? There also seems to be https://json-schema.org/understanding-json-schema/reference/regular_expressions.html but I don't think it's implemented at the moment

[toasted-nutbread]

We could do that, but a message telling the user to adjust the Interface server option is probably the most foolproof. This will handle other cases, such as other relative URLS, or if a HTTP server is running on http://127.0.0.1:8765 which isn't Anki.

[siikamiika]

Yeah, agreed. The verbose error message you added is also great for future bug reports.

[toasted-nutbread]

Added improved error message details in 2dbb24e and info about the default value. @Ayokana if you are still experiencing this issue, try this solution to see if it fixes it.

[toasted-nutbread]

Friendly reminder @FooSoft that there is still a testing release waiting to be released on stable. #327 (comment)

The only issue that I noticed in that release was #337, which sometimes causes some tags to not show up depending on the results mode, but I don't think this is a blocker.