New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MAL API is down #588

Open
Akamaru opened this Issue May 24, 2018 · 65 comments

Comments

Projects
None yet
@Akamaru

Akamaru commented May 24, 2018

It seems that MyAnimeList has had a vulnerability in the API and has now disabled it.
Taiga does not work anymore and shows an error.

MyAnimeList returned an error: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>404 Not Found</title> </head><body> <h1>Not Foud</h1> <p>The requested URL /api/account/verify_credentials.xml was not found on this server.</p> <p>Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocu...

image

I know it's not an taiga issue and erengy can't fix this. We have to wait for MAL.

@Hyoretsu

This comment has been minimized.

Hyoretsu commented May 24, 2018

Looks like it wasn't just me

@taigalover

This comment has been minimized.

taigalover commented May 24, 2018

Phew, wasn't just me either

@erengy erengy changed the title from MAL API Issue to MAL API is down May 24, 2018

@moyitpro

This comment has been minimized.

moyitpro commented May 24, 2018

This is like a train wreck. The new official MAL API stop working and now this. This is very frustrating.

@erengy

This comment has been minimized.

Owner

erengy commented May 24, 2018

It's a serious issue. They completely disabled the API without prior notice to third-party developers, and even went as far as removing the API documentation page. This affects all applications that make use of the official API. It's undoubtedly a new low in their track record.

There's no telling when the issue is going to be resolved, but contacting DeNA's customer support seems to be the best way to file your complaints at the moment. From what I can tell, this whole thing is related to DeNA (MAL's parent company) rather than the MAL staff.

Note that you can continue using Taiga while the MAL API is down, to some extent. But you won't be able to sync your list, retrieve new anime information, etc. Taiga will send any queued updates to MAL when the API becomes available.


Relevant forum topics:
https://myanimelist.net/forum/?topicid=1731317
https://myanimelist.net/forum/?topicid=1731329

@moyitpro

This comment has been minimized.

moyitpro commented May 24, 2018

Ugh, not this again. I guess I should focus more on adding AniList support to Hachidori.

@onlybrad

This comment has been minimized.

onlybrad commented May 24, 2018

I don't even remember what email I used to create my MAL account. So not only I can't recuperate my account, I can't even resync my anime list with a new account because the API is down...

@erengy

This comment has been minimized.

Owner

erengy commented May 24, 2018

We've been told that the API outage is for an indefinite period of time. MAL staff are unable to comment further on the issue; they encourage us to speak with DeNA's customer support instead. Apparently there's been some disagreement between them and DeNA on the matter.

We currently don't know what this is all about, but it wasn't due to a security breach. If I had to speculate, it might be related to GDPR (aka "we've updated our privacy policy" regulation) coming into effect tomorrow. Perhaps they couldn't work out the issues in time and ended up shutting down the API entirely. In any case though, DeNA handled the situation quite poorly so far. We're hoping to get a proper response tomorrow.

In the best-case scenario, the API will come back online soon and Taiga will continue working as usual. In the worst-case (?) scenario, many third-party application developers, including myself, will stop using MAL altogether and suggest other people to do the same.

@guft

This comment has been minimized.

guft commented May 25, 2018

Yes I believe it is GDPR related. Over the last week many online services have updated their privacy policies and forced users to reset passwords and reconfirm permissions. Considering that MAL's entire purpose is tracking stats and information about its users, it makes sense they would have to jump through a large number of hoops to be GDPR compliant. Taking the API down is a drastic step, but considering GDPR's fines for noncompliance are 20 million euros or 4% of global revenue (whichever is greater) it makes sense they would do this if they are afraid they are not in compliance. However, keeping the site on lockdown is not good for their business either, so if they can't satisfy regulator demands they will likely resort to blocking EU users and restoring the API everywhere else, which is what many other online services have resorted to doing in the last week.

@Ruhrpottpatriot

This comment has been minimized.

Ruhrpottpatriot commented May 25, 2018

GDPR is a bullshit excuse! That regulation is in effect since TWO years, today the transition period ended that was put into place by the EU to allow relevant services to migrate their stuff. That means, they had two years time to make everything GDPR compliant, yet sat on their asses and did nothing.

Is Kitsu still working? If so MAL has seen the last of me.

While we're at it. How do I clear the sync queue?

@moyitpro

This comment has been minimized.

moyitpro commented May 25, 2018

@Ruhrpottpatriot

MAL importing on Kitsu still work and Taiga and other third party clients that work with Kitsu should still work.

@Azraelle

This comment has been minimized.

Azraelle commented May 25, 2018

REEEEE

@Kovaelin

This comment has been minimized.

Kovaelin commented May 25, 2018

MAL's going to lose a lot of users if they don't fix their API.

@Ruhrpottpatriot

This comment has been minimized.

Ruhrpottpatriot commented May 25, 2018

@moyitpro Yeah, I figured that out. But not I have to manually update my list, since Taiga first pulls the list, then pushes it back. That resulted in a loss of data for me.

@erengy Could you implement a way to force push the list to the selected online service. Sometimes the user knows that the online service is outdated and just wants to overwrite it with new data.

@Zenithtb

This comment has been minimized.

Zenithtb commented May 25, 2018

@Ruhrpottpatriot - interesting idea - using Taiga as a pull-me-push-you of data to be able to change service... I like your thinking!

@taigalover

This comment has been minimized.

taigalover commented May 26, 2018

Taiga will send any queued updates to MAL when the API becomes available.

So do we now use our old password or new password to login to Taiga? So when (if) the API becomes available, taiga can send the queued updates?

@Azraelle

This comment has been minimized.

Azraelle commented May 26, 2018

Anilist is adding back xml import. I'm jumping the ship.

@Ruhrpottpatriot

This comment has been minimized.

Ruhrpottpatriot commented May 26, 2018

@Azraelle I heard that MAL has an xml export function multiple times now. Where can I find it?

@Azraelle

This comment has been minimized.

Azraelle commented May 26, 2018

@asakurato

This comment has been minimized.

asakurato commented May 26, 2018

screenshot_gmail_20180526-220221
Just received this from DeNa (MAL support), so maybe there is hope

@Akamaru

This comment has been minimized.

Akamaru commented May 26, 2018

@Ruhrpottpatriot

This comment has been minimized.

Ruhrpottpatriot commented May 26, 2018

Which, is a complete bullshit excuse as an admin (Xinil) has stated in the forum, that there was no security breach.

@tophf

This comment has been minimized.

tophf commented May 27, 2018

A weird workaround could be writing a Chrome/Firefox extension that communicates with Taiga via nativeMessaging and keeps MAL site in a hidden iframe with a content script inside that can supposedly do everything the API provided via page DOM. It can even run in a shell tray when all browser windows are closed unless the user explicitly exited the browser via Ctrl-Shift-Q or Exit command. Such extension seems relatively easy to implement.

@spillerrec

This comment has been minimized.

spillerrec commented May 27, 2018

Thanks for this issue even though it is not due to Taiga, as I came here to see what was going on.

To me it sounds like it might be a third party website that stole peoples login information, though the timing is suspiciously close to the GDPR deadline. (Correction for above, it is up to 20 million euros or 4% of revenue, whichever is greater.)
I do think it is a very bad idea that the API uses the username/password for credentials, now that they apparently have a store and can save peoples credit card information. Very bad idea... Some sort of API key system (such as SSH keys or whatever) where you can have multiple keys and set permissions for each would be much better.

I would give it a bit more time before putting in too much work on a workaround. I would prefer if everything was done in Taiga though, so there is no dependency on browsers and IPC. Just too much that can go wrong and I fear it wouldn't work in Wine either. I doubt their internal API is so weird that you can't simulate the requests without using a full-featured web browser.

For convenience, here is the MAL link to export your anime/manga list:
https://myanimelist.net/panel.php?go=export

@marwanpro

This comment has been minimized.

marwanpro commented May 27, 2018

Fine I will develop a C# API tomorrow.

@moyitpro

This comment has been minimized.

moyitpro commented May 27, 2018

@spillerrec
I have worked with the new MAL API before they closed down the new official MAL API. It was using OAuth2 with PCE challenge, which is more secure than using Basic Auth. If the username and password thing was an issue, they should of retrofit OAuth2 to the old API and require developers to register their apps to obtain a OAuth2 client and secret to the old API until the new API is ready.

@KrisKamweru

This comment has been minimized.

KrisKamweru commented May 28, 2018

I think I'll just be switching over to Kitsu. Would there be any reason why I should not, i.e. is Kitsu inferior in any way to MAL?

@erengy

This comment has been minimized.

Owner

erengy commented Jun 8, 2018

MAL API has been offline for two weeks now. Our current situation is this:

Our dev team is currently focused on getting the website fully reviewed right now, so it's difficult to say when we may have news about the API.

They still have to review character and staff databases, user profiles and blogs, forums and clubs, videos, reviews, recommendations, news, articles, store... At their current pace, that's going to take them at least two more weeks. So, you should set your expectations accordingly.

MAL's API has never been a priority for them, because they didn't use it themselves. Contrast this with AniList and Kitsu: Their websites consume their own public APIs, so they have to keep it working.

@erengy

This comment has been minimized.

Owner

erengy commented Jun 14, 2018

It's been three weeks now. Still no news about the API. Their latest tweet says:

Unofficial apps (e.g. iMAL, MyAniList, MALClient, PocketMAL) will still not be working properly as our public API is still disabled. We are aware this is an important issue for users, and apologize for not having any new information to provide yet.

I won't be posting any more updates here until something substantial happens.

@Fcort237

This comment has been minimized.

Fcort237 commented Jul 9, 2018

Hello, very interesting your post, I want to thank you, but I would like you to answer me the following: which API anime database you currently use for your anime website, I'm starting in this I already have a movie website, but I want Can you make an anime one?

@Nevalopo

This comment has been minimized.

Nevalopo commented Jul 10, 2018

I feel so lost without Taiga :(

I hope they fix their API really soon or i might have to go check out other anime databases

@Fcort237

This comment was marked as spam.

Fcort237 commented Jul 10, 2018

@Fcort237

This comment was marked as spam.

Fcort237 commented Jul 10, 2018

@qgustavor

This comment was marked as resolved.

qgustavor commented Jul 10, 2018

@Fcort237 No te entiendo: ¿no conoces AniList y Kitsu? Ambos funcionan con el Taiga. También existe AniDB y Anime Planet. No soy el responsable de Taiga, pero por mi experiencia con el GitHub recomiendo que no quede enviando varios mensajes y espere que alguien te responda. Probablemente aún no te respondieron porque están sin tiempo. En cuanto a crear otra base de datos mi opinión es que ya hay varios de ellos, crear otro más no sería una buena idea.

@haliliceylan

This comment has been minimized.

haliliceylan commented Jul 21, 2018

MAL API is still offline...

@Volatar

This comment has been minimized.

Volatar commented Jul 22, 2018

iMAL, the iOS app, switched to spoofing the web interface to restore partial functionality to it's app. Might be time to consider that.

@qgustavor

This comment has been minimized.

qgustavor commented Jul 22, 2018

@Volatar Browser spoofing isn't hard to implement, but it's hard to maintain: as MyAnimeList never had a complete API clients used to spoof browsers and parse data which the original API don't provided (example 1, example 2). Because of this most clients already have tools in order to parse HTML.

One of the features the official API had is updating the anime list: it's not hard to implement spoofing a browser, just do a GET request to the edit anime page, edit the <form> as you want, add the CSRF token to it and then submit it. In the other hand it's likely to break as MyAnimeList is updating the entire website to fix "issues" and because there are some people thinking they're doing that in order to kill third party applications. I don't think they will put a captcha on this page - users would hate this - but there other things that they can do in order to make spoofing harder.

@Volatar

This comment has been minimized.

Volatar commented Jul 22, 2018

@qgustavor The question then becomes "is it worth it" I guess.

I for one would love Taiga to be able push the two months of queued syncing it's been waiting on so I could then think about moving to a difference service if nothing else.

I can only speak from the user perspective (with some coding knowledge, so I do know how difficult these things can be), but man, I really miss Tagia. Without it I honestly find myself discouraged from watching anime these days. Went from watching 14 shows last season, to two this season. I just can't keep track of more than that without tools.

I wish I could contribute myself.

@Nevalopo

This comment has been minimized.

Nevalopo commented Jul 22, 2018

I just went to https://myanimelist.net/panel.php?go=export To export my anime list from MAL and then imported it at kitsu now everything is working flawlessly agian after changing the service to Kitsu in Taiga.

@spillerrec

This comment has been minimized.

spillerrec commented Jul 23, 2018

@Volatar erengy added an export option from Taiga, so you should be able to use that to move to a different service without loosing your 2 months of progress I believe. See one of the posts above for details.

Tools → Export anime list → Export as MyAnimeList XML.

@Zorua

This comment has been minimized.

Zorua commented Jul 23, 2018

@spillerrec I'll let you know that even though

Taiga is up to date!
Current version: 1.3.0

the menu entry is not there.

@Akamaru

This comment has been minimized.

Akamaru commented Jul 23, 2018

@Zorua see the comment from 30. May here

I added an export feature to Taiga in 6af570e to make things a bit easier:

  1. Get the latest build of Taiga.
  2. Use the new feature via Tools → Export anime list → Export as MyAnimeList XML.
  3. Upload the generated XML file to AniList or Kitsu.
    This file will include your queued updates. However, my previous warning still stands:

Note that Taiga may not have your entire list data (e.g. how many times you've re-watched a series) due to limitations of MAL API (sigh).

@MatiasPi

This comment has been minimized.

MatiasPi commented Aug 2, 2018

I know that this has already been said but I'll emphasize it: There is no good reason to continue using MAL, or waiting for them to re-enable their API. Export your list to Kitsu and use Taiga that way. I've been using Kitsu for about a month and I can see it has a much better looking website, has all the features MAL has and more, and their API is fully functional. I have not looked back since.

Only problem I had with Kitsu was the simplified ranking system, but I later found out that you can change it to a 10 point system in the settings, just like MAL. It is fully integrated into Taiga as well. I freaking love Kitsu/Taiga.

MAL used to be great, but it is time to move on, sadly.

@aftadizca

This comment has been minimized.

aftadizca commented Aug 4, 2018

Use Anilist now. Good feature..

@erengy

This comment has been minimized.

Owner

erengy commented Aug 6, 2018

If you have a large amount of changes waiting to be synced with MAL, here's something new you can do:

  1. Upgrade to Taiga v1.3.1, if you haven't done so already.
  2. Export your anime list via Tools → Export anime list → Export as MyAnimeList XML.
  3. Import that file to MyAnimeList by choosing the MyAnimeList Import option.
  4. Check your online anime list at MAL to make sure it's up to date now.
  5. Go to History page of Taiga, right-click on the page, then select Clear queue → Merge.

Note that MAL will overwrite the entries you import, and Taiga doesn't have all the data for your list entries (e.g. how many times you've re-watched a series) due to limitations of MAL API. This shouldn't cause any issues for most people, though.

Also make sure to select the Merge option rather than the Delete option at the final step. Otherwise your local list will lose all the changes you made since MAL API went offline.

@Kovaelin

This comment has been minimized.

Kovaelin commented Aug 8, 2018

Another month before we get an update on the API. https://twitter.com/myanimelist/status/1027084541268295681

@ifonefox

This comment has been minimized.

ifonefox commented Aug 9, 2018

There's a little more detail in the forum post:

What about the API?
MAL is concentrating on bringing the site back to its full functionality before work on the API can begin. At this time, the work on the API has not yet been scheduled, but we will provide an update later this month.

At least its better than radio silence on the API

@erengy

This comment has been minimized.

Owner

erengy commented Aug 31, 2018

The previously-announced announcement announces a series of announcements regarding the API in the upcoming weeks:

We are currently reviewing the steps necessary to complete development of a new API, which would allow third party applications to function again. Up until this point, third party applications have made use of an older API to interface with MAL, allowing them to interact with the site and function. When the API was disabled at the end of May, this caused the majority of third party applications to stop functioning, as they were no longer able to interact with the site.

At this point we have no other information to share, but aim to provide bi-weekly updates on the situation.

In addition, after 99 days of silence, a DeNA representative brought forward an apology in an internal post, claiming their limited resources in a billion-dollar company as the reason the API was put on-hold for so long. They promised to provide actual new information in time, while mentioning that they do not have a release date for the new API yet. We will see how long it takes for them to deliver.

@Volatar

This comment has been minimized.

Volatar commented Sep 16, 2018

It's been four months now and the API is still not restored. I was patiently waiting, but I am now done.

As @erengy said

a DeNA representative brought forward an apology in an internal post, claiming their limited resources in a billion-dollar company as the reason the API was put on-hold for so long.

When it comes down to it, API users are the absolutely last priority for this kind of company. Facebook revealed for it's IPO that they simply could not figure out how to monetize (via advertising) their mobile users, which had that that time (IIRC) become over 2/3rds of the userbase. API users are even lower on the totem pole. You can try and deliver ads to API users, and the application developers will throw the ads away. Reddit also ran into this problem, and ended up trying to solve it by buying up and killing the largest mobile apps that were using their API. Twitter bought their largest API application so they could put ads in it. They all know they can't fully kill their API's -- no one can if they want the vocal and influential power users that draw others to stick to their service, as such people do not use the web interfaces as a rule, but they can do their best to discourage the common man from such applications.

MAL has gone a step beyond all of those and actually killed their API. Not permanently (or so they say) but I have no faith at this point they actually even want to bring it back.

Thank you everyone for keeping up with this thread and it's discussion.

Today I evaluate the competition and decide where I am moving my database to, and I am not going back.

👋

EDIT: I do hope I can find a good iOS app for whatever I move to though...

EDIT2: AniList impressed me right off the bat, and they have not only mobile apps, but they actually list Taiga among their apps list. That's just awesome. I have made the move.

o7 MAL. It was fun while it lasted, but AniList has a night mode :3

@moyitpro

This comment has been minimized.

moyitpro commented Sep 16, 2018

Actually, there was an update recently regarding the API

We are currently working a new terms of use for third party application developers. Also, we have mostly finished reviewing all the tasks necessary for completing development. We are planning to start work on the remaining development some time next week.

I wouldn't lose hope yet, but it will be a while and we still don't know if they will restore the beta or not.

@BlackJoe23

This comment has been minimized.

BlackJoe23 commented Sep 21, 2018

Ye also I hate all the other services AI's myanimelist has the most decent desktop friendly one. And the other don't even store your entry update dates and have a shittier database. mal is the most anidb like without being out there to use.

@Volatar

This comment has been minimized.

Volatar commented Sep 26, 2018

@moyitpro I was directly responding to that terrible ambiguous MAL update on their API

@timw4mail

This comment has been minimized.

timw4mail commented Oct 12, 2018

If you really want MAL updates, and you use Kitsu, you can enable syncing on your Kitsu account. They use screen-scraping, but their syncing continues to work in spite of the MAL API nonsense.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment