Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stopped working after MIQ website update #71

Closed
alistairgraham opened this issue Jul 23, 2021 · 21 comments · Fixed by #75
Closed

Stopped working after MIQ website update #71

alistairgraham opened this issue Jul 23, 2021 · 21 comments · Fixed by #75

Comments

@alistairgraham
Copy link

miq-nz-booking-assistance-Windows-v0.10.1.exe has stopped refreshing the page. I think it might be because it's waiting for an element with the class .flatpickr-input to exist but it never exists now?

await page.waitForSelector('.flatpickr-input');

@Hijzij
Copy link

Hijzij commented Jul 23, 2021

Have the same problem: stopped working as off 1430hrs NZtime Friday 23July.
Closed and restarted programme: no success
Restarted laptop: no success
Removed app and reloaded: no success.......
Hopefully some-one can help us.

@acidack
Copy link

acidack commented Jul 23, 2021

yeap they've changed the way the backend once again making it harder to scrape.. none of the tools are working currently as a result.

@cummins070
Copy link

cummins070 commented Jul 23, 2021

Same problem here, it was working earlier today, had it running all day (based in the US). Hopefully it's an easy fix!

@necastroz
Copy link

I got this issue ...... any help ? Refused to apply inline style because it violates the following Content Security Policy directive: "default-src 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-3juv2Ft1FaY3xWYNGXExi9oTqA3RQ4gGhU='), or a nonce ('nonce-...') is required to enable inline execution. Note that hashes do not apply to event handlers, style attributes and javascript: navigations unless the 'unsafe-hashes' keyword is present. Note also that 'style-src' was not explicitly set, so 'default-src' is used as a fallback.

@Lyannel12
Copy link

same help please.. :(

@nedmi
Copy link

nedmi commented Jul 23, 2021

I also have not been able to get back online line as well checked 9.0 it also does not scan the month request

@stilljake
Copy link
Contributor

The MIQ website has been updated and all the flatpickr calendar elements have changed.
Screenshot 2021-07-23 at 05 05 53

You are correct @alistairgraham in thinking .flatpickr-input never exists because it doesn't anymore. Probably this was an attempt by them to combat bots.

They have done the same thing to the calendar at https://allocation.miq.govt.nz/portal/ so none of the tools will work anymore.

The MIQ date notifications twitter bot is broken too.

I managed to check the page source when a date became available and it seems like they are also now obfuscating 'data-arrival-dates' which means the current method of for checking for dates matches will no longer work either.
Screenshot 2021-07-23 at 05 08 44

@jvolker
Copy link
Owner

jvolker commented Jul 23, 2021

miq-nz-booking-assistance-Windows-v0.10.1.exe has stopped refreshing the page. I think it might be because it's waiting for an element with the class .flatpickr-input to exist but it never exists now?

await page.waitForSelector('.flatpickr-input');

Looks like the calendar has id="accommodation-calendar" now. Maybe it's possible to check for any dates that do not have the class q14-disabled?

image

Please only comment here if you have any new information. It really helps anyone working on improving/fixing this tool to not get a flood of messages. Thanks.

@jvolker jvolker changed the title MIQBookingAssistance.exe stopped working Stopped working after MIQ website update Jul 23, 2021
@stilljake
Copy link
Contributor

stilljake commented Jul 23, 2021

Looks like the calendar has id="accommodation-calendar" now. Maybe it's possible to check for any dates that do not have the class q14-disabled?

It looks like unique class tags are being generated for each session

Screenshot 2021-07-23 at 10 23 02

and see my previous comment where it shows `s9r`

@MannerlyApple94
Copy link

and see my previous comment where it shows s9r

Not sure if this helps, but it seems it cycles through a limited number of class tags, at least for me, the only variations I'm getting are:

  • s9r-day
  • yyh-day
  • bki-day
  • g10-day
  • it7-day
  • vv8-day
  • mi4-day

@nz029
Copy link

nz029 commented Jul 23, 2021

i have a script that is running fine, would you need that mate?

@cummins070
Copy link

i have a script that is running fine, would you need that mate?

Sharing caring :)

@alistairgraham
Copy link
Author

@MannerlyApple94 I've had a look at the CSS and there are 8 combinations. The eighth is q14-day.

@dvilsbosss
Copy link

i have a script that is running fine, would you need that mate?

@nz029 hopefully its still working. waiting for you to share.

@alexDrinkwater
Copy link
Collaborator

@nz029 Can you create a PR?

@alexDrinkwater
Copy link
Collaborator

alexDrinkwater commented Jul 23, 2021

Here is the method that parses the data-arrival-dates attribute.

                key: "getAvailableDates",
                value: function() {
                    if (this.calendarElement.getAttribute("data-arrival-dates").length <= 0)
                        return [];
                    for (var e = [], t = [58501527, 28741588, 61219430, 20294527, 57613046, 20913046, 23093500, 40765602], n = this.calendarElement.getAttribute("data-arrival-dates").split("_"), a = 0; a < n.length; a++) {
                        n[a] = n[a].split("").reverse().join(""),
                        n[a] = "" + (Number(n[a]) - t[a % t.length]);
                        var i = n[a].substring(0, 4) + "-" + n[a].substring(4, 6) + "-" + n[a].substring(6, 8);
                        e.includes(i) || e.push(i)
                    }
                    var o = r()(this.getMaxDate(), "YYYY-MM-DD");
                    return e.filter((function(e) {
                        return r()(e, "YYYY-MM-DD") < o
                    }
                    ))
                }

I'm assuming the t = [58501527, 28741588, 61219430, 20294527, 57613046, 20913046, 23093500, 40765602] array will be unique to each session or refresh.

@nz029
Copy link

nz029 commented Jul 23, 2021

@nz029 Can you create a PR?

i am a civil engineer alex.... know nothing about coding or the coding language. But necessity can teach you anything..... haha
what do you mean by PR? :/ :(

PS: I haven't created that that script, just using it.
I have messaged it to jvolker on twitter

@alexDrinkwater alexDrinkwater mentioned this issue Jul 23, 2021
@alexDrinkwater
Copy link
Collaborator

@nz029 Thanks nz029. Not sure which twitter handle you used because jvolker on twitter is a different person to on github. Either way I think I've got a fix that bypasses miq spoofing the css classes.

@nz029
Copy link

nz029 commented Jul 23, 2021

@nz029 Thanks nz029. Not sure which twitter handle you used because jvolker on twitter is a different person to on github. Either way I think I've got a fix that bypasses miq spoofing the css classes.

I sent it to the correct handle mentioned on his profile.
https://twitter.com/jeremiasvolker

@alexDrinkwater
Copy link
Collaborator

@nz029 Thanks nz029. Not sure which twitter handle you used because jvolker on twitter is a different person to on github. Either way I think I've got a fix that bypasses miq spoofing the css classes.

I sent it to the correct handle mentioned on his profile.
https://twitter.com/jeremiasvolker

@jvolker
Copy link
Owner

jvolker commented Jul 25, 2021

Thanks a lot, everyone, for fixing the script! Especially @alexDrinkwater for implementing the fix and creating a new release.

I did receive the script on Twitter but have been off for 1.5 days. Please post anything related to this script here instead. Then everyone can work with it right away.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.