Get 404 when use function getMediasByTag #70 #71

Closed
senol01 opened this Issue Mar 15, 2017 · 33 comments

Comments

Projects
None yet
9 participants
@senol01

senol01 commented Mar 15, 2017

InstagramScraper\Exception\InstagramException Object ( [message:protected] => Response code is 404. Body:

Unbelievable, but the problem is solved if you install cookies (I loaded https://www.instagram.com/explore/tags/video/?__a=1&max_id=1 and copied them from the query, then inserted in /Unirest/Request.php on line 236)

$headers = [ 'cookie' => 'simple cookies' ];

public static function get($url, $headers = array(), $parameters = null, $username = null, $password = null) { $headers = [ 'cookie' => 'simple cookies' ]; return self::send(Method::GET, $url, $parameters, $headers, $username, $password); }

`require '/vendor/autoload.php';

use InstagramScraper\Instagram;

$medias = Instagram::getMediasByTag('zara', 30);
echo json_encode($medias);`

@raiym

This comment has been minimized.

Show comment
Hide comment
@raiym

raiym Mar 15, 2017

Member

Hello @senol01 thank you.

It seems that Instagram started require auth for those two links:

    const MEDIA_JSON_BY_LOCATION_ID = 'https://www.instagram.com/explore/locations/{{facebookLocationId}}/?__a=1&max_id={{maxId}}'; // doesn't work
    const MEDIA_JSON_BY_TAG = 'https://www.instagram.com/explore/tags/{tag}/?__a=1&max_id={max_id}'; // doesn't work
Member

raiym commented Mar 15, 2017

Hello @senol01 thank you.

It seems that Instagram started require auth for those two links:

    const MEDIA_JSON_BY_LOCATION_ID = 'https://www.instagram.com/explore/locations/{{facebookLocationId}}/?__a=1&max_id={{maxId}}'; // doesn't work
    const MEDIA_JSON_BY_TAG = 'https://www.instagram.com/explore/tags/{tag}/?__a=1&max_id={max_id}'; // doesn't work
@filippodicostanzo

This comment has been minimized.

Show comment
Hide comment
@filippodicostanzo

filippodicostanzo Mar 15, 2017

From today instagram disabled access at your website without login.

There is any solution for bypass this problem?

From today instagram disabled access at your website without login.

There is any solution for bypass this problem?

@filippodicostanzo

This comment has been minimized.

Show comment
Hide comment
@filippodicostanzo

filippodicostanzo Mar 15, 2017

@senol01 how to install the cookie?

I insert the $headers = [ 'cookie' => 'simple cookies' ]; in the specific line but i recive the same error!

InstagramScraper\Exception\InstagramException Object ( [message:protected] => Response code is 404. Body:

Any solution for this issue?

@senol01 how to install the cookie?

I insert the $headers = [ 'cookie' => 'simple cookies' ]; in the specific line but i recive the same error!

InstagramScraper\Exception\InstagramException Object ( [message:protected] => Response code is 404. Body:

Any solution for this issue?

@pinotarantino

This comment has been minimized.

Show comment
Hide comment
@filippodicostanzo

This comment has been minimized.

Show comment
Hide comment
@filippodicostanzo

filippodicostanzo Mar 15, 2017

@pinotarantino how can i try?

where I insert the url?

Do you can specific me all the steps?

If you're aren't logged in instagram the url isn't working. And when i call the function in my php file return InstagramScraper\Exception\InstagramException Object ( [message:protected] => Response code is 404. Body:

filippodicostanzo commented Mar 15, 2017

@pinotarantino how can i try?

where I insert the url?

Do you can specific me all the steps?

If you're aren't logged in instagram the url isn't working. And when i call the function in my php file return InstagramScraper\Exception\InstagramException Object ( [message:protected] => Response code is 404. Body:

@pinotarantino

This comment has been minimized.

Show comment
Hide comment
@pinotarantino

pinotarantino Mar 15, 2017

if you click on the link I posted above it works, also https://www.instagram.com/explore/tags/fitness/?__a=1 works; you get the json data... if this works, maybe they're stopping instagram-php-scraper? I don't know...

if you click on the link I posted above it works, also https://www.instagram.com/explore/tags/fitness/?__a=1 works; you get the json data... if this works, maybe they're stopping instagram-php-scraper? I don't know...

@filippodicostanzo

This comment has been minimized.

Show comment
Hide comment
@filippodicostanzo

filippodicostanzo Mar 15, 2017

If you are not logged into instagram the link doesn't work and return
"Spiacenti, questa pagina non è disponibile
È possibile che il link che hai seguito sia corrotto o che la pagina sia stata rimossa. Torna su Instagram."

If you are not logged into instagram the link doesn't work and return
"Spiacenti, questa pagina non è disponibile
È possibile che il link che hai seguito sia corrotto o che la pagina sia stata rimossa. Torna su Instagram."

@maxmaxme

This comment has been minimized.

Show comment
Hide comment
@maxmaxme

maxmaxme Mar 15, 2017

@fabianoroberto

  1. Open https://www.instagram.com/explore/tags/fitness/?__a=1&max_id=1
  2. Copy all cookies
  3. Insert in /Unirest/Request.php on line 236 this code:
    $headers = [ 'cookie' => 'simple cookies' ];

Sample:
2017-03-15 19 44 13

@fabianoroberto

  1. Open https://www.instagram.com/explore/tags/fitness/?__a=1&max_id=1
  2. Copy all cookies
  3. Insert in /Unirest/Request.php on line 236 this code:
    $headers = [ 'cookie' => 'simple cookies' ];

Sample:
2017-03-15 19 44 13

@filippodicostanzo

This comment has been minimized.

Show comment
Hide comment
@filippodicostanzo

filippodicostanzo Mar 15, 2017

@maxmaxme how can i get this string for the cookie?

@maxmaxme how can i get this string for the cookie?

@maxmaxme

This comment has been minimized.

Show comment
Hide comment
@maxmaxme

maxmaxme Mar 15, 2017

@filippodicostanzo

  1. F12 (CMD+Option+I)
  2. Console
  3. Write: console.log(document.cookie)

maxmaxme commented Mar 15, 2017

@filippodicostanzo

  1. F12 (CMD+Option+I)
  2. Console
  3. Write: console.log(document.cookie)
@filippodicostanzo

This comment has been minimized.

Show comment
Hide comment
@filippodicostanzo

filippodicostanzo Mar 15, 2017

@maxmaxme thank you but for me doesn't work

return anytime 404 :(

@maxmaxme thank you but for me doesn't work

return anytime 404 :(

@senol01

This comment has been minimized.

Show comment
Hide comment
@senol01

senol01 Mar 15, 2017

maxmaxme did you try if it works?
console.log(document.cookie)
mid=WMl5gAAEAAHXmW0Bp5VYmEgeERhA; ig_pr=1; ig_vw=1366; ds_user_id=4211931173; s_network=""; csrftoken=BREKQUyRvTIOfomdbj2WvpYlAhZippmw
have only become this

senol01 commented Mar 15, 2017

maxmaxme did you try if it works?
console.log(document.cookie)
mid=WMl5gAAEAAHXmW0Bp5VYmEgeERhA; ig_pr=1; ig_vw=1366; ds_user_id=4211931173; s_network=""; csrftoken=BREKQUyRvTIOfomdbj2WvpYlAhZippmw
have only become this

@maxmaxme

This comment has been minimized.

Show comment
Hide comment
@maxmaxme

maxmaxme Mar 15, 2017

@filippodicostanzo @senol01
Sorry. I took cookies from another place.

  1. Open https://www.instagram.com/explore/tags/fitness/?__a=1&max_id=1
  2. F12 (CMD+Option+I)
  3. Network
  4. F5
  5. Click on "?__a=1&max_id=1"
  6. Copy Cookies from "Request Headers"

2017-03-15 20 39 48

@filippodicostanzo @senol01
Sorry. I took cookies from another place.

  1. Open https://www.instagram.com/explore/tags/fitness/?__a=1&max_id=1
  2. F12 (CMD+Option+I)
  3. Network
  4. F5
  5. Click on "?__a=1&max_id=1"
  6. Copy Cookies from "Request Headers"

2017-03-15 20 39 48

@senol01

This comment has been minimized.

Show comment
Hide comment
@senol01

senol01 Mar 15, 2017

Bro thank you very much
https://www.instagram.com/explore/tags/usa/?__a=1
But i have one questition more, how i can find the popular Medias?

senol01 commented Mar 15, 2017

Bro thank you very much
https://www.instagram.com/explore/tags/usa/?__a=1
But i have one questition more, how i can find the popular Medias?

@maxmaxme

This comment has been minimized.

Show comment
Hide comment
@maxmaxme

maxmaxme Mar 15, 2017

@senol01 Ask this question to the developer @postaddictme

maxmaxme commented Mar 15, 2017

@senol01 Ask this question to the developer @postaddictme

@senol01

This comment has been minimized.

Show comment
Hide comment
@senol01

senol01 Mar 15, 2017

for maxmaxme

senol01 commented Mar 15, 2017

for maxmaxme

@pinotarantino

This comment has been minimized.

Show comment
Hide comment
@pinotarantino

pinotarantino Mar 15, 2017

Thanks to senol01 and maxmaxme, your solution worked for me...! ;) Do you think that this escamotage could resist for long time to Instagram countermeasure? Anyway, Thank you very much!

Thanks to senol01 and maxmaxme, your solution worked for me...! ;) Do you think that this escamotage could resist for long time to Instagram countermeasure? Anyway, Thank you very much!

@filippodicostanzo

This comment has been minimized.

Show comment
Hide comment
@filippodicostanzo

filippodicostanzo Mar 15, 2017

@maxmaxme & @senol01 thank you're saved my week :)

Anynone knows the duration of these cookie?

@maxmaxme & @senol01 thank you're saved my week :)

Anynone knows the duration of these cookie?

@ximrx

This comment has been minimized.

Show comment
Hide comment
@ximrx

ximrx Mar 16, 2017

This is not a solution, is this library going to work by adding authentication and API key or is it going to be shut down?

ximrx commented Mar 16, 2017

This is not a solution, is this library going to work by adding authentication and API key or is it going to be shut down?

@raiym

This comment has been minimized.

Show comment
Hide comment
@raiym

raiym Mar 16, 2017

Member

Thanks for support guys.
Really appreciate it.
For now only to urls are require auth:

const MEDIA_JSON_BY_LOCATION_ID = 'https://www.instagram.com/explore/locations/{{facebookLocationId}}/?__a=1&max_id={{maxId}}'; // doesn't work
const MEDIA_JSON_BY_TAG = 'https://www.instagram.com/explore/tags/{tag}/?__a=1&max_id={max_id}'; // doesn't work

related to tags, locations and get account by id.

All other functions work as before. (at least for now)
Of course I do understand needs about tags and locations.

We already have private instagram api: https://github.com/mgp25/Instagram-API developed by @mgp25

So I don't know for now is postaddict library needs auth or not?

Actually, I will be glad to hear your suggestions and feeback

Member

raiym commented Mar 16, 2017

Thanks for support guys.
Really appreciate it.
For now only to urls are require auth:

const MEDIA_JSON_BY_LOCATION_ID = 'https://www.instagram.com/explore/locations/{{facebookLocationId}}/?__a=1&max_id={{maxId}}'; // doesn't work
const MEDIA_JSON_BY_TAG = 'https://www.instagram.com/explore/tags/{tag}/?__a=1&max_id={max_id}'; // doesn't work

related to tags, locations and get account by id.

All other functions work as before. (at least for now)
Of course I do understand needs about tags and locations.

We already have private instagram api: https://github.com/mgp25/Instagram-API developed by @mgp25

So I don't know for now is postaddict library needs auth or not?

Actually, I will be glad to hear your suggestions and feeback

@raiym

This comment has been minimized.

Show comment
Hide comment
@raiym

raiym Mar 16, 2017

Member

Well.
I have decided that library will support web login and scraping tags and locations will be fixed

And I think about adding official API support as well

Member

raiym commented Mar 16, 2017

Well.
I have decided that library will support web login and scraping tags and locations will be fixed

And I think about adding official API support as well

@filippodicostanzo

This comment has been minimized.

Show comment
Hide comment
@filippodicostanzo

filippodicostanzo Mar 16, 2017

@raiym thank you! after this intensive work you can explain how to install this for scraping tags please :)

@raiym thank you! after this intensive work you can explain how to install this for scraping tags please :)

@raiym raiym self-assigned this Mar 16, 2017

@raiym raiym added the bug label Mar 16, 2017

@aik27

This comment has been minimized.

Show comment
Hide comment
@aik27

aik27 Mar 16, 2017

Contributor

Seems, this query (for tagged data) work without any authorization (only csrftoken cookie required).

ig_hashtag(' . $tag . ') { media.first(' . $limit . ') { count, nodes { id, caption, code, comments.last(10) { count, nodes { id, created_at, text, user { id, profile_pic_url, username } } }, date, dimensions { height, width }, filter_name, display_src, id, is_video, likes { count }, owner { id, username, full_name, profile_pic_url }, thumbnail_src, video_url, location { name, id } }, page_info} }

Contributor

aik27 commented Mar 16, 2017

Seems, this query (for tagged data) work without any authorization (only csrftoken cookie required).

ig_hashtag(' . $tag . ') { media.first(' . $limit . ') { count, nodes { id, caption, code, comments.last(10) { count, nodes { id, created_at, text, user { id, profile_pic_url, username } } }, date, dimensions { height, width }, filter_name, display_src, id, is_video, likes { count }, owner { id, username, full_name, profile_pic_url }, thumbnail_src, video_url, location { name, id } }, page_info} }

@raiym

This comment has been minimized.

Show comment
Hide comment
@raiym

raiym Mar 16, 2017

Member

almost done. please wait another hour or two

Member

raiym commented Mar 16, 2017

almost done. please wait another hour or two

@matkozikowski

This comment has been minimized.

Show comment
Hide comment
@matkozikowski

matkozikowski Mar 16, 2017

@raiym
So if you fix the problem we will able to get all images by tag ?
thx.

@raiym
So if you fix the problem we will able to get all images by tag ?
thx.

@raiym

This comment has been minimized.

Show comment
Hide comment
@raiym

raiym Mar 16, 2017

Member
Member

raiym commented Mar 16, 2017

@raiym

This comment has been minimized.

Show comment
Hide comment
@raiym

raiym Mar 16, 2017

Member
Member

raiym commented Mar 16, 2017

@matkozikowski

This comment has been minimized.

Show comment
Hide comment
@matkozikowski

matkozikowski Mar 16, 2017

Thats fine. After fix it please let us know what we should change :)

Thats fine. After fix it please let us know what we should change :)

@aik27

This comment has been minimized.

Show comment
Hide comment
@aik27

aik27 Mar 16, 2017

Contributor

@raiym, really, don't rush with login and password. Some commercial plugins works without authorization. Just emulate browser request, get csrftoken cookie and grab tags and location data by http://instagram.com/query/ endpoint.

Contributor

aik27 commented Mar 16, 2017

@raiym, really, don't rush with login and password. Some commercial plugins works without authorization. Just emulate browser request, get csrftoken cookie and grab tags and location data by http://instagram.com/query/ endpoint.

@raiym

This comment has been minimized.

Show comment
Hide comment
@raiym

raiym Mar 16, 2017

Member

Thank you guys.
I already have done it.

https://github.com/postaddictme/instagram-php-scraper/releases/tag/v0.5.0

I am planning to add session caching (because now every time php dies login required). Maybe for now you will cache sessionId on your own

Member

raiym commented Mar 16, 2017

Thank you guys.
I already have done it.

https://github.com/postaddictme/instagram-php-scraper/releases/tag/v0.5.0

I am planning to add session caching (because now every time php dies login required). Maybe for now you will cache sessionId on your own

@pinotarantino

This comment has been minimized.

Show comment
Hide comment
@pinotarantino

pinotarantino Mar 16, 2017

Thank you veruy much raym, very good job, just installed now! It works fine but with a little change in my case:

  • $medias = self::$instagram->getTopMediasByTagName("$kw"); // doesn't work, problem with self. "Fatal error: Cannot access self:: when no class scope is active"
  • $medias = $instagram->getTopMediasByTagName("$kw"); // works ok without 'self::'

Thank you again! ;)

Thank you veruy much raym, very good job, just installed now! It works fine but with a little change in my case:

  • $medias = self::$instagram->getTopMediasByTagName("$kw"); // doesn't work, problem with self. "Fatal error: Cannot access self:: when no class scope is active"
  • $medias = $instagram->getTopMediasByTagName("$kw"); // works ok without 'self::'

Thank you again! ;)

@jetonr

This comment has been minimized.

Show comment
Hide comment
@jetonr

jetonr Mar 16, 2017

I created a dummy instagram account and got only the sessionId from the coockie and it works when I send the request.
My question is will this work when making multiple requests from different domains and do they ever expire?

jetonr commented Mar 16, 2017

I created a dummy instagram account and got only the sessionId from the coockie and it works when I send the request.
My question is will this work when making multiple requests from different domains and do they ever expire?

@raiym

This comment has been minimized.

Show comment
Hide comment
@raiym

raiym Mar 20, 2017

Member

@jetonr i think it will work, but because Instagram links session for ip and browser (I think)you need to perform login on each server
But I suggest to use one account only from one ip.

Member

raiym commented Mar 20, 2017

@jetonr i think it will work, but because Instagram links session for ip and browser (I think)you need to perform login on each server
But I suggest to use one account only from one ip.

@raiym raiym closed this Mar 20, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment