Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Send batch indexing requests using PHP(forming homogeneous batch requests) #1567

Closed
bochuanWorkland opened this issue Dec 14, 2018 · 7 comments
Assignees
Labels
type: question Request for information or clarification. Not an issue.

Comments

@bochuanWorkland
Copy link

bochuanWorkland commented Dec 14, 2018

Based on the Indexing API documentation: Using the Indexing API sending a batch request to the Indexing API, should use the following endpoint:
https://indexing.googleapis.com/batch
However, accessing this url will show this error:
404. That’s an error.
The requested URL /batch was not found on this server.

I followed this documentation(https://developers.google.com/api-client-library/php/guide/batch)
and here is my code:

use Google_Client; 
use Google_Service_Drive;
use Google_Http_Batch;
use GuzzleHttp\Psr7\Request as CreateRequest;

$client = new \Google_Client();

// privateKey.json is the private key that I created for my service account.
$client->setAuthConfig('/var/www/html/privateKey.json');
$client->addScope('https://www.googleapis.com/auth/indexing');

// setting the useBatch parameter on the Google_Client to true
$client->setUseBatch(true);

// using the Google_Http_Batch class
$batch = new \Google_Http_Batch($client);

// Get a Guzzle HTTP Client and do the authorization
$httpClient = $client->authorize();

// after authorization done, send the request to Google api
$endpoint = "https://indexing.googleapis.com/batch";

// Define content here
$content = json_encode(array(
    "url" => "https://example.com/jobs/".$jobs[$i]["job_id"],
    "type" => "URL_UPDATED"
),JSON_FORCE_OBJECT);

$newRequest = new CreateRequest('POST', $endpoint, ['body' => $content]);
$batch->add($newRequest, $jobs[$i]["job_id"]);
$batchResults = $batch->execute();

and here is the running result:
//======================= result =====================//

"response-2018112103" => Google_Service_Exception {#189
    #errors: null
    #message: "Not Found"
    #code: 404
    #file: "/var/www/html/vendor/google/apiclient/src/Google/Http/REST.php"
    #line: 118
    trace: {
      /var/www/html/vendor/google/apiclient/src/Google/Http/REST.php:118 {}
      /var/www/html/vendor/google/apiclient/src/Google/Http/Batch.php:175 {}
      /var/www/html/vendor/google/apiclient/src/Google/Http/Batch.php:132 {}
      /var/www/html/app/Http/Controllers/DiffuseController.php:244 {
        › 
        › $batchResults = $batch->execute();
        › dd($batchResults);
      }

//====================== The End of result ==============//

Then I dive into this documentation (https://developers.googleblog.com/2018/03/discontinuing-support-for-json-rpc-and.html).
Instead of using Google_Http_Batch Class which comes from Google API Client Libraries, I use GuzzleHttp\Pool Class, and GuzzleHttp\Psr7\Request Class, and I changed the endpoint from www.googleapis.com/batch to www.googleapis.com/batch/indexing/v3.
I am currently forming homogeneous batch requests and here is my code:

use GuzzleHttp\Pool;
use GuzzleHttp\Psr7\Request as CreateRequest;

require_once '/var/www/html/vendor/autoload.php';
$client = new \Google_Client();

// privateKey.json is the private key that you created for your service account.
$client->setAuthConfig('/var/www/html/privateKey.json');
$client->addScope('https://www.googleapis.com/auth/indexing');

// setting the useBatch parameter on the Google_Client to true
$client->setUseBatch(true);

// Get a Guzzle HTTP Client and do the authorization
$httpClient = $client->authorize(); // this will return a Guzzle http object

// after authorization done, send the request to Google api
$endpoint = "https://googleapis.com/batch/indexing/v3";

// Define content here
$content = json_encode(array(
    "url" => "https://example.com/jobs/".$jobs[$i]["job_id"],
    "type" => "URL_UPDATED"
),JSON_FORCE_OBJECT);

//$newRequest = $httpClient->post($endpoint, ['body' => $content]);
$newRequest = new CreateRequest('POST', $endpoint, ['body' => $content]);
array_push($batchRequests, $newRequest);

$responses = Pool::batch($httpClient, $batchRequests, array('concurrency' => 15,));
foreach ($responses as $response) {
    Log::info("the response infomation: ".$response->getStatusCode());
}

and here is the running result:
// =================== result ======================//

array:3 [
  0 => Response {#201
    -reasonPhrase: "Not Found"
    -statusCode: 404
    -headers: array:5 [
      "Content-Type" => array:1 [
        0 => "text/html; charset=UTF-8"
      ]
      "Referrer-Policy" => array:1 [
        0 => "no-referrer"
      ]
      "Content-Length" => array:1 [
        0 => "1578"
      ]
      "Date" => array:1 [
        0 => "Fri, 14 Dec 2018 17:09:52 GMT"
      ]
      "Alt-Svc" => array:1 [
        0 => "quic=":443"; ma=2592000; v="44,43,39,35""
      ]
    ]
    -headerNames: array:5 [
      "content-type" => "Content-Type"
      "referrer-policy" => "Referrer-Policy"
      "content-length" => "Content-Length"
      "date" => "Date"
      "alt-svc" => "Alt-Svc"
    ]
    -protocol: "1.1"
    -stream: Stream {#194
      -stream: stream resource @16
        wrapper_type: "PHP"
        stream_type: "TEMP"
        mode: "w+b"
        unread_bytes: 0
        seekable: true
        uri: "php://temp"
        options: []
      }
      -size: null
      -seekable: true
      -readable: true
      -writable: true
      -uri: "php://temp"
      -customMetadata: []
    }
  }
  1 => Response {#170
    -reasonPhrase: "Not Found"
    -statusCode: 404
    -headers: array:5 [
      "Content-Type" => array:1 [
        0 => "text/html; charset=UTF-8"
      ]
      "Referrer-Policy" => array:1 [
        0 => "no-referrer"
      ]
      "Content-Length" => array:1 [
        0 => "1578"
      ]
      "Date" => array:1 [
        0 => "Fri, 14 Dec 2018 17:09:52 GMT"
      ]
      "Alt-Svc" => array:1 [
        0 => "quic=":443"; ma=2592000; v="44,43,39,35""
      ]
    ]
    -headerNames: array:5 [
      "content-type" => "Content-Type"
      "referrer-policy" => "Referrer-Policy"
      "content-length" => "Content-Length"
      "date" => "Date"
      "alt-svc" => "Alt-Svc"
    ]
    -protocol: "1.1"
    -stream: Stream {#202
      -stream: stream resource @22
        wrapper_type: "PHP"
        stream_type: "TEMP"
        mode: "w+b"
        unread_bytes: 0
        seekable: true
        uri: "php://temp"
        options: []
      }
      -size: null
      -seekable: true
      -readable: true
      -writable: true
      -uri: "php://temp"
      -customMetadata: []
    }
  }
  2 => Response {#168
    -reasonPhrase: "Not Found"
    -statusCode: 404
    -headers: array:5 [
      "Content-Type" => array:1 [
        0 => "text/html; charset=UTF-8"
      ]
      "Referrer-Policy" => array:1 [
        0 => "no-referrer"
      ]
      "Content-Length" => array:1 [
        0 => "1578"
      ]
      "Date" => array:1 [
        0 => "Fri, 14 Dec 2018 17:09:52 GMT"
      ]
      "Alt-Svc" => array:1 [
        0 => "quic=":443"; ma=2592000; v="44,43,39,35""
      ]
    ]
    -headerNames: array:5 [
      "content-type" => "Content-Type"
      "referrer-policy" => "Referrer-Policy"
      "content-length" => "Content-Length"
      "date" => "Date"
      "alt-svc" => "Alt-Svc"
    ]
    -protocol: "1.1"
    -stream: Stream {#215
      -stream: stream resource @27
        wrapper_type: "PHP"
        stream_type: "TEMP"
        mode: "w+b"
        unread_bytes: 0
        seekable: true
        uri: "php://temp"
        options: []
      }
      -size: null
      -seekable: true
      -readable: true
      -writable: true
      -uri: "php://temp"
      -customMetadata: []
    }
  }
]

// =================== The End of result ================= //

I am not sure which endpoint I can use in order to send batch indexing requests using PHP(homogeneous batch requests).

Thanks for any help!

@mattwhisenhunt mattwhisenhunt added the type: question Request for information or clarification. Not an issue. label Dec 14, 2018
@danielgsims
Copy link

Hey @bochuanWorkland! Sorry we haven't got an answer to you yet. I added some code formatting to your comment to help make it easier to read. I'll start looking into your issue and see if I can find out what's going on.

@danielgsims danielgsims self-assigned this Jan 20, 2019
@Darkbelg
Copy link

i'm also having this issue. The default endpoint is no longer correct. And it's not really documented how and where to find the new one.

In order to do batch results for the youtube api i have to do this:
$batch = new Google_Http_Batch($client,false,null,"batch/youtube/v3/");

The batch path plus the service path seem to be the same not sure if this is just coincidence;

@danielgsims danielgsims removed their assignment Feb 15, 2019
@millyhung
Copy link

The default api base path of Google_Client is https://www.googleapis.com, so it uses https://www.googleapis.com/batch as the endpoint.
You could change it by setConfig() in Google_Client before create $batch:

$client->setConfig('base_path', 'https://indexing.googleapis.com');

Or create a batch instance by Google_Service_Indexing.

$client = new \Google_Client();

$client->setAuthConfig('/var/www/html/privateKey.json');
$client->addScope('https://www.googleapis.com/auth/indexing');
$client->setUseBatch(true);

$service = new \Google_Service_Indexing($client);
$batch = $service->createBatch();

// add request
$postBody = new \Google_Service_Indexing_UrlNotification();
$postBody->setType('URL_UPDATED');
$postBody->setUrl('https://example.com/jobs/' . $jobs[$i]['job_id']);
$batch ->add($service->urlNotifications->publish($postBody));
// ---- add request

$results = $batch->execute(); // it does authorize in execute()

@tmatsuo
Copy link

tmatsuo commented Apr 16, 2019

Seems like @millyhung posted a solution to the original problem. Thanks @millyhung !
Let's close this, but @bochuanWorkland feel free to reopen if the solution doesn't work for you.

@tmatsuo tmatsuo closed this as completed Apr 16, 2019
@tuliofindi
Copy link

I also have the same problem. I followed all the documentation provided by google. But I do not get the desired result. I would like to get clearer information about this code @millyhung

@millyhung
Copy link

I can't find the sample codes or documentations I saw. They may have been redirected to Github.

The original problem Not Found was caused by the wrong endpoint.
I trace the source codes. Google_Http_Batch::execute() L96 $request->getRequestTarget() returns path instead of full url.
And at L121, request will be sent to $this->rootUrl . '/' . $this->batchPath.
What is the $this->rootUrl? See L61, it determines by the base_path of Google_Client, and the default value is https://www.googleapis.com. (See Google_Client L46 and L100)
But Indexing uses the different endpoint https://indexing.googleapis.com/batch (documentation).
So we need to change the base_path of client before creating the instance of Google_Http_Batch or pass the $rootUrl to the constructor of Google_Http_Batch.

@googleapis googleapis deleted a comment from oleg972 Jun 24, 2020
@googleapis googleapis deleted a comment from oleg972 Jun 24, 2020
@linxinemily
Copy link

linxinemily commented Nov 19, 2020

I encountered the same problem, when I followed the solution provided by @millyhung , the problem was solved!
thanks @millyhung !!
(however, the documentation is really lack of these detailed information...)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: question Request for information or clarification. Not an issue.
Projects
None yet
Development

No branches or pull requests

8 participants