Skip to content
Google BigQuery for Laravel
Branch: master
Clone or download
Pull request Compare This branch is 12 commits ahead of prologuetech:master.
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.

Google BigQuery for Laravel

This package aims to wrap laravel functionality around Google's BigQuery.


Update: Add functions to convert Big Query Model to Eloquent and run bind sql query. Update Google cloud dependancy.


Via Composer

$ composer require ynizon/big


Publish our config file into your application:

php artisan vendor:publish --provider="Prologuetech\Big\BigServiceProvider"

You should have a config/prologue-big.php file to configure defaults.

Laravel 5.4.x

Older versions of Laravel require you to add our big service provider to your application providers array in config/app.php:


You now have access to a familiar laravel experience, enjoy!

Google Authentication

The Google SDK supports Application Default Credentials (ADC) and thus this package does as well. You may leave your auth_file field inside of your config file null to use ADC. Credentials fetcher is not currently supported but may be added in the future.

For more information see the adc docs.

How to use


By default we use the following global config options with BigQuery.

$this->options = [
    'useLegacySql' => false,
    'useQueryCache' => false,


When creating tables in BQ we automatically flip a Eloquent model schema for you. Let's cover an example of archiving data from our events table into BQ using laravel's chunk method.

$datasetId = 'test';
$tableId = 'events';

// Create our BQ helper
$big = new Big();

// Create table, we will pass in a mocked model to mutate into BQ schema
// Note: create table will only make a new table if it does not exist

/** @var Google\Cloud\BigQuery\Table $table */
$table = $big->createFromModel($datasetId, $tableId, new Event());

// Let's stream our events into BQ in large chunks
// Note: notArchived() is a simple scope, use whatever scopes you have on your model
Event::notArchived()->chunk(1000, function ($events) use ($big, $table) {
    // Prepare our rows
    $rows = $big->prepareData($events);

    // Stream into BQ, you may also pass in any options with a 3rd param.
    // Note: By default we use: 'ignoreUnknownValues' => true
    $big->insert($table, $rows);

    // Get our current id's
    /** @var Illuminate\Support\Collection $events */
    $ids = $events->pluck('id')->toArray();

    // Update these event's as processed
    Event::whereIn('id', $ids)->update([
        'system_processed' => 1

That's it! You now have a replica of your events table in BigQuery, enjoy!


Instantiating Big will automatically setup a Google ServiceBuilder and give us direct access to BigQuery through our internals via $big->query. However there are many helpers built into Big that make interacting with BigQuery a piece of cake (or a tasty carrot if you're into that kind of thing).

For example when running a query on BigQuery we must use the reload method in a loop to poll results. Big comes with a useful method run so all you need to do is this:

$query = 'SELECT count(id) FROM';

$big = new Big();
$results = $big->run($query);

Example to get Eloquent model with pagination from Big Query query.

$products = Product::whereIn("account_id",$accounts_id);
$arrLikeFields[] = "title";
$products = $products->where("title","LIKE","%".$q."%");
$sql = $big->bindCountQuery($query, array("products"), $arrLikeFields);

//Get count
$totalProducts = $big->run($sql);
$totalProducts = $totalProducts[0]["nb"];

//Get products
$sql = $big->bindQuery($query->orderBy("updated_at"), array("products"),15, $currentPage, $arrLikeFields);
$products = Big::unflipModel(new Product(), $big->run($sql));

$url = "/";
if (isset($_SERVER["REDIRECT_URL"])){
$products = new LengthAwarePaginator($products, $totalProducts, 15, $currentPage, ['path'=>url($url)]);

When using run we automatically poll BigQuery and return all results as a laravel collection object for you so you can enjoy your results as a refreshing cup of Laravel.

Change log

Please see CHANGELOG for more information on what has changed recently.


The MIT License (MIT). Please see License File for more information.

You can’t perform that action at this time.