Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Want to see Resource Information detail #41

Closed
halnique opened this issue Sep 8, 2022 · 3 comments · Fixed by #46
Closed

Want to see Resource Information detail #41

halnique opened this issue Sep 8, 2022 · 3 comments · Fixed by #46
Labels
enhancement New feature or request

Comments

@halnique
Copy link

halnique commented Sep 8, 2022

There does not seem to be much information available in the emulator.
In my use case, requires Job and Table information.
Can you make it so that I get the same information as BigQuery?

https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs
https://cloud.google.com/bigquery/docs/reference/rest/v2/tables

Job in BigQuery

$ bq query --use_legacy_sql=false --project_id=my-project --job_id=dummy-job-id 'select * from `dataset1`.`table_a`'
[
  {
    "id": "1", 
    "name": "alice"
  }, 
  {
    "id": "2", 
    "name": "bob"
  }
]
$ bq show --project_id=my-project --job=true dummy-job-id
{
  "configuration": {
    "jobType": "QUERY", 
    "query": {
      "destinationTable": {
        "datasetId": "<datasetId>", 
        "projectId": "my-project", 
        "tableId": "<tableId>"
      }, 
      "priority": "INTERACTIVE", 
      "query": "select * from `dataset1`.`table_a`", 
      "useLegacySql": false, 
      "writeDisposition": "WRITE_TRUNCATE"
    }
  }, 
  "etag": "<etag>", 
  "id": "my-project:<location>.dummy-job-id", 
  "jobReference": {
    "jobId": "dummy-job-id", 
    "location": "<location>", 
    "projectId": "my-project"
  }, 
  "kind": "bigquery#job", 
  "selfLink": "https://bigquery.googleapis.com/bigquery/v2/projects/my-project/jobs/dummy-job-id?location=<location>", 
  "statistics": {
    "creationTime": "1662625182766", 
    "endTime": "1662625182927", 
    "query": {
      "cacheHit": true, 
      "statementType": "SELECT", 
      "totalBytesBilled": "0", 
      "totalBytesProcessed": "0"
    }, 
    "startTime": "1662625182805", 
    "totalBytesProcessed": "0"
  }, 
  "status": {
    "state": "DONE"
  }, 
  "user_email": "<my email address>"
}

Job in emulator

$ bq --api=http://localhost:9050/ query --use_legacy_sql=false --project_id=test --job_id=dummy-job-id 'select * from `dataset1`.`table_a`'
[
  {
    "id": "1", 
    "name": "alice"
  }, 
  {
    "id": "2", 
    "name": "bob"
  }
]
$ bq --api=http://localhost:9050/ show --project_id=test --job=true dummy-job-id{
  "configuration": {
    "query": {
      "query": "select * from `dataset1`.`table_a`", 
      "useLegacySql": false
    }
  }, 
  "jobReference": {
    "jobId": "dummy-job-id", 
    "projectId": "test"
  }, 
  "status": {
    "state": "DONE"
  }
}

emulator logs

2022-09-08T09:08:50.000Z	INFO	server/middleware.go:41	GET /discovery/v1/apis/bigquery/v2/rest	{"query": ""}
2022-09-08T09:09:04.985Z	INFO	server/middleware.go:41	POST /projects/test/jobs	{"query": "alt=json"}
2022-09-08T09:09:04.993Z	INFO	contentdata/repository.go:144		{"query": "select * from `dataset1`.`table_a`", "values": []}
2022-09-08T09:09:04.994Z	DEBUG	contentdata/repository.go:210	query result	{"rows": [[1,"alice"],[2,"bob"]]}
2022-09-08T09:09:05.007Z	INFO	server/middleware.go:41	GET /projects/test/queries/dummy-job-id	{"query": "timeoutMs=0&startIndex=0&alt=json&maxResults=100"}

2022-09-08T09:09:10.764Z	INFO	server/middleware.go:41	GET /discovery/v1/apis/bigquery/v2/rest	{"query": ""}
2022-09-08T09:09:25.770Z	INFO	server/middleware.go:41	GET /projects/test/jobs/dummy-job-id	{"query": "alt=json"}

Table in BigQuery

$ bq show --project_id=my-project dataset1.table_a
{
  "creationTime": "1662527315669", 
  "etag": "<etag>", 
  "id": "my-project:dataset1.table_a", 
  "kind": "bigquery#table", 
  "lastModifiedTime": "1662625126801", 
  "location": "<location>", 
  "numActiveLogicalBytes": "28", 
  "numActivePhysicalBytes": "3853", 
  "numBytes": "28", 
  "numLongTermBytes": "0", 
  "numLongTermLogicalBytes": "0", 
  "numLongTermPhysicalBytes": "0", 
  "numRows": "2", 
  "numTimeTravelPhysicalBytes": "1071", 
  "numTotalLogicalBytes": "28", 
  "numTotalPhysicalBytes": "3853", 
  "schema": {
    "fields": [
      {
        "mode": "NULLABLE", 
        "name": "id", 
        "type": "INTEGER"
      }, 
      {
        "mode": "NULLABLE", 
        "name": "name", 
        "type": "STRING"
      }
    ]
  }, 
  "selfLink": "https://bigquery.googleapis.com/bigquery/v2/projects/my-project/datasets/dataset1/tables/table_a", 
  "tableReference": {
    "datasetId": "dataset1", 
    "projectId": "my-project", 
    "tableId": "table_a"
  }, 
  "type": "TABLE"
}

Table in emulator

$ bq --api=http://localhost:9050/ show --project_id=test dataset1.table_a
{}

emulator logs

2022-09-08T09:06:49.803Z	INFO	server/middleware.go:41	GET /discovery/v1/apis/bigquery/v2/rest	{"query": ""}
2022-09-08T09:07:04.792Z	INFO	server/middleware.go:41	GET /projects/test/datasets/dataset1/tables/table_a	{"query": "alt=json"}
@goccy
Copy link
Owner

goccy commented Sep 8, 2022

OK, Please let me know what field names you need as it is difficult to keep all the information the same as BigQuery.

@halnique
Copy link
Author

halnique commented Sep 9, 2022

For the time being, I would like to use the following fields.

Job

  • statistics.totalBytesProcessed

Table

  • numRows
  • schema.fields

@goccy goccy added the enhancement New feature or request label Sep 12, 2022
@goccy goccy closed this as completed in #46 Sep 15, 2022
@goccy
Copy link
Owner

goccy commented Sep 16, 2022

Thank you for providing the information !
I fixed this problem with v0.1.12

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants