Workflow job fails with no error message

[gcameron89777]

I have a workflow (not self hosted runner) that when mimicked on local works fine. The job in question pulls data from google big query and then saves it as a file data.json, and then in a subsequent step as an artifact.

When testing on local, the file size is 1.5gb and it takes about 30 - 40 minutes to run.

The job fails in github actions and there's no error message. The output in the console exactly matches what I see on local, which is also on an ubuntu docker container.

My main.yml for the culprit job:

env:
  rundate: $(date --date="2 days ago" '+%Y%m%d')

jobs:
  get-bq-data:
    runs-on: ubuntu-latest
    timeout-minutes: 600
    name: Get GA4 data from BQ
    steps:
      - name: Set env
        run: echo "RUNDATE=${{ env.rundate }}" >> $GITHUB_ENV   
      - name: Checkout current repo
        uses: actions/checkout@v2
      - name: Run extract and save data
        uses: ./bq-cli # Uses an action in this directory
        with:
          SERVICE_JSON: ${{ secrets.SERVICE_JSON }}
      - name: check json is present
        run: ls -lR
      - name: Upload data as artifact
        uses: actions/upload-artifact@v2
        with:
          name: bq-data
          path: data.json.gz

My entrypoint.sh which gets run during step Run extract and save data:

#!/bin/bash

# get service json from secret
echo "${INPUT_SERVICE_JSON}" > /service.json

# auth
gcloud auth activate-service-account --key-file=/service.json --project=ga4-extract

# config
gcloud config set project ga4-extract

# run dummy query. Hacky workaround for initialization output
# after first run this output no longer gets added
# e.g. https://github.com/GoogleCloudPlatform/terraform-google-secured-data-warehouse/issues/35
bq query 'select 1'

# actual query
bq query --max_rows=1000000 --use_legacy_sql=false --format=json --batch=true 'select * from `ga4-extract.analytics_123456789.events_'"$RUNDATE"'`;' >gzip >data.json.gz

It's the last command that github actions does not seem to get past.

On local here's what my terminal looks like when running this:

root@9ca5b0ebad67:/# bq query --max_rows=1000000 --use_legacy_sql=false --format=json --batch=true 'select * from `ga4-extract.analytics_302644320.events_'"$RUNDATE"'`;' >gzip >data.json.gz
Waiting on bqjob_r6f9fabb03b94b240_0000018253d04f87_1 ... (0s) Current status: DONE   

After 30 - 40 minutes the query completes and if I ls -l I can see the data.json file.

On the actions console, I see the same output as above (last line in screen shot below) and the command seems to be running. But after 30 - 40 minutes, the job fails and the console gives me no information. Here's a screen shot:

Why did the job fail? There's no error message? Or, how can I investigate to find out more info? I'm unsure how to debug this with the given information.

[Terence-art]

Unpack the delete all outlinks and see what you get

[gcameron89777]

Hi @Terence-art thanks for responding to my post. I do not understand your instruction, could you clarify what you mean?

[Terence-art]

I'm in win env still suppose enclosed content needs to be disclosed first then, inside the container there should be any sort of code that depends to other code in or outside the box so, you need the crude version removing all dependencies to other files and after you can work on it, no?

[gcameron89777]

I ran the steps using debug mode and found some new information. The culprit step returns this output:

Waiting on bqjob_r78a5eea2534158c7_000001825dc46a8f_1 ... (0s) Current status: DONE   
##[debug]Docker Action run completed with exit code 247
##[debug]Finishing: Run extract and save data

After some searching of that exit code, it sounds like my container is running out of memory. Since this doesn't happen on local, presumably it's a github actions limit? Is there someway that I can increase the containers memory? My action.yml currently looks like this:

# action.yml

name: 'get data'
description: 'Get data using Big Query'
inputs:
  SERVICE_JSON:
    description: 'The service account json'
    required: true
outputs:
  data-json:
    description: 'json output of the returned table'
runs:
  using: 'docker'
  image: 'Dockerfile'
  entrypoint: '/entrypoint.sh'
  args:
    - ${{ inputs.SERVICE_JSON }}