When Code PushUp is first integrated in the same PR as the GitHub Action, then the comparison fails because of missing Code PushUp configuration in base branch:

Workaround is to integrate GitHub Action after Code PushUp config is already in base branch, or to ignore error in initial PR. However, this isn't a very smooth integration experience.