Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Consolidate coverage generation #13398

Closed
6 tasks done
nstng opened this issue Jul 25, 2022 · 10 comments · Fixed by #14127 or #14264
Closed
6 tasks done

Consolidate coverage generation #13398

nstng opened this issue Jul 25, 2022 · 10 comments · Fixed by #14127 or #14264
Assignees
@LKreutzer
Labels
bazel changes for the Bazelification effort willfix This will be worked on and shouldn't be reaped

Comments

@nstng
Copy link
Contributor

nstng commented Jul 25, 2022 •
edited by LKreutzer

Currently the MME coverage is generated by Make and all others by bazel. Can we do everything with bazel instead of splitting it up? This would make the agw-workflow much faster.

Blocked by #13399

  • all places where AGW coverage is generated with make is replaced with bazel coverage generation
  • create a new workflow and move coverage generation there ~agw-coverage.yml
    • use native build - other configs are not necessary
    • see if codecov actions can be upgraded
    • make sure the same codecov flags are used!
    • Unused Make targets should be removed
@nstng nstng added bazel changes for the Bazelification effort willfix This will be worked on and shouldn't be reaped labels Jul 25, 2022
@nstng nstng self-assigned this Sep 7, 2022
@nstng nstng removed their assignment Sep 20, 2022
@LKreutzer LKreutzer self-assigned this Oct 11, 2022
@LKreutzer LKreutzer mentioned this issue Oct 11, 2022
Merged
1 task
@LKreutzer
Copy link
Contributor

LKreutzer commented Oct 24, 2022 •
edited

All python tests "fail" in CI with:

FAIL: //lte/gateway/python/magma/subscriberdb/tests:sid_utils_tests (see /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/python/magma/subscriberdb/tests/sid_utils_tests/test.log)
INFO: From Testing //lte/gateway/python/magma/subscriberdb/tests:sid_utils_tests:
==================== Test output for //lte/gateway/python/magma/subscriberdb/tests:sid_utils_tests:
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0 -- /usr/bin/python3.8
cachedir: .pytest_cache
rootdir: /tmp/bazel/sandbox/processwrapper-sandbox/1803/execroot/__main__/bazel-out/k8-fastbuild/bin/lte/gateway/python/magma/subscriberdb/tests/sid_utils_tests.runfiles/__main__
plugins: cov-3.0.0, flaky-3.7.0
collecting ... collected 1 item
lte/gateway/python/magma/subscriberdb/tests/sid_utils_tests.py::SIDTests::test_str_conversion PASSED
============================== 1 passed in 0.17s ===============================
GCov does not exist at the given path: ''
/tmp/bazel/sandbox/processwrapper-sandbox/1803/execroot/__main__/bazel-out/k8-fastbuild/bin/external/remote_coverage_tools/Main: Cannot locate runfiles directory. (Set $JAVA_RUNFILES to inhibit searching.)
================================================================================

Tests show up as passing in the logging and the job status is green. However the summary shows them as failed and there are no coverage files in CI.

See e.g. https://github.com/magma/magma/actions/runs/3311758394/jobs/5467599473

@LKreutzer
Copy link
Contributor

LKreutzer commented Oct 24, 2022 •
edited

Locally with empty disk-cache:

$ bazel coverage //lte/gateway/python/magma/subscriberdb/tests:sid_utils_tests
Starting local Bazel server and connecting to it...
INFO: Invocation ID: 344463b2-15db-4404-86f3-6d5dc27f1688
INFO: Reading 'startup' options from /workspaces/magma/.bazelrc: --output_base=/tmp/bazel, --host_jvm_args=-Xmx8g
INFO: Options provided by the client:
  Inherited 'common' options: --isatty=1 --terminal_columns=182
INFO: Reading rc options for 'coverage' from /workspaces/magma/bazel/bazelrcs/cache.bazelrc:
  Inherited 'common' options: --config=disk_cache
INFO: Reading rc options for 'coverage' from /workspaces/magma/.bazelrc:
  Inherited 'build' options: --announce_rc --color=yes --cxxopt=-std=c++14 --strip=never --per_file_copt=^lte/gateway/c/.*$@-g --define=folly_so=1 --test_env=PATH=/bin:/usr/bin:/usr/local/bin:/usr/sbin --test_env=MAGMA_ROOT --test_env=S1AP_TESTER_ROOT --per_file_copt=^lte/gateway/c/core/.*$@-DPACKAGE_BUGREPORT="TBD" --per_file_copt=^lte/gateway/c/core/.*$@-DPACKAGE_VERSION="0.1" --per_file_copt=^lte/gateway/c/core/oai/tasks/amf/.*$@-DPACKAGE_NAME="AMF" --per_file_copt=^lte/gateway/c/core/oai/tasks/mme_app/.*$@-DPACKAGE_NAME="MME" --per_file_copt=^lte/gateway/c/core/oai/tasks/sgw/.*$@-DPACKAGE_NAME="S/P-GW" --javacopt=-source 8 --javacopt=-target 8 --java_runtime_version=remotejdk_11
INFO: Reading rc options for 'coverage' from /workspaces/magma/.bazelrc:
  Inherited 'test' options: --test_output=errors --test_env=GO_TEST_WRAP_TESTV=1 --per_file_copt=^lte/gateway/c/core/.*$@-DMME_UNIT_TEST
INFO: Reading rc options for 'coverage' from /workspaces/magma/.bazelrc:
  'coverage' options: --config=coverage
INFO: Found applicable config definition common:disk_cache in file /workspaces/magma/.bazelrc: --repository_cache=/var/cache/bazel-cache-repo
INFO: Found applicable config definition build:disk_cache in file /workspaces/magma/.bazelrc: --disk_cache=/var/cache/bazel-cache
INFO: Found applicable config definition build:coverage in file /workspaces/magma/.bazelrc: --combined_report=lcov --instrumentation_filter=//(orc8r|lte)/gateway/(c|python)[/:],-//(orc8r|lte)/protos[/:],-/*/test[/:]
INFO: Analyzed target //lte/gateway/python/magma/subscriberdb/tests:sid_utils_tests (78 packages loaded, 2024 targets configured).
INFO: Found 1 test target...
INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/field_mask_pb2.py:
external/com_google_protobuf/python: warning: directory does not exist.
INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/any_pb2.py:
external/com_google_protobuf/python: warning: directory does not exist.
INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/timestamp_pb2.py:
external/com_google_protobuf/python: warning: directory does not exist.
INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/wrappers_pb2.py:
external/com_google_protobuf/python: warning: directory does not exist.
INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/empty_pb2.py:
external/com_google_protobuf/python: warning: directory does not exist.
INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/source_context_pb2.py:
external/com_google_protobuf/python: warning: directory does not exist.
INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/struct_pb2.py:
external/com_google_protobuf/python: warning: directory does not exist.
INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/duration_pb2.py:
external/com_google_protobuf/python: warning: directory does not exist.
INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/descriptor_pb2.py:
external/com_google_protobuf/python: warning: directory does not exist.
INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/compiler/plugin_pb2.py:
external/com_google_protobuf/python: warning: directory does not exist.
INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/type_pb2.py:
external/com_google_protobuf/python: warning: directory does not exist.
INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/api_pb2.py:
external/com_google_protobuf/python: warning: directory does not exist.
INFO: LCOV coverage report is located at /tmp/bazel/execroot/__main__/bazel-out/_coverage/_coverage_report.dat
 and execpath is bazel-out/_coverage/_coverage_report.dat
INFO: From Coverage report generation:
Oct 24, 2022 10:10:11 AM com.google.devtools.coverageoutputgenerator.Main getTracefiles
INFO: Found 1 tracefiles.
Oct 24, 2022 10:10:11 AM com.google.devtools.coverageoutputgenerator.Main parseFilesSequentially
INFO: Parsing file bazel-out/k8-fastbuild/testlogs/lte/gateway/python/magma/subscriberdb/tests/sid_utils_tests/coverage.dat
Oct 24, 2022 10:10:11 AM com.google.devtools.coverageoutputgenerator.Main getGcovInfoFiles
INFO: No gcov info file found.
Oct 24, 2022 10:10:11 AM com.google.devtools.coverageoutputgenerator.Main getGcovJsonInfoFiles
INFO: No gcov json file found.
Oct 24, 2022 10:10:11 AM com.google.devtools.coverageoutputgenerator.Main getProfdataFileOrNull
INFO: No .profdata file found.
Target //lte/gateway/python/magma/subscriberdb/tests:sid_utils_tests up-to-date:
  bazel-bin/lte/gateway/python/magma/subscriberdb/tests/sid_utils_tests-runner.py
  bazel-bin/lte/gateway/python/magma/subscriberdb/tests/sid_utils_tests
INFO: Elapsed time: 109.143s, Critical Path: 16.24s
INFO: 240 processes: 18 internal, 220 processwrapper-sandbox, 2 worker.
INFO: Build completed successfully, 240 total actions
//lte/gateway/python/magma/subscriberdb/tests:sid_utils_tests            PASSED in 0.9s
  /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/python/magma/subscriberdb/tests/sid_utils_tests/coverage.dat

INFO: Build completed successfully, 240 total actions

Content of the file is:

TN:
SF:/workspaces/magma/lte/gateway/python/magma/subscriberdb/sid.py
DA:14,1
DA:17,1
DA:22,1
DA:23,1
DA:34,1
DA:35,1
DA:36,0
DA:41,1
DA:42,1
DA:53,1
DA:54,1
DA:55,1
DA:56,1
DA:57,1
DA:58,1
LF:15
LH:14
end_of_record

@LKreutzer
Copy link
Contributor

LKreutzer commented Oct 24, 2022 •
edited

Local run with clean remote cache after removing local cache and bazel clean --expunge:

INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/descriptor_pb2.py:
external/com_google_protobuf/python: warning: directory does not exist.
INFO: From ProtoCompile external/com_google_protobuf/python/google/protobuf/empty_pb2.py:
external/com_google_protobuf/python: warning: directory does not exist.
FAIL: //lte/gateway/python/magma/subscriberdb/tests:sid_utils_tests (see /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/python/magma/subscriberdb/tests/sid_utils_tests/test.log)
INFO: From Testing //lte/gateway/python/magma/subscriberdb/tests:sid_utils_tests:
==================== Test output for //lte/gateway/python/magma/subscriberdb/tests:sid_utils_tests:
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0 -- /usr/bin/python3.8
cachedir: .pytest_cache
rootdir: /tmp/bazel/sandbox/processwrapper-sandbox/220/execroot/__main__/bazel-out/k8-fastbuild/bin/lte/gateway/python/magma/subscriberdb/tests/sid_utils_tests.runfiles/__main__
plugins: cov-3.0.0, flaky-3.7.0
collecting ... collected 1 item

lte/gateway/python/magma/subscriberdb/tests/sid_utils_tests.py::SIDTests::test_str_conversion PASSED

============================== 1 passed in 0.12s ===============================
GCov does not exist at the given path: ''
/tmp/bazel/sandbox/processwrapper-sandbox/220/execroot/__main__/bazel-out/k8-fastbuild/bin/external/remote_coverage_tools/Main: Cannot locate runfiles directory. (Set $JAVA_RUNFILES to inhibit searching.)
================================================================================
INFO: LCOV coverage report is located at /tmp/bazel/execroot/__main__/bazel-out/_coverage/_coverage_report.dat
 and execpath is bazel-out/_coverage/_coverage_report.dat
INFO: From Coverage report generation:
Oct 24, 2022 11:00:57 AM com.google.devtools.coverageoutputgenerator.Main getTracefiles
INFO: Found 1 tracefiles.
Oct 24, 2022 11:00:57 AM com.google.devtools.coverageoutputgenerator.Main parseFilesSequentially
INFO: Parsing file bazel-out/k8-fastbuild/testlogs/lte/gateway/python/magma/subscriberdb/tests/sid_utils_tests/coverage.dat
Oct 24, 2022 11:00:57 AM com.google.devtools.coverageoutputgenerator.Main getGcovInfoFiles
INFO: No gcov info file found.
Oct 24, 2022 11:00:57 AM com.google.devtools.coverageoutputgenerator.Main getGcovJsonInfoFiles
INFO: No gcov json file found.
Oct 24, 2022 11:00:57 AM com.google.devtools.coverageoutputgenerator.Main getProfdataFileOrNull
INFO: No .profdata file found.
Target //lte/gateway/python/magma/subscriberdb/tests:sid_utils_tests up-to-date:
  bazel-bin/lte/gateway/python/magma/subscriberdb/tests/sid_utils_tests-runner.py
  bazel-bin/lte/gateway/python/magma/subscriberdb/tests/sid_utils_tests
INFO: Elapsed time: 198.732s, Critical Path: 121.34s
INFO: 236 processes: 14 internal, 220 processwrapper-sandbox, 2 worker.
INFO: Build completed, 1 test FAILED, 236 total actions
//lte/gateway/python/magma/subscriberdb/tests:sid_utils_tests            FAILED in 0.5s
  /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/python/magma/subscriberdb/tests/sid_utils_tests/test.log
  /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/python/magma/subscriberdb/tests/sid_utils_tests/coverage.dat

INFO: Build completed, 1 test FAILED, 236 total actions

TN:
SF:/workspaces/magma/lte/gateway/python/magma/subscriberdb/sid.py
DA:14,1
DA:17,1
DA:22,1
DA:23,1
DA:34,1
DA:35,1
DA:36,0
DA:41,1
DA:42,1
DA:53,1
DA:54,1
DA:55,1
DA:56,1
DA:57,1
DA:58,1
LF:15
LH:14
end_of_record

@LKreutzer
Copy link
Contributor

Coverage files are identical with remote and disk caches.
Local run only fails with remote cache.

@LKreutzer
Copy link
Contributor

Might be relevant:

@LKreutzer
Copy link
Contributor

For comparison C++ test locally with remote cache:

_coverage/_cc_coverage.gcov: Permission denied
rm: cannot remove '*.gcov': No such file or directory
cp: cannot create regular file '/tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/bazel-out/k8-fastbuild/bin/lte/gateway/c/session_manager/_objs/stored_state/StoredState.pic.gcno': Permission denied
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 154: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/gcov: No such file or directory
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 159: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/gcov: No such file or directory
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 169: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/_cc_coverage.gcov: Permission denied
rm: cannot remove '*.gcov': No such file or directory
cp: cannot create regular file '/tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/bazel-out/k8-fastbuild/bin/lte/gateway/c/session_manager/_objs/upf_msg_manage_handler/UpfMsgManageHandler.pic.gcno': Permission denied
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 154: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/gcov: No such file or directory
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 159: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/gcov: No such file or directory
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 169: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/_cc_coverage.gcov: Permission denied
rm: cannot remove '*.gcov': No such file or directory
cp: cannot create regular file '/tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/bazel-out/k8-fastbuild/bin/lte/gateway/c/session_manager/_objs/utilities/Utilities.pic.gcno': Permission denied
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 154: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/gcov: No such file or directory
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 159: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/gcov: No such file or directory
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 169: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/_cc_coverage.gcov: Permission denied
rm: cannot remove '*.gcov': No such file or directory
cp: cannot create regular file '/tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/bazel-out/k8-fastbuild/bin/lte/gateway/c/session_manager/test/_objs/protobuf_creators/ProtobufCreators.pic.gcno': Permission denied
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 154: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/gcov: No such file or directory
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 159: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/gcov: No such file or directory
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 169: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/_cc_coverage.gcov: Permission denied
rm: cannot remove '*.gcov': No such file or directory
cp: cannot create regular file '/tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/bazel-out/k8-fastbuild/bin/lte/gateway/c/session_manager/test/_objs/sessiond_integ_test/test_sessiond_integ.pic.gcno': Permission denied
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 154: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/gcov: No such file or directory
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 159: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/gcov: No such file or directory
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 169: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/_cc_coverage.gcov: Permission denied
rm: cannot remove '*.gcov': No such file or directory
cp: cannot create regular file '/tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/bazel-out/k8-fastbuild/bin/orc8r/gateway/c/common/async_grpc/_objs/async_grpc_receiver/GRPCReceiver.pic.gcno': Permission denied
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 154: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/gcov: No such file or directory
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 159: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/gcov: No such file or directory
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 169: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/_cc_coverage.gcov: Permission denied
rm: cannot remove '*.gcov': No such file or directory
cp: cannot create regular file '/tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/bazel-out/k8-fastbuild/bin/orc8r/gateway/c/common/config/_objs/mconfig_loader/MConfigLoader.pic.gcno': Permission denied
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 154: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/gcov: No such file or directory
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 159: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/gcov: No such file or directory
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 169: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/_cc_coverage.gcov: Permission denied
rm: cannot remove '*.gcov': No such file or directory
cp: cannot create regular file '/tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/bazel-out/k8-fastbuild/bin/orc8r/gateway/c/common/config/_objs/service_config_loader/ServiceConfigLoader.pic.gcno': Permission denied
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 154: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/gcov: No such file or directory
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 159: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/gcov: No such file or directory
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 169: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/_cc_coverage.gcov: Permission denied
rm: cannot remove '*.gcov': No such file or directory
cp: cannot create regular file '/tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/bazel-out/k8-fastbuild/bin/orc8r/gateway/c/common/config/_objs/yaml_utils/YAMLUtils.pic.gcno': Permission denied
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 154: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/gcov: No such file or directory
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 159: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/gcov: No such file or directory
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 169: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/_cc_coverage.gcov: Permission denied
rm: cannot remove '*.gcov': No such file or directory
cp: cannot create regular file '/tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/bazel-out/k8-fastbuild/bin/orc8r/gateway/c/common/eventd/_objs/eventd_client/EventdClient.pic.gcno': Permission denied
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 154: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/gcov: No such file or directory
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 159: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/gcov: No such file or directory
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 169: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/_cc_coverage.gcov: Permission denied
rm: cannot remove '*.gcov': No such file or directory
cp: cannot create regular file '/tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/bazel-out/k8-fastbuild/bin/orc8r/gateway/c/common/sentry/_objs/sentry_wrapper/SentryWrapper.pic.gcno': Permission denied
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 154: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/gcov: No such file or directory
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 159: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/gcov: No such file or directory
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 169: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/_cc_coverage.gcov: Permission denied
rm: cannot remove '*.gcov': No such file or directory
cp: cannot create regular file '/tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/bazel-out/k8-fastbuild/bin/orc8r/gateway/c/common/service303/_objs/service303/MagmaService.pic.gcno': Permission denied
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 154: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/gcov: No such file or directory
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 159: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/gcov: No such file or directory
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 169: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/_cc_coverage.gcov: Permission denied
rm: cannot remove '*.gcov': No such file or directory
cp: cannot create regular file '/tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/bazel-out/k8-fastbuild/bin/orc8r/gateway/c/common/service303/_objs/service303/MetricsHelpers.pic.gcno': Permission denied
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 154: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/gcov: No such file or directory
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 159: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/gcov: No such file or directory
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 169: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/_cc_coverage.gcov: Permission denied
rm: cannot remove '*.gcov': No such file or directory
cp: cannot create regular file '/tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/bazel-out/k8-fastbuild/bin/orc8r/gateway/c/common/service303/_objs/service303/MetricsSingleton.pic.gcno': Permission denied
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 154: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/gcov: No such file or directory
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 159: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/gcov: No such file or directory
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 169: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/_cc_coverage.gcov: Permission denied
rm: cannot remove '*.gcov': No such file or directory
cp: cannot create regular file '/tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/bazel-out/k8-fastbuild/bin/orc8r/gateway/c/common/service303/_objs/service303/ProcFileUtils.pic.gcno': Permission denied
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 154: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/gcov: No such file or directory
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 159: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/gcov: No such file or directory
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 169: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/_cc_coverage.gcov: Permission denied
rm: cannot remove '*.gcov': No such file or directory
cp: cannot create regular file '/tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/bazel-out/k8-fastbuild/bin/orc8r/gateway/c/common/service_registry/_objs/service_registry/ServiceRegistrySingleton.pic.gcno': Permission denied
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 154: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/gcov: No such file or directory
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 159: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/gcov: No such file or directory
external/bazel_tools/tools/test/collect_cc_coverage.sh: line 169: /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/_coverage/_cc_coverage.gcov: Permission denied
rm: cannot remove '*.gcov': No such file or directory
Oct 24, 2022 11:26:36 AM com.google.devtools.coverageoutputgenerator.Main getTracefiles
INFO: No lcov file found.
Oct 24, 2022 11:26:36 AM com.google.devtools.coverageoutputgenerator.Main getGcovInfoFiles
INFO: No gcov info file found.
Oct 24, 2022 11:26:36 AM com.google.devtools.coverageoutputgenerator.Main getGcovJsonInfoFiles
INFO: No gcov json file found.
Oct 24, 2022 11:26:36 AM com.google.devtools.coverageoutputgenerator.Main getProfdataFileOrNull
INFO: No .profdata file found.
Oct 24, 2022 11:26:36 AM com.google.devtools.coverageoutputgenerator.Main runWithArgs
WARNING: There was no coverage found.
java.io.IOException: Permission denied
        at java.base/java.io.UnixFileSystem.createFileExclusively(Native Method)
        at java.base/java.io.File.createTempFile(File.java:2129)
        at com.google.testing.coverage.JacocoCoverageRunner.getUniquePath(JacocoCoverageRunner.java:345)
        at com.google.testing.coverage.JacocoCoverageRunner.access$000(JacocoCoverageRunner.java:81)
        at com.google.testing.coverage.JacocoCoverageRunner$2.run(JacocoCoverageRunner.java:519)
================================================================================
INFO: LCOV coverage report is located at /tmp/bazel/execroot/__main__/bazel-out/_coverage/_coverage_report.dat
 and execpath is bazel-out/_coverage/_coverage_report.dat
INFO: From Coverage report generation:
Oct 24, 2022 11:26:41 AM com.google.devtools.coverageoutputgenerator.Main getTracefiles
INFO: Found 1 tracefiles.
Oct 24, 2022 11:26:41 AM com.google.devtools.coverageoutputgenerator.Main parseFilesSequentially
INFO: Parsing file bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/coverage.dat
Oct 24, 2022 11:26:41 AM com.google.devtools.coverageoutputgenerator.Main getGcovInfoFiles
INFO: No gcov info file found.
Oct 24, 2022 11:26:41 AM com.google.devtools.coverageoutputgenerator.Main getGcovJsonInfoFiles
INFO: No gcov json file found.
Oct 24, 2022 11:26:41 AM com.google.devtools.coverageoutputgenerator.Main getProfdataFileOrNull
INFO: No .profdata file found.
Target //lte/gateway/c/session_manager/test:sessiond_integ_test up-to-date:
  bazel-bin/lte/gateway/c/session_manager/test/sessiond_integ_test
INFO: Elapsed time: 767.394s, Critical Path: 354.72s
INFO: 2588 processes: 511 internal, 2 local, 2075 processwrapper-sandbox.
INFO: Build completed successfully, 2588 total actions
//lte/gateway/c/session_manager/test:sessiond_integ_test                  FLAKY, failed in 1 out of 2 in 1.2s
  Stats over 2 runs: max = 1.2s, min = 1.1s, avg = 1.2s, dev = 0.0s
  /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/test_attempts/attempt_1.log
  /tmp/bazel/execroot/__main__/bazel-out/k8-fastbuild/testlogs/lte/gateway/c/session_manager/test/sessiond_integ_test/coverage.dat

INFO: Build completed successfully, 2588 total actions

SF:lte/gateway/c/session_manager/AAAClient.cpp
FN:38,_ZN12_GLOBAL__N_121create_deactivate_reqERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES7_
FN:46,_ZN12_GLOBAL__N_123create_add_sessions_reqERKSt13unordered_mapINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt6vectorISt10unique_ptrIN5magma12SessionStateESt14default_deleteISA_EESaISD_EESt4hashIS6_ESt8equal_toIS6_ESaISt4pairIKS6_SF_EEE
FN:103,_ZN3aaa14AsyncAAAClient12add_sessionsERKSt13unordered_mapINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt6vectorISt10unique_ptrIN5magma12SessionStateESt14default_deleteISB_EESaISE_EESt4hashIS7_ESt8equal_toIS7_ESaISt4pairIKS7_SG_EEE
FN:121,_ZN3aaa14AsyncAAAClient16add_sessions_rpcERKNS_6protos20add_sessions_requestESt8functionIFvN4grpc6StatusENS1_9acct_respEEE
FN:86,_ZN3aaa14AsyncAAAClient17terminate_sessionERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_
FN:130,_ZN3aaa14AsyncAAAClient21terminate_session_rpcERKNS_6protos25terminate_session_requestESt8functionIFvN4grpc6StatusENS1_9acct_respEEE
FN:78,_ZN3aaa14AsyncAAAClientC2ESt10shared_ptrIN4grpc7ChannelEE
FN:81,_ZN3aaa14AsyncAAAClientC2Ev
FN:110,_ZZN3aaa14AsyncAAAClient12add_sessionsERKSt13unordered_mapINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt6vectorISt10unique_ptrIN5magma12SessionStateESt14default_deleteISB_EESaISE_EESt4hashIS7_ESt8equal_toIS7_ESaISt4pairIKS7_SG_EEEENKUlN4grpc6StatusENS_6protos9acct_respEE_clEST_SV_
FN:89,_ZZN3aaa14AsyncAAAClient17terminate_sessionERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_ENKUlN4grpc6StatusENS_6protos9acct_respEE_clESA_SC_
FNDA:0,_ZN12_GLOBAL__N_121create_deactivate_reqERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES7_
FNDA:0,_ZN12_GLOBAL__N_123create_add_sessions_reqERKSt13unordered_mapINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt6vectorISt10unique_ptrIN5magma12SessionStateESt14default_deleteISA_EESaISD_EESt4hashIS6_ESt8equal_toIS6_ESaISt4pairIKS6_SF_EEE
FNDA:0,_ZN3aaa14AsyncAAAClient12add_sessionsERKSt13unordered_mapINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt6vectorISt10unique_ptrIN5magma12SessionStateESt14default_deleteISB_EESaISE_EESt4hashIS7_ESt8equal_toIS7_ESaISt4pairIKS7_SG_EEE
FNDA:0,_ZN3aaa14AsyncAAAClient16add_sessions_rpcERKNS_6protos20add_sessions_requestESt8functionIFvN4grpc6StatusENS1_9acct_respEEE
FNDA:0,_ZN3aaa14AsyncAAAClient17terminate_sessionERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_
FNDA:0,_ZN3aaa14AsyncAAAClient21terminate_session_rpcERKNS_6protos25terminate_session_requestESt8functionIFvN4grpc6StatusENS1_9acct_respEEE
FNDA:0,_ZN3aaa14AsyncAAAClientC2ESt10shared_ptrIN4grpc7ChannelEE
FNDA:0,_ZN3aaa14AsyncAAAClientC2Ev
FNDA:0,_ZZN3aaa14AsyncAAAClient12add_sessionsERKSt13unordered_mapINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt6vectorISt10unique_ptrIN5magma12SessionStateESt14default_deleteISB_EESaISE_EESt4hashIS7_ESt8equal_toIS7_ESaISt4pairIKS7_SG_EEEENKUlN4grpc6StatusENS_6protos9acct_respEE_clEST_SV_
FNDA:0,_ZZN3aaa14AsyncAAAClient17terminate_sessionERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_ENKUlN4grpc6StatusENS_6protos9acct_respEE_clESA_SC_
FNF:10
FNH:0
DA:38,0
DA:40,0
DA:43,0
DA:46,0
DA:48,0
DA:49,0
DA:50,0
DA:51,0
DA:52,0
DA:53,0
DA:55,0
DA:56,0
DA:57,0
DA:58,0
DA:59,0
DA:61,0
DA:62,0
DA:63,0
DA:64,0
DA:65,0
DA:66,0
DA:67,0
DA:68,0
DA:71,0
DA:78,0
DA:79,0
DA:81,0
DA:83,0
DA:84,0
DA:86,0
DA:88,0
DA:89,0
DA:91,0
DA:92,0
DA:93,0
DA:95,0
DA:96,0
DA:97,0
DA:99,0
DA:100,0
DA:103,0
DA:104,0
DA:105,0
DA:106,0
DA:107,0
DA:108,0
DA:110,0
.
.
.

@LKreutzer
Copy link
Contributor

LKreutzer commented Oct 24, 2022 •
edited

Running bazel coverage //lte/gateway/python/magma/subscriberdb/tests:sid_utils_tests --experimental_split_coverage_postprocessing --experimental_fetch_all_coverage_outputs
results in a successfull test, but with an empty coverage file.

@LKreutzer
Copy link
Contributor

LKreutzer commented Oct 24, 2022 •
edited

Locally with remote cache:
bazel coverage //lte/gateway/python/magma/subscriberdb/tests:sid_utils_tests --experimental_fetch_all_coverage_outputs
Returns a green status and filled coverage file.
After cleaning the local cache and using a new cache key for the remote cache it does not work.

@LKreutzer
Copy link
Contributor

The solution for python seems to be to not use any remote_download_optimization.

@LKreutzer LKreutzer mentioned this issue Oct 25, 2022
Merged
1 task
@LKreutzer
Copy link
Contributor

Make targets will be removed in the PR #14264

@LKreutzer LKreutzer reopened this Nov 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@LKreutzer LKreutzer

Labels
bazel changes for the Bazelification effort willfix This will be worked on and shouldn't be reaped
Projects
None yet
Milestone
No milestone
2 participants
@LKreutzer @nstng