Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[tf.data] refactor serialization tests based on CheckpointTestBase in kernel_tests #47318

Merged
merged 12 commits into from Mar 2, 2021
Merged

[tf.data] refactor serialization tests based on CheckpointTestBase in kernel_tests #47318

merged 12 commits into from Mar 2, 2021

Conversation

kvignesh1420
Copy link
Member

@kvignesh1420 kvignesh1420 commented Feb 22, 2021

This PR is a pre-requisite for #47314 where serializability tests are being cleaned up and moved to kernel_tests.

  • Renamed DatasetSerializationTestBase to CheckpointTestBase to be consistent with the future test cases.
  • Refactored all serialization tests to use the new checkpoint_test_base target.
  • modified targets in BUILD files.

cc: @jsimsa

@kvignesh1420
Copy link
Member Author

kvignesh1420 commented Feb 22, 2021

@jsimsa I tried to run the serialization tests once before refactoring them with checkpoint_test_base.

When I run the following,

$ bazel test --runs_per_test=1 //tensorflow/python/data/experimental/kernel_tests/serialization:auto_shard_dataset_serialization_test

I get

WARNING: All specified test targets were excluded by filters
INFO: Analyzed 0 targets (0 packages loaded, 0 targets configured).
INFO: Found 0 test targets...
INFO: Elapsed time: 0.276s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Build completed successfully, 1 total action

Is there any issue with the current test targets or am I missing something?

UPDATE: The tests are being filtered out even after the changes.

@kvignesh1420 kvignesh1420 changed the title [tf.data] rename and move CheckpointTestBase to kernel_tests [tf.data] refactor serialization tests based on CheckpointTestBase in kernel_tests Feb 22, 2021
@jsimsa
Copy link
Contributor

jsimsa commented Feb 22, 2021

You see the tests not being executed because they are all marked as "no_oss" (e.g. https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/data/experimental/kernel_tests/serialization/BUILD#L39). If you remove the tag, the test should execute.

@google-ml-butler google-ml-butler bot added kokoro:force-run Tests on submitted change ready to pull PR ready for merge process labels Feb 22, 2021
@kokoro-team kokoro-team removed the kokoro:force-run Tests on submitted change label Feb 22, 2021
@rthadur
Copy link
Contributor

rthadur commented Feb 22, 2021

@kvignesh1420 can you please check sanity build failures ?

@rthadur rthadur removed the ready to pull PR ready for merge process label Feb 22, 2021
@rthadur rthadur added this to Assigned Reviewer in PR Queue via automation Feb 22, 2021
PR Queue automation moved this from Assigned Reviewer to Approved by Reviewer Feb 23, 2021
@google-ml-butler google-ml-butler bot added kokoro:force-run Tests on submitted change ready to pull PR ready for merge process labels Feb 23, 2021
@kokoro-team kokoro-team removed the kokoro:force-run Tests on submitted change label Feb 23, 2021
@gbaned gbaned added comp:data tf.data related issues ready to pull PR ready for merge process and removed ready to pull PR ready for merge process labels Feb 23, 2021
@kvignesh1420
Copy link
Member Author

@rthadur I have fixed the ci sanity issues.

@rthadur rthadur removed the ready to pull PR ready for merge process label Feb 23, 2021
@google-ml-butler google-ml-butler bot added kokoro:force-run Tests on submitted change ready to pull PR ready for merge process labels Feb 23, 2021
@kokoro-team kokoro-team removed the kokoro:force-run Tests on submitted change label Feb 23, 2021
@kvignesh1420
Copy link
Member Author

@jsimsa any update on this PR? Anything I can help with? Will continue with #47314 once this gets merged.

@jsimsa
Copy link
Contributor

jsimsa commented Mar 2, 2021

@kvignesh1420 there are internal projects that import the serialization module this PR moves and I will need to address the conflicts before your PR is merged, I hope to take care of that today

@kvignesh1420
Copy link
Member Author

Thanks @jsimsa

@copybara-service copybara-service bot merged commit c3c0352 into tensorflow:master Mar 2, 2021
PR Queue automation moved this from Approved by Reviewer to Merged Mar 2, 2021
@kvignesh1420 kvignesh1420 deleted the checkpoint-base branch March 4, 2021 13:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: yes comp:data tf.data related issues ready to pull PR ready for merge process size:L CL Change Size: Large
Projects
PR Queue
  
Merged
Development

Successfully merging this pull request may close these issues.

None yet

5 participants