Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BEAM-13471] [Playground] Tag existing unit-tests #16325

Merged
merged 7 commits into from
Jan 12, 2022
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,14 @@
*/
package org.apache.beam.examples.complete;

// beam-playground:
// name: TfIdfTest
// description: Unit-test for the TfIdf example.
// multifile: false
// pipeline_options:
// categories:
// - Side Input
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// - Side Input
// - Side Input
// - Flatten


import java.net.URI;
import java.util.Arrays;
import org.apache.beam.sdk.coders.StringDelegateCoder;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,14 @@
*/
package org.apache.beam.examples.complete.game;

// beam-playground:
// name: GameStatsTest
// description: Unit-test for the GameStats example.
// multifile: false
// pipeline_options:
// categories:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// categories:
// categories:
// - Testing

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

//

import java.io.Serializable;
import java.util.Arrays;
import java.util.List;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,14 @@
*/
package org.apache.beam.examples.complete.kafkatopubsub;

// beam-playground:
// name: KafkaToPubsubTest
// description: Unit-test for the KafkaToPubsub example.
// multifile: false
// pipeline_options:
// categories:
// - Streaming

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's remove this file, it doesn't test anything related to Beam

import static org.apache.beam.examples.complete.kafkatopubsub.KafkaPubsubConstants.PASSWORD;
import static org.apache.beam.examples.complete.kafkatopubsub.KafkaPubsubConstants.USERNAME;
import static org.apache.beam.examples.complete.kafkatopubsub.kafka.consumer.Utils.getKafkaCredentialsFromVault;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,15 @@
*/
package org.apache.beam.examples.cookbook;

// beam-playground:
// name: DistinctExampleTest
// description: Unit-test for the DistinctExample example.
// multifile: false
// pipeline_options:
// categories:
// - IO
// - Options
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// - IO
// - Options
// - Core Transforms
// - Filtering


import java.util.Arrays;
import java.util.List;
import org.apache.beam.sdk.coders.StringUtf8Coder;
Expand Down
8 changes: 8 additions & 0 deletions sdks/go/pkg/beam/combine_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,14 @@

package beam_test

// beam-playground:
// name: CombineTest
// description: Unit-test for the Combine example.
// multifile: false
// pipeline_options:
// categories:
// - Combiners

import (
"reflect"
"testing"
Expand Down
8 changes: 8 additions & 0 deletions sdks/go/pkg/beam/core/util/symtab/symtab_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,14 @@

package symtab

// beam-playground:
// name: SymtabTest
// description: Unit-test for the Symtab example.
// multifile: false
// pipeline_options:
// categories:
// - Side Input

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's remove this file, it doesn't test anything related to Beam

import (
"io/ioutil"
"os"
Expand Down
7 changes: 7 additions & 0 deletions sdks/go/pkg/beam/partition_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,13 @@

package beam_test

// beam-playground:
// name: PartitionTest
// description: Unit-test for the Partition example.
// multifile: false
// pipeline_options:
// categories:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// categories:
// categories:
// - Testing

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.


import (
"testing"

Expand Down
8 changes: 8 additions & 0 deletions sdks/go/pkg/beam/testing/ptest/ptest.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,14 @@
// Package ptest contains utilities for pipeline unit testing.
package ptest

// beam-playground:
// name: PTest
// description: Contains utilities for pipeline unit testing.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need utilities? Do they test anything?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, the package doesn't really test anything. Remove tag.

// multifile: false
// pipeline_options:
// categories:
// - Testing

import (
"context"
"flag"
Expand Down
8 changes: 8 additions & 0 deletions sdks/go/pkg/beam/transforms/filter/distinct_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,14 @@

package filter_test

// beam-playground:
// name: DistinctTest
// description: Unit-test for the Distinct example.
// multifile: false
// pipeline_options:
// categories:
// - Filtering
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// - Filtering
// - Filtering
// - Core Transforms


import (
"testing"

Expand Down
8 changes: 8 additions & 0 deletions sdks/go/pkg/beam/transforms/filter/filter_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,14 @@

package filter_test

// beam-playground:
// name: FilterTest
// description: Unit-test for the Filter example.
// multifile: false
// pipeline_options:
// categories:
// - Filtering

import (
"testing"

Expand Down
10 changes: 10 additions & 0 deletions sdks/go/test/integration/integration.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,16 @@
// environment setup, or by calling gradle commands in :sdks:go:test.
package integration

// beam-playground:
// name: CheckFiltersTest
// description: Unit-test checks if an integration test is filtered to be skipped, either
// because the intended runner does not support it, or the test is sickbayed.
// multifile: false
// pipeline_options:
// categories:
// - Filtering
// - Testing

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's remove this file, it doesn't test anything related to Beam

import (
"fmt"
"regexp"
Expand Down
8 changes: 8 additions & 0 deletions sdks/go/test/integration/synthetic/synthetic_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,14 @@

package synthetic

// beam-playground:
// name: SyntheticTest
// description: Unit-test for the Synthetic example.
// multifile: false
// pipeline_options:
// categories:
// - Testing

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like this test is something internal, let's remove it. It might be not so interesting for Playground users

import (
"testing"

Expand Down
9 changes: 9 additions & 0 deletions sdks/go/test/integration/xlang/xlang_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,15 @@

package xlang

// beam-playground:
// name: XLangTest
// description: Unit-test for the XLang example.
// multifile: false
// pipeline_options:
// categories:
// - Combiners
// - Testing
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// - Testing
// - Testing

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand why it is displayed like this, but in the code it is correct.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's remove this file as well, it's hard to understand and it also seems like not related to Playground users


import (
"fmt"
"reflect"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,14 @@
*/
package org.apache.beam.sdk.io;

// beam-playground:
// name: FileBasedSourceTest
// description: Unit-test for the FileBasedSource example.
// multifile: false
// pipeline_options:
// categories:
// - IO

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's remove this file, it tests Beam internals

import static org.apache.beam.sdk.testing.SourceTestUtils.assertSplitAtFractionExhaustive;
import static org.apache.beam.sdk.testing.SourceTestUtils.assertSplitAtFractionFails;
import static org.apache.beam.sdk.testing.SourceTestUtils.assertSplitAtFractionSucceedsAndConsistent;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,14 @@
*/
package org.apache.beam.sdk.schemas.transforms;

// beam-playground:
// name: ConvertTest
// description: Unit-test for the Convert example.
// multifile: false
// pipeline_options:
// categories:
// - Schemas

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's remove this file, it tests Beam internals

import java.util.Arrays;
import java.util.Map;
import java.util.Objects;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,14 @@
*/
package org.apache.beam.sdk.testing;

// beam-playground:
// name: TestStreamTest
// description: Unit-test for the TestStream example.
// multifile: false
// pipeline_options:
// categories:
// - Testing
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// - Testing
// - Streaming


import static org.apache.beam.sdk.transforms.windowing.Window.into;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.allOf;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,15 @@
*/
package org.apache.beam.sdk.transforms;

// beam-playground:
// name: CombineFnsTest
// description: Unit-test for the CombineFn example.
// multifile: false
// pipeline_options:
// categories:
// - Combiners
// - Core Transforms

import static org.apache.beam.sdk.transforms.display.DisplayDataMatchers.hasDisplayItem;
import static org.apache.beam.sdk.transforms.display.DisplayDataMatchers.includesDisplayDataFor;
import static org.hamcrest.MatcherAssert.assertThat;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,15 @@
*/
package org.apache.beam.sdk.transforms;

// beam-playground:
// name: FilterTest
// description: Unit-test for the Filter example.
// multifile: false
// pipeline_options:
// categories:
// - Filtering
// - Core Transforms

import static org.apache.beam.sdk.transforms.display.DisplayDataMatchers.hasDisplayItem;
import static org.hamcrest.MatcherAssert.assertThat;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,15 @@
*/
package org.apache.beam.sdk.transforms.windowing;

// beam-playground:
// name: WindowingTest
// description: Unit-test for the Windowing kata.
// multifile: false
// pipeline_options:
// categories:
// - Windowing
// - Streaming

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's remove this file, it tests Beam internals

import java.io.File;
import java.io.FileOutputStream;
import java.io.PrintStream;
Expand Down
8 changes: 8 additions & 0 deletions sdks/python/apache_beam/examples/cookbook/combiners_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,14 @@

# pytype: skip-file

# beam-playground:
# name: CombinersTest
# description: Unit-test to showcase combiners.
# multifile: false
# pipeline_options:
# categories:
# - Combiners

import logging
import unittest

Expand Down
8 changes: 8 additions & 0 deletions sdks/python/apache_beam/examples/cookbook/filters_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,14 @@

# pytype: skip-file

# beam-playground:
# name: FiltersTest
# description: Unit-test for the Filters example.
# multifile: false
# pipeline_options:
# categories:
# - Filtering

import logging
import unittest

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,14 @@

# pytype: skip-file

# beam-playground:
# name: StreamingWordCountITTest
# description: Unit-test for the streaming WordCount example.
# multifile: false
# pipeline_options:
# categories:
# - Streaming
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# - Streaming
# - Streaming
# - IO


import logging
import unittest
import uuid
Expand Down
8 changes: 8 additions & 0 deletions sdks/python/apache_beam/examples/wordcount_minimal_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,14 @@

# pytype: skip-file

# beam-playground:
# name: WordCountMinimalTest
# description: Unit-test for the minimal WordCount example.
# multifile: false
# pipeline_options:
# categories:
# - IO
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# - IO
# - IO
# - Combiners


import collections
import logging
import re
Expand Down
8 changes: 8 additions & 0 deletions sdks/python/apache_beam/examples/wordcount_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,14 @@

# pytype: skip-file

# beam-playground:
# name: WordCountTest
# description: Unit-test for the WordCount example.
# multifile: false
# pipeline_options:
# categories:
# - IO
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# - IO
# - IO
# - Combiners


import collections
import logging
import re
Expand Down
8 changes: 8 additions & 0 deletions sdks/python/apache_beam/io/filesystemio_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,14 @@

# pytype: skip-file

# beam-playground:
# name: FileSystemIOTest
# description: Unit-test for the FileSystemIO example.
# multifile: false
# pipeline_options:
# categories:
# - IO

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's remove this file, it tests Beam internals

import io
import logging
import multiprocessing
Expand Down
8 changes: 8 additions & 0 deletions sdks/python/apache_beam/metrics/metric_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,14 @@

# pytype: skip-file

# beam-playground:
# name: MetricTest
# description: Unit-test for the Metric example.
# multifile: false
# pipeline_options:
# categories:
# - Metrics
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# - Metrics
# - Metrics

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto


import unittest

import hamcrest as hc
Expand Down