Skip to content
This repository has been archived by the owner on Oct 17, 2021. It is now read-only.

Add stubs and fakes for Dataset #317

Merged
merged 5 commits into from
Sep 11, 2018
Merged

Conversation

caisq
Copy link
Contributor

@caisq caisq commented Sep 7, 2018

DEV


This change is Reviewable

Copy link
Member

@davidsoergel davidsoergel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 1 approvals obtained (waiting on @caisq and @davidsoergel)


src/engine/dataset_stub.ts, line 47 at r1 (raw file):

  /**
   * The size of each batch generated by the iteartor.

nit: iterator


src/engine/dataset_stub.ts, line 115 at r1 (raw file):

/**
 * A fake dataset with configurable feature and target shapes.

I don't know what tests you have in mind, but this all seems unnecessarily complex. I'd just hardcode 10 trivial examples, maybe in batches of 3 (to exercise 'small last batch' logic).

Now that you already wrote this, we could keep it-- but even so I'm not sure it's worth the comprehension & maintenance cost. Do you think a much simpler fake would be sufficient?


src/engine/dataset_stub.ts, line 127 at r1 (raw file):

    tfc.util.assert(
        config.batchSize > 0 && Number.isInteger(config.batchSize),
        `batchSize must be a positive ineger, but got ${config.batchSize}`);

nit: integer


src/engine/dataset_stub_test.ts, line 17 at r1 (raw file):

import {FakeNumericDataset} from './dataset_stub';

describeMathCPUAndGPU('FakeNumericDataset', () => {

Wow, tests for the fake? Very thorough :)


src/engine/dataset_stub_test.ts, line 22 at r1 (raw file):

        {xShape: [3], yShape: [1], batchSize: 8, numBatches: 5});
    for (let k = 0; k < 2; ++k) {
      // Run twice to make sure that calling iteartor() multiple times works.

nit: iterator


src/engine/training.ts, line 35 at r1 (raw file):

import {SymbolicTensor} from './topology';

I thought lint wants two blank lines here-- maybe this is a mistake?

Copy link
Contributor Author

@caisq caisq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 1 approvals obtained (waiting on @caisq)


src/engine/dataset_stub.ts, line 47 at r1 (raw file):

Previously, davidsoergel (David Soergel) wrote…

nit: iterator

Done.


src/engine/dataset_stub.ts, line 115 at r1 (raw file):

Previously, davidsoergel (David Soergel) wrote…

I don't know what tests you have in mind, but this all seems unnecessarily complex. I'd just hardcode 10 trivial examples, maybe in batches of 3 (to exercise 'small last batch' logic).

Now that you already wrote this, we could keep it-- but even so I'm not sure it's worth the comprehension & maintenance cost. Do you think a much simpler fake would be sufficient?

If we hardcode 10 examples, there will be quite some code duplication, potentially leading to higher maintenance cost. The API of the fake class here is flexible enough to support

  1. Different numbers of input tensors and output tensors
  2. Different shapes of the tensors
    which should satisfy most test cases. If we find we need other fake datasets, we can extend this existing class later without unnecessary code duplication.

src/engine/dataset_stub.ts, line 127 at r1 (raw file):

Previously, davidsoergel (David Soergel) wrote…

nit: integer

Done.


src/engine/dataset_stub_test.ts, line 17 at r1 (raw file):

Previously, davidsoergel (David Soergel) wrote…

Wow, tests for the fake? Very thorough :)

Ack. :)


src/engine/training.ts, line 35 at r1 (raw file):

Previously, davidsoergel (David Soergel) wrote…

I thought lint wants two blank lines here-- maybe this is a mistake?

Hmm. It looks it doesn't matter whether it's 1 or 2 lines. Reverted this change.

Copy link
Member

@davidsoergel davidsoergel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 3 of 3 files at r2.
Reviewable status: 0 of 1 approvals obtained (waiting on @davidsoergel and @caisq)

@caisq caisq merged commit 848abf7 into tensorflow:master Sep 11, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
2 participants