Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow seed to be set for serialized pipeline #1844

Merged
merged 3 commits into from
Apr 2, 2020
Merged

Allow seed to be set for serialized pipeline #1844

merged 3 commits into from
Apr 2, 2020

Conversation

szalpal
Copy link
Member

@szalpal szalpal commented Mar 30, 2020

Similarly to other Pipeline params (refer the other ctor), this PR allows seed to be set for ctor with serialized pipeline.

Few comments required:

I'm aware, that -1 might be legitimate seed value, thus prolly shouldn't be a default value, but not in our code:

    if (seed < 0) {
      using Clock = std::chrono::high_resolution_clock;
      seed = Clock::now().time_since_epoch().count();
    }

Without optional there's no clean way to handle default seed in this case. I could go with int64_t* seed = nullptr, but alongside the rest of parameters, it would be obscure. using optional = std::pair<T, bool> also looks questinably compliated. Plus, we go with -1 as default seed anyway: in the other ctor.

I have an impression, that the seed was used incorrectly. Let's say, that -1 is passed to the Init(...). original_seed_ is set to -1. Then, in the snippet I pasted above, seed is set according to clock. Finally, when somebody serializes the pipeline (SerializeToProtobuf), the -1 is set as the original_seed_, instead of the clock-wise value assigned before. It might be intentional though, please double-check.

Signed-off-by: Michał Szołucha szal@szal-mbp.local

Michał Szołucha added 2 commits March 30, 2020 05:01
Signed-off-by: Michał Szołucha <szal@szal-mbp.local>
Signed-off-by: Michał Szołucha <szal@szal-mbp.local>
@szalpal
Copy link
Member Author

szalpal commented Mar 30, 2020

!build

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [1220604]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [1220604]: BUILD FAILED

@@ -96,7 +96,7 @@ class DLL_PUBLIC Pipeline {
}

DLL_PUBLIC Pipeline(const string &serialized_pipe, int batch_size = -1, int num_threads = -1,
int device_id = -1, bool pipelined_execution = true,
int device_id = -1, int64_t seed = -1, bool pipelined_execution = true,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now any code that uses the old Pipeline(const string &serialized_pipe signature is brokenand no one knows why.

Copy link
Contributor

@JanuszL JanuszL Mar 30, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about moving it to the last position?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True, changed

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now it is inconsistent in a different way with other constructors.

@JanuszL
Copy link
Contributor

JanuszL commented Mar 30, 2020

I have an impression, that the seed was used incorrectly. Let's say, that -1 is passed to the Init(...). original_seed_ is set to -1. Then, in the snippet I pasted above, seed is set according to clock. Finally, when somebody serializes the pipeline (SerializeToProtobuf), the -1 is set as the original_seed_, instead of the clock-wise value assigned before. It might be intentional though, please double-check.

seed (int, optional, default = -1) – Seed used for random number generation. Leaving the default value for this parameter results in random seed.
So user has been warned. We also use std::seed_seq ss{seed}; to generate per operator seeds and it doesn't accept negative values AFAIK.

@JanuszL
Copy link
Contributor

JanuszL commented Mar 30, 2020

I have an impression, that the seed was used incorrectly. Let's say, that -1 is passed to the Init(...). original_seed_ is set to -1. Then, in the snippet I pasted above, seed is set according to clock. Finally, when somebody serializes the pipeline (SerializeToProtobuf), the -1 is set as the original_seed_, instead of the clock-wise value assigned before. It might be intentional though, please double-check.

seed (int, optional, default = -1) – Seed used for random number generation. Leaving the default value for this parameter results in random seed.
So user has been warned. We also use std::seed_seq ss{seed}; to generate per operator seeds and it doesn't accept negative values AFAIK.

That is true that others FWs accepts also negative values, but I don't know if this is used by anyone. If we want to truly support this we need to rework std::seed_seq ss{seed};, I don't have any strong opinion.

Signed-off-by: Michał Szołucha <szal@szal-mbp.local>
@szalpal
Copy link
Member Author

szalpal commented Mar 30, 2020

!build

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [1221209]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [1221209]: BUILD FAILED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [1221209]: BUILD PASSED

@szalpal szalpal merged commit 2154f57 into NVIDIA:master Apr 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants