Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Debug operators #29

Merged
merged 9 commits into from Mar 3, 2022
Merged

Debug operators #29

merged 9 commits into from Mar 3, 2022

Conversation

blakestier
Copy link
Contributor

First pass adding some debugging to our operators.

Copy link
Contributor

@whoahbot whoahbot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great! Thanks for putting this together.

Copy link
Contributor

@davidselassie davidselassie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for writing these up! I have one question on the macro and some thoughts about what might be relevant to log. But if you want to tackle these later, that's fine too.

src/operators/mod.rs Outdated Show resolved Hide resolved
fn type_name_of<T>(_: T) -> &'static str {
std::any::type_name::<T>()
}
let name = type_name_of(f);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like the goal of this macro is to get the name of the containing function? How does this work?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is the goal although the rust function isn't as precise as I'd like.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As in, does this print out the name of this function f() instead of the calling function like map()? Or does that somehow access the calling function? Or is this a kind of hack where f() has a fully qualified name like bytewax::map::f() and so we get access to the enclosing function? I'd like to understand how this works.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Candidly, the internet helped me with this, but I believe you can wrap the function in your own code and thereby access the type of the function which resolves to the fully qualified name.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that makes sense. Hence the slice to -3 to remove f() as the last step.

src/operators/mod.rs Outdated Show resolved Hide resolved
src/operators/mod.rs Outdated Show resolved Hide resolved

// These are all shims which map the Timely Rust API into equivalent
// calls to Python functions through PyO3.

pub(crate) fn map(mapper: &TdPyCallable, item: TdPyAny) -> TdPyAny {
debug!("{}, mapper:{:?}, item:{:?}", log_func!(), mapper, item);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice to log out epoch as well in all of these so you can flip a single logging switch and see a complete trace of execution. But that would be a bigger change because it would require changing the Timely operators we use and re-jiggering some of the shim functions, so I don't think we need to do it now. Just a note for posterity!

src/operators/mod.rs Outdated Show resolved Hide resolved
fn type_name_of<T>(_: T) -> &'static str {
std::any::type_name::<T>()
}
let name = type_name_of(f);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As in, does this print out the name of this function f() instead of the calling function like map()? Or does that somehow access the calling function? Or is this a kind of hack where f() has a fully qualified name like bytewax::map::f() and so we get access to the enclosing function? I'd like to understand how this works.

src/operators/mod.rs Outdated Show resolved Hide resolved
src/operators/mod.rs Outdated Show resolved Hide resolved
src/operators/mod.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@davidselassie davidselassie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! Thanks

@blakestier blakestier merged commit c2ebe99 into main Mar 3, 2022
@blakestier blakestier deleted the log branch March 3, 2022 00:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants