[Relay] Extract intermediate node by its expression ID #12646

sisleyli · 2022-08-30T11:47:21Z

Hi,
When I actually develop operators for some models, I often need to do some verification.
And when I encounter a situation where the input and output values of the entire model do not match the golden values, I often need to dump some intermediate nodes to confirm which node the problem starts from.
As we know, we can easily print Relay IR by print(mod["main"]),

Relay IR represents each node as "%xx %10 %20", I feel comfortable if we can dump nodes with "%xx" as a parameter.
So I develop a pass that can dump op by dump_op_by_num(mod,xx).

Because we generally use dump OP for analysis purposes, I set it as an relay.analysis API.
What do you think? Do we need such an API?

I am looking forward to your reply. Thank you!

sisleyli · 2022-08-30T11:53:51Z

Hi @masahi ! Do you think we need such an analysis pass? Thank you!

masahi · 2022-08-31T00:49:39Z

I think such pass is useful indeed, but I don't have a good idea on how to identify the intermediate node that will become the output node in the extracted subgraph.

Your approach that requests an integer ID from users and incrementing the count as we go through ops is probably ok for chain-like models, but for more general models where there is no clear ordering on nodes, this would not work.

sisleyli · 2022-08-31T02:24:50Z

Hi masashi, thanks for your reply.
The more general models you mean are RNN/LSTM models, or models like below?

               0
            /  |  \
           /   |   \
         1     2     3

The integer ID that I request is the number that RelayPrinter alloc to every node.(which we can see this id by print(mod["main"]) easily). The original intention of this method is to use the printed relay IR information to dump the specified op. So once the order of printing is determined, I think we just need to make sure that the integer ID specified by this method is consistent with the printed variable id.

masahi · 2022-08-31T07:34:07Z

ok, I agree that using the order from print works in practice.

I don't like the name "DumpOPByName". How about this signature: ExtractIntermediateExpr(mod, expr_id)

masahi · 2022-08-31T07:39:41Z

btw, have you used debug_executor? I heard that it can also dump output tensors of intermediate nodes. Your use case may be covered by debug_executor as well. cc @areusch

sisleyli · 2022-08-31T08:45:46Z

expr_id

I agree ExtractIntermediateExpr(mod, expr_id) is better, I have modified my code.

I have used debug_executor. Do you mean m.debug_get_output(node,out=tvm_output)?
I remembered the param node was not in Relay level, I will check later. Thank you!

sisleyli · 2022-08-31T09:25:53Z

Hi masahi, when I use debug_executor, There are many nodes called tvmgen_default_fused_xxxxx, it seems that the operators that debugger dump have already been fused(I didn't use any pass). Therefore it is not easy for me to dump the output of a specific op in Relay level.
I'm not sure if there are some params to control this fusion. cc @areusch

masahi · 2022-08-31T09:33:48Z

I see, that makes sense (that debug_executor only allows outputs from post fusion).

Please rebase against main, otherwise CI fails.

sisleyli · 2022-08-31T13:50:47Z

I see, CI has passed now, please review again when you have free time. Thank you!

areusch · 2022-08-31T20:25:45Z

ah yeah this is related to what we discussed a few weeks at community meeting. There hasn't been a follow-on edge ID proposal yet.

masahi · 2022-09-01T01:46:01Z

python/tvm/relay/analysis/analysis.py

+
+    This function is used for extracting Relay Expr
+    by the local variable number of the main function
+    that we can see in `print(mod["main"])`.


It would be nice to have an example here. local variable number is not a word in TVM, we should just refer to them as expression ID or something.

masahi · 2022-09-01T01:46:15Z

python/tvm/relay/analysis/analysis.py

+    ----------
+    mod : tvm.IRModule
+
+    expr_id : the Expr Number that we want to extract


Number -> ID

masahi · 2022-09-01T02:44:30Z

python/tvm/relay/analysis/analysis.py

+    --------
+    .. code-block:: python
+
+        # Supposed our module is printed like this:


masahi · 2022-09-01T02:44:46Z

python/tvm/relay/analysis/analysis.py

+
+
+def extract_intermdeiate_expr(mod, expr_id):
+    """Extract Relay Expr by the expression ID


masahi · 2022-09-01T02:45:43Z

src/relay/analysis/extract_intermediate_expr.cc

+  IRModule Extract() {
+    VisitExpr(this->mod_->Lookup("main"));
+
+    // ensure the target op num we want to dump is valid.


num -> id

dump -> extract

masahi · 2022-09-01T02:46:04Z

src/relay/analysis/extract_intermediate_expr.cc

+  using MixedModeVisitor::VisitExpr_;
+
+  const IRModule mod_;
+  /*! \brief the OP Number that we want to dump. */


OP Number -> expression ID

dump -> extract

masahi · 2022-09-01T02:46:26Z

@sisleyli More doc comments, but otherwise looks good

sisleyli · 2022-09-01T05:36:27Z

Hi @masahi, CI shows that python: i386 2 of 5 fail. But I just modified some comments compared to the previous commit. I don't know where the problem is. Could you please help me? Thank you!

masahi · 2022-09-01T06:06:45Z

@tvm-bot rerun

modify doc comments

sisleyli · 2022-09-01T10:29:07Z

Hi @masahi , I see that CI has passed. is it able to be merged, or need more work? Thank you!

[Relay] Extract Intermediate Expr by relay expr ID for analysis modify doc comments Co-authored-by: Bin Li <binli1@amd.com>

masahi reviewed Sep 1, 2022

View reviewed changes

sisleyli force-pushed the feat_dump_relay_node branch from 249d34b to 49aeb26 Compare September 1, 2022 02:41

masahi reviewed Sep 1, 2022

View reviewed changes

python/tvm/relay/analysis/analysis.py Outdated

--------

.. code-block:: python

# Supposed our module is printed like this:

Copy link

Member

masahi Sep 1, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suppose

masahi reviewed Sep 1, 2022

View reviewed changes

masahi changed the title ~~[Analysis] Add dump op by relay var number for analysis~~ [Relay] Extract intermediate node by its expression ID Sep 1, 2022

[Relay] Extract Intermediate Expr by relay expr ID for analysis

710ecfa

modify doc comments

sisleyli force-pushed the feat_dump_relay_node branch from fbc9d5a to 710ecfa Compare September 1, 2022 08:01

masahi approved these changes Sep 1, 2022

View reviewed changes

masahi merged commit 38ba8c0 into apache:main Sep 1, 2022

AndrewZhaoLuo mentioned this pull request Oct 4, 2022

TVM v0.10.0.rc0 Release Candidate Notes #12979

Closed

xinetzone pushed a commit to daobook/tvm that referenced this pull request Nov 25, 2022

[Relay] Extract intermediate node by its expression ID (apache#12646)

ae4a970

[Relay] Extract Intermediate Expr by relay expr ID for analysis modify doc comments Co-authored-by: Bin Li <binli1@amd.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Relay] Extract intermediate node by its expression ID #12646

[Relay] Extract intermediate node by its expression ID #12646

sisleyli commented Aug 30, 2022

sisleyli commented Aug 30, 2022 •

edited

masahi commented Aug 31, 2022 •

edited

sisleyli commented Aug 31, 2022

masahi commented Aug 31, 2022 •

edited

masahi commented Aug 31, 2022

sisleyli commented Aug 31, 2022

sisleyli commented Aug 31, 2022

masahi commented Aug 31, 2022 •

edited

sisleyli commented Aug 31, 2022

areusch commented Aug 31, 2022

masahi Sep 1, 2022

masahi Sep 1, 2022

masahi Sep 1, 2022

masahi Sep 1, 2022

masahi Sep 1, 2022

masahi Sep 1, 2022

masahi commented Sep 1, 2022

sisleyli commented Sep 1, 2022

masahi commented Sep 1, 2022

sisleyli commented Sep 1, 2022



		def extract_intermdeiate_expr(mod, expr_id):
		"""Extract Relay Expr by the expression ID

[Relay] Extract intermediate node by its expression ID #12646

[Relay] Extract intermediate node by its expression ID #12646

Conversation

sisleyli commented Aug 30, 2022

sisleyli commented Aug 30, 2022 • edited

masahi commented Aug 31, 2022 • edited

sisleyli commented Aug 31, 2022

masahi commented Aug 31, 2022 • edited

masahi commented Aug 31, 2022

sisleyli commented Aug 31, 2022

sisleyli commented Aug 31, 2022

masahi commented Aug 31, 2022 • edited

sisleyli commented Aug 31, 2022

areusch commented Aug 31, 2022

masahi Sep 1, 2022

Choose a reason for hiding this comment

masahi Sep 1, 2022

Choose a reason for hiding this comment

masahi Sep 1, 2022

Choose a reason for hiding this comment

masahi Sep 1, 2022

Choose a reason for hiding this comment

masahi Sep 1, 2022

Choose a reason for hiding this comment

masahi Sep 1, 2022

Choose a reason for hiding this comment

masahi commented Sep 1, 2022

sisleyli commented Sep 1, 2022

masahi commented Sep 1, 2022

sisleyli commented Sep 1, 2022

sisleyli commented Aug 30, 2022 •

edited

masahi commented Aug 31, 2022 •

edited

masahi commented Aug 31, 2022 •

edited

masahi commented Aug 31, 2022 •

edited