Fix #1786: use `_interopRequireWildcard` for import when required by an export #1829

cspotcode · 2021-06-17T14:54:40Z

Based on advice in #1826
Fixes #1786
Might also fix #1790

The ImportAnalyzer and CommonJs folders need to share a Scope. To facilitate this, I refactored each module folder into two Fold structs, for example, CommonJs and CommonJsWorker. The worker is created in fold_module, given a &mut Scope to do its work, and then dropped afterward. This let me avoid adding borrow() and deref() calls all over the place.

CLAassistant · 2021-06-17T14:54:45Z

All committers have signed the CLA.

cspotcode · 2021-06-18T22:32:21Z

I have fixed the merge and the test; everything is passing now.

kdy1

Thanks!

src/builder.rs

ecmascript/transforms/module/tests/common_js.rs

ecmascript/transforms/module/src/util.rs

ecmascript/transforms/module/src/umd.rs

ecmascript/transforms/module/src/amd.rs

ecmascript/transforms/module/src/common_js.rs

ecmascript/transforms/module/src/import_analysis.rs

ecmascript/transforms/module/src/umd.rs

cspotcode · 2021-06-19T03:29:39Z

Thanks, I should be able to push the requested changes soon. Regarding merging the visitors, it seemed very inconvenient to constantly borrow and dereference from the `RC<RefCell<>>`. This is why I opted for using two visitors. Is there a good way to avoid this? Can I change `chain()` to accept a smaller visitor factory, and then teach chain to invoke the factory and use the value it returns? This would avoid the messy borrowing refactor and also avoid creating new visitors.

…

On Fri, Jun 18, 2021, 10:42 PM 강동윤 ***@***.***> wrote: ***@***.**** requested changes on this pull request. Thanks! ------------------------------ In src/builder.rs <#1829 (comment)>: > @@ -177,18 +180,20 @@ impl<'a, 'b, P: swc_ecma_visit::Fold> PassBuilder<'a, 'b, P> { )) }; + let scope_instance: Scope = Default::default(); Can you changed this to Scope::default()? ------------------------------ In ecmascript/transforms/module/tests/common_js.rs <#1829 (comment)>: > @@ -4782,6 +4787,60 @@ test!( console.log('aFunc: ', (0, _testlibrary).aFunc(1, 2)); " ); +test!( + syntax(), + |_| { + let scope_instance: Scope = Default::default(); Please change this to Scope::default() ------------------------------ In ecmascript/transforms/module/src/util.rs <#1829 (comment)>: > @@ -78,7 +78,7 @@ impl Default for Lazy { } #[derive(Clone, Default)] -pub(super) struct Scope { If possible, please hide all fields with pub(crate) (on fields) ------------------------------ In ecmascript/transforms/module/src/umd.rs <#1829 (comment)>: > + let mut scope_ref_mut = self.scope.borrow_mut(); + let scope = scope_ref_mut.deref_mut(); + let mut umd = UmdWorker { + cm: &mut self.cm, + root_mark: self.root_mark, + in_top_level: self.in_top_level, + config: &mut self.config, + scope, + exports: &mut self.exports, + }; + umd.fold_module(module) + } +} + +struct UmdWorker<'a> { + cm: &'a mut Lrc<SourceMap>, &mut is not necessary in &mut Arc<T> / &mut Rc<T> ------------------------------ In ecmascript/transforms/module/src/amd.rs <#1829 (comment)>: > exports: Exports, } +impl Fold for Amd { Can you merge Amd and AmdWorker? Creating a new visitor is really bad for binary size. ------------------------------ In ecmascript/transforms/module/src/common_js.rs <#1829 (comment)>: > in_top_level: bool, } impl Fold for CommonJs { noop_fold_type!(); + fn fold_module(&mut self, module: Module) -> Module { Can you merge CommonJs and CommonJsWorker? Creating a new visitor is really bad for binary size. ------------------------------ In ecmascript/transforms/module/src/import_analysis.rs <#1829 (comment)>: > } -impl Fold for ImportAnalyzer { +impl Fold for ImportAnalyzerWorker<'_> { noop_fold_type!(); fn fold_module(&mut self, module: Module) -> Module { Can you merge visitors? ------------------------------ In ecmascript/transforms/module/src/umd.rs <#1829 (comment)>: > exports: Exports, } impl Fold for Umd { noop_fold_type!(); + fn fold_module(&mut self, module: Module) -> Module { Can you merge visitors? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1829 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAC35OF5NMT5EIVS2KXCIWLTTP7YHANCNFSM4633C44A> .

kdy1 · 2021-06-19T05:50:03Z

Is there a good way to avoid this?

No. As far as I know, runtime-borrows are cumbersome because they should be (in rust).

Can I change chain() to accept a smaller visitor factory, and then teach chain to invoke the factory and use the value it returns? This would avoid the messy borrowing refactor and also avoid creating new visitors.

I think you should create an alternative maco like chain_factory! instead of modifying it.

But this is a complex problem. You want to pass down RefMut or &mut instead of Rc<RefCell>, right?
If then, it would require creating a new visitor from the macro and it becomes even worse.
I already tried something similar.

This is because you can't pass down RefMut to the second one until the first one finishes. So you can't create a chained visitor with shared data.

kdy1 · 2021-06-19T15:09:45Z

I think this will conflict with #1834
I'll postpone #1834 if you are going to resolve issues soon.

cspotcode · 2021-06-19T15:11:18Z

Yes I should be able to resolve these issues this weekend, today or tomorrow.

…

On Sat, Jun 19, 2021 at 11:09 AM 강동윤 ***@***.***> wrote: I think this will conflict with #1834 <#1834> I'll postpone #1834 <#1834> if you are going to resolve issues soon. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1829 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAC35OFUEHGTNZK6TBY7NGDTTSXMLANCNFSM4633C44A> .

kdy1 · 2021-06-19T15:12:10Z

Great, thanks!

cspotcode · 2021-06-22T05:05:45Z

I was so obsessed with learning Rust, I attempted something slightly different than merging the visitors. I implemented a FoldFactory which can give you a dyn Fold when you need to run the visitor. For backwards-compatibility, every Fold also implements FoldFactory by returning a reference to itself. Chains can be a mix of FoldFactory and Fold. The chain logic has been update to expect FoldFactory and to create Folds when the visitor runs, not sooner.

This allows CommonJsFactory to be a FoldFactory, not a full visitor. When it runs, it creates a CommonJs visitor with a mutable borrow of Scope. This borrow expires when the visitor finishes, but for the entire duration, CommonJs does not need to do extra work to dereference or borrow Scope.

This change adds dyn Fold, and I'm not sure if that will be bad for binary size. Is dyn Fold adding another visitor?

I noticed that the sourcemap tests require node 12 or older. Node 14 and 16 changed their stack trace output. Should I make a PR adding this requirement to CONTRIBUTING.md?

cspotcode · 2021-06-22T05:09:12Z

Sorry, I forgot to change the other Fold visitors, I'll push that code soon...

kdy1 · 2021-06-22T05:22:04Z

I'm not sure about such change.
Can you compare the size and performance of master and your branch?

kdy1 · 2021-06-22T05:27:51Z

I looked at the code of amd pass, and it's two visitor and it's not related to dyn Fold nor FoldFactory.
Did you patch amd pass?

kdy1 · 2021-06-22T06:01:56Z

I profiled the size on mac os.

master:

21550312  6 22 14:48 ./target/release/libnode.dylib

It's 21045 KB

pr:

58555296  6 22 15:00 ./target/release/libnode.dylib

It's 57182 KB

More than triple.

I also found that this pr makes compilation much slower (14m -> 27m)
Seems reasonable as there are much more code to generate.

cspotcode · 2021-06-22T06:38:41Z

Yeah I accidentally forgot to change some of the visitors, I'm still working on that.

cspotcode · 2021-06-23T15:18:10Z

ecmascript/transforms/module/src/util.rs

-                    .entry(($i.sym.clone(), $i.span.ctxt()))
-            };
-        }
-


This macro began raising an error that I did not understand, and I could not figure out how to fix it unless I stopped using the macro. I think the error was caused by folder.scope_mut() returning a RefMut<Scope>.

cspotcode · 2021-06-23T15:23:46Z

ecmascript/transforms/module/src/util.rs

@@ -20,8 +21,8 @@ use swc_ecma_visit::{Fold, FoldWith, VisitWith};

 pub(super) trait ModulePass: Fold {
    fn config(&self) -> &Config;
-    fn scope(&self) -> &Scope;
-    fn scope_mut(&mut self) -> &mut Scope;


As far as I can tell, this change is necessary because we cannot return a &Scope without also passing ownership of the Ref that it comes from.

cspotcode · 2021-06-23T15:27:49Z

I've removed all the *Workers and FoldFactory. ImportAnalyzer and CommonJs borrow and deref the scope as necessary. Tests are passing locally, so I think this is ready for another review.

kdy1

Mostly LGMT. Thanks a lot!

kdy1 · 2021-06-24T00:40:14Z

ecmascript/transforms/module/src/amd.rs

                            });
+                            drop(scope);


Why is this required?

Without it I get this error:

error[E0502]: cannot borrow `*self` as mutable because it is also borrowed as immutable --> ecmascript/transforms/module/src/amd.rs:361:54 | 308 | let mut scope_ref_mut = self.scope.borrow_mut(); | ---------- immutable borrow occurs here ... 361 | None => Box::new(Expr::Ident(orig.clone()).fold_with(self)), | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ mutable borrow occurs here ... 409 | } | - immutable borrow might be used here, when `scope_ref_mut` is dropped and runs the destructor for type `RefMut<'_, Scope>`

As far as I understand, scope and scope_ref_mut constitute an immutable borrow of self which will not be dropped till near the bottom of this function. This conflicts with mutable borrows made by fold_with(self). But if I drop scope and scope_ref_mut earlier, there is no conflict.

kdy1 · 2021-06-24T00:40:37Z

ecmascript/transforms/module/src/amd.rs

                                            .import_types
                                            .entry(src.value.clone())
                                            .or_insert(false);
                                    }
                                }
-
+                                drop(scope);


kdy1 · 2021-06-24T00:41:21Z

ecmascript/transforms/module/src/common_js.rs

                        }
                        _ => {}
                    }
-
+                    drop(scope);


Same as above

kdy1 · 2021-06-24T00:41:35Z

ecmascript/transforms/module/src/common_js.rs

                            });

                            stmts.reserve(export.specifiers.len());

+                            drop(scope);


Same as above

kdy1

LGTM. Thanks!

cspotcode added 5 commits June 17, 2021 09:53

Initial commit

0003034

Add test

151215c

Fix code style and test

fe09a7d

cleanup

25f8719

Merge remote-tracking branch 'origin/master' into ab/fix-1786

34aa71e

cspotcode mentioned this pull request Jun 17, 2021

Doesn't export the default value when interopRequireDefault is applied #1790

Closed

kdy1 and others added 4 commits June 18, 2021 12:25

Merge branch 'master' into ab/fix-1786

874aec6

Merge branch 'master' into ab/fix-1786

baae92b

Fix git merge

cba9bb3

Fix test

9b28d06

kdy1 requested changes Jun 19, 2021

View reviewed changes

Implement pr feedback

5d0d053

kdy1 added this to the v1.2.62 milestone Jun 20, 2021

Addressing PR feedback

6e3ba94

cspotcode added 3 commits June 23, 2021 10:29

Revert FoldFactory; borrow and deref scope manually wherever needed

765fad0

Review and cleanup

bec3423

Fix import_analysis and cleanup

08ad8b4

cspotcode commented Jun 23, 2021

View reviewed changes

kdy1 reviewed Jun 24, 2021

View reviewed changes

Merge branch 'master' into ab/fix-1786

e966aff

kdy1 approved these changes Jun 24, 2021

View reviewed changes

kdy1 merged commit a31ca40 into swc-project:master Jun 24, 2021

kdy1 mentioned this pull request Jun 24, 2021

fix(es/transforms/module/common_js): fix duplicate named exports iss… #1846

Merged

swc-project locked as resolved and limited conversation to collaborators Nov 9, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix #1786: use `_interopRequireWildcard` for import when required by an export #1829

Fix #1786: use `_interopRequireWildcard` for import when required by an export #1829

cspotcode commented Jun 17, 2021 •

edited

Loading

CLAassistant commented Jun 17, 2021 •

edited

Loading

cspotcode commented Jun 18, 2021

kdy1 left a comment

cspotcode commented Jun 19, 2021 via email

kdy1 commented Jun 19, 2021

kdy1 commented Jun 19, 2021

cspotcode commented Jun 19, 2021 via email

kdy1 commented Jun 19, 2021

cspotcode commented Jun 22, 2021

cspotcode commented Jun 22, 2021

kdy1 commented Jun 22, 2021

kdy1 commented Jun 22, 2021 •

edited

Loading

kdy1 commented Jun 22, 2021

cspotcode commented Jun 22, 2021

cspotcode Jun 23, 2021

cspotcode Jun 23, 2021

cspotcode commented Jun 23, 2021

kdy1 left a comment

kdy1 Jun 24, 2021

cspotcode Jun 24, 2021

kdy1 Jun 24, 2021

kdy1 Jun 24, 2021

kdy1 Jun 24, 2021

kdy1 left a comment

Fix #1786: use _interopRequireWildcard for import when required by an export #1829

Fix #1786: use _interopRequireWildcard for import when required by an export #1829

Conversation

cspotcode commented Jun 17, 2021 • edited Loading

CLAassistant commented Jun 17, 2021 • edited Loading

cspotcode commented Jun 18, 2021

kdy1 left a comment

Choose a reason for hiding this comment

cspotcode commented Jun 19, 2021 via email

kdy1 commented Jun 19, 2021

kdy1 commented Jun 19, 2021

cspotcode commented Jun 19, 2021 via email

kdy1 commented Jun 19, 2021

cspotcode commented Jun 22, 2021

cspotcode commented Jun 22, 2021

kdy1 commented Jun 22, 2021

kdy1 commented Jun 22, 2021 • edited Loading

kdy1 commented Jun 22, 2021

cspotcode commented Jun 22, 2021

cspotcode Jun 23, 2021

Choose a reason for hiding this comment

cspotcode Jun 23, 2021

Choose a reason for hiding this comment

cspotcode commented Jun 23, 2021

kdy1 left a comment

Choose a reason for hiding this comment

kdy1 Jun 24, 2021

Choose a reason for hiding this comment

cspotcode Jun 24, 2021

Choose a reason for hiding this comment

kdy1 Jun 24, 2021

Choose a reason for hiding this comment

kdy1 Jun 24, 2021

Choose a reason for hiding this comment

kdy1 Jun 24, 2021

Choose a reason for hiding this comment

kdy1 left a comment

Choose a reason for hiding this comment

Fix #1786: use `_interopRequireWildcard` for import when required by an export #1829

Fix #1786: use `_interopRequireWildcard` for import when required by an export #1829

cspotcode commented Jun 17, 2021 •

edited

Loading

CLAassistant commented Jun 17, 2021 •

edited

Loading

kdy1 commented Jun 22, 2021 •

edited

Loading