[EXPERIMENT] rustdoc: Cleanup `external_traits` #90365

camelid · 2021-10-28T04:27:33Z

This is preparation for cleaning up `external_traits`.

camelid · 2021-10-28T04:29:00Z

src/librustdoc/fold.rs

-        {
-            let external_traits = { std::mem::take(&mut *c.external_traits.borrow_mut()) };
-            for (k, mut v) in external_traits {
-                v.trait_.items =
-                    v.trait_.items.into_iter().filter_map(|i| self.fold_item(i)).collect();
-                c.external_traits.borrow_mut().insert(k, v);
-            }
-        }


I think this is the only part of the change that could break something. Locally, 2 tests failed, but with strange found crate std compiled by an incompatible version of rustc errors that may be unrelated. We'll see if CI passes or not.

Removing this altogether does not seem right, you're no longer running any passes on associated trait items at all. Try keeping the same logic from before but just removing the borrow_mut().

Ok, I'll try that. If it works, it'll at least be incremental progress over the current situation :)

Wait, just removing the borrow_mut won't work. The main purpose of this change is to stop storing external_traits in Crate, and thus allow it to not be a Rc<RefCell<...>>. What change are you suggesting?

Do you have access to a DocContext from within this method? If not, you can change DocFolder to add a method returning a reference to it.

Ah, I see what you mean. That's an interesting idea, I'll see if that's possible.

rust-log-analyzer · 2021-10-28T04:45:34Z

The job x86_64-gnu-llvm-12 failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

warning: `tidy` is not installed; diffs will not be generated

running 478 tests
i................................................i.................................................. 100/478
...................................................................F...............................F 200/478
.................i.................................................................i................ 400/478
..............................................................................
Some tests failed in compiletest suite=rustdoc mode=rustdoc host=x86_64-unknown-linux-gnu target=x86_64-unknown-linux-gnu
failures:
failures:

---- [rustdoc] rustdoc/intra-doc/cross-crate/traits.rs stdout ----

error: htmldocck failed!
status: exit status: 1
command: "/usr/bin/python3" "/checkout/src/etc/htmldocck.py" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/intra-doc/cross-crate/traits" "/checkout/src/test/rustdoc/intra-doc/cross-crate/traits.rs"
------------------------------------------

------------------------------------------
stderr:
stderr:
------------------------------------------
12: @has check failed
 `XPATH PATTERN` did not match
     // @has 'traits/struct.SomeStruct.html' '//a[@href="../inner/trait.SomeTrait.html"]' 'a trait'
Encountered 1 errors

------------------------------------------



---- [rustdoc] rustdoc/issue-13698.rs stdout ----

error: htmldocck failed!
status: exit status: 1
command: "/usr/bin/python3" "/checkout/src/etc/htmldocck.py" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc/issue-13698" "/checkout/src/test/rustdoc/issue-13698.rs"
------------------------------------------

------------------------------------------
stderr:
stderr:
------------------------------------------
7: @!has check failed
 `XPATH PATTERN` did not match
 // @!has issue_13698/struct.Foo.html '//*[@id="method.foo"]' 'fn foo'
Encountered 1 errors

------------------------------------------

---
test result: FAILED. 472 passed; 2 failed; 4 ignored; 0 measured; 0 filtered out; finished in 36.87s



command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--rustdoc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" "--src-base" "/checkout/src/test/rustdoc" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/rustdoc" "--stage-id" "stage2-x86_64-unknown-linux-gnu" "--suite" "rustdoc" "--mode" "rustdoc" "--target" "x86_64-unknown-linux-gnu" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/usr/lib/llvm-12/bin/FileCheck" "--nodejs" "/usr/bin/node" "--host-rustcflags" "-Crpath -O -Cdebuginfo=0  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--target-rustcflags" "-Crpath -O -Cdebuginfo=0  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--docck-python" "/usr/bin/python3" "--lldb-python" "/usr/bin/python3" "--gdb" "/usr/bin/gdb" "--quiet" "--llvm-version" "12.0.0" "--llvm-components" "aarch64 aarch64asmparser aarch64codegen aarch64desc aarch64disassembler aarch64info aarch64utils aggressiveinstcombine all all-targets amdgpu amdgpuasmparser amdgpucodegen amdgpudesc amdgpudisassembler amdgpuinfo amdgpuutils analysis arm armasmparser armcodegen armdesc armdisassembler arminfo armutils asmparser asmprinter avr avrasmparser avrcodegen avrdesc avrdisassembler avrinfo binaryformat bitreader bitstreamreader bitwriter bpf bpfasmparser bpfcodegen bpfdesc bpfdisassembler bpfinfo cfguard codegen core coroutines coverage debuginfocodeview debuginfodwarf debuginfogsym debuginfomsf debuginfopdb demangle dlltooldriver dwarflinker engine executionengine extensions filecheck frontendopenacc frontendopenmp fuzzmutate globalisel hellonew hexagon hexagonasmparser hexagoncodegen hexagondesc hexagondisassembler hexagoninfo instcombine instrumentation interfacestub interpreter ipo irreader jitlink lanai lanaiasmparser lanaicodegen lanaidesc lanaidisassembler lanaiinfo libdriver lineeditor linker lto mc mca mcdisassembler mcjit mcparser mips mipsasmparser mipscodegen mipsdesc mipsdisassembler mipsinfo mirparser msp430 msp430asmparser msp430codegen msp430desc msp430disassembler msp430info native nativecodegen nvptx nvptxcodegen nvptxdesc nvptxinfo objcarcopts object objectyaml option orcjit orcshared orctargetprocess passes perfjitevents powerpc powerpcasmparser powerpccodegen powerpcdesc powerpcdisassembler powerpcinfo profiledata remarks riscv riscvasmparser riscvcodegen riscvdesc riscvdisassembler riscvinfo runtimedyld scalaropts selectiondag sparc sparcasmparser sparccodegen sparcdesc sparcdisassembler sparcinfo support symbolize systemz systemzasmparser systemzcodegen systemzdesc systemzdisassembler systemzinfo tablegen target textapi transformutils vectorize webassembly webassemblyasmparser webassemblycodegen webassemblydesc webassemblydisassembler webassemblyinfo windowsmanifest x86 x86asmparser x86codegen x86desc x86disassembler x86info xcore xcorecodegen xcoredesc xcoredisassembler xcoreinfo xray" "--system-llvm" "--cc" "" "--cxx" "" "--cflags" "" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--channel" "nightly" "--color" "always"


Build completed unsuccessfully in 0:15:41

jyn514 · 2021-10-28T09:11:23Z

src/librustdoc/core.rs

@@ -494,7 +495,8 @@ crate fn run_global_ctxt(

    let render_options = ctxt.render_options;
    let mut cache = ctxt.cache;
-    krate = tcx.sess.time("create_format_cache", || cache.populate(krate, tcx, &render_options));
+    cache.traits = ctxt.external_traits;


Hmm, this implies ctxt.cache and ctxt.external_traits don't need to be separate fields, right? Could we store them in the cache to start so it's less stateful? Right now this depends on the field being assigned just before populate and nowhere else for correctness.

I'll try that.

jyn514 · 2021-10-28T09:12:14Z

src/librustdoc/formats/cache.rs

        tcx: TyCtxt<'_>,
+        mut krate: clean::Crate,


nit: there's no need to reorder these parameters

Yeah, though generally we pass tcx as the first argument. I can revert this change if you'd prefer.

I haven't spotted much of a pattern for this; I would rather keep the old order to reduce churn.

Ok, I'll change it back then.

jyn514 · 2021-10-28T09:18:07Z

src/librustdoc/fold.rs

-        {
-            let external_traits = { std::mem::take(&mut *c.external_traits.borrow_mut()) };
-            for (k, mut v) in external_traits {
-                v.trait_.items =
-                    v.trait_.items.into_iter().filter_map(|i| self.fold_item(i)).collect();
-                c.external_traits.borrow_mut().insert(k, v);
-            }
-        }


Removing this altogether does not seem right, you're no longer running any passes on associated trait items at all. Try keeping the same logic from before but just removing the borrow_mut().

bors · 2021-10-31T09:38:48Z

☔ The latest upstream changes (presumably #90391) made this pull request unmergeable. Please resolve the merge conflicts.

camelid · 2021-12-14T23:56:25Z

I'm going to close this for now since it's not the right approach.

jyn514 · 2021-12-16T14:41:01Z

@camelid did you get a chance to try out #90365 (comment) ? I think with that change this would be the right approach :)

camelid · 2021-12-17T02:35:17Z

Yeah, I did. The issue is that it causes multiple mutable borrows of self to be live simultaneously in the folder/visitor: one reference for the DocContext (because it's stored as a field of the folder) and one reference for the visiting of the external traits.

I don't think there's a good way to solve that problem without coming up with a different approach.

camelid added 4 commits October 27, 2021 21:11

Use DocContext to initialize Cache.traits

5082584

This is preparation for cleaning up `external_traits`.

Remove Crate.external_traits

e1b14c0

Remove Rc wrapper

d17cce4

Remove RefCell wrapper

64358a8

camelid commented Oct 28, 2021

View reviewed changes

jyn514 suggested changes Oct 28, 2021

View reviewed changes

camelid mentioned this pull request Oct 30, 2021

rustdoc: Compute some fields of clean::Crate on-demand to reduce size #90391

Merged

jyn514 self-assigned this Nov 9, 2021

camelid closed this Dec 14, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[EXPERIMENT] rustdoc: Cleanup `external_traits` #90365

[EXPERIMENT] rustdoc: Cleanup `external_traits` #90365

camelid commented Oct 28, 2021

camelid Oct 28, 2021

jyn514 Oct 28, 2021

camelid Oct 28, 2021

camelid Oct 28, 2021

jyn514 Oct 28, 2021

camelid Oct 28, 2021

rust-log-analyzer commented Oct 28, 2021

jyn514 Oct 28, 2021

camelid Oct 28, 2021

jyn514 Oct 28, 2021

camelid Oct 28, 2021

jyn514 Oct 28, 2021

camelid Oct 28, 2021

jyn514 Oct 28, 2021

bors commented Oct 31, 2021

camelid commented Dec 14, 2021

jyn514 commented Dec 16, 2021

camelid commented Dec 17, 2021

[EXPERIMENT] rustdoc: Cleanup external_traits #90365

[EXPERIMENT] rustdoc: Cleanup external_traits #90365

Conversation

camelid commented Oct 28, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rust-log-analyzer commented Oct 28, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bors commented Oct 31, 2021

camelid commented Dec 14, 2021

jyn514 commented Dec 16, 2021

camelid commented Dec 17, 2021

[EXPERIMENT] rustdoc: Cleanup `external_traits` #90365

[EXPERIMENT] rustdoc: Cleanup `external_traits` #90365