Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

too many exported symbols linker error on Windows #2346

Closed
1 task done
nipzu opened this issue Jul 23, 2023 · 17 comments · Fixed by #3601
Closed
1 task done

too many exported symbols linker error on Windows #2346

nipzu opened this issue Jul 23, 2023 · 17 comments · Fixed by #3601

Comments

@nipzu
Copy link

nipzu commented Jul 23, 2023

Prerequisites

  • Put an X between the brackets on this line if you have done all of the following:
    • Checked that your issue isn't already filed.
    • Reduced the issue to a self-contained, reproducible test case.

Description

Trying to build an executable with a sizeable dependency (mathlib) fails with a linker error.

Steps to Reproduce

  1. Create a new project using Lake
lake new lean-too-many-symbols
cd lean-too-many-symbols
  1. Add mathlib4 as a dependency
echo "import Lake
open Lake DSL

require mathlib from git
  ""https://github.com/leanprover-community/mathlib4""

package «lean-too-many-symbols»

@[default_target]
lean_exe «lean-too-many-symbols» {
  root := ``Main
}" > lakefile.lean
curl -L https://raw.githubusercontent.com/leanprover-community/mathlib4/master/lean-toolchain -o lean-toolchain
  1. Import from mathlib
echo "import Mathlib.Data.Vector

def main : IO Unit := pure ()" > Main.lean
  1. Try to build the executable
lake update
lake build

Expected behavior: I expected Lean to succesfully build an executable.

Actual behavior: The following linker error is produced:

error: > c:\Users\[me]\.elan\toolchains\leanprover--lean4---nightly-2023-07-12\bin\leanc.exe -o .\build\bin\lean-too-many-symbols.exe .\build\ir\Main.o .\lake-packages\mathlib\build\ir\Mathlib\Mathport\Rename.o .\lake-packages\std\build\ir\Std\Util\ExtendedBinder.o .\lake-packages\std\build\ir\Std\Classes\SetNotation.o .\lake-packages\std\build\ir\Std\Tactic\OpenPrivate.o .\lake-packages\std\build\ir\Std\Tactic\NoMatch.o .\lake-packages\std\build\ir\Std\Data\Option\Init\Lemmas.o .\lake-packages\std\build\ir\Std\Tactic\HaveI.o .\lake-packages\std\build\ir\Std\Tactic\Unreachable.o .\lake-packages\std\build\ir\Std\Linter\UnreachableTactic.o .\lake-packages\std\build\ir\Std\Lean\TagAttribute.o .\lake-packages\std\build\ir\Std\Lean\AttributeExtra.o .\lake-packages\std\build\ir\Std\Linter\UnnecessarySeqFocus.o .\lake-packages\std\build\ir\Std\Linter.o .\lake-packages\std\build\ir\Std\Util\TermUnsafe.o .\lake-packages\std\build\ir\Std\Tactic\GuardExpr.o .\lake-packages\std\build\ir\Std\Tactic\ByCases.o .\lake-packages\std\build\ir\Std\Tactic\SeqFocus.o .\lake-packages\std\build\ir\Std\Lean\Name.o .\lake-packages\std\build\ir\Std\Lean\Format.o .\lake-packages\std\build\ir\Std\Lean\Position.o .\lake-packages\std\build\ir\Std\Tactic\TryThis.o .\lake-packages\std\build\ir\Std\Tactic\ShowTerm.o .\lake-packages\std\build\ir\Std\Lean\Parser.o .\lake-packages\std\build\ir\Std\Tactic\SimpTrace.o .\lake-packages\std\build\ir\Std\Lean\Meta\Basic.o .\lake-packages\std\build\ir\Std\Lean\Tactic.o .\lake-packages\std\build\ir\Std\Tactic\Basic.o .\lake-packages\std\build\ir\Std\Lean\NameMapAttribute.o .\lake-packages\std\build\ir\Std\Tactic\Lint\Basic.o .\lake-packages\std\build\ir\Std\Data\List\Init\Lemmas.o .\lake-packages\std\build\ir\Std\Data\Array\Init\Basic.o .\lake-packages\std\build\ir\Std\Data\Ord.o .\lake-packages\std\build\ir\Std\Data\Array\Basic.o .\lake-packages\std\build\ir\Std\Tactic\Lint\Misc.o .\lake-packages\std\build\ir\Std\Logic.o .\lake-packages\std\build\ir\Std\Classes\LawfulMonad.o .\lake-packages\std\build\ir\Std\Data\Array\Init\Lemmas.o .\lake-packages\std\build\ir\Std\Data\List\Basic.o .\lake-packages\std\build\ir\Std\Control\ForInStep\Basic.o .\lake-packages\std\build\ir\Std\Control\ForInStep\Lemmas.o .\lake-packages\std\build\ir\Std\Data\Nat\Init\Lemmas.o .\lake-packages\std\build\ir\Std\Classes\Dvd.o .\lake-packages\std\build\ir\Std\Data\Nat\Basic.o .\lake-packages\std\build\ir\Std\Data\Nat\Lemmas.o .\lake-packages\std\build\ir\Std\Data\Option\Basic.o .\lake-packages\std\build\ir\Std\Tactic\RCases.o .\lake-packages\std\build\ir\Std\Lean\Command.o .\lake-packages\std\build\ir\Std\Tactic\Ext\Attr.o .\lake-packages\std\build\ir\Std\Data\Option\Lemmas.o .\lake-packages\std\build\ir\Std\Classes\BEq.o .\lake-packages\std\build\ir\Std\Tactic\Ext.o .\lake-packages\std\build\ir\Std\Lean\Meta\LCtx.o .\lake-packages\std\build\ir\Std\Tactic\Simpa.o .\lake-packages\std\build\ir\Std\Data\List\Lemmas.o .\lake-packages\mathlib\build\ir\Mathlib\Init\Data\Nat\Notation.o .\lake-packages\mathlib\build\ir\Mathlib\Init\Data\List\Basic.o .\lake-packages\mathlib\build\ir\Mathlib\Data\String\Defs.o .\lake-packages\mathlib\build\ir\Mathlib\Data\Array\Defs.o .\lake-packages\mathlib\build\ir\Mathlib\Data\KVMap.o .\lake-packages\std\build\ir\Std\Lean\Expr.o .\lake-packages\mathlib\build\ir\Mathlib\Lean\Expr\Basic.o .\lake-packages\mathlib\build\ir\Mathlib\Lean\Expr\Traverse.o .\lake-packages\mathlib\build\ir\Mathlib\Util\MemoFix.o .\lake-packages\mathlib\build\ir\Mathlib\Lean\Expr\ReplaceRec.o .\lake-packages\mathlib\build\ir\Mathlib\Lean\EnvExtension.o .\lake-packages\mathlib\build\ir\Mathlib\Lean\Meta\Simp.o .\lake-packages\std\build\ir\Std\Lean\Delaborator.o .\lake-packages\std\build\ir\Std\Tactic\CoeExt.o .\lake-packages\std\build\ir\Std\Util\LibraryNote.o .\lake-packages\std\build\ir\Std\Tactic\Lint\Simp.o .\lake-packages\std\build\ir\Std\Tactic\Lint\TypeClass.o .\lake-packages\std\build\ir\Std\Tactic\Lint\Frontend.o .\lake-packages\std\build\ir\Std\Tactic\Lint.o .\lake-packages\mathlib\build\ir\Mathlib\Lean\Meta.o .\lake-packages\mathlib\build\ir\Mathlib\Tactic\Relation\Rfl.o .\lake-packages\mathlib\build\ir\Mathlib\Tactic\Relation\Symm.o .\lake-packages\mathlib\build\ir\Mathlib\Lean\Elab\Tactic\Basic.o .\lake-packages\mathlib\build\ir\Mathlib\Tactic\Relation\Trans.o .\lake-packages\mathlib\build\ir\Mathlib\Lean\Expr.o .\lake-packages\mathlib\build\ir\Mathlib\Tactic\Eqns.o .\lake-packages\mathlib\build\ir\Mathlib\Lean\Message.o .\lake-packages\Qq\build\ir\Qq\ForLean\ReduceEval.o .\lake-packages\Qq\build\ir\Qq\ForLean\ToExpr.o .\lake-packages\Qq\build\ir\Qq\Typ.o .\lake-packages\Qq\build\ir\Qq\Macro.o .\lake-packages\Qq\build\ir\Qq\MetaM.o .\lake-packages\mathlib\build\ir\Mathlib\Tactic\Simps\NotationClass.o .\lake-packages\std\build\ir\Std\Data\Char.o .\lake-packages\std\build\ir\Std\Data\String\Basic.o .\lake-packages\mathlib\build\ir\Mathlib\Tactic\RunCmd.o .\lake-packages\mathlib\build\ir\Mathlib\Lean\Linter.o .\lake-packages\mathlib\build\ir\Mathlib\Tactic\Simps\Basic.o .\lake-packages\mathlib\build\ir\Mathlib\Tactic\ToAdditive.o .\lake-packages\mathlib\build\ir\Mathlib\Init\ZeroOne.o .\lake-packages\mathlib\build\ir\Mathlib\Util\CompileInductive.o .\lake-packages\mathlib\build\ir\Mathlib\Init\Data\Nat\Basic.o .\lake-packages\mathlib\build\ir\Mathlib\Init\Data\Nat\Div.o .\lake-packages\std\build\ir\Std\WF.o .\lake-packages\mathlib\build\ir\Mathlib\Tactic\Alias.o .\lake-packages\std\build\ir\Std\Classes\Cast.o .\lake-packages\std\build\ir\Std\Classes\Order.o .\lake-packages\std\build\ir\Std\Data\Nat\Gcd.o .\lake-packages\std\build\ir\Std\Data\Int\Basic.o .\lake-packages\std\build\ir\Std\Tactic\NormCast\Ext.o .\lake-packages\std\build\ir\Std\Tactic\NormCast\Lemmas.o .\lake-packages\std\build\ir\Std\Data\Int\Lemmas.o .\lake-packages\std\build\ir\Std\Data\Int\DivMod.o .\lake-packages\std\build\ir\Std\Data\Rat\Basic.o .\lake-packages\std\build\ir\Std\Classes\RatCast.o .\lake-packages\std\build\ir\Std\CodeAction\Attr.o .\lake-packages\std\build\ir\Std\Lean\InfoTree.o .\lake-packages\std\build\ir\Std\CodeAction\Basic.o .\lake-packages\std\build\ir\Std\CodeAction\Misc.o .\lake-packages\std\build\ir\Std\CodeAction.o .\lake-packages\std\build\ir\Std\Control\ForInStep.o .\lake-packages\std\build\ir\Std\Data\Array\Lemmas.o .\lake-packages\std\build\ir\Std\Data\Array\Merge.o .\lake-packages\std\build\ir\Std\Data\AssocList.o .\lake-packages\std\build\ir\Std\Data\BinomialHeap\Basic.o .\lake-packages\std\build\ir\Std\Data\BinomialHeap\Lemmas.o .\lake-packages\std\build\ir\Std\Data\BinomialHeap.o .\lake-packages\std\build\ir\Std\Data\DList.o .\lake-packages\std\build\ir\Std\Data\Fin\Basic.o .\lake-packages\std\build\ir\Std\Data\Fin\Lemmas.o .\lake-packages\std\build\ir\Std\Data\HashMap\Basic.o .\lake-packages\std\build\ir\Std\Data\HashMap\WF.o .\lake-packages\std\build\ir\Std\Data\HashMap.o .\lake-packages\std\build\ir\Std\Data\PairingHeap.o .\lake-packages\std\build\ir\Std\Data\Prod\Lex.o .\lake-packages\std\build\ir\Std\Data\RBMap\Basic.o .\lake-packages\std\build\ir\Std\Data\RBMap\WF.o .\lake-packages\std\build\ir\Std\Data\RBMap.o .\lake-packages\std\build\ir\Std\Data\RBMap\Alter.o .\lake-packages\std\build\ir\Std\Data\RBMap\Lemmas.o .\lake-packages\std\build\ir\Std\Data\Range\Lemmas.o .\lake-packages\std\build\ir\Std\Data\Rat\Lemmas.o .\lake-packages\std\build\ir\Std\Data\Rat.o .\lake-packages\std\build\ir\Std\Data\String\Lemmas.o .\lake-packages\std\build\ir\Std\Data\String.o .\lake-packages\std\build\ir\Std\Lean\HashMap.o .\lake-packages\std\build\ir\Std\Lean\HashSet.o .\lake-packages\std\build\ir\Std\Lean\Meta\AssertHypotheses.o .\lake-packages\std\build\ir\Std\Lean\Meta\Clear.o .\lake-packages\std\build\ir\Std\Lean\Meta\Expr.o .\lake-packages\std\build\ir\Std\Lean\PersistentHashMap.o .\lake-packages\std\build\ir\Std\Lean\Meta\DiscrTree.o .\lake-packages\std\build\ir\Std\Lean\Meta\Inaccessible.o .\lake-packages\std\build\ir\Std\Lean\Meta\InstantiateMVars.o .\lake-packages\std\build\ir\Std\Lean\MonadBacktrack.o .\lake-packages\std\build\ir\Std\Lean\Meta\SavedState.o .\lake-packages\std\build\ir\Std\Lean\Meta\UnusedNames.o .\lake-packages\std\build\ir\Std\Lean\PersistentHashSet.o .\lake-packages\std\build\ir\Std\Tactic\Congr.o .\lake-packages\std\build\ir\Std\Tactic\Exact.o .\lake-packages\std\build\ir\Std\Tactic\GuardMsgs.o .\lake-packages\std\build\ir\Std\Tactic\Instances.o .\lake-packages\std\build\ir\Std\Tactic\PrintDependents.o .\lake-packages\std\build\ir\Std\Tactic\SqueezeScope.o .\lake-packages\std\build\ir\Std\Tactic\Where.o .\lake-packages\std\build\ir\Std.o .\lake-packages\mathlib\build\ir\Mathlib\Tactic\Cases.o .\lake-packages\mathlib\build\ir\Mathlib\Tactic\Basic.o .\lake-packages\mathlib\build\ir\Mathlib\Mathport\Attributes.o .\lake-packages\mathlib\build\ir\Mathlib\Init\Logic.o .\lake-packages\mathlib\build\ir\Mathlib\Init\Data\Ordering\Basic.o .\lake-packages\mathlib\build\ir\Mathlib\Init\Algebra\Classes.o .\lake-packages\mathlib\build\ir\Mathlib\Tactic\Core.o .\lake-packages\mathlib\build\ir\Mathlib\Tactic\SplitIfs.o .\lake-packages\mathlib\build\ir\Mathlib\Init\Algebra\Order.o .\lake-packages\mathlib\build\ir\Mathlib\Init\Algebra\Functions.o .\lake-packages\mathlib\build\ir\Mathlib\Init\Data\Nat\Lemmas.o .\lake-packages\mathlib\build\ir\Mathlib\Init\Data\List\Lemmas.o .\lake-packages\mathlib\build\ir\Mathlib\Data\Vector.o
error: stderr:
ld.lld: error: too many exported symbols (got 68125, max 65535)
clang: error: linker command failed with exit code 1 (use -v to see invocation)
error: external command `c:\Users\[me]\.elan\toolchains\leanprover--lean4---nightly-2023-07-12\bin\leanc.exe` exited with code 1

Running the leanc command with -v, the actual ld.lld invocation is:

> c:/Users/[me]/.elan/toolchains/leanprover--lean4---nightly-2023-07-12/bin/ld.lld --sysroot=c:\\Users\\[me]\\.elan\\toolchains\\leanprover--lean4---nightly-2023-07-12 -m i386pep -Bdynamic -o .\\build\\bin\\lean-too-many-symbols.exe c:\\Users\\[me]\\.elan\\toolchains\\leanprover--lean4---nightly-2023-07-12/lib/crt2.o c:\\Users\\[me]\\.elan\\toolchains\\leanprover--lean4---nightly-2023-07-12/lib/crtbegin.o -Lc:\\Users\\[me]\\.elan\\toolchains\\leanprover--lean4---nightly-2023-07-12/lib -Lc:\\Users\\[me]\\.elan\\toolchains\\leanprover--lean4---nightly-2023-07-12\\lib\\lean -Lc:\\Users\\[me]\\.elan\\toolchains\\leanprover--lean4---nightly-2023-07-12/x86_64-w64-mingw32/lib -Lc:\\Users\\[me]\\.elan\\toolchains\\leanprover--lean4---nightly-2023-07-12/x86_64-w64-mingw32/mingw/lib -Lc:\\Users\\[me]\\.elan\\toolchains\\leanprover--lean4---nightly-2023-07-12/lib -Lc:\\Users\\[me]\\.elan\\toolchains\\leanprover--lean4---nightly-2023-07-12/x86_64-w64-mingw32/sys-root/mingw/lib -Lc:/Users/[me]/.elan/toolchains/leanprover--lean4---nightly-2023-07-12/lib/clang/15.0.1/lib/windows .\\build\\ir\\Main.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Mathport\\Rename.o .\\lake-packages\\std\\build\\ir\\Std\\Util\\ExtendedBinder.o .\\lake-packages\\std\\build\\ir\\Std\\Classes\\SetNotation.o .\\lake-packages\\std\\build\\ir\\Std\\Tactic\\OpenPrivate.o .\\lake-packages\\std\\build\\ir\\Std\\Tactic\\NoMatch.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\Option\\Init\\Lemmas.o .\\lake-packages\\std\\build\\ir\\Std\\Tactic\\HaveI.o .\\lake-packages\\std\\build\\ir\\Std\\Tactic\\Unreachable.o .\\lake-packages\\std\\build\\ir\\Std\\Linter\\UnreachableTactic.o .\\lake-packages\\std\\build\\ir\\Std\\Lean\\TagAttribute.o .\\lake-packages\\std\\build\\ir\\Std\\Lean\\AttributeExtra.o .\\lake-packages\\std\\build\\ir\\Std\\Linter\\UnnecessarySeqFocus.o .\\lake-packages\\std\\build\\ir\\Std\\Linter.o .\\lake-packages\\std\\build\\ir\\Std\\Util\\TermUnsafe.o .\\lake-packages\\std\\build\\ir\\Std\\Tactic\\GuardExpr.o .\\lake-packages\\std\\build\\ir\\Std\\Tactic\\ByCases.o .\\lake-packages\\std\\build\\ir\\Std\\Tactic\\SeqFocus.o .\\lake-packages\\std\\build\\ir\\Std\\Lean\\Name.o .\\lake-packages\\std\\build\\ir\\Std\\Lean\\Format.o .\\lake-packages\\std\\build\\ir\\Std\\Lean\\Position.o .\\lake-packages\\std\\build\\ir\\Std\\Tactic\\TryThis.o .\\lake-packages\\std\\build\\ir\\Std\\Tactic\\ShowTerm.o .\\lake-packages\\std\\build\\ir\\Std\\Lean\\Parser.o .\\lake-packages\\std\\build\\ir\\Std\\Tactic\\SimpTrace.o .\\lake-packages\\std\\build\\ir\\Std\\Lean\\Meta\\Basic.o .\\lake-packages\\std\\build\\ir\\Std\\Lean\\Tactic.o .\\lake-packages\\std\\build\\ir\\Std\\Tactic\\Basic.o .\\lake-packages\\std\\build\\ir\\Std\\Lean\\NameMapAttribute.o .\\lake-packages\\std\\build\\ir\\Std\\Tactic\\Lint\\Basic.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\List\\Init\\Lemmas.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\Array\\Init\\Basic.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\Ord.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\Array\\Basic.o .\\lake-packages\\std\\build\\ir\\Std\\Tactic\\Lint\\Misc.o .\\lake-packages\\std\\build\\ir\\Std\\Logic.o .\\lake-packages\\std\\build\\ir\\Std\\Classes\\LawfulMonad.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\Array\\Init\\Lemmas.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\List\\Basic.o .\\lake-packages\\std\\build\\ir\\Std\\Control\\ForInStep\\Basic.o .\\lake-packages\\std\\build\\ir\\Std\\Control\\ForInStep\\Lemmas.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\Nat\\Init\\Lemmas.o .\\lake-packages\\std\\build\\ir\\Std\\Classes\\Dvd.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\Nat\\Basic.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\Nat\\Lemmas.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\Option\\Basic.o .\\lake-packages\\std\\build\\ir\\Std\\Tactic\\RCases.o .\\lake-packages\\std\\build\\ir\\Std\\Lean\\Command.o .\\lake-packages\\std\\build\\ir\\Std\\Tactic\\Ext\\Attr.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\Option\\Lemmas.o .\\lake-packages\\std\\build\\ir\\Std\\Classes\\BEq.o .\\lake-packages\\std\\build\\ir\\Std\\Tactic\\Ext.o .\\lake-packages\\std\\build\\ir\\Std\\Lean\\Meta\\LCtx.o .\\lake-packages\\std\\build\\ir\\Std\\Tactic\\Simpa.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\List\\Lemmas.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Init\\Data\\Nat\\Notation.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Init\\Data\\List\\Basic.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Data\\String\\Defs.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Data\\Array\\Defs.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Data\\KVMap.o .\\lake-packages\\std\\build\\ir\\Std\\Lean\\Expr.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Lean\\Expr\\Basic.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Lean\\Expr\\Traverse.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Util\\MemoFix.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Lean\\Expr\\ReplaceRec.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Lean\\EnvExtension.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Lean\\Meta\\Simp.o .\\lake-packages\\std\\build\\ir\\Std\\Lean\\Delaborator.o .\\lake-packages\\std\\build\\ir\\Std\\Tactic\\CoeExt.o .\\lake-packages\\std\\build\\ir\\Std\\Util\\LibraryNote.o .\\lake-packages\\std\\build\\ir\\Std\\Tactic\\Lint\\Simp.o .\\lake-packages\\std\\build\\ir\\Std\\Tactic\\Lint\\TypeClass.o .\\lake-packages\\std\\build\\ir\\Std\\Tactic\\Lint\\Frontend.o .\\lake-packages\\std\\build\\ir\\Std\\Tactic\\Lint.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Lean\\Meta.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Tactic\\Relation\\Rfl.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Tactic\\Relation\\Symm.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Lean\\Elab\\Tactic\\Basic.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Tactic\\Relation\\Trans.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Lean\\Expr.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Tactic\\Eqns.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Lean\\Message.o .\\lake-packages\\Qq\\build\\ir\\Qq\\ForLean\\ReduceEval.o .\\lake-packages\\Qq\\build\\ir\\Qq\\ForLean\\ToExpr.o .\\lake-packages\\Qq\\build\\ir\\Qq\\Typ.o .\\lake-packages\\Qq\\build\\ir\\Qq\\Macro.o .\\lake-packages\\Qq\\build\\ir\\Qq\\MetaM.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Tactic\\Simps\\NotationClass.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\Char.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\String\\Basic.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Tactic\\RunCmd.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Lean\\Linter.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Tactic\\Simps\\Basic.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Tactic\\ToAdditive.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Init\\ZeroOne.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Util\\CompileInductive.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Init\\Data\\Nat\\Basic.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Init\\Data\\Nat\\Div.o .\\lake-packages\\std\\build\\ir\\Std\\WF.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Tactic\\Alias.o .\\lake-packages\\std\\build\\ir\\Std\\Classes\\Cast.o .\\lake-packages\\std\\build\\ir\\Std\\Classes\\Order.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\Nat\\Gcd.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\Int\\Basic.o .\\lake-packages\\std\\build\\ir\\Std\\Tactic\\NormCast\\Ext.o .\\lake-packages\\std\\build\\ir\\Std\\Tactic\\NormCast\\Lemmas.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\Int\\Lemmas.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\Int\\DivMod.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\Rat\\Basic.o .\\lake-packages\\std\\build\\ir\\Std\\Classes\\RatCast.o .\\lake-packages\\std\\build\\ir\\Std\\CodeAction\\Attr.o .\\lake-packages\\std\\build\\ir\\Std\\Lean\\InfoTree.o .\\lake-packages\\std\\build\\ir\\Std\\CodeAction\\Basic.o .\\lake-packages\\std\\build\\ir\\Std\\CodeAction\\Misc.o .\\lake-packages\\std\\build\\ir\\Std\\CodeAction.o .\\lake-packages\\std\\build\\ir\\Std\\Control\\ForInStep.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\Array\\Lemmas.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\Array\\Merge.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\AssocList.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\BinomialHeap\\Basic.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\BinomialHeap\\Lemmas.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\BinomialHeap.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\DList.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\Fin\\Basic.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\Fin\\Lemmas.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\HashMap\\Basic.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\HashMap\\WF.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\HashMap.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\PairingHeap.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\Prod\\Lex.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\RBMap\\Basic.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\RBMap\\WF.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\RBMap.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\RBMap\\Alter.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\RBMap\\Lemmas.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\Range\\Lemmas.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\Rat\\Lemmas.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\Rat.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\String\\Lemmas.o .\\lake-packages\\std\\build\\ir\\Std\\Data\\String.o .\\lake-packages\\std\\build\\ir\\Std\\Lean\\HashMap.o .\\lake-packages\\std\\build\\ir\\Std\\Lean\\HashSet.o .\\lake-packages\\std\\build\\ir\\Std\\Lean\\Meta\\AssertHypotheses.o .\\lake-packages\\std\\build\\ir\\Std\\Lean\\Meta\\Clear.o .\\lake-packages\\std\\build\\ir\\Std\\Lean\\Meta\\Expr.o .\\lake-packages\\std\\build\\ir\\Std\\Lean\\PersistentHashMap.o .\\lake-packages\\std\\build\\ir\\Std\\Lean\\Meta\\DiscrTree.o .\\lake-packages\\std\\build\\ir\\Std\\Lean\\Meta\\Inaccessible.o .\\lake-packages\\std\\build\\ir\\Std\\Lean\\Meta\\InstantiateMVars.o .\\lake-packages\\std\\build\\ir\\Std\\Lean\\MonadBacktrack.o .\\lake-packages\\std\\build\\ir\\Std\\Lean\\Meta\\SavedState.o .\\lake-packages\\std\\build\\ir\\Std\\Lean\\Meta\\UnusedNames.o .\\lake-packages\\std\\build\\ir\\Std\\Lean\\PersistentHashSet.o .\\lake-packages\\std\\build\\ir\\Std\\Tactic\\Congr.o .\\lake-packages\\std\\build\\ir\\Std\\Tactic\\Exact.o .\\lake-packages\\std\\build\\ir\\Std\\Tactic\\GuardMsgs.o .\\lake-packages\\std\\build\\ir\\Std\\Tactic\\Instances.o .\\lake-packages\\std\\build\\ir\\Std\\Tactic\\PrintDependents.o .\\lake-packages\\std\\build\\ir\\Std\\Tactic\\SqueezeScope.o .\\lake-packages\\std\\build\\ir\\Std\\Tactic\\Where.o .\\lake-packages\\std\\build\\ir\\Std.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Tactic\\Cases.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Tactic\\Basic.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Mathport\\Attributes.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Init\\Logic.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Init\\Data\\Ordering\\Basic.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Init\\Algebra\\Classes.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Tactic\\Core.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Tactic\\SplitIfs.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Init\\Algebra\\Order.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Init\\Algebra\\Functions.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Init\\Data\\Nat\\Lemmas.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Init\\Data\\List\\Lemmas.o .\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Data\\Vector.o -Bstatic -lgmp -lunwind -Bdynamic --start-group -lleancpp -lLean --end-group --start-group -lInit -lleanrt --end-group -lc++ -lc++abi -lgmp -lucrtbase --stack 104857600 -lm -lbcrypt -lmingw32 c:/Users/[me]/.elan/toolchains/leanprover--lean4---nightly-2023-07-12/lib/clang/15.0.1/lib/windows/libclang_rt.builtins-x86_64.a -lmoldname -lmingwex -lpthread -ladvapi32 -lshell32 -luser32 -lkernel32 -lmingw32 c:/Users/[me]/.elan/toolchains/leanprover--lean4---nightly-2023-07-12/lib/clang/15.0.1/lib/windows/libclang_rt.builtins-x86_64.a -lmoldname -lmingwex -lkernel32 c:\\Users\\[me]\\.elan\\toolchains\\leanprover--lean4---nightly-2023-07-12/lib/crtend.o
ld.lld: error: too many exported symbols (got 68125, max 65535)

Reproduces how often: This problem reproduces every time on my Windows machine. On Linux, the problem does not occur.

Versions

> elan --version
elan 2.0.0 (001eb2cfe 2023-07-03)
> lake --version
Lake version 4.1.0-pre (Lean version 4.0.0-nightly-2023-07-12)
> lean --version
Lean (version 4.0.0-nightly-2023-07-12, commit 6e904421304c, Release)

Windows 10 Home 22H2

Additional Information

The symbol limit check was added to ld.lld in https://reviews.llvm.org/D86701.

@Kha
Copy link
Member

Kha commented Jul 23, 2023

It would be nice to be able to "unexport" dllexport symbols when linking executables but I don't know if that is possible. The symbol closure might also be artificially inflated here by #467

@banbh
Copy link

banbh commented Sep 4, 2023

I get the same problem when building https://github.com/paulcadman/lean4-leetcode on a similar Windows machine with a slightly newer toolchain. Happy to provide details if they would help.

@Kha
Copy link
Member

Kha commented Sep 5, 2023

If ayone could check whether there is a simple linker flag that avoids this issue, that would be very helpful! I hope at least --strip-all would work?

@banbh
Copy link

banbh commented Sep 6, 2023

Happy to try it. How can I change the linker flags? I'm currently using lake to drive the build.

(As you can guess, I'm not very familiar with the lean toolchain.)

@Kha
Copy link
Member

Kha commented Sep 7, 2023

Thanks! If you run lake build -v, it should show you the full cmdline. You can copy and modify it then.

@banbh
Copy link

banbh commented Sep 8, 2023

lake build -v prints out the call to leanc.exe. Adding the flag you mentioned to the compiler invocation results in an error stating that clang has no such flag. How do make it pass an extra flag to the linker?

@banbh
Copy link

banbh commented Sep 8, 2023

I tried appending -Wl,--strip-all to the leanc invocation but got the same error: "ld.lld: error: too many exported symbols (got 65866, max 65535)".

@Kha
Copy link
Member

Kha commented Sep 8, 2023

Mmh, that's disappointing, but thanks for checking. I would really hope there is some other combination of flags but I'm not that familiar with MinGW linking.

@banbh
Copy link

banbh commented Sep 10, 2023

An easier way of doing what I did is to edit the lakefile, adding the following line:

  moreLinkArgs := #["-Wl,--strip-all"]

to the package block. Sadly, lake build behaves as before.

@somombo
Copy link

somombo commented Sep 20, 2023

As disscussed here on zulip this issue might also be manifesting itself as
unspecified system_category error
To reproduce this, just change import Mathlib.Data.Vector to import Mathlib or import Mathlib.Data.Real.Basic in step 3 of the original post above.

@somombo
Copy link

somombo commented Sep 20, 2023

The symbol limit check was added to ld.lld in https://reviews.llvm.org/D86701.

Looks like clang v11.1.0 was the last version that did not include this check (see commit info)

I have naively tried to run the raw clang command using clang v11.0.0 and and it still failed with:

Std\\Tactic\\Lint.o" ".\\lake-packages\\std\\build\\ir\\Std\\Tactic\\PrintDependents.o" ".\\lake-packages\\std\\build\\ir\\Std\\Tactic\\Replace.o" ".\\lake-packages\\std\\build\\ir\\Std\\Tactic\\SqueezeScope.o" ".\\lake-packages\\std\\build\\ir\\Std\\Tactic\\Where.o" ".\\lake-packages\\std\\build\\ir\\Std\\Util\\Pickle.o" ".\\lake-packages\\std\\build\\ir\\Std\\WF.o" ".\\lake-packages\\std\\build\\ir\\Std.o" ".\\lake-packages\\mathlib\\build\\ir\\Mathlib\\Tactic\\Basic.o" -Bstatic gmp.lib unwind.lib -Bdynamic --start-group leancpp.lib Lean.lib --end-group --start-group Init.lib leanrt.lib --end-group Lake.lib c++.lib c++abi.lib gmp.lib ucrtbase.lib --stack 104857600 m.lib bcrypt.lib
lld-link: warning: ignoring unknown argument '-Bstatic'
lld-link: warning: ignoring unknown argument '-Bdynamic'
lld-link: warning: ignoring unknown argument '--start-group'
lld-link: warning: ignoring unknown argument '--end-group'
lld-link: warning: ignoring unknown argument '--start-group'
lld-link: warning: ignoring unknown argument '--end-group'
lld-link: warning: ignoring unknown argument '--stack'
lld-link: error: could not open 'gmp.lib': no such file or directory
lld-link: error: could not open 'unwind.lib': no such file or directory
lld-link: error: could not open 'leancpp.lib': no such file or directory
lld-link: error: could not open 'Lean.lib': no such file or directory
lld-link: error: could not open 'Init.lib': no such file or directory
lld-link: error: could not open 'leanrt.lib': no such file or directory
lld-link: error: could not open 'Lake.lib': no such file or directory
lld-link: error: could not open 'c++.lib': no such file or directory
lld-link: error: could not open 'c++abi.lib': no such file or directory
lld-link: error: could not open 'gmp.lib': no such file or directory
lld-link: error: could not open 'ucrtbase.lib': no such file or directory
lld-link: error: could not open '104857600': no such file or directory
lld-link: error: could not open 'm.lib': no such file or directory
clang: error: linker command failed with exit code 1 (use -v to see invocation)

@digama0
Copy link
Collaborator

digama0 commented Sep 20, 2023

The check is not the problem here, it is there for a good reason: The DLL format itself does not support more than 65536 entries. If you backtrack to before the error message was added, you will just get a silently corrupted DLL file or similar.

@semorrison
Copy link
Collaborator

Just linking to another place someone has encountered this. That example doesn't even require Mathlib. :-(

@Kha
Copy link
Member

Kha commented Dec 14, 2023

We really should fix that then. I see two possibilities:

  • We do find a way to convince the linker to drop the export table for executables after all. After looking into the lld code, it really seems like there is no built-in way to do that; I opened lld option for discarding the export table llvm/llvm-project#75451 but I wouldn't expect this to be fixed by tomorrow.
  • We compile object files both with and without dllexport annotations. I assume that if this was done as a new Lake facet, we could ensure that we only create the versions actually needed for our eventual build targets, i.e. shared vs static/leanExe. /cc @tydeu

@tydeu
Copy link
Member

tydeu commented Dec 14, 2023

@Kha

I assume that if this was done as a new Lake facet, we could ensure that we only create the versions actually needed for our eventual build targets, i.e. shared vs static/leanExe. /cc @tydeu

Sounds good to me.

@awson
Copy link
Contributor

awson commented Mar 9, 2024

In my "Native Windows GHC" I've solved a similar problem.

The ONLY viable long-term solution (I've thoroughly analyzed all the previous, all failed, attempts to solve the problem) is to NOT use the standard windows symbols exporting scheme.

So I've devised my own one.

GHC case is much, MUCH more complicated than the Lean one, and I believe it won't be that hard to port the whole idea to Lean.

The idea is simple: we build our own table of Lean-generated exported symbols, and only export this single table symbol (along with standard exports from C++ code).

We also generate a stub for each Lean-generated exported symbol, and also the tiny patcher function, which puts an actual address into this stub. All these patchers are put into constructors section, and c-runtime runs them all before any of them is used.

This how static linking against such a DLL works (and I used some tricks to make a client exe link ONLY those stubs and patchers, that are used by the exe).

To make dynamic loading from such a DLL possible, we shall add some metadata, mapping each symbol name to its offset in the table.

All of this is about libleanshared.dll.

This particular issue is resolved trivially: since no object files contains any export directive, the exe doesn't try to export anything.

We also won't need #3601 and won't need separate libInit_shared.dll.

Also, I believe, it will help to solve #1148.

But having said that I wonder how you ever build Lean on Windows?

Msys2 doc looks outdated (and wrong).

I managed to configure things looking into ci scripts.

But it first failed with llvm-ar.exe: Argument list too long when building libLean.a. Well, I did this manually using response file.

Then it failed with ld.lld: error: undefined symbol: lean_get_githash. Well I linked libleanrt.a manually.

And now it finally fails with ld.lld: error: too many exported symbols (got 66386, max 65535) when linking libleanshared.dll at stage 0.

But how is it possible if we have https://github.com/leanprover/lean4-nightly/releases/download/nightly-2024-03-08/lean-4.8.0-nightly-2024-03-08-windows.tar.zst built after stage0 is regenerated?

@Kha Kha linked a pull request Mar 15, 2024 that will close this issue
@Kha
Copy link
Member

Kha commented Mar 15, 2024

@awson That is useful to know but it seems like a solution we would like to avoid if at all possible

@Kha Kha closed this as completed in #3601 Mar 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants