Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve precompilation with SnoopCompile.jl #11

Merged
merged 17 commits into from
Nov 27, 2022
Merged

Conversation

MilesCranmer
Copy link
Member

cc @mkitti. I'll do the same with SymbolicRegression.jl and then we can try to make a PySR startup image.

@MilesCranmer
Copy link
Member Author

Probably want to export test_all_combinations too so that SymbolicRegression.jl can precompile the safe operators it uses.

@coveralls
Copy link

coveralls commented Nov 23, 2022

Pull Request Test Coverage Report for Build 3558692731

  • 86 of 109 (78.9%) changed or added relevant lines in 2 files are covered.
  • 4 unchanged lines in 1 file lost coverage.
  • Overall coverage decreased (-1.3%) to 90.472%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/precompile.jl 81 104 77.88%
Files with Coverage Reduction New Missed Lines %
src/EvaluateEquation.jl 4 98.18%
Totals Coverage Status
Change from base Build 3459442572: -1.3%
Covered Lines: 959
Relevant Lines: 1060

💛 - Coveralls

@MilesCranmer
Copy link
Member Author

Looks like StaticArrays and VectorizationBase are pretty expensive to load. Maybe they should be conditional somehow.

julia> @time_imports using DynamicExpressions
      2.2 ms  ArrayInterfaceCore
      0.1 ms  IfElse
     18.3 ms  Static
      0.2 ms  Compat
      9.4 ms  ArrayInterface
      0.4 ms  Adapt
     47.9 ms  OffsetArrays
      0.5 ms  ArrayInterfaceOffsetArrays
      1.7 ms  StaticArraysCore
    387.0 ms  StaticArrays
      0.3 ms  ArrayInterfaceStaticArraysCore
      0.7 ms  ArrayInterfaceStaticArrays
      0.1 ms  SIMDTypes
      1.1 ms  ManualMemory
      2.6 ms  LayoutPointers
     10.3 ms  CPUSummary 41.10% compilation time
      0.1 ms  BitTwiddlingConvenienceFunctions
     79.8 ms  HostCPUFeatures 54.53% compilation time
    335.9 ms  VectorizationBase
      2.6 ms  SLEEFPirates
      0.1 ms  UnPack
     32.1 ms  ThreadingUtilities 76.48% compilation time
     10.9 ms  PolyesterWeave 70.63% compilation time
      1.8 ms  DocStringExtensions 56.20% compilation time
      2.5 ms  CloseOpenIntervals
      0.1 ms  SnoopPrecompile
     56.5 ms  ChainRulesCore
      4.1 ms  IrrationalConstants
      0.6 ms  DiffRules
      1.7 ms  DiffResults
     16.3 ms  Preferences
      0.8 ms  OpenLibm_jll
      0.2 ms  NaNMath
      0.5 ms  ChangesOfVariables
      0.9 ms  InverseFunctions
      0.5 ms  LogExpFunctions
      0.2 ms  JLLWrappers
      6.1 ms  CompilerSupportLibraries_jll
      2.6 ms  OpenSpecFun_jll 71.57% compilation time
     11.9 ms  SpecialFunctions
     45.6 ms  MacroTools
      0.3 ms  CommonSubexpressions
    103.2 ms  ForwardDiff
      2.1 ms  SIMDDualNumbers
    142.4 ms  LoopVectorization
      0.2 ms  TermInterface
      0.1 ms  Reexport
      0.2 ms  ExprTools
     48.4 ms  TimerOutputs
      0.2 ms  AutoHashEquals
      5.2 ms  OrderedCollections
      0.3 ms  Parameters
     54.8 ms  DataStructures
      0.6 ms  ConstructionBase
     26.2 ms  Setfield
      5.9 ms  SplittablesBase
      0.3 ms  ArgCheck
     24.2 ms  InitialValues
      0.2 ms  Requires
      0.1 ms  DataValueInterfaces
      1.0 ms  DataAPI
      0.1 ms  IteratorInterfaceExtensions
      0.1 ms  TableTraits
     11.3 ms  Tables
      0.2 ms  ZygoteRules
     67.8 ms  BangBang 68.88% compilation time (29% recompilation)
      6.9 ms  MicroCollections
      2.4 ms  Referenceables
     64.4 ms  Baselet
      0.1 ms  CompositionsBase
      0.1 ms  DefineSingletons
     70.6 ms  Transducers 63.13% compilation time (22% recompilation)
     17.1 ms  ThreadsX
     96.0 ms  Metatheory 83.54% compilation time
     13.7 ms  AbstractTrees
      4.1 ms  Combinatorics
    286.8 ms  MutableArithmetics
     61.2 ms  MultivariatePolynomials
     27.3 ms  DynamicPolynomials
      1.2 ms  Bijections
     87.8 ms  RecipesBase
    130.5 ms  FillArrays
      2.4 ms  GPUArraysCore
     23.7 ms  RecursiveArrayTools
      1.0 ms  PreallocationTools
     17.2 ms  LabelledArrays
    159.3 ms  SymbolicUtils
      0.1 ms  RealDot
     16.2 ms  StructArrays
     14.5 ms  ChainRules
     55.2 ms  IRTools
      8.2 ms  AbstractFFTs
    156.3 ms  Zygote 11.31% compilation time
    539.5 ms  DynamicExpressions

@MilesCranmer
Copy link
Member Author

Tested with PySR too; everything is a go

@MilesCranmer MilesCranmer changed the title Adds some SnoopPrecompile-ation Improve precompilation with SnoopCompile.jl Nov 27, 2022
@MilesCranmer MilesCranmer merged commit 34271cb into master Nov 27, 2022
@MilesCranmer MilesCranmer deleted the snoopcompile branch November 27, 2022 15:04
@mkitti
Copy link

mkitti commented Nov 27, 2022

You might want to use StaticArraysCore

@MilesCranmer
Copy link
Member Author

Actually StaticArrays is an indirect dependency; whatever package uses it (maybe LoopVectorization?) would have to update their end. I guess I can’t really change it from here though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants