Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

package does not precompile #101

Closed
pfarndt opened this issue Jun 9, 2023 · 17 comments
Closed

package does not precompile #101

pfarndt opened this issue Jun 9, 2023 · 17 comments

Comments

@pfarndt
Copy link

pfarndt commented Jun 9, 2023

With julia versions 1.9.0 and 1.9.1 the package does not precompile and is not usable under linux - see output below.

I tested it with julia version 1.8.5 under linux and it precompiles and runs.
It also runs on Mac (1.8.5 and 1.9.x).

Thanks for taking care.

test_FASTX) pkg> test FASTX
     Testing FASTX
      Status `/scratch/local/jl_d17NoU/Project.toml`
  [47718e42] BioGenerics v0.1.2
  [7e6ae17a] BioSequences v3.1.4
  [c2308a5c] FASTX v2.1.0 `~/.julia/dev/FASTX`
  [3372ea36] FormatSpecimens v1.1.0
  [e0db7c4e] ReTest v0.3.2
  [354b36f9] StringViews v1.3.1
  [3bb67fe8] TranscodingStreams v0.9.13
  [9a3f8284] Random `@stdlib/Random`
      Status `/scratch/local/jl_d17NoU/Manifest.toml`
  [67c07d97] Automa v0.8.2
  [47718e42] BioGenerics v0.1.2
  [7e6ae17a] BioSequences v3.1.4
  [3c28c6f8] BioSymbols v5.1.3
  [c2308a5c] FASTX v2.1.0 `~/.julia/dev/FASTX`
  [3372ea36] FormatSpecimens v1.1.0
  [bd334432] InlineTest v0.2.0
  [aea7be01] PrecompileTools v1.1.2
  [21216c6a] Preferences v1.4.0
  [e0db7c4e] ReTest v0.3.2
  [fdea26ae] SIMD v3.4.5
  [7b38b023] ScanByte v0.3.3
  [354b36f9] StringViews v1.3.1
  [3bb67fe8] TranscodingStreams v0.9.13
  [7200193e] Twiddle v1.1.2
  [0dad84c5] ArgTools v1.1.1 `@stdlib/ArgTools`
  [56f22d72] Artifacts `@stdlib/Artifacts`
  [2a0f44e3] Base64 `@stdlib/Base64`
  [ade2ca70] Dates `@stdlib/Dates`
  [8ba89e20] Distributed `@stdlib/Distributed`
  [f43a241f] Downloads v1.6.0 `@stdlib/Downloads`
  [7b1f6079] FileWatching `@stdlib/FileWatching`
  [b77e0a4c] InteractiveUtils `@stdlib/InteractiveUtils`
  [b27032c2] LibCURL v0.6.3 `@stdlib/LibCURL`
  [76f85450] LibGit2 `@stdlib/LibGit2`
  [8f399da3] Libdl `@stdlib/Libdl`
  [56ddb016] Logging `@stdlib/Logging`
  [d6f4376e] Markdown `@stdlib/Markdown`
  [ca575930] NetworkOptions v1.2.0 `@stdlib/NetworkOptions`
  [44cfe95a] Pkg v1.9.0 `@stdlib/Pkg`
  [de0858da] Printf `@stdlib/Printf`
  [3fa0cd96] REPL `@stdlib/REPL`
  [9a3f8284] Random `@stdlib/Random`
  [ea8e919c] SHA v0.7.0 `@stdlib/SHA`
  [9e88b42a] Serialization `@stdlib/Serialization`
  [6462fe0b] Sockets `@stdlib/Sockets`
  [fa267f1f] TOML v1.0.3 `@stdlib/TOML`
  [a4e569a6] Tar v1.10.0 `@stdlib/Tar`
  [8dfed614] Test `@stdlib/Test`
  [cf7118a7] UUIDs `@stdlib/UUIDs`
  [4ec0a83e] Unicode `@stdlib/Unicode`
  [deac9b47] LibCURL_jll v7.84.0+0 `@stdlib/LibCURL_jll`
  [29816b5a] LibSSH2_jll v1.10.2+0 `@stdlib/LibSSH2_jll`
  [c8ffd9c3] MbedTLS_jll v2.28.2+0 `@stdlib/MbedTLS_jll`
  [14a3606d] MozillaCACerts_jll v2022.10.11 `@stdlib/MozillaCACerts_jll`
  [83775a58] Zlib_jll v1.2.13+0 `@stdlib/Zlib_jll`
  [8e850ede] nghttp2_jll v1.48.0+0 `@stdlib/nghttp2_jll`
  [3f19e933] p7zip_jll v17.4.0+0 `@stdlib/p7zip_jll`
Precompiling project...
  ✗ FASTX
  ✗ FASTX → BioSequencesExt
  14 dependencies successfully precompiled in 35 seconds. 3 already precompiled.

ERROR: The following 1 direct dependency failed to precompile:

FASTX [c2308a5c-f048-11e8-3e8a-31650f418d12]

Failed to precompile FASTX [c2308a5c-f048-11e8-3e8a-31650f418d12] to "/home/arndt/.julia/compiled/v1.9/FASTX/jl_B443Ke".
LLVM ERROR: Do not know how to split the result of this operator!


[116426] signal (6.-6): Aborted
in expression starting at /home/arndt/.julia/dev/FASTX/src/workload.jl:3
__pthread_kill_implementation at /scratch/local/bee-buczek/glibc/glibc-2.36-1/source/nptl/pthread_kill.c:44
raise at /scratch/local/bee-buczek/glibc/glibc-2.36-1/source/signal/../sysdeps/posix/raise.c:26
abort at /scratch/local/bee-buczek/glibc/glibc-2.36-1/source/stdlib/abort.c:79
_ZN4llvm18report_fatal_errorERKNS_5TwineEb at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm18report_fatal_errorEPKcb at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm16DAGTypeLegalizer17SplitVectorResultEPNS_6SDNodeEj at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm16DAGTypeLegalizer3runEv at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm12SelectionDAG13LegalizeTypesEv at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm16SelectionDAGISel17CodeGenAndEmitDAGEv at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm16SelectionDAGISel20SelectAllBasicBlocksERKNS_8FunctionE at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm16SelectionDAGISel20runOnMachineFunctionERNS_15MachineFunctionE.part.902 at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN12_GLOBAL__N_115X86DAGToDAGISel20runOnMachineFunctionERN4llvm15MachineFunctionE at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm19MachineFunctionPass13runOnFunctionERNS_8FunctionE at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm13FPPassManager13runOnFunctionERNS_8FunctionE at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm13FPPassManager11runOnModuleERNS_6ModuleE at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm6legacy15PassManagerImpl3runERNS_6ModuleE at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm3orc14SimpleCompilerclERNS_6ModuleE at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
operator() at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/jitlayers.cpp:1206
_ZN4llvm3orc14IRCompileLayer4emitESt10unique_ptrINS0_29MaterializationResponsibilityESt14default_deleteIS3_EENS0_16ThreadSafeModuleE at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm3orc16IRTransformLayer4emitESt10unique_ptrINS0_29MaterializationResponsibilityESt14default_deleteIS3_EENS0_16ThreadSafeModuleE at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
emit at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/jitlayers.cpp:631
_ZN4llvm3orc31BasicIRLayerMaterializationUnit11materializeESt10unique_ptrINS0_29MaterializationResponsibilityESt14default_deleteIS3_EE at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm3orc19MaterializationTask3runEv at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm6detail18UniqueFunctionBaseIvJSt10unique_ptrINS_3orc4TaskESt14default_deleteIS4_EEEE8CallImplIPFvS7_EEEvPvRS7_ at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm3orc16ExecutionSession22dispatchOutstandingMUsEv at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm3orc16ExecutionSession17OL_completeLookupESt10unique_ptrINS0_21InProgressLookupStateESt14default_deleteIS3_EESt10shared_ptrINS0_23AsynchronousSymbolQueryEESt8functionIFvRKNS_8DenseMapIPNS0_8JITDylibENS_8DenseSetINS0_15SymbolStringPtrENS_12DenseMapInfoISF_vEEEENSG_ISD_vEENS_6detail12DenseMapPairISD_SI_EEEEEE at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm3orc25InProgressFullLookupState8completeESt10unique_ptrINS0_21InProgressLookupStateESt14default_deleteIS3_EE at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm3orc16ExecutionSession19OL_applyQueryPhase1ESt10unique_ptrINS0_21InProgressLookupStateESt14default_deleteIS3_EENS_5ErrorE at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm3orc16ExecutionSession6lookupENS0_10LookupKindERKSt6vectorISt4pairIPNS0_8JITDylibENS0_19JITDylibLookupFlagsEESaIS8_EENS0_15SymbolLookupSetENS0_11SymbolStateENS_15unique_functionIFvNS_8ExpectedINS_8DenseMapINS0_15SymbolStringPtrENS_18JITEvaluatedSymbolENS_12DenseMapInfoISI_vEENS_6detail12DenseMapPairISI_SJ_EEEEEEEEESt8functionIFvRKNSH_IS6_NS_8DenseSetISI_SL_EENSK_IS6_vEENSN_IS6_SV_EEEEEE at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm3orc16ExecutionSession6lookupERKSt6vectorISt4pairIPNS0_8JITDylibENS0_19JITDylibLookupFlagsEESaIS7_EERKNS0_15SymbolLookupSetENS0_10LookupKindENS0_11SymbolStateESt8functionIFvRKNS_8DenseMapIS5_NS_8DenseSetINS0_15SymbolStringPtrENS_12DenseMapInfoISK_vEEEENSL_IS5_vEENS_6detail12DenseMapPairIS5_SN_EEEEEE at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm3orc16ExecutionSession6lookupERKSt6vectorISt4pairIPNS0_8JITDylibENS0_19JITDylibLookupFlagsEESaIS7_EENS0_15SymbolStringPtrENS0_11SymbolStateE at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm3orc16ExecutionSession6lookupENS_8ArrayRefIPNS0_8JITDylibEEENS0_15SymbolStringPtrENS0_11SymbolStateE at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm3orc16ExecutionSession6lookupENS_8ArrayRefIPNS0_8JITDylibEEENS_9StringRefENS0_11SymbolStateE at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
addModule at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/jitlayers.cpp:1420
jl_add_to_ee at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/jitlayers.cpp:1815
_jl_compile_codeinst at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/jitlayers.cpp:241
jl_generate_fptr_impl at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/jitlayers.cpp:460
jl_compile_method_internal at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2348 [inlined]
jl_compile_method_internal at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2237
_jl_invoke at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2750 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2940
macro expansion at /home/arndt/.julia/dev/FASTX/src/workload.jl:37 [inlined]
macro expansion at /home/arndt/.julia/packages/PrecompileTools/0yi7r/src/workloads.jl:74 [inlined]
macro expansion at /home/arndt/.julia/dev/FASTX/src/workload.jl:9 [inlined]
top-level scope at /home/arndt/.julia/packages/PrecompileTools/0yi7r/src/workloads.jl:136
jl_toplevel_eval_flex at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/toplevel.c:903
jl_toplevel_eval_flex at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/toplevel.c:856
ijl_toplevel_eval_in at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/toplevel.c:971
eval at ./boot.jl:370 [inlined]
include_string at ./loading.jl:1899
_jl_invoke at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2758 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2940
_include at ./loading.jl:1959
include at ./Base.jl:457
jfptr_include_31033 at /project/evogen-computing/julia/julia-1.9.1/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2758 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2940
jl_apply at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/julia.h:1879 [inlined]
jl_f__call_latest at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/builtins.c:774
include at /home/arndt/.julia/dev/FASTX/src/FASTX.jl:1
unknown function (ip: 0x7fa0aa703582)
_jl_invoke at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2758 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2940
jl_apply at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/julia.h:1879 [inlined]
do_call at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/interpreter.c:126
eval_value at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/interpreter.c:226
eval_stmt_value at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/interpreter.c:177 [inlined]
eval_body at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/interpreter.c:624
jl_interpret_toplevel_thunk at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/interpreter.c:762
jl_toplevel_eval_flex at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/toplevel.c:912
jl_eval_module_expr at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/toplevel.c:203 [inlined]
jl_toplevel_eval_flex at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/toplevel.c:715
jl_toplevel_eval_flex at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/toplevel.c:856
ijl_toplevel_eval_in at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/toplevel.c:971
eval at ./boot.jl:370 [inlined]
include_string at ./loading.jl:1899
_jl_invoke at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2758 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2940
_include at ./loading.jl:1959
include at ./Base.jl:457 [inlined]
include_package_for_output at ./loading.jl:2045
jfptr_include_package_for_output_32423 at /project/evogen-computing/julia/julia-1.9.1/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2758 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2940
jl_apply at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/julia.h:1879 [inlined]
do_call at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/interpreter.c:126
eval_value at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/interpreter.c:226
eval_stmt_value at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/interpreter.c:177 [inlined]
eval_body at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/interpreter.c:624
jl_interpret_toplevel_thunk at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/interpreter.c:762
jl_toplevel_eval_flex at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/toplevel.c:912
jl_toplevel_eval_flex at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/toplevel.c:856
ijl_toplevel_eval_in at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/toplevel.c:971
eval at ./boot.jl:370 [inlined]
include_string at ./loading.jl:1899
include_string at ./loading.jl:1909 [inlined]
exec_options at ./client.jl:305
_start at ./client.jl:522
jfptr__start_29455 at /project/evogen-computing/julia/julia-1.9.1/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2758 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2940
jl_apply at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/julia.h:1879 [inlined]
true_main at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/jlapi.c:573
jl_repl_entrypoint at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/jlapi.c:717
main at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/cli/loader_exe.c:59
__libc_start_call_main at /scratch/local/bee-buczek/glibc/glibc-2.36-1/source/csu/../sysdeps/nptl/libc_start_call_main.h:58
__libc_start_main_impl at /scratch/local/bee-buczek/glibc/glibc-2.36-1/source/csu/../csu/libc-start.c:381
unknown function (ip: 0x401098)
Allocations: 9101736 (Pool: 9092749; Big: 8987); GC: 10
Stacktrace:
 [1] pkgerror(msg::String)
   @ Pkg.Types /project/evogen-computing/julia/julia-1.9.1/share/julia/stdlib/v1.9/Pkg/src/Types.jl:69
 [2] precompile(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; internal_call::Bool, strict::Bool, warn_loaded::Bool, already_instantiated::Bool, timing::Bool, kwargs::Base.Pairs{Symbol, Base.TTY, Tuple{Symbol}, NamedTuple{(:io,), Tuple{Base.TTY}}})
   @ Pkg.API /project/evogen-computing/julia/julia-1.9.1/share/julia/stdlib/v1.9/Pkg/src/API.jl:1581
 [3] precompile(pkgs::Vector{Pkg.Types.PackageSpec}; io::Base.TTY, kwargs::Base.Pairs{Symbol, Bool, Tuple{Symbol}, NamedTuple{(:warn_loaded,), Tuple{Bool}}})
   @ Pkg.API /project/evogen-computing/julia/julia-1.9.1/share/julia/stdlib/v1.9/Pkg/src/API.jl:156
 [4] precompile(; name::Nothing, uuid::Nothing, version::Nothing, url::Nothing, rev::Nothing, path::Nothing, mode::Pkg.Types.PackageMode, subdir::Nothing, kwargs::Base.Pairs{Symbol, Bool, Tuple{Symbol}, NamedTuple{(:warn_loaded,), Tuple{Bool}}})
   @ Pkg.API /project/evogen-computing/julia/julia-1.9.1/share/julia/stdlib/v1.9/Pkg/src/API.jl:171
 [5] top-level scope
   @ none:6
     Testing Precompilation of test environment failed. Continuing to tests
     Testing Running tests...
LLVM ERROR: Do not know how to split the result of this operator!


[116507] signal (6.-6): Aborted
in expression starting at /home/arndt/.julia/dev/FASTX/src/workload.jl:3
__pthread_kill_implementation at /scratch/local/bee-buczek/glibc/glibc-2.36-1/source/nptl/pthread_kill.c:44
raise at /scratch/local/bee-buczek/glibc/glibc-2.36-1/source/signal/../sysdeps/posix/raise.c:26
abort at /scratch/local/bee-buczek/glibc/glibc-2.36-1/source/stdlib/abort.c:79
_ZN4llvm18report_fatal_errorERKNS_5TwineEb at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm18report_fatal_errorEPKcb at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm16DAGTypeLegalizer17SplitVectorResultEPNS_6SDNodeEj at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm16DAGTypeLegalizer3runEv at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm12SelectionDAG13LegalizeTypesEv at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm16SelectionDAGISel17CodeGenAndEmitDAGEv at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm16SelectionDAGISel20SelectAllBasicBlocksERKNS_8FunctionE at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm16SelectionDAGISel20runOnMachineFunctionERNS_15MachineFunctionE.part.902 at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN12_GLOBAL__N_115X86DAGToDAGISel20runOnMachineFunctionERN4llvm15MachineFunctionE at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm19MachineFunctionPass13runOnFunctionERNS_8FunctionE at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm13FPPassManager13runOnFunctionERNS_8FunctionE at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm13FPPassManager11runOnModuleERNS_6ModuleE at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm6legacy15PassManagerImpl3runERNS_6ModuleE at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm3orc14SimpleCompilerclERNS_6ModuleE at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
operator() at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/jitlayers.cpp:1206
_ZN4llvm3orc14IRCompileLayer4emitESt10unique_ptrINS0_29MaterializationResponsibilityESt14default_deleteIS3_EENS0_16ThreadSafeModuleE at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm3orc16IRTransformLayer4emitESt10unique_ptrINS0_29MaterializationResponsibilityESt14default_deleteIS3_EENS0_16ThreadSafeModuleE at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
emit at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/jitlayers.cpp:631
_ZN4llvm3orc31BasicIRLayerMaterializationUnit11materializeESt10unique_ptrINS0_29MaterializationResponsibilityESt14default_deleteIS3_EE at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm3orc19MaterializationTask3runEv at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm6detail18UniqueFunctionBaseIvJSt10unique_ptrINS_3orc4TaskESt14default_deleteIS4_EEEE8CallImplIPFvS7_EEEvPvRS7_ at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm3orc16ExecutionSession22dispatchOutstandingMUsEv at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm3orc16ExecutionSession17OL_completeLookupESt10unique_ptrINS0_21InProgressLookupStateESt14default_deleteIS3_EESt10shared_ptrINS0_23AsynchronousSymbolQueryEESt8functionIFvRKNS_8DenseMapIPNS0_8JITDylibENS_8DenseSetINS0_15SymbolStringPtrENS_12DenseMapInfoISF_vEEEENSG_ISD_vEENS_6detail12DenseMapPairISD_SI_EEEEEE at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm3orc25InProgressFullLookupState8completeESt10unique_ptrINS0_21InProgressLookupStateESt14default_deleteIS3_EE at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm3orc16ExecutionSession19OL_applyQueryPhase1ESt10unique_ptrINS0_21InProgressLookupStateESt14default_deleteIS3_EENS_5ErrorE at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm3orc16ExecutionSession6lookupENS0_10LookupKindERKSt6vectorISt4pairIPNS0_8JITDylibENS0_19JITDylibLookupFlagsEESaIS8_EENS0_15SymbolLookupSetENS0_11SymbolStateENS_15unique_functionIFvNS_8ExpectedINS_8DenseMapINS0_15SymbolStringPtrENS_18JITEvaluatedSymbolENS_12DenseMapInfoISI_vEENS_6detail12DenseMapPairISI_SJ_EEEEEEEEESt8functionIFvRKNSH_IS6_NS_8DenseSetISI_SL_EENSK_IS6_vEENSN_IS6_SV_EEEEEE at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm3orc16ExecutionSession6lookupERKSt6vectorISt4pairIPNS0_8JITDylibENS0_19JITDylibLookupFlagsEESaIS7_EERKNS0_15SymbolLookupSetENS0_10LookupKindENS0_11SymbolStateESt8functionIFvRKNS_8DenseMapIS5_NS_8DenseSetINS0_15SymbolStringPtrENS_12DenseMapInfoISK_vEEEENSL_IS5_vEENS_6detail12DenseMapPairIS5_SN_EEEEEE at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm3orc16ExecutionSession6lookupERKSt6vectorISt4pairIPNS0_8JITDylibENS0_19JITDylibLookupFlagsEESaIS7_EENS0_15SymbolStringPtrENS0_11SymbolStateE at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm3orc16ExecutionSession6lookupENS_8ArrayRefIPNS0_8JITDylibEEENS0_15SymbolStringPtrENS0_11SymbolStateE at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
_ZN4llvm3orc16ExecutionSession6lookupENS_8ArrayRefIPNS0_8JITDylibEEENS_9StringRefENS0_11SymbolStateE at /project/evogen-computing/julia/julia-1.9.1/bin/../lib/julia/libLLVM-14jl.so (unknown line)
addModule at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/jitlayers.cpp:1420
jl_add_to_ee at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/jitlayers.cpp:1815
_jl_compile_codeinst at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/jitlayers.cpp:241
jl_generate_fptr_impl at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/jitlayers.cpp:460
jl_compile_method_internal at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2348 [inlined]
jl_compile_method_internal at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2237
_jl_invoke at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2750 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2940
macro expansion at /home/arndt/.julia/dev/FASTX/src/workload.jl:37 [inlined]
macro expansion at /home/arndt/.julia/packages/PrecompileTools/0yi7r/src/workloads.jl:74 [inlined]
macro expansion at /home/arndt/.julia/dev/FASTX/src/workload.jl:9 [inlined]
top-level scope at /home/arndt/.julia/packages/PrecompileTools/0yi7r/src/workloads.jl:136
jl_toplevel_eval_flex at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/toplevel.c:903
jl_toplevel_eval_flex at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/toplevel.c:856
ijl_toplevel_eval_in at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/toplevel.c:971
eval at ./boot.jl:370 [inlined]
include_string at ./loading.jl:1899
_jl_invoke at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2758 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2940
_include at ./loading.jl:1959
include at ./Base.jl:457
jfptr_include_31033 at /project/evogen-computing/julia/julia-1.9.1/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2758 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2940
jl_apply at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/julia.h:1879 [inlined]
jl_f__call_latest at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/builtins.c:774
include at /home/arndt/.julia/dev/FASTX/src/FASTX.jl:1
unknown function (ip: 0x7efdb69033b2)
_jl_invoke at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2758 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2940
jl_apply at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/julia.h:1879 [inlined]
do_call at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/interpreter.c:126
eval_value at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/interpreter.c:226
eval_stmt_value at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/interpreter.c:177 [inlined]
eval_body at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/interpreter.c:624
jl_interpret_toplevel_thunk at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/interpreter.c:762
jl_toplevel_eval_flex at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/toplevel.c:912
jl_eval_module_expr at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/toplevel.c:203 [inlined]
jl_toplevel_eval_flex at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/toplevel.c:715
jl_toplevel_eval_flex at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/toplevel.c:856
ijl_toplevel_eval_in at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/toplevel.c:971
eval at ./boot.jl:370 [inlined]
include_string at ./loading.jl:1899
_jl_invoke at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2758 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2940
_include at ./loading.jl:1959
include at ./Base.jl:457 [inlined]
include_package_for_output at ./loading.jl:2045
jfptr_include_package_for_output_32443 at /project/evogen-computing/julia/julia-1.9.1/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2758 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2940
jl_apply at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/julia.h:1879 [inlined]
do_call at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/interpreter.c:126
eval_value at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/interpreter.c:226
eval_stmt_value at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/interpreter.c:177 [inlined]
eval_body at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/interpreter.c:624
jl_interpret_toplevel_thunk at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/interpreter.c:762
jl_toplevel_eval_flex at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/toplevel.c:912
jl_toplevel_eval_flex at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/toplevel.c:856
ijl_toplevel_eval_in at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/toplevel.c:971
eval at ./boot.jl:370 [inlined]
include_string at ./loading.jl:1899
include_string at ./loading.jl:1909 [inlined]
exec_options at ./client.jl:305
_start at ./client.jl:522
jfptr__start_29455 at /project/evogen-computing/julia/julia-1.9.1/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2758 [inlined]
ijl_apply_generic at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/gf.c:2940
jl_apply at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/julia.h:1879 [inlined]
true_main at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/jlapi.c:573
jl_repl_entrypoint at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/src/jlapi.c:717
main at /cache/build/default-amdci4-6/julialang/julia-release-1-dot-9/cli/loader_exe.c:59
__libc_start_call_main at /scratch/local/bee-buczek/glibc/glibc-2.36-1/source/csu/../sysdeps/nptl/libc_start_call_main.h:58
__libc_start_main_impl at /scratch/local/bee-buczek/glibc/glibc-2.36-1/source/csu/../csu/libc-start.c:381
unknown function (ip: 0x401098)
Allocations: 9102979 (Pool: 9093983; Big: 8996); GC: 14
ERROR: LoadError: Failed to precompile FASTX [c2308a5c-f048-11e8-3e8a-31650f418d12] to "/home/arndt/.julia/compiled/v1.9/FASTX/jl_bNz5in".
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:35
  [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
    @ Base ./loading.jl:2296
  [3] compilecache
    @ ./loading.jl:2163 [inlined]
  [4] _require(pkg::Base.PkgId, env::String)
    @ Base ./loading.jl:1805
  [5] _require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:1660
  [6] macro expansion
    @ ./loading.jl:1648 [inlined]
  [7] macro expansion
    @ ./lock.jl:267 [inlined]
  [8] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1611
  [9] include(fname::String)
    @ Base.MainInclude ./client.jl:478
 [10] top-level scope
    @ ~/.julia/dev/FASTX/test/runtests.jl:8
 [11] include(fname::String)
    @ Base.MainInclude ./client.jl:478
 [12] top-level scope
    @ none:6
in expression starting at /home/arndt/.julia/dev/FASTX/test/FASTXTests.jl:1
in expression starting at /home/arndt/.julia/dev/FASTX/test/runtests.jl:8
ERROR: Package FASTX errored during testing

(test_FASTX) pkg> 
@jakobnissen
Copy link
Member

Dear @pfarndt

I've just tested it on Linux on Julia v1.9.1 where it works fine and passes all tests.
It looks like there is an issue with LLVM. Did you install Julia via a Linux distribution package manager like pamac? The people who create the Julia packages for Linux's package managers tends to ignore the installation instructions of Julia and package Julia with the wrong version of LLVM, say.
To check it, try installing Julia one of the recommended ways (try using juliaup).

@pfarndt
Copy link
Author

pfarndt commented Jun 9, 2023

I did not use any package manager. I used:

URL="https://julialang-s3.julialang.org/bin/linux/x64/1.9/julia-1.9.1-linux-x86_64.tar.gz"
wget $URL -O /tmp/julia.tar.gz

to download and then unpack everything. The last time I checked juliaup did not work properly under linux.

@jakobnissen
Copy link
Member

This appears to be an issue with Julia, not FASTX.jl. For good measure, could you show the result of versioninfo()? It's probably JuliaLang/julia#47176 or JuliaLang/julia#49653

@pfarndt
Copy link
Author

pfarndt commented Jun 9, 2023

Here my versioning.

Note: I use JULIA_CPU_TARGET = generic because I run julia on different CPUs in our cluster with a shared filesystem. Without this option I run in multiple other problems JuliaLang/julia#49829 because different CPUs precompile some code at the same time.

julia> versioninfo()
Julia Version 1.9.1
Commit 147bdf428cd (2023-06-07 08:27 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 128 × AMD EPYC 7601 32-Core Processor
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-14.0.6 (ORCJIT, znver1)
  Threads: 1 on 128 virtual cores
Environment:
  JULIA_CPU_TARGET = generic
  JULIA_REVISE_POLL = 1

@pfarndt
Copy link
Author

pfarndt commented Jun 9, 2023

I just convinced myself. It is the JULIA_CPU_TARGET = generic which cases this issue. If I unset this variable all precompiles fine.

@jakobnissen
Copy link
Member

jakobnissen commented Jun 9, 2023

Ok. This might be due to either the dependency SIMD.jl or ScanByte.jl. My money is on ScanByte.jl (both dependencies of FASTX.jl). This package will call into LLVM to detect your CPU features and compile code for that platform. It might play very poorly with JULIA_CPU_TARGET.

Unfortunately, I still can't reproduce. Can you reproduce with simply:

$ export JULIA_CPU_TARGET=generic
$ julia --project=.

(FASTX) pkg> test

? Or do you need to compile a custom sysimage? I'm not too familiar with JULIA_CPU_TARGET

@jakobnissen
Copy link
Member

Okay I need to look into this. Maybe the solution is to ditch ScanByte.jl although that would make me sad. Also, FASTX.jl currently doesn't use any of ScanByte's functionality, and the package does not precompile any methods, so it's weird this happens.
Thanks for the bug report

@pfarndt
Copy link
Author

pfarndt commented Jun 9, 2023

I am also somewhat out of my comfort zone here. Probably it also depends on the CPU type. See JuliaLang/julia#49829

@pfarndt
Copy link
Author

pfarndt commented Jun 9, 2023

I am happy to check out some commits and test them on our system.

@jakobnissen
Copy link
Member

jakobnissen commented Jun 9, 2023

Can you perhaps make a new project, adding FASTX and Automa 0.8.0. This version of Automa should work with FASTX, yet it does not have ScanByte or SIMD as a dependency. Try seeing if the test suite runs with that.

One way of doing this would be to clone FASTX.jl for development, enter the repo, then launch julia and add Automa@0.8.0, then launch julia --project=. and run pkg> test

@jakobnissen
Copy link
Member

jakobnissen commented Jun 9, 2023

The next step after that, if FASTX runs with Automa 0.8.0, would be to add Automa 0.8.0, then manually add ScanByte.jl. Then you have ScanByte, but not SIMD. Try to see if FASTX runs, then also try to run some ScanByte code:

using ScanByte
memchr(rand(UInt8, 100), Val(ByteSet(filter(i -> rand(Bool), 0x00:0xff))))

@pfarndt
Copy link
Author

pfarndt commented Jun 9, 2023

(FASTX) pkg> add Automa@0.8.0
   Resolving package versions...
    Updating `~/.julia/dev/FASTX/Project.toml`
⌃ [67c07d97] ↓ Automa v0.8.2 ⇒ v0.8.0
    Updating `~/.julia/dev/FASTX/Manifest.toml`
⌃ [67c07d97] ↓ Automa v0.8.2 ⇒ v0.8.0
  [34da2185] + Compat v4.6.1
  [864edb3b] + DataStructures v0.18.13
  [bac558e1] + OrderedCollections v1.6.0
  [cf7118a7] + UUIDs
        Info Packages marked with ⌃ have new versions available and may be upgradable.
Precompiling project...
  ✗ FASTX
  0 dependencies successfully precompiled in 8 seconds. 11 already precompiled.
  1 dependency errored. To see a full report either run `import Pkg; Pkg.precompile()` or load the package

Note however, my dev version of FASTX still has ScanByte

(FASTX) pkg> st
Project FASTX v2.1.0
Status `~/.julia/dev/FASTX/Project.toml`
⌃ [67c07d97] Automa v0.8.0
  [47718e42] BioGenerics v0.1.2
  [aea7be01] PrecompileTools v1.1.2
  [7b38b023] ScanByte v0.3.3
  [354b36f9] StringViews v1.3.1
  [3bb67fe8] TranscodingStreams v0.9.13
Info Packages marked with ⌃ have new versions available and may be upgradable.

@pfarndt
Copy link
Author

pfarndt commented Jun 13, 2023

Sorry for the delay - I added ScanByte manually and the suggested code runs - see below.

julia> versioninfo()
Julia Version 1.9.1
Commit 147bdf428cd (2023-06-07 08:27 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 128 × AMD EPYC 7601 32-Core Processor
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-14.0.6 (ORCJIT, znver1)
  Threads: 1 on 128 virtual cores
Environment:
  JULIA_CPU_TARGET = generic
  JULIA_REVISE_POLL = 1

(FASTX) pkg> st
Project FASTX v2.1.0
Status `~/.julia/dev/FASTX/Project.toml`
⌃ [67c07d97] Automa v0.8.0
  [47718e42] BioGenerics v0.1.2
  [aea7be01] PrecompileTools v1.1.2
  [7b38b023] ScanByte v0.3.3
  [354b36f9] StringViews v1.3.1
  [3bb67fe8] TranscodingStreams v0.9.13
Info Packages marked with ⌃ have new versions available and may be upgradable.

julia> using ScanByte

julia> memchr(rand(UInt8, 100), Val(ByteSet(filter(i -> rand(Bool), 0x00:0xff))))
1

FASTX itself still refuses to precompile. I hope that helps.

@jakobnissen
Copy link
Member

OK - then it is presumably SIMD.jl which bugs out, although I can't be sure.

One last thing to check is if a old version of FASTX before the use of SIMD works: For example, FASTX 1.2.0 together with Automa 0.7.0.

I'll leave this issue open until we for sure track down the cause.

@pfarndt
Copy link
Author

pfarndt commented Jun 13, 2023

As you expected FASTX 1.2.0 together with Automa 0.7.0 precompiles and runs.

(FASTX) pkg> test 
     Testing FASTX
      Status `/scratch/local/jl_FneBEp/Project.toml`
⌅ [67c07d97] Automa v0.7.0
  [47718e42] BioGenerics v0.1.2
⌅ [7e6ae17a] BioSequences v2.0.6
⌅ [3c28c6f8] BioSymbols v4.0.2
  [c2308a5c] FASTX v1.2.0 `~/tmp/test_FASTXp/FASTX.jl`
  [3372ea36] FormatSpecimens v1.1.0
  [3bb67fe8] TranscodingStreams v0.9.13
  [8dfed614] Test `@stdlib/Test`
      Status `/scratch/local/jl_FneBEp/Manifest.toml`
⌅ [67c07d97] Automa v0.7.0
  [47718e42] BioGenerics v0.1.2
⌅ [7e6ae17a] BioSequences v2.0.6
⌅ [3c28c6f8] BioSymbols v4.0.2
  [861a8166] Combinatorics v1.0.2
  [34da2185] Compat v4.6.1
  [864edb3b] DataStructures v0.18.13
  [c2308a5c] FASTX v1.2.0 `~/tmp/test_FASTXp/FASTX.jl`
  [3372ea36] FormatSpecimens v1.1.0
  [1cb3b9ac] IndexableBitVectors v1.0.0
  [bac558e1] OrderedCollections v1.6.0
⌅ [860ef19b] StableRNGs v0.1.2
  [3bb67fe8] TranscodingStreams v0.9.13
  [7200193e] Twiddle v1.1.2
  [0dad84c5] ArgTools v1.1.1 `@stdlib/ArgTools`
  [56f22d72] Artifacts `@stdlib/Artifacts`
  [2a0f44e3] Base64 `@stdlib/Base64`
  [ade2ca70] Dates `@stdlib/Dates`
  [f43a241f] Downloads v1.6.0 `@stdlib/Downloads`
  [7b1f6079] FileWatching `@stdlib/FileWatching`
  [b77e0a4c] InteractiveUtils `@stdlib/InteractiveUtils`
  [b27032c2] LibCURL v0.6.3 `@stdlib/LibCURL`
  [76f85450] LibGit2 `@stdlib/LibGit2`
  [8f399da3] Libdl `@stdlib/Libdl`
  [56ddb016] Logging `@stdlib/Logging`
  [d6f4376e] Markdown `@stdlib/Markdown`
  [ca575930] NetworkOptions v1.2.0 `@stdlib/NetworkOptions`
  [44cfe95a] Pkg v1.9.0 `@stdlib/Pkg`
  [de0858da] Printf `@stdlib/Printf`
  [3fa0cd96] REPL `@stdlib/REPL`
  [9a3f8284] Random `@stdlib/Random`
  [ea8e919c] SHA v0.7.0 `@stdlib/SHA`
  [9e88b42a] Serialization `@stdlib/Serialization`
  [6462fe0b] Sockets `@stdlib/Sockets`
  [fa267f1f] TOML v1.0.3 `@stdlib/TOML`
  [a4e569a6] Tar v1.10.0 `@stdlib/Tar`
  [8dfed614] Test `@stdlib/Test`
  [cf7118a7] UUIDs `@stdlib/UUIDs`
  [4ec0a83e] Unicode `@stdlib/Unicode`
  [deac9b47] LibCURL_jll v7.84.0+0 `@stdlib/LibCURL_jll`
  [29816b5a] LibSSH2_jll v1.10.2+0 `@stdlib/LibSSH2_jll`
  [c8ffd9c3] MbedTLS_jll v2.28.2+0 `@stdlib/MbedTLS_jll`
  [14a3606d] MozillaCACerts_jll v2022.10.11 `@stdlib/MozillaCACerts_jll`
  [83775a58] Zlib_jll v1.2.13+0 `@stdlib/Zlib_jll`
  [8e850ede] nghttp2_jll v1.48.0+0 `@stdlib/nghttp2_jll`
  [3f19e933] p7zip_jll v17.4.0+0 `@stdlib/p7zip_jll`
        Info Packages marked with ⌅ have new versions available but compatibility constraints restrict them from upgrading.
Precompiling project...
  7 dependencies successfully precompiled in 23 seconds. 10 already precompiled.
  1 dependency had warnings during precompilation:
┌ BioSequences [7e6ae17a-c86d-528c-b3b9-7f778a29fe59]
│  [ Info: Compiling bit-parallel GC counter for LongSequence{<:NucleicAcidAlphabet}
│  [ Info: Compiling bit-parallel mismatch counter for LongSequence{<:NucleicAcidAlphabet}
│  [ Info: Compiling bit-parallel match counter for LongSequence{<:NucleicAcidAlphabet}
│  [ Info: Compiling bit-parallel ambiguity counter...
│  [ Info: 	For a single LongSequence{<:NucleicAcidAlphabet}
│  [ Info: 	For a pair of LongSequence{<:NucleicAcidAlphabet}s
│  [ Info: Compiling bit-parallel certainty counter for LongSequence{<:NucleicAcidAlphabet}
│  [ Info: Compiling bit-parallel gap counter for LongSequence{<:NucleicAcidAlphabet}
└  
     Testing Running tests...
Test Summary: | Pass  Total  Time
FASTA         |  304    304  5.1s
Test Summary: | Pass  Total  Time
FASTQ         |  201    201  3.2s
Test Summary:  | Pass  Total  Time
Quality scores |   10     10  0.1s
     Testing FASTX tests passed 

(FASTX) pkg> 

MillironX added a commit to MillironX/HapLink.jl that referenced this issue Jun 27, 2023
Automa is never used within HapLink's code, HOWEVER, Automa v0.8
introduces a compilation bug and we must avoid it here at all costs.
(See BioJulia/FASTX.jl#101). In order to
manipulate Automa's compatibility resolution, add it as a proper
dependency and set its compatibility to v0.7.
MillironX added a commit to ksumngs/HapLink.jl that referenced this issue Jun 28, 2023
Automa is never used within HapLink's code, HOWEVER, Automa v0.8
introduces a compilation bug and we must avoid it here at all costs.
(See BioJulia/FASTX.jl#101). In order to
manipulate Automa's compatibility resolution, add it as a proper
dependency and set its compatibility to v0.7.
@jakobnissen
Copy link
Member

I was able to finally reproduce the problem, and fixed it in ScanByte 0.4.0. I will make a release of Automa shortly which bumps ScanByte. Could you check (in a couple of hours from now), if the problem persists?

@jakobnissen
Copy link
Member

Closing as solved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants