Split REPLMode into an package extension called REPLExt

[mkitti]

This PR separates Pkg.REPLMode into an package extension called PkgREPLMode.

• REPL.jl is now a weak dependency of Pkg.jl
• Some code duplication was necessary in DocView.jl. Perhaps this could be a separate package used by Pkg.jl and REPL.jl.
• This pull request includes precompilation code for the REPL Mode as in Precompile REPLMode.repl_init, add force_compile #3711

[IanButterworth]

Suggested change

	module PkgREPLMode
	module REPLExt

I really don't like the trend to put the parent package name first and this should bear the Ext name

[IanButterworth]

Also, nit but it's a "package extension" not an "extension package"

[mkitti]

This requires coordination with JuliaLang/julia#52467

[IanButterworth]

Would this break usage like this?

julia -e 'import Pkg; pkg"add Foo"'

[mkitti]

Would this break usage like this?\n\njulia -e 'import Pkg; pkg"add Foo"'

Yes. Although I think it has been quite clear that this is not the intended use.

julia -e 'using Pkg; pkg"status"'
┌ Warning: The Pkg REPL mode is intended for interactive use only, and should not be used from scripts. It is recommended to use the functional API instead.

To make it work, they would need to load it as follows.

julia -e 'using Pkg, REPL; pkg"status"'

[mkitti]

Since it is a macro, could we just add @eval using REPL to the top of the macro?

[IanButterworth]

I think that breakage is very reasonable tbh just might come up in pkgeval

[fredrikekre]

Does anything in that code path even use REPL though? Why hinge this functionality on whether REPL is loaded?

[mkitti]

Does anything in that code path even use REPL though? Why hinge this functionality on whether REPL is loaded?

Yes, it is very dependent on the REPL.

From master,

Pkg.jl/src/REPLMode/REPLMode.jl

Lines 459 to 470 in f7f222f

	function MiniREPL()
	MiniREPL(TextDisplay(stdout_f()), REPL.Terminals.TTYTerminal(get(ENV, "TERM", Sys.iswindows() ? "" : "dumb"), stdin, stdout_f(), stderr_f()))
	end
	REPL.REPLDisplay(repl::MiniREPL) = repl.display
	const minirepl = Ref{MiniREPL}()
	__init__() = minirepl[] = MiniREPL()
	macro pkg_str(str::String)
	:($(do_cmd)(minirepl[], $str; do_rethrow=true))
	end

This branch,

Pkg.jl/ext/REPLExt/REPLExt.jl

Lines 458 to 492 in 4a784ae

	# Provide a string macro pkg"cmd" that can be used in the same way
	# as the REPLExt `pkg> cmd`. Useful for testing and in environments
	# where we do not have a REPL, e.g. IJulia.
	struct MiniREPL <: REPL.AbstractREPL
	display::TextDisplay
	t::REPL.Terminals.TTYTerminal
	end
	function MiniREPL()
	MiniREPL(TextDisplay(stdout_f()), REPL.Terminals.TTYTerminal(get(ENV, "TERM", Sys.iswindows() ? "" : "dumb"), stdin, stdout_f(), stderr_f()))
	end
	REPL.REPLDisplay(repl::MiniREPL) = repl.display
	const minirepl = Ref{MiniREPL}()
	function __init__()
	minirepl[] = MiniREPL()
	if isdefined(Base, :active_repl)
	repl_init(Base.active_repl)
	else
	atreplinit() do repl
	if isinteractive() && repl isa REPL.LineEditREPL
	isdefined(repl, :interface) || (repl.interface = REPL.setup_interface(repl))
	repl_init(repl)
	end
	end
	end
	push!(empty!(REPL.install_packages_hooks), try_prompt_pkg_add)
	end
	macro pkg_str(str::String)
	:($(do_cmd)(minirepl[], $str; do_rethrow=true))
	end
	pkgstr(str::String) = do_cmd(minirepl[], str; do_rethrow=true)

[fredrikekre]

Thats surprising, looks like it is only for the help view though, perhaps it could be refactored.

On another note, typically the entry point to the extension code is some type that you pass to the function, but in this case it will just magically work if REPL happen to be loaded, right?

[mkitti]

The pkg"" commands go through the Pkg REPL mode interface and are thus meant to take advantage of the "interactive" code. I personally think the the macro should be @pkg_cmd rather than @pkg_str but there might be historical reasons for this.

On another note, typically the entry point to the extension code is some type that you pass to the function, but in this case it will just magically work if REPL happen to be loaded, right?

Yes. There are actually hooks on both sides. I just did some refactoring in 03a7da0 to make the initialization a bit more robust.

Also there is a hook on the REPL side that needs some modification for this pull request to work.
JuliaLang/julia#52467

I kept the name PkgREPLMode in REPL because referring to REPLExt inside REPL seems confusing and silly.
https://github.com/JuliaLang/julia/blob/f115ef7d66aa8261d199e642281bf6f4e70cda0b/stdlib/REPL/src/REPL.jl#L1138-L1144

[mkitti]

Maintaining this branch as other pull requests get merged is going to be a challenge since so many files are affected.

[KristofferC]

This probably breaks the RadioMenu stuff in this file. However, I think that is more or less useless and should be removed.

[mkitti]

Is this the only section that is affected?

https://github.com/JuliaLang/Pkg.jl/blob/989b507d26b4f6fd7f28043a772c22e3d5bcfa4b/src/Types.jl#L1076C18-L1084

We could probably create some kind of hook to move that into the extension if we still need it. Perhaps the more pressing thing is developing a test to cover that part of the code.

[mkitti]

Addressed in d9138bd

[KristofferC]

Regarding the pkg"" stuff, it should be possible I think with some tweaks to make that work without a REPL. I don't think there is anything that is really needs from the REPL module. Some packages (unfortunately) use the pkg"" way to inform users how to install their package, e.g. https://github.com/bat/BAT.jl/blob/v2.0.5/docs/src/installation.md#installing-batjl-and-related-julia-packages.

Avoiding having to load the REPL for Pkg will be a very nice thing.

[KristofferC]

Subsumed by #3777