Installing xrepl should not install the web server

[lexi-lambda]

As discussed with some people at RacketCon, doing raco pkg install xrepl from a minimal installation currently installs a lot of things, including the web server, draw, typed racket, r5rs, and r6rs. I’m not sure that all of these things can be decoupled from the main documentation, but it would be nice if I could install some utilities from minimal Racket without installing these things.

[samth]

Well, raco pkg install xrepl-lib will install what you probably wanted.

More broadly, I think there are a few possible choices here:

1. The status quo.
2. racket-doc doesn't reference Typed Racket/the web server/anything that's not in a minimal install.
3. racket-doc gets split up into multiple packages, and the reference avoids having so many external links.
4. The links are changed so that they use some indirect link form that avoids a build-time dependency. This means not just using @racket[] and for-label in the relevant docs.
5. raco pkg install xrepl installs built versions of everything and thus doesn't install the build-deps.
6. Doc links automatically do some form of 4 so that xrepl-doc doesn't have a build-dep on racket-doc.

I'm actually surprised that 5 didn't just happen. But which one (or combination) are you thinking of?

[lexi-lambda]

I’m not sure what the ideal solution is, but I’d personally like to be able to have the xrepl docs installed without needing most of the main distribution to be installed, too. It also seems unlikely that it would actually be possible to decouple the guide and the reference since they’re so commonly cross-referenced, which is a good thing.

I’d personally be okay if racket-doc referenced fewer packages, but it could probably harm discoverability, and it would make the documentation worse for people who just install the main distribution anyway (which I would assume is probably the majority of Racketeers, but I could be wrong). I think the idea of a sort of “weak link” between pieces of documentation sounds like a good compromise to me, since it wouldn’t negatively impact anyone who has those pieces of documentation installed, anyway. The only significant question is how the links would render when the relevant documentation isn’t installed.

As an aside, I think (5) didn’t happen because the package server is not currently configured to distribute built versions of packages, so it always installs sources.

[mflatt]

With respect to (5): For a release, the release-specific package catalog does distribute built packages for anything in the main distribution. Still, "built" means "sources plus built artifacts", so all dependencies are installed for the "sources" part. Use --binary or even --binary-lib for a binary install from a built package, which will skip build-time dependencies.

[sorawee]

Here's what get installed if we use --binary on xrepl (I didn't use xrepl-lib since @lexi-lambda wants docs too):

 dependencies of xrepl:
   xrepl-lib
   xrepl-doc
 dependencies of xrepl-lib:
   readline-lib
   scribble-text-lib
 dependencies of xrepl-doc:
   sandbox-lib
   scribble-lib
 dependencies of scribble-text-lib:
   scheme-lib
   at-exp-lib
 dependencies of sandbox-lib:
   errortrace-lib
 dependencies of scribble-lib:
   compatibility-lib
   scribble-html-lib
   planet-lib
   net-lib
   draw-lib
   syntax-color-lib
   typed-racket-lib
 dependencies of errortrace-lib:
   source-syntax
 dependencies of planet-lib:
   srfi-lite-lib
 dependencies of draw-lib:
   draw-x86_64-macosx-3
 dependencies of syntax-color-lib:
   parser-tools-lib
   option-contract-lib
 dependencies of typed-racket-lib:
   pconvert-lib
   string-constants-lib

It is still a lot, but at least the web server and r5rs are not among the dependencies anymore.

scribble-lib has typed-racket-lib as a dependency because one file in scribble-lib is implemented using Typed Racket: https://github.com/racket/scribble/blob/master/scribble-lib/scribble/blueboxes.rkt.

[Metaxal]

A little more information.

Out of the ~1500 user packages on pkgs, more than 1100 depend indirectly on drracket, which I don't believe is right.

Even a package as small as "text-table"—meant for mere text output—which depends directly only on
'("base" "racket-doc" "rackunit-lib" "scribble-lib")
indirectly depends on 148 collections, including
draw, drracket, eli-tester, gui, icons, pict, math, web-server-lib, ...

My preferred solution would be to keep racket-doc for backward compatibility of course, but have raco setup --check-pkg-deps and raco setup --fix-pkg-deps to replace "racket-doc" with the minimum set of packages needed to fix the documentation.

Another solution is to at least provide a racket-nogui-doc, so that non-gui packages don't trigger gui dependencies.

List of packages that text-table depends on, transitively:

'("2d-lib"
  "at-exp-lib"
  "base"
  "cext-lib"
  "class-iop-lib"
  "com-win32-i386"
  "com-win32-x86_64"
  "compatibility"
  "compatibility-doc"
  "compatibility-lib"
  "compiler-lib"
  "data-doc"
  "data-enumerate-lib"
  "data-lib"
  "db-doc"
  "db-lib"
  "db-ppc-macosx"
  "db-win32-i386"
  "db-win32-x86_64"
  "db-x86_64-linux-natipkg"
  "deinprogramm-signature"
  "distributed-places"
  "distributed-places-doc"
  "distributed-places-lib"
  "draw"
  "draw-doc"
  "draw-i386-macosx-3"
  "draw-lib"
  "draw-ppc-macosx-3"
  "draw-ttf-x86_64-linux-natipkg"
  "draw-win32-i386-3"
  "draw-win32-x86_64-3"
  "draw-x11-x86_64-linux-natipkg"
  "draw-x86_64-linux-natipkg-3"
  "draw-x86_64-macosx-3"
  "drracket"
  "drracket-plugin-lib"
  "drracket-tool-doc"
  "drracket-tool-lib"
  "dynext-lib"
  "eli-tester"
  "errortrace-doc"
  "errortrace-lib"
  "future-visualizer"
  "gui"
  "gui-doc"
  "gui-i386-macosx"
  "gui-lib"
  "gui-pkg-manager-lib"
  "gui-ppc-macosx"
  "gui-win32-i386"
  "gui-win32-x86_64"
  "gui-x86_64-linux-natipkg"
  "gui-x86_64-macosx"
  "htdp-lib"
  "html-lib"
  "icons"
  "images-gui-lib"
  "images-lib"
  "macro-debugger"
  "macro-debugger-text-lib"
  "math-doc"
  "math-i386-macosx"
  "math-lib"
  "math-ppc-macosx"
  "math-win32-i386"
  "math-win32-x86_64"
  "math-x86_64-linux-natipkg"
  "math-x86_64-macosx"
  "mzscheme-doc"
  "net-cookies-doc"
  "net-cookies-lib"
  "net-doc"
  "net-lib"
  "option-contract-lib"
  "parser-tools-doc"
  "parser-tools-lib"
  "pconvert-lib"
  "pict"
  "pict-doc"
  "pict-lib"
  "pict-snip-lib"
  "plai-lib"
  "planet-doc"
  "planet-lib"
  "plot-compat"
  "plot-doc"
  "plot-gui-lib"
  "plot-lib"
  "profile-doc"
  "profile-lib"
  "quickscript"
  "r5rs-doc"
  "r5rs-lib"
  "r6rs-doc"
  "r6rs-lib"
  "racket"
  "racket-doc"
  "racket-i386-macosx-3"
  "racket-index"
  "racket-lib"
  "racket-ppc-macosx-3"
  "racket-win32-i386-3"
  "racket-win32-x86_64-3"
  "racket-x86_64-linux-natipkg-3"
  "racket-x86_64-macosx-3"
  "rackunit-doc"
  "rackunit-gui"
  "rackunit-lib"
  "rackunit-typed"
  "readline"
  "readline-doc"
  "readline-lib"
  "sandbox-lib"
  "sasl-lib"
  "scheme-lib"
  "scribble-doc"
  "scribble-html-lib"
  "scribble-lib"
  "scribble-text-lib"
  "serialize-cstruct-lib"
  "slideshow-doc"
  "slideshow-lib"
  "snip-lib"
  "source-syntax"
  "srfi-doc"
  "srfi-lib"
  "srfi-lite-lib"
  "string-constants-doc"
  "string-constants-lib"
  "syntax-color"
  "syntax-color-doc"
  "syntax-color-lib"
  "testing-util-lib"
  "tex-table"
  "trace"
  "typed-racket-compatibility"
  "typed-racket-doc"
  "typed-racket-lib"
  "typed-racket-more"
  "unix-socket-lib"
  "web-server-doc"
  "web-server-lib"
  "wxme-lib"
  "xrepl"
  "xrepl-doc"
  "xrepl-lib"
  "zo-lib")

[samth]

I think the right solution here is number 4 from my comment a while ago, the infrastructure for which is already implemented. I forget the exact scribble code to do this, but it should be relatively easy to significantly reduce the dependencies of racket-doc this way.

[Metaxal]

Yes, that would be perfect (I thought it would be no nowhere close implemented). I assume this is not document yet?

[samth]

It's documented (and has been around a while), see the #:indirect keyword for racketmodname and the #:indirect? keyword argument to seclink.

[Metaxal]

Aha! Thanks, I'll take a look. If it works well, I believe this solution should be standardized and better documented, as it's clear that almost no package uses this method.

[samth]

See also other-doc, although that should probably be made more flexible in terms of not always adding "the" and "documentation".

[mfelleisen]

This means not just using @racket[] and for-label in the relevant docs.

What kind of work does it imply for the existing docs?

[samth]

Something like this commit if it's just a reference to a separate manual.

For references that use particular identifiers, it would be more challenging.

[mfelleisen]

Even if all of us are willing to edit the core packages, this looks like significant work for the pkgs authors — more than they might be willing to tackle.

[samth]

I think people can manage their own dependencies how they want, but using indirect links is the right tool to fix this bug.

[mfelleisen]

Should we put this on the agenda?

[samth]

I don't think much further discussion is needed here; just writing some patches.

[rfindler]

Maybe one such patch would be to the docs for the section links and module links docs to suggest using weak links?

[Metaxal]

Looks like there's at least one reference I'm not seeing.

I'm defining

(define-syntax-rule (racket arg)
   (racketmodname arg #:indirect))

to make sure all rackets are indirect, but it's still complaining about The Guide and Reference:

raco setup: found undeclared dependency:
raco setup:   mode: build (of documentation)
raco setup:   for package: "text-table"
raco setup:   on package: "racket-doc"
raco setup:   from document: "text-table"
raco setup:   to document: "reference"
raco setup: found undeclared dependency:
raco setup:   mode: build (of documentation)
raco setup:   for package: "text-table"
raco setup:   on package: "racket-doc"
raco setup:   from document: "text-table"
raco setup:   to document: "guide"
raco setup: --- summary of package problems ---                    [15:28:55]
raco setup: undeclared dependency detected
raco setup:   for package: "text-table"
raco setup:   on package for build:
raco setup:    "racket-doc"

I'm also using defproc, defform, codeblock, verbatim, title and author, but that's all.

Is it one of these?

The small file can be found here.

[samth]

You need the dependency on racket-doc to link identifiers like displayln.

[Metaxal]

Can't this be made indirect? Also I'm currently using @racketmodname[(displayln (table->string args ...)) #:indirect] which I thought would have this effect.

[samth]

racketmodname links to modules; it doesn't format code. There's not currently a way to make the kind of links there indirect. Also, I believe what creates the explicit dependency is the use of (require (for-label racket/base)).

[rfindler]

From where I sit, it looks like we can probably productively trim dependencies from racket-doc to other things but it doesn't seem to make a lot of sense to have documentation that has code snippets that doesn't depend on racket-doc (without harder work anyway).

[samth]

See #3215 for some progress here.

[Metaxal]

Is there a chance that reducing the doc deps will make compiling the docs faster for a small collection?

[samth]

No, I don't expect this to change documentation build time.