testing: Make all files use lexical-binding

Mainly, add the corresponding cookie, but also add various `require`s
so that the compiler knows which vars should be trated as dynbound.
This does not fix all the warnings, but does try to eliminate
all those about "unused" variables.  For the variables truly unused,
the patch usually adds an underscore to their name to silence the warning.

Some of the fixes affect files which already used `lexical-binding`.
Not sure why the test worked before: maybe because the tests were run
without compiling them first (which could cause some of the
missing `require`d packages to be autoloaded before we got to the
problematic code, thus hiding the problem)?

I found some suspicious code, for which I added FIXMEs.

There are also a few changes to the main files.

* lisp/org-protocol.el (org-protocol-check-filename-for-protocol):
Don't call `server-edit` if it's not yet defined.  [ Needed to get
the tests to pass. ]

* lisp/ob-core.el (org-babel-temporary-directory)
(org-babel-temporary-stable-directory): Always define (and use nil
if we don't want to create a directory for it).  Simplify the code based
on the fact that (defvar V E) only evaluates E if V is not yet `boundp`.
(org-babel-temp-file, org-babel-temp-stable-file)
(org-babel-remove-temporary-directory)
(org-babel-remove-temporary-stable-directory): Adjust accordingly.

* lisp/org.el (org-log-beginning): Add FIXME.

* testing/org-test.el: Require `org` and `org-id`.
(org-id-locations-file): Don't `defconst` it.
(org-test-at-id, org-test-in-example-file, org-test-at-marker)
(org-test-with-temp-text, org-test-with-temp-text-in-file): Move edebug
specs into `declare` (and simplify them).
(org-test-with-tramp-remote-dir--worker): Declare dynbound tramp vars.
(org--compile-when): Fix quoting of `exp`.
(org-test-load): Tweak regexps.

* testing/org-batch-test-init.el: Tweak regexp, remove dead code and
add a FIXME about it.

* testing/lisp/test-ox.el: Require `ox` instead of
erroring out if it's not already loaded.  Also require `org-inlinetask`.
(org-test-with-parsed-data): Silence warnings when `info` is not used.
(test-org-export/bind-keyword): Add FIXME.

* testing/lisp/test-ox-publish.el: Require `org-test` and `ox-publish`.
(test-org-publish/resolve-external-link): Expose lambdas to
the compiler.  Remove unused var `ids`.
(test-org-publish/get-project-from-filename): Remove unused var `file`.

* testing/lisp/test-org.el: Require `org-macs`, `org`,
`org-inlinetask`, `org-refile`, and `org-agenda`.
(test-org/org-read-date): Declare `org-time-was-given` as dynbound.
(test-org/set-regexps-and-options): Add FIXME.

* testing/lisp/test-org-timer.el: Require `org-timer`.

* testing/lisp/test-org-table.el: Require `ox`.

* testing/lisp/test-org-protocol.el: Require `org-protocol` instead of
erroring out if it's not already loaded.  Also require `capture`, and
add missing `provide` statement.

* testing/lisp/test-org-pcomplete.el: Require `org`.

* testing/lisp/test-org-list.el: Require `org-list` and `org`.

* testing/lisp/test-org-lint.el: Require `org-footnote` and `org-lint`.

* testing/lisp/test-org-footnote.el: Require `org-footnote`.

* testing/lisp/test-org-element.el: Require `org-element` instead of
erroring out if it's not already loaded.  Also require `org` and
`org-inlinetask`.

* testing/lisp/test-org-duration.el: Require `org-duration`.

* testing/lisp/test-org-datetree.el: Require `org-datetree`.

* testing/lisp/test-org-colview.el: Require `org-colview`,
`org-duration`, and `org-inlinetask`.

* testing/lisp/test-org-clock.el: Require `org-duration` and `org-clock`.

* testing/lisp/test-org-archive.el: Require `org-archive`.

* testing/lisp/test-org-agenda.el
(test-org-agenda/bulk-custom-arg-func): Add FIXME.

* testing/lisp/test-ol.el: Require `ol` and `org-id`.
(test-org-link/store-link): Declare `org-store-link-props` and add FIXME.

* testing/lisp/test-oc.el (test-org-cite/export-capability): Add FIXME.

* testing/lisp/test-ob.el: Require `ob-core`, `org-src`, `ob-ref`,
and `org-table`.
(test-ob/eval-header-argument): Rename `foo` to `test-ob--foo` and
declare it as dynbound.
(test-ob/blocks-with-spaces, test-ob/specific-colnames): Add FIXME.
(test-ob/noweb-expansions-in-cache):
Declare `noweb-expansions-in-cache-var` as dynbound.

* testing/lisp/test-ob-tangle.el: Require `org` and `ob-tangle`.

* testing/lisp/test-ob-shell.el:
* testing/lisp/test-ob-python.el: Require `ob-core`.

* testing/lisp/test-ob-lob.el: Require `ob-lob`.
(temporary-value-for-test): Declare as dynbound.

* testing/lisp/test-ob-plantuml.el: Require `ob-plantuml` instead of
erroring out if it's not already loaded.
* testing/lisp/test-ob-lilypond.el: Require `ob-lilypond` instead of
erroring out if it's not already loaded.  Use `with-current-buffer`.

* testing/lisp/test-ob-julia.el: Require `ob-core`.

* testing/lisp/test-ob-java.el (org-babel-temporary-directory):
Remove dead code now that `org-babel-temporary-directory` is always bound.

* testing/lisp/test-ob-exp.el: Require `ob-exp`, `org-src`, and `org-test`.
(ob-exp/evaluate-all-executables-in-order):
Declare `*evaluation-collector*` as dynbound.

* testing/lisp/test-ob-emacs-lisp.el (ob-emacs-lisp/dynamic-lexical-edit)
(ob-emacs-lisp/dynamic-lexical-execute):
Rename dynbound var to `ob-emacs--x` and declare it as such.

* testing/lisp/test-ob-R.el: Require `ob-core`.
(ess-ask-for-ess-directory, ess-history-file): Declare vars.

lisp/ob-core.el

@@ -2850,7 +2850,7 @@ CONTEXT may be one of :tangle, :export or :eval."
 (defvar org-babel-expand-noweb-references--cache nil
   "Noweb reference cache used during expansion.")
 (defvar org-babel-expand-noweb-references--cache-buffer nil
-  "Cons (buffer . modified-tick) for cached noweb references.
+  "Cons (BUFFER . MODIFIED-TICK) for cached noweb references.
 See `org-babel-expand-noweb-references--cache'.")
 (defun org-babel-expand-noweb-references (&optional info parent-buffer)
   "Expand Noweb references in the body of the current source code block.
@@ -3194,33 +3194,25 @@ additionally processed by `shell-quote-argument'."
   (let ((f (org-babel-local-file-name (expand-file-name name))))
     (if no-quote-p f (shell-quote-argument f))))
 
-(defvar org-babel-temporary-directory)
-(unless (or noninteractive (boundp 'org-babel-temporary-directory))
-  (defvar org-babel-temporary-directory
-    (or (and (boundp 'org-babel-temporary-directory)
-	     (file-exists-p org-babel-temporary-directory)
-	     org-babel-temporary-directory)
-	(make-temp-file "babel-" t))
-    "Directory to hold temporary files created to execute code blocks.
+(defvar org-babel-temporary-directory
+  (unless noninteractive
+    (make-temp-file "babel-" t))
+  "Directory to hold temporary files created to execute code blocks.
 Used by `org-babel-temp-file'.  This directory will be removed on
-Emacs shutdown."))
-
-(defvar org-babel-temporary-stable-directory)
-(unless (or noninteractive (boundp 'org-babel-temporary-stable-directory))
-  (defvar org-babel-temporary-stable-directory
-    (or (and (boundp 'org-babel-temporary-stable-directory)
-	     (file-exists-p org-babel-temporary-stable-directory)
-	     org-babel-temporary-stable-directory)
-        (let (dir)
-          (while (or (not dir) (file-exists-p dir))
-            (setq dir (expand-file-name
-                       (format "babel-stable-%d" (random 1000))
-                       (temporary-file-directory))))
-          (make-directory dir)
-          dir))
-    "Directory to hold temporary files created to execute code blocks.
+Emacs shutdown.")
+
+(defvar org-babel-temporary-stable-directory
+  (unless noninteractive
+    (let (dir)
+      (while (or (not dir) (file-exists-p dir))
+        (setq dir (expand-file-name
+                   (format "babel-stable-%d" (random 1000))
+                   (temporary-file-directory))))
+      (make-directory dir)
+      dir))
+  "Directory to hold temporary files created to execute code blocks.
 Used by `org-babel-temp-file'.  This directory will be removed on
-Emacs shutdown."))
+Emacs shutdown.")
 
 (defcustom org-babel-remote-temporary-directory "/tmp/"
   "Directory to hold temporary files on remote hosts."
@@ -3258,7 +3250,7 @@ of `org-babel-temporary-directory'."
 		      prefix org-babel-remote-temporary-directory))))
         (make-temp-file prefix nil suffix))
     (let ((temporary-file-directory
-	   (or (and (boundp 'org-babel-temporary-directory)
+	   (or (and org-babel-temporary-directory
 		    (file-exists-p org-babel-temporary-directory)
 		    org-babel-temporary-directory)
 	       temporary-file-directory)))
@@ -3277,7 +3269,7 @@ constructed like the following: PREFIXDATAhashSUFFIX."
         (with-temp-file path)
         path)
     (let* ((temporary-file-directory
-	    (or (and (boundp 'org-babel-temporary-stable-directory)
+	    (or (and org-babel-temporary-stable-directory
 		     (file-exists-p org-babel-temporary-stable-directory)
 		     org-babel-temporary-stable-directory)
 	        temporary-file-directory))
@@ -3290,7 +3282,7 @@ constructed like the following: PREFIXDATAhashSUFFIX."
 
 (defun org-babel-remove-temporary-directory ()
   "Remove `org-babel-temporary-directory' on Emacs shutdown."
-  (when (and (boundp 'org-babel-temporary-directory)
+  (when (and org-babel-temporary-directory
 	     (file-exists-p org-babel-temporary-directory))
     ;; taken from `delete-directory' in files.el
     (condition-case nil
@@ -3307,13 +3299,12 @@ constructed like the following: PREFIXDATAhashSUFFIX."
 	  (delete-directory org-babel-temporary-directory))
       (error
        (message "Failed to remove temporary Org-babel directory %s"
-		(if (boundp 'org-babel-temporary-directory)
-		    org-babel-temporary-directory
-		  "[directory not defined]"))))))
+		(or org-babel-temporary-directory
+		    "[directory not defined]"))))))
 
 (defun org-babel-remove-temporary-stable-directory ()
   "Remove `org-babel-temporary-stable-directory' and on Emacs shutdown."
-  (when (and (boundp 'org-babel-temporary-stable-directory)
+  (when (and org-babel-temporary-stable-directory
 	     (file-exists-p org-babel-temporary-stable-directory))
     (let ((org-babel-temporary-directory
            org-babel-temporary-stable-directory))

lisp/org-protocol.el

@@ -137,7 +137,6 @@
 
 (declare-function org-publish-get-project-from-filename "ox-publish"
 		  (filename &optional up))
-(declare-function server-edit "server" (&optional arg))
 
 (defvar org-capture-link-is-already-stored)
 (defvar org-capture-templates)
@@ -671,7 +670,8 @@ CLIENT is ignored."
 		       (new-style (not (= ?: (aref (match-string 1 fname) 0)))))
                   (when (plist-get (cdr prolist) :kill-client)
 		    (message "Greedy org-protocol handler.  Killing client.")
-		    (server-edit))
+		    ;; If not fboundp, there's no client to kill.
+		    (if (fboundp 'server-edit) (server-edit)))
                   (when (fboundp func)
                     (unless greedy
                       (throw 'fname

lisp/org.el

@@ -10325,7 +10325,8 @@ narrowing."
          ;; When current headline is at the end of buffer and does not
          ;; end with trailing newline the above can move to the
          ;; beginning of the headline.
-         (when (< (point) endpos)) (goto-char endpos)))))
+         (when (< (point) endpos)) ;; FIXME: Spurious extra paren?
+         (goto-char endpos)))))
    (if (bolp) (point) (line-beginning-position 2))))
 
 (defun org-add-log-setup (&optional purpose state prev-state how extra)
@@ -14431,9 +14432,11 @@ D may be an absolute day number, or a calendar-type list (month day year)."
     (cond ((stringp result) (split-string result "; "))
 	  ((and (consp result)
 		(not (consp (cdr result)))
-		(stringp (cdr result))) (cdr result))
+		(stringp (cdr result)))
+	   (cdr result))
 	  ((and (consp result)
-		(stringp (car result))) result)
+		(stringp (car result)))
+	   result)
 	  (result entry))))
 
 (defun org-diary-to-ical-string (frombuf)

testing/examples/babel.el

@@ -1,3 +1,4 @@
+;; -*- lexical-binding: t; -*-
 (string-match-p "^#[[:digit:]]+$" "#123")
 
 ;; [[id:73115FB0-6565-442B-BB95-50195A499EF4][detangle:1]]

testing/lisp/test-ob-C.el

@@ -1,4 +1,4 @@
-;;; test-ob-C.el --- tests for ob-C.el
+;;; test-ob-C.el --- tests for ob-C.el  -*- lexical-binding: t; -*-
 
 ;; Copyright (c) 2010-2014, 2019 Sergey Litvinov, Thierry Banel
 ;; Authors: Sergey Litvinov, Thierry Banel

testing/lisp/test-ob-R.el

@@ -1,4 +1,4 @@
-;;; test-ob-R.el --- tests for ob-R.el
+;;; test-ob-R.el --- tests for ob-R.el  -*- lexical-binding: t; -*-
 
 ;; Copyright (c) 2011-2014, 2019 Eric Schulte
 ;; Authors: Eric Schulte
@@ -20,8 +20,12 @@
 
 ;;; Code:
 (org-test-for-executable "R")
+(require 'ob-core)
 (unless (featurep 'ess)
   (signal 'missing-test-dependency "ESS"))
+(defvar ess-ask-for-ess-directory)
+(defvar ess-history-file)
+
 (unless (featurep 'ob-R)
   (signal 'missing-test-dependency "Support for R code blocks"))
 
@@ -138,7 +142,7 @@ x
 4.0 * 3.5
 log(10)
 log10(10)
-(3 + 1) * 5
+\(3 + 1) * 5
 3^-1
 1/0
 #+end_src"

testing/lisp/test-ob-awk.el

@@ -1,4 +1,4 @@
-;;; test-ob-awk.el --- tests for ob-awk.el
+;;; test-ob-awk.el --- tests for ob-awk.el  -*- lexical-binding: t; -*-
 
 ;; Copyright (c) 2010-2014, 2019 Sergey Litvinov
 ;; Authors: Sergey Litvinov

testing/lisp/test-ob-clojure.el

@@ -1,4 +1,4 @@
-;;; test-ob-clojure.el
+;;; test-ob-clojure.el  -*- lexical-binding: t; -*-
 
 ;; Copyright (c) 2018-2022 Free Software Foundation, Inc.
 ;; Authors: stardiviner

testing/lisp/test-ob-emacs-lisp.el

@@ -1,4 +1,4 @@
-;;; test-ob-emacs-lisp.el
+;;; test-ob-emacs-lisp.el  -*- lexical-binding: t; -*-
 
 ;; Copyright (c) 2012-2022 Free Software Foundation, Inc.
 ;; Authors: Eric Schulte, Martyn Jago
@@ -87,22 +87,24 @@
 
     (should (string= "dynamic" (execute "
 #+begin_src emacs-lisp :lexical no :results verbatim
-(let ((x 'dynamic)) (funcall (let ((x 'lexical)) (lambda () x))))
+\(let ((x 'dynamic)) (funcall (let ((x 'lexical)) (lambda () x))))
 #+end_src")))
 
     (should (string= "lexical" (execute "
 #+begin_src emacs-lisp :lexical yes :results verbatim
-(let ((x 'dynamic)) (funcall (let ((x 'lexical)) (lambda () x))))
+\(let ((x 'dynamic)) (funcall (let ((x 'lexical)) (lambda () x))))
 #+end_src")))
 
-    (should (string= "dynamic" (let ((x 'dynamic)) (execute "
+    (defvar ob-emacs--x)
+
+    (should (string= "dynamic" (let ((ob-emacs--x 'dynamic)) (execute "
 #+begin_src emacs-lisp :lexical no :results verbatim
-x
+ob-emacs--x
 #+end_src"))))
 
-    (should (string= "lexical" (let ((x 'dynamic)) (execute "
-#+begin_src emacs-lisp :lexical '((x . lexical)) :results verbatim
-x
+    (should (string= "lexical" (let ((ob-emacs--x 'dynamic)) (execute "
+#+begin_src emacs-lisp :lexical '((ob-emacs--x . lexical)) :results verbatim
+ob-emacs--x
 #+end_src"))))
 
     ;; Src block execution uses `eval'. As of 2019-02-26, `eval' does
@@ -132,22 +134,23 @@ x
 
     (should (eq 'dynamic (execute "
 #+begin_src emacs-lisp :lexical no :results verbatim
-(let ((x 'dynamic)) (funcall (let ((x 'lexical)) (lambda () x))))
+\(let ((x 'dynamic)) (funcall (let ((x 'lexical)) (lambda () x))))
 #+end_src")))
 
     (should (eq 'lexical (execute "
 #+begin_src emacs-lisp :lexical yes :results verbatim
-(let ((x 'dynamic)) (funcall (let ((x 'lexical)) (lambda () x))))
+\(let ((x 'dynamic)) (funcall (let ((x 'lexical)) (lambda () x))))
 #+end_src")))
 
-    (should (eq 'dynamic (let ((x 'dynamic)) (execute "
+    (defvar ob-emacs--x)
+    (should (eq 'dynamic (let ((ob-emacs--x 'dynamic)) (execute "
 #+begin_src emacs-lisp :lexical no :results verbatim
-x
+ob-emacs--x
 #+end_src"))))
 
-    (should (eq 'lexical (let ((x 'dynamic)) (execute "
-#+begin_src emacs-lisp :lexical '((x . lexical)) :results verbatim
-x
+    (should (eq 'lexical (let ((ob-emacs--x 'dynamic)) (execute "
+#+begin_src emacs-lisp :lexical '((ob-emacs--x . lexical)) :results verbatim
+ob-emacs--x
 #+end_src"))))
 
     (should (equal nil (execute "

testing/lisp/test-ob-eshell.el

@@ -1,4 +1,4 @@
-;;; test-ob-eshell.el
+;;; test-ob-eshell.el  -*- lexical-binding: t; -*-
 
 ;; Copyright (c) 2018 stardiviner
 ;; Authors: stardiviner

testing/lisp/test-ob-exp.el

@@ -1,4 +1,4 @@
-;;; test-ob-exp.el
+;;; test-ob-exp.el  -*- lexical-binding: t; -*-
 
 ;; Copyright (c) 2010-2015, 2019 Eric Schulte
 ;; Authors: Eric Schulte
@@ -24,6 +24,10 @@
 
 ;;; Code:
 
+(require 'ob-exp)
+(require 'org-src)
+(require 'org-test "../testing/org-test")
+
 (defmacro org-test-with-expanded-babel-code (&rest body)
   "Execute BODY while in a buffer with all Babel code evaluated.
 Current buffer is a copy of the original buffer."
@@ -136,7 +140,7 @@ a table."
     '(property-drawer plain-list src-block fixed-width src-block plain-list)
     (org-test-at-id "5daa4d03-e3ea-46b7-b093-62c1b7632df3"
       (org-narrow-to-subtree)
-      (mapcar 'org-element-type
+      (mapcar #'org-element-type
 	      (org-element-map
 		  (org-test-with-expanded-babel-code
 		   (org-element-parse-buffer 'greater-element))
@@ -179,6 +183,7 @@ a table."
 	 nil t))))))
 
 (ert-deftest ob-exp/evaluate-all-executables-in-order ()
+  (defvar *evaluation-collector*)
   (should
    (equal '(5 4 3 2 1)
 	  (let ((org-export-use-babel t) *evaluation-collector*)

testing/lisp/test-ob-fortran.el

@@ -1,4 +1,4 @@
-;;; test-ob-fortran.el --- tests for ob-fortran.el
+;;; test-ob-fortran.el --- tests for ob-fortran.el  -*- lexical-binding: t; -*-
 
 ;; Copyright (c) 2010-2014, 2019 Sergey Litvinov
 ;; Authors: Sergey Litvinov

testing/lisp/test-ob-header-arg-defaults.el

@@ -1,4 +1,4 @@
-;;; test-ob-header-arg-defaults.el --- tests for default header args from properties
+;;; test-ob-header-arg-defaults.el --- tests for default header args from properties  -*- lexical-binding: t; -*-
 
 ;; Copyright (c) 2013, 2014, 2019 Achim Gratz
 ;; Authors: Achim Gratz

testing/lisp/test-ob-java.el

@@ -1,4 +1,4 @@
-;;; test-ob-java.el --- tests for ob-java.el
+;;; test-ob-java.el --- tests for ob-java.el  -*- lexical-binding: t; -*-
 
 ;; Copyright (c) 2020-2022 Free Software Foundation, Inc.
 ;; Authors: Eric Schulte
@@ -22,12 +22,7 @@
 
 ;;; Code:
 (require 'org-test "../testing/org-test")
-
 (require 'ob-core)
-(defvar org-babel-temporary-directory ; from ob-core
-  (if (boundp 'org-babel-temporary-directory)
-    org-babel-temporary-directory
-  (temporary-file-directory)))
 
 (org-test-for-executable "java")
 (org-test-for-executable "javac")

testing/lisp/test-ob-julia.el

@@ -1,4 +1,4 @@
-;;; test-ob-python.el --- tests for ob-python.el
+;;; test-ob-python.el --- tests for ob-python.el  -*- lexical-binding: t; -*-
 
 ;; Copyright (c) 2011-2014, 2019, 2021 Eric Schulte
 ;; Authors: Pedro Bruel, based on test-ob-python.el by Eric Schulte
@@ -21,6 +21,7 @@
 
 ;;; Code:
 (org-test-for-executable "julia")
+(require 'ob-core)
 (unless (featurep 'ob-julia)
   (signal 'missing-test-dependency "Support for julia code blocks"))
 

testing/lisp/test-ob-lilypond.el

@@ -1,4 +1,4 @@
-;;; test-ob-lilypond.el --- tests for ob-lilypond.el
+;;; test-ob-lilypond.el --- tests for ob-lilypond.el  -*- lexical-binding: t; -*-
 
 ;; Copyright (c) 2010-2014, 2019 Martyn Jago
 ;; Authors: Martyn Jago
@@ -19,11 +19,9 @@
 ;; along with this program.  If not, see <https://www.gnu.org/licenses/>.
 
 ;;; Code:
-(unless (featurep 'ob-lilypond)
-  (signal 'missing-test-dependency "Support for Lilypond code blocks"))
+(require 'ob-lilypond)
 
-(save-excursion
-  (set-buffer (get-buffer-create "test-ob-lilypond.el"))
+(with-current-buffer (get-buffer-create "test-ob-lilypond.el")
   (setq org-babel-lilypond-here
         (file-name-directory
          (or load-file-name (buffer-file-name)))))

testing/lisp/test-ob-lob.el

@@ -1,4 +1,4 @@
-;;; test-ob-lob.el --- test for ob-lob.el
+;;; test-ob-lob.el --- test for ob-lob.el  -*- lexical-binding: t; -*-
 
 ;; Copyright (c) 2010-2015, 2019 Eric Schulte
 ;; Authors: Eric Schulte
@@ -19,6 +19,7 @@
 ;; along with this program.  If not, see <https://www.gnu.org/licenses/>.
 
 (eval-and-compile (require 'cl-lib))
+(require 'ob-lob)
 
 
 ;;; Tests
@@ -135,6 +136,8 @@ for export
 #+end_example"
     (should (progn (org-babel-exp-process-buffer) t))))
 
+(defvar temporary-value-for-test)
+
 (ert-deftest test-ob-lob/caching-call-line ()
   (let ((temporary-value-for-test 0))
     (org-test-with-temp-text "

testing/lisp/test-ob-lua.el

@@ -1,4 +1,4 @@
-;;; test-ob-lua.el --- tests for ob-lua.el
+;;; test-ob-lua.el --- tests for ob-lua.el  -*- lexical-binding: t; -*-
 
 ;; Copyright (c) 2016, 2019 Thibault Marin
 ;; Authors: Thibault Marin