projectile-discover-projects-in-search-path recursively

[DougBeney]

I organize my ~/Code folder by language. So it's directory structure might look like...

├── browser-extensions
│   ├── My Cool Extension
│   └── My Other Cool Extension
├── Jekyll
│   └──  Some Website
├── python
│   └── Some Python Project

I use this to set my project search path.

(setq projectile-project-search-path '("~/Code"))

Instead of adding each sub-folder to my projectile search path (I have quite a few languages/categories in there), I would love to be able to run projectile-discover-projects-in-search-path and have it go recursively.

Any thoughts?

EDIT

For my unique situation, I came up with a pretty simple solution.

  (setq projectile-project-search-path (cddr (directory-files "~/Code" t)))

(The cddr bit removes the first two items from the returned list which are . and .. directories.)

Could also use the function directory-files-recursively which will catch any projects nested even deeper. :)

[stfl]

thank you for the workaround.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contribution and understanding!

[mattiasb]

I do this FWIW:

;;;###autoload
(defun mb-cmd-projectile-index-projects ()
  "Index my project directories."
  (interactive)

  (mapc #'projectile-add-known-project
        (mb-f-find-git-projects "~/" 5))

  (projectile-cleanup-known-projects))

(defun mb-f-find-git-projects (dir &optional depth)
  "Find all git projects under DIR.
Optionally only search as deep as DEPTH."
  (let* ((depth-flag (if depth (format "-maxdepth %d" depth) ""))
         (cmd (format "find %s %s -name '.git' -type d" dir depth-flag))
         (result (split-string (shell-command-to-string cmd))))
    (mapcar (lambda (s) (substring s 0 -4)) result)))

;; This is from memory, I have another construct for simplifying keybindings.
(define-key 'projectile-command-map (kbd "i") #'mb-cmd-projectile-index-projects)

[mohkale]

I think it'd be nice if projectile-project-search-path used the same form as magit-repository-directories. It lets you specify both the path to look for a project and the number of subdirectories to traverse until you find one.

[juergenhoetzel]

I also have this issue. I'm using: ghq which requires to descent 3 levels.

  (setq projectile-project-search-path (cddr (directory-files "~/Code" t)))

Nice workaround there is also a predefined regex to filter out dotfiles:

(directory-files "~/ghq" nil directory-files-no-dot-files-regexp)

For ghqI use:

(setq projectile-project-search-path
			     (directory-files-recursively "~/ghq" directory-files-no-dot-files-regexp t
							  (lambda (dir)
							    (eq (length (split-string dir "/")) 3))))

[juergenhoetzel]

This adds support for magit-style path definitions:

[Fix #1500] Add support for recursive project discovery

[mohkale]

@juergenhoetzel

Thanks for this. Just worth mentioning the magit and projectile style paths are not the same. Magit uses a depth of 0 to mean add this directory literally. Projectile uses a depth of 0 to say look in this directories subdirectories. That is projectile always inspects subdirectories of the directories in the path where-as magit lets you add a directory exactly.

The inconsistency isn't ideal and frankly I do think magits approach of 0 meaning don't look any deeper makes more sense. I'd rather not break compatibility with @juergenhoetzel work but is there any desire to make it so that by default if you have a string in projectile-project-search-path then you search with depth 1, and when you try to search with depth 0 you just add that directory itself instead of looking up a sub-directory? That would fix the issue and the only compatibility that breaks is with the relatively new cons feature of projectile-project-search-path. Thoughts?

Edit:

For those wondering why you'd want add an exact path to the search path instead of its container, I keep my dotfiles in ~/.config/dotfiles. The ~/.config directory doesn't often contain repos but I don't want to have to search every directory in it for one repo that I already know exists.

[mohkale]

Something like this is what I meant:

@@ -1028,15 +1028,14 @@ The cache is created both in memory and on the hard drive."
 
 If DEPTH is non-nil recursively descend exactly DEPTH levels below DIRECTORY and
 discover projects there."
-  (if (file-exists-p directory)
-      (let ((subdirs (directory-files directory t)))
-        (dolist (dir subdirs)
-          (when (and (file-directory-p dir)
-                     (not (member (file-name-nondirectory dir) '(".." "."))))
-            (if (and (numberp depth) (> depth 0))
-	        (projectile-discover-projects-in-directory dir (1- depth))
-              (when (projectile-project-p dir)
-	        (projectile-add-known-project dir))))))
+  (if (file-directory-p directory)
+      (if (and (numberp depth) (> depth 0))
+          (dolist (dir (directory-files directory t))
+            (when (and (file-directory-p dir)
+                       (not (member (file-name-nondirectory dir) '(".." "."))))
+	            (projectile-discover-projects-in-directory dir (1- depth))))
+        (when (projectile-project-p directory)
+          (projectile-add-known-project directory)))
     (message "Project search path directory %s doesn't exist" directory)))
 
 ;;;###autoload
@@ -1047,7 +1046,7 @@ Invoked automatically when `projectile-mode' is enabled."
   (dolist (path projectile-project-search-path)
     (if (consp path)
 	(projectile-discover-projects-in-directory (car path) (cdr path))
-      (projectile-discover-projects-in-directory path 0))))
+      (projectile-discover-projects-in-directory path 1))))
 
 
 (defun delete-file-projectile-remove-from-cache (filename &optional _trash)

However there seems to be a bug in projectile-merge-known-projects. It never changes from the value on disk for me 😢.

Edit:

The bug seems to have been because the directory of my projectile-known-projects-file didn't exsit. Otherwise the patch above works fine.

[bbatsov]

It's still fine to change the code, as we haven't released anything yet. I've never used this functionality in Magit myself, but given this is a search path I don't think that a depth of 0 makes sense for it, as you won't be searching for anything this way.

[juergenhoetzel]

It's still fine to change the code, as we haven't released anything yet. I've never used this functionality in Magit myself, but given this is a search path I don't think that a depth of 0 makes sense for it, as you won't be searching for anything this way.

I was undecided myself whether 0 or 1 should represent the default search depth. But because of the existing docstring:

List of folders where projectile is automatically going to LOOK for projects.

I decided and entry of "~/basedir" should be equal to ("~/basedir" . 0)

because LOOKING in my opinion already implies to descend 1 level down.

But maybe that's just my point of view and I made a typical Off-by-one-Error :-)

[mohkale]

@bbatsov

but given this is a search path I don't think that a depth of 0 makes sense for it, as you won't be searching for anything this way.

That makes sense but since you'll still be checking that the passed in directory exists and that it's a project I'm hoping the meaning gets across. The alternative would be to advise projectile-discover-projects-in-search-path to check a list of specific directories after the fact which isn't ideal.

Would you accept a PR with the changes I describe above?

@juergenhoetzel

I was undecided myself whether 0 or 1 should represent the default search depth. But because of the existing docstring:

The reasoning makes sense, it's just I've just aliased my magit-repository-directories variable to projectile-project-search-path and because their so similair it would be strange to have two exact definitions with all the depth numbers being one lower than the other. I can always just abandon the projectile approach and adapt something similair using magits search features.

But maybe that's just my point of view and I made a typical Off-by-one-Error :-)

Personally I feel depth 0 meaning don't descend gives users more control than the alternative. Atm you can always specify "search this directory" but you can't specify "add this directory".