Skeletor #35

alexanderbates · 2020-09-15T09:27:14Z

R wrapper for a skeleton pipeline in python to skeletonise neuron meshes, e.g. from FlyWire. Also, documentation and article describing how to install related python modules (updated version of Istvan Taiusz's flyconnectome slack post)

jefferis

Hi @alexanderbates, thanks for the PR. I have made a few comments implying changes. Could you respond to them or ping me if you have queries? Best, G.

jefferis · 2020-09-16T15:49:47Z

R/flywire-skeleton.R

+      savedir <- tempdir()
+      ff=file.path(savedir, paste0(id, '.obj'))
+      reticulate::py_run_string(sprintf("m.export('%s')",ff), ...)
+      res=sapply(ff, readobj::read.obj, convert.rgl = TRUE, simplify = FALSE)


Since nat 1.10.2 (which is required by fafbseg) read.neurons will read meshes.

jefferis · 2020-09-16T16:10:49Z

vignettes/articles/FAFB-FlyWire_cache/html/__packages

@@ -0,0 +1,10 @@
+base


@alexanderbates I'm not really sure about the wisdom of including rmarkdown caches here. Was this intentional?

Ah no, it was not intentional. Somehow this was generated when I ran build_site().

jefferis · 2020-09-17T10:59:58Z

vignettes/articles/accessing-graphene-server.Rmd

 You can gain access to the `Graphene/Chunkedgraph` server like below:

 1. Visit https://globalv1.flywire-daf.com/auth/api/v1/refresh_token,
 2. Copy the token present there, let's say it was "xxyyzz"
-3. create a file named `chunkedgraph-secret.json` that has the following contents
+3. create a file named `chunkedgraph-secret.json` at location `~.cloudvolume/secrets/chunkedgraph-secret.json`. For example, in Terminal: 


typo ~.cloudvolume/secrets/chunkedgraph-secret.json -> ~/.cloudvolume/secrets/chunkedgraph-secret.json

jefferis · 2020-09-17T11:01:33Z

vignettes/articles/accessing-graphene-server.Rmd

-4. Save the file in the location `~.cloudvolume/secrets/chunkedgraph-secret.json` 
-5. Now you have access to the chunked graph server
+
+You can also use the fafbseg function `set_chunkedgraph_token` to add your token to this file.


@alexanderbates I am about to change this function's name to flywire_set_token. Furthermore, I think there is really no point in this lengthy description as flywire_set_token/set_chunkedgraph_token does everything you need including getting the new token from the flywire website.

Ah okay! I just updated these old instructions a bit because i found it helpful

jefferis · 2020-09-17T11:04:18Z

docs/pkgdown.yml

@@ -1,10 +1,10 @@
-pandoc: 2.7.3
-pkgdown: 1.5.1


@alexanderbates can you please remove all changes to the docs/ folder. I would rather regenerate this myself after merging to master. Otherwise there are so many changes I find it hard to see the wood from the trees. There is however one thing that is missing, you could please edit _pkgdown.yml (note leading underscore) to add your new exported function(s). Thanks!

Is there an easy way to revert all changes within a specified folder?

https://stackoverflow.com/questions/43544919/git-revert-all-committed-files-in-a-directory-to-master

* we shouldn't need to do this if we are skeletonising .obj files on disk

* 'skeletor' of github.com:natverse/fafbseg: Replace WHO with WH0

* for skeletonisation and radii calculation

* 'skeletor' of github.com:natverse/fafbseg: Update skeletor to work with more methods arguments conflicts due to double edits including in errors where there were logic changes on both sides that needed integrating

* 'skeletor' of github.com:natverse/fafbseg: Add theta argument for mesh cleaning w.r.t skeleton

jefferis · 2020-10-05T19:01:17Z

R/flywire-skeleton.R

@@ -21,7 +22,12 @@
 #' You will need to have the \code{ncollpyde}
 #' python3 module installed. You can get this with \code{pip3 install ncollpyde}. If you get issues
 #' related to this module, best to set this to \code{FALSE}.
-#' @param radius Logical. Whether or not to return radius information for each skeleton node.
+#' @param theta numeric. Used if \code(clean==TRUE). For each twig we generate the dotproduct between the tangent


@alexanderbates, this should be \code{clean=TRUE}. Can you fix?

If you don't feel like running R CMD check before pushing (which you should), you should at least run devtools::check_man() to make sure that the docs aren't broken!

jefferis · 2020-10-05T19:21:14Z

@alexanderbates for reference a conditional test along these lines

test_that("skeletor works", {
  skip_if_not_installed('reticulate')
  cv_available=try(check_cloudvolume_reticulate, silent = TRUE)
  skip_if_not(cv_available, 'skipping tests requiring cloudvolume')
  res=skeletor(obj="testdata/720575940637333875.obj")
  # some more interesting expectation
  expect_is(res, 'neuron')
})

…keletor

jefferis · 2020-10-06T08:32:46Z

R/flywire-skeleton.R

+      savedir <- if(!is.null(save.obj)){
+        save.obj
+      }else{
+        tempdir()


tempdir() is the root temporary folder of the R session. You should not delete it (as you found out the other day when you were deleting bridging registrations). It is better to make your own temporary folder

dir.create(td<-tempfile()) td

@alexanderbates: you should also add an on.exit hook to delete that folder on exit

on.exit(unlink(td, recursive=TRUE))

jefferis · 2020-10-06T08:39:35Z

R/flywire-skeleton.R

@@ -364,8 +364,7 @@ py_skeletor <- function(id,
  if(reroot){
    neuron = reroot_hairball(neuron, k.soma.search = k.soma.search, radius.soma.search = radius.soma.search, brain = brain)
  }
-  if(mesh3d|save.obj){
-    if(is.null(mesh)){
+  if(mesh3d|!is.null(save.obj)){


@alexanderbates: as a general rule it is better to use || or && for this kind of logic in if statements as those operators short circuit i.e. they stop evaluating when the value of the expression is certain. So if you do

if(shortcheck() | longcheck())

both are evaluated even if shortcheck() returns TRUE whereas

if(shortcheck() || longcheck())

would stop evaluating if shortcheck() returned TRUE

If you want convincing, here is some code

fastcheck <- function(x=T) { message("fast") x } slowcheck <- function(x=T) { message("slow") x } message("|| operator") if(fastcheck() || slowcheck()) { message("if!") } else { message("else!") } message("| operator") if(fastcheck() | slowcheck()) { message("if!") } else { message("else!") }

|| operator fast if! | operator fast slow if!

Thank you, I'll also apply this more generally.

alexanderbates added 8 commits September 11, 2020 13:50

Improved flywire token instructions

a7df59d

Merge branch 'master' of https://github.com/jefferis/fafbseg

e2a6ab9

Instructions from I.Taisz on installing cloudvolume and meshparty

49893f3

Update .gitignore

39caa8e

R wrapper for skeletor pipeline

a486961

Skeletor documentation

bbf03c5

Instructions for installing required python modules

1e5b06f

Merge branch 'master' of https://github.com/jefferis/fafbseg

8fb7043

alexanderbates requested a review from jefferis September 15, 2020 09:27

alexanderbates added 20 commits September 15, 2020 11:24

Update flywire-skeleton.R

b58e8bc

Update skeletor.Rd

dab9810

Update flywire-skeleton.R

0e86c80

Update installing-cloudvolume-meshparty.Rmd

618252d

Update skeletor.Rd

d71cd65

Update flywire-skeleton.R

9be2cfe

Update installing-cloudvolume-meshparty.Rmd

000bd53

Update installing-cloudvolume-meshparty.Rmd

4bcb2a2

Add neuronmesh class to skeletor outputs

f2bb8c8

Update flywire-skeleton.R

5e1565b

Update flywire-skeleton.R

9e0fe8f

Update flywire-skeleton.R

3c8fb69

Update flywire-skeleton.R

f3fe4c3

Update flywire-skeleton.R

68d242f

Update flywire-skeleton.R

810315e

Update installing-cloudvolume-meshparty.Rmd

1f93158

Add heal option to R skeletor pipeline

15acbc1

Update flywire-skeleton.R

1755b6b

Update flywire-skeleton.R

6d6ce9d

Update flywire-skeleton.R

5db44f9

jefferis requested changes Sep 17, 2020

View reviewed changes

jefferis and others added 4 commits September 23, 2020 16:28

only initialise cloudvolume connection if essential

6ef0c93

* we shouldn't need to do this if we are skeletonising .obj files on disk

Replace WHO with WH0

8a95f21

Merge branch 'skeletor' of github.com:natverse/fafbseg into skeletor

a6972d8

* 'skeletor' of github.com:natverse/fafbseg: Replace WHO with WH0

Update skeletor to work with more methods arguments

6b7635e

* for skeletonisation and radii calculation

alexanderbates requested a review from jefferis October 1, 2020 18:22

jefferis and others added 3 commits October 1, 2020 21:41

Merge branch 'skeletor' of github.com:natverse/fafbseg into skeletor

11afbc5

* 'skeletor' of github.com:natverse/fafbseg: Update skeletor to work with more methods arguments conflicts due to double edits including in errors where there were logic changes on both sides that needed integrating

Add theta argument for mesh cleaning w.r.t skeleton

dd80276

Merge branch 'skeletor' of github.com:natverse/fafbseg into skeletor

28f203f

* 'skeletor' of github.com:natverse/fafbseg: Add theta argument for mesh cleaning w.r.t skeleton

jefferis reviewed Oct 5, 2020

View reviewed changes

alexanderbates added 10 commits October 5, 2020 20:41

Fix documentation build error

4886b55

Add test for skeletor

f10dfd4

Merge branch 'skeletor' of https://github.com/jefferis/fafbseg into s…

fc7df53

…keletor

Travis update to install python libraries for skeletor

75eb89e

Update .travis.yml

0ff3734

Remove travis python install for now

3179ab4

Update skeletor.Rd

96996e3

Update flywire-skeleton.R

d502262

Save .obj files in at specified path with skeletor

55204f6

Update flywire-skeleton.R

9d8a939

jefferis requested changes Oct 6, 2020

View reviewed changes

jefferis reviewed Oct 6, 2020

View reviewed changes

alexanderbates added 6 commits October 6, 2020 11:10

Improved handling of temporary save files in skeletor

8caa344

Update flywire-skeleton.R

f9b6e3d

Updated skeltor R wrapper for skeletor 0.2.4

90197b6

Update flywire-skeleton.R

bbd2cc8

Update skeletor.Rd

9ad31f6

Update flywire-skeleton.R

a0951bd

jefferis merged commit 416593a into master Oct 9, 2020

alexanderbates deleted the skeletor branch October 10, 2020 14:40

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Skeletor #35

Skeletor #35

alexanderbates commented Sep 15, 2020

jefferis left a comment

jefferis Sep 16, 2020

alexanderbates Sep 17, 2020

jefferis Sep 16, 2020

alexanderbates Sep 17, 2020

jefferis Sep 17, 2020

alexanderbates Sep 17, 2020

jefferis Sep 17, 2020

alexanderbates Sep 17, 2020

jefferis Sep 17, 2020

alexanderbates Sep 17, 2020

alexanderbates Sep 19, 2020

jefferis Oct 5, 2020

jefferis Oct 5, 2020

jefferis commented Oct 5, 2020

jefferis Oct 6, 2020

jefferis Oct 6, 2020

alexanderbates Oct 6, 2020

jefferis Oct 6, 2020 •

edited

jefferis Oct 6, 2020

alexanderbates Oct 6, 2020

Skeletor #35

Skeletor #35

Conversation

alexanderbates commented Sep 15, 2020

jefferis left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jefferis commented Oct 5, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jefferis Oct 6, 2020 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jefferis Oct 6, 2020 •

edited