How to use a local genome?

[Rcperez]

Hi!

Great repo. I would be grateful to use your repo in my work but I can't seem to the library to use my local file.

I tried this hack:

import pandas as pd
from jbrowse_jupyter import launch, create

hg38 = create('view', genome='hg38')

new_assembly = hg38.set_assembly(myfile.fa, overwrite = True)

The error messsage was:

/usr/local/lib/python3.7/dist-packages/jbrowse_jupyter/jbrowse_config.py in set_assembly(self, assembly_data, **kwargs)
176 self.config["assembly"] = assembly_config
177 else:
--> 178 raise TypeError("Local files are not currently supported.")
179
180 # ============ Tracks =============

TypeError: Local files are not currently supported.

Please advise.

Thank you!

/Rolando

[teresam856]

Hello @Rcperez!
Thank you for checking out the package. We have not implemented local file support, but it is something we are planning on working on. Here is the issue if you would like to track it #5.

However, I came up with a quick temporary solution. You can serve your local data via an http server to host your local data (simple http server or flask), then you can use the url and pass it to your component. (Note: this only works as a developing environment, and would be hard to deploy or use in a production setting)

I created this file in the root dir of my computer to run an http server to serve my local files. Credit goes to: https://stackoverflow.com/questions/21956683/enable-access-control-on-simple-http-server

test_server.py

#!/usr/bin/env python3
from http.server import HTTPServer, SimpleHTTPRequestHandler, test
import sys

class CORSRequestHandler (SimpleHTTPRequestHandler):
    def end_headers (self):
        self.send_header('Access-Control-Allow-Origin', '*')
        SimpleHTTPRequestHandler.end_headers(self)

if __name__ == '__main__':
    test(CORSRequestHandler, HTTPServer, port=8000)

$ python test_server.py

While that server is running...

Replace the path "http://[::]:8000/<path/to/your/genome>" with the path to the file of your genome
browser.py

from dash import html, Dash
from jbrowse_jupyter import create, create_component
app = Dash(__name__)

conf = create("config")
new_assembly = conf.set_assembly("http://[::]:8000/<path/to/your/genome>", overwrite = True)

# Note: set_assembly assumes there will be a .fai file in the same directory unless you pass a specific path
# create a dash component
component = create_component(hg38.get_config(), dash_comp="LGV")

# launch the component
app.layout = html.Div(
    [component],
    id='test'
)

if __name__ == "__main__":
    app.run_server(port=3001, debug=True)

$ python browser.py

[Rcperez]

Great to hear and thank you so much. Best, Rolando

…

On Jan 26, 2022, at 6:28 PM, Teresa Martinez ***@***.***> wrote: Hello @Rcperez <https://github.com/Rcperez>! Thank you for checking out the package. We have not implemented local file support, but it is something we are planning on working on. Here is the issue if you would like to track it #5 <#5>. However, I came up with a quick temporary solution. You can serve your local data via an http server to host your local data (simple http server or flask), then you can use the url and pass it to your component. (Note: this only works as a developing environment, and would be hard to deploy or use in a production setting) I created this file in the root dir of my computer to run an http server to serve my local files. Credit goes to: https://stackoverflow.com/questions/21956683/enable-access-control-on-simple-http-server <https://stackoverflow.com/questions/21956683/enable-access-control-on-simple-http-server> test_server.py #!/usr/bin/env python3 from http.server import HTTPServer, SimpleHTTPRequestHandler, test import sys class CORSRequestHandler (SimpleHTTPRequestHandler): def end_headers (self): self.send_header('Access-Control-Allow-Origin', '*') SimpleHTTPRequestHandler.end_headers(self) if __name__ == '__main__': test(CORSRequestHandler, HTTPServer, port=8000) $ python test_server.py While that server is running... browser.py from dash import html, Dash from jbrowse_jupyter import create, create_component app = Dash(__name__) conf = create("config") new_assembly = conf.set_assembly("http://[::]:8000/<path/to/your/genome>", overwrite = True) # Note: set_assembly assumes there will be a .fai file in the same directory unless you pass a specific path # create a dash component component = create_component(hg38.get_config(), dash_comp="LGV") # launch the component app.layout = html.Div( [component], id='test' ) if __name__ == "__main__": app.run_server(port=3001, debug=True) $ python browser.py — Reply to this email directly, view it on GitHub <#25 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE6A7Y64F4RH5DA4OW3YUB3UYCUTDANCNFSM5M4OXTMQ>. You are receiving this because you were mentioned.

[teresam856]

Let us know how that works for you. And feel free to contact us through our chat on gitter https://gitter.im/GMOD/jbrowse2. :)

[Rcperez]

Hello, I’m new to Dash, Flask, servers, etc. so I’ve not gotten this to work. I created the files you suggested on my computer and I’m basically stuck there. Should I now launch browse-jupyter from my computer? How then do I set the assembly to my local genome? My intention is to run the colab notebook and to use my own genome in the colab notebook. The solution you provided works when I am running local jbrowse-jupyter correct, not the Google Colab notebook template you provide? Thanks so much! /rp

…

On Feb 1, 2022, at 1:28 PM, Teresa Martinez ***@***.***> wrote: Let us know how that works for you. And feel free to contact us through our chat on gitter https://gitter.im/GMOD/jbrowse2 <https://gitter.im/GMOD/jbrowse2>. :) — Reply to this email directly, view it on GitHub <#25 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE6A7Y4GMZS5PP6CLDOD2KTUZBGALANCNFSM5M4OXTMQ>. You are receiving this because you were mentioned.

[teresam856]

@Rcperez Yes that is correct. The solution provided would work when running the browser.ipynb or the browser.py from your local environment and not the colab link. Binder/Colab/Google have restrictions about mixed content. This means that it will not accept the data from the http server since colab is under https. Another workaround would be to launch a specific https server.However, https servers require more configuration such as certs and keys to enable/serve your own https data.

[Rcperez]

Understood, I will try that and report back asap! /rp

…

On Feb 8, 2022, at 7:59 PM, Teresa Martinez ***@***.***> wrote: @Rcperez <https://github.com/Rcperez> Yes that is correct. The solution provided would work when running the browser.ipynb or the browser.py from your local environment and not the colab link. Binder/Colab/Google have restrictions about mixed content. This means that it will not accept the data from the http server since colab is under https. Another workaround would be to launch a specific https server. Https servers require more configuration such as certs and keys to enable/serve your own https data. — Reply to this email directly, view it on GitHub <#25 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE6A7Y4VWSNPDCWQDP7FRJTU2HRDDANCNFSM5M4OXTMQ>. You are receiving this because you were mentioned.

[cmdcolin]

Just wanted to note that this issue came up during chat. my idea:

cmdcolin: the interaction between jbrowse (which operates by fetching files from a server) and jupyter (which doesnt really provide a server to fetch files from by default) is a little tricky sometimes
 2 replies
cmdcolin: same thing basically applies to jbrowseR
cmdcolin: so the solution is either "spin up a server" is something that is mentioned in docs but that may only work on a localhost (and possibly not on google colab in jupyter or shinyapps.io for r)
cmdcolin: the alternative to spinning up a server is essentially all FromConfigAdapters, which may suffer scalability challenges
cmdcolin: note that @jbrowse/img is an interesting alternative because it can literally run server side, true ssr style, even perhaps on e.g. colab server side, so an @jbrowse/img integration with notebooks may be interesting

xref for @jbrowse/img, it is a command line tool that just generates svg output, not the full interactive browser, but could be useful in a REPL environment like jupyter, and further down the line, we could possibly make a @jbrowse/img that is interactive https://www.npmjs.com/package/@jbrowse/img

[Rcperez]

very interesting, thank you.

…

On Feb 9, 2022, at 8:44 AM, Colin Diesh ***@***.***> wrote: Just wanted to note that this issue came up during chat. my idea: cmdcolin: the interaction between jbrowse (which operates by fetching files from a server) and jupyter (which doesnt really provide a server to fetch files from by default) is a little tricky sometimes 2 replies cmdcolin: same thing basically applies to jbrowseR cmdcolin: so the solution is either "spin up a server" is something that is mentioned in docs but that may only work on a localhost (and possibly not on google colab in jupyter or shinyapps.io for r) cmdcolin: the alternative to spinning up a server is essentially all FromConfigAdapters, which may suffer scalability challenges cmdcolin: note that @jbrowse/img is an interesting alternative because it can literally run server side, true ssr style, even perhaps on e.g. colab server side, so an @jbrowse/img integration with notebooks may be interesting xref for @jbrowse/img, it is a command line tool that just generates svg output, not the full interactive browser, but could be useful in a REPL environment like jupyter, and further down the line, we could possibly make a @jbrowse/img that is interactive ***@***.***/img ***@***.***/img> — Reply to this email directly, view it on GitHub <#25 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE6A7YZZTRC4BYRIZOBLZKDU2KKXJANCNFSM5M4OXTMQ>. You are receiving this because you were mentioned.

[Rcperez]

happy to report the local server solution seems to have worked. I will continue tinkering. modified the code very slightly as I was getting an error for “config” should be “conf” and initialized the hg38 variable. browser.py from dash import html, Dash from jbrowse_jupyter import create, create_component app = Dash(__name__) conf = create("conf") new_assembly = conf.set_assembly("http://[::]:8000/<path/to/your/genome> <http://[::]:8000/%3Cpath/to/your/genome%3E>", overwrite = True) # Note: set_assembly assumes there will be a .fai file in the same directory unless you pass a specific path # create a dash component hg38 = create('LGV', genome='hg38’) component = create_component(hg38.get_config(), dash_comp="LGV") # launch the component app.layout = html.Div( [component], id='test' ) if __name__ == "__main__": app.run_server(port=3001, debug=True)

…

On Feb 9, 2022, at 10:21 AM, rolando perez ***@***.***> wrote: very interesting, thank you. > On Feb 9, 2022, at 8:44 AM, Colin Diesh ***@***.*** ***@***.***>> wrote: > > > Just wanted to note that this issue came up during chat. my idea: > > cmdcolin: the interaction between jbrowse (which operates by fetching files from a server) and jupyter (which doesnt really provide a server to fetch files from by default) is a little tricky sometimes > 2 replies > cmdcolin: same thing basically applies to jbrowseR > cmdcolin: so the solution is either "spin up a server" is something that is mentioned in docs but that may only work on a localhost (and possibly not on google colab in jupyter or shinyapps.io <http://shinyapps.io/> for r) > cmdcolin: the alternative to spinning up a server is essentially all FromConfigAdapters, which may suffer scalability challenges > cmdcolin: note that @jbrowse/img is an interesting alternative because it can literally run server side, true ssr style, even perhaps on e.g. colab server side, so an @jbrowse/img integration with notebooks may be interesting > xref for @jbrowse/img, it is a command line tool that just generates svg output, not the full interactive browser, but could be useful in a REPL environment like jupyter, and further down the line, we could possibly make a @jbrowse/img that is interactive ***@***.***/img ***@***.***/img> > — > Reply to this email directly, view it on GitHub <#25 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE6A7YZZTRC4BYRIZOBLZKDU2KKXJANCNFSM5M4OXTMQ>. > You are receiving this because you were mentioned. >

[teresam856]

Awesome. From what you posted above, were you able to create a component with your conf, such as component = create_component(conf.get_config(), dash_comp="LGV")? Also here are our sphinx docs with additional tutorials and a guide of our API. @Rcperez