Wigbigwig #5704

fubar2 · 2024-01-10T07:07:59Z

FOR CONTRIBUTOR:

- I have read the CONTRIBUTING.md document and this tool is appropriate for the tools-iuc repo.
- License permits unrestricted use (educational + commercial)
- This PR adds a new tool or tool collection

What:

Allow history reference genomes to be used to make bedgraph files and thus bigwig tracks.
Update ucsc-wigtobigwig to release 447 - no longer allows stdin so needs the grep output as an input file.
Add profile and a citation so it passes planemo lint

Why:

The existing converter can only be used with a built-in reference genome.
VGP workflows typically use history reference genomes and bigwig is handy for JBrowse2 to speed up loading times for >10M bed feature tracks.

Moving this to the IUC was suggested by @mvdbeek in galaxyproject/galaxy#17261 (comment)

We shouldn't update build-in tools in such major ways. Can you export this to the IUC if there's no other tool doing this ?

Ross agrees - the codebase is designed to be flavoured with plugin tools. Moving text/genome built in tools to IUC suites makes a lot of sense to me in the long term since not every server wants or needs them. They will all possibly want the upload tool.
Simpler and modular is going to help make the codebase more sustainable and the IUC seems a natural way to deal with these tools as they need fixing going forward. Not arguing for rip and replace - not broken enough and no resources...

blastxml doesn't crash - hard to test..

…the config :(

…nditionals. Such fun was had discovering this.

…- a test would be needed.

…apter...

…a but need a proper reference/maf test

…ere.

fubar2 · 2024-01-11T00:37:27Z

@bgruening no, I had accidentally edited the moab/tool_data_table_conf.xml.test so planemo was mixing things up. If you look higher up you'll see planemo linting moab. Seems fine now but it was an interesting experience to track it down...

No idea. Maybe another rebase?

bernt-matthias

Would be cool to have this included here: https://github.com/galaxyproject/tools-iuc/tree/main/tools/ucsc_tools

bernt-matthias · 2024-01-11T09:29:12Z

tools/wigtobigwig/wigtobigwig.xml

@@ -0,0 +1,133 @@
+<tool id="wigtobigwig" name="Wig-BedGraph-to-bigWig" version="1.1.2" profile="22.05">


Please us a version macro and used here and in the requirement version.

Can you open a PR removing the tool from the Galaxy code base / at least from the tool conf https://github.com/galaxyproject/galaxy/blob/fd45e1bd9fc1f351bb2bb203659758b13173d94c/lib/galaxy/config/sample/tool_conf.xml.sample#L78

Maybe we can prefix the tool name by UCSC?

Can you open a PR removing the tool from the Galaxy code base / at least from the tool conf https://github.com/galaxyproject/galaxy/blob/fd45e1bd9fc1f351bb2bb203659758b13173d94c/lib/galaxy/config/sample/tool_conf.xml.sample#L78

No, this needs to stay for reproducibility.

You can hide it though (hidden="True" in the tool conf file)

bernt-matthias · 2024-01-11T09:30:24Z

tools/wigtobigwig/wigtobigwig.xml

+        #else:
+              -clip
+        #end if
+        2>&1 || echo "Error running wigToBigWig." >&2


Would prefer that stderr is not redirected to stdout.

This is what the converter code has been since lasted touched 6 years ago :)
Not my idea but of course.

bernt-matthias · 2024-01-11T09:36:51Z

tools/wigtobigwig/wigtobigwig.xml

@@ -0,0 +1,133 @@
+<tool id="wigtobigwig" name="Wig-BedGraph-to-bigWig" version="1.1.2" profile="22.05">
+    <description>bedGraph or Wig to bigWig converter</description>


Just make this the tool name and remove the descrpition.

bernt-matthias · 2024-01-11T09:38:17Z

tools/wigtobigwig/wigtobigwig.xml

+            </when>
+       </conditional>
+        <conditional name="settings">
+                <param name="settingsType" type="select" label="Converter settings to use" help="Default settings should usually be used.">


Can you make the indentation consistent?

bernt-matthias · 2024-01-11T09:39:01Z

tools/wigtobigwig/wigtobigwig.xml

+        <conditional name="hist_or_builtin">
+            <param  name="genome_type_select" type="select"  label="Reference genome source"
+                help="Was the Wig/bedgraph input constructed using a genome from your history, or a built-in genome?">
+                <option selected="True" value="indexed">Input data is for a built-in genome</option>


Can you add a test for the indexed case?

I cannot.
I have tried. Cannot get anyting to work in CI even if it passes biocontainer and ordinary tests - so someone will need to help out for that to work. Please fix if you know how. It's an odd situation because the genome is being read as metadata from the input, Not sure how to do that in tests so advice appreciated..

Can you add the test case that should run .. then we can have a look on the error.

will do tomorrow. Late here.

Turns out to not be easy. chromInfo seems to be a special case in Galaxy. I added a test but disabled it.

fubar2 · 2024-01-11T10:17:56Z

Would be cool to have this included here: https://github.com/galaxyproject/tools-iuc/tree/main/tools/ucsc_tools

@bernt-matthias - it's up to the iuc to decide where it goes. I don't mind at all. How does the iuc decide where it wants this?

bernt-matthias · 2024-01-11T10:31:58Z

How does the iuc decide where it wants this?

There is no standard. I just like to have tools coming from the same source close together.

bgruening · 2024-01-13T22:30:31Z

@fubar2 I can take this over if you like. Thanks for your work!

fubar2 · 2024-01-14T00:39:26Z

I can take this over if you like.
@bgruening - that would be wonderful - yes please.

bgruening · 2024-01-14T10:50:51Z

Ok, I moved the wrapper and fixed the review comments.

bernt-matthias

chromInfo also works (tested locally) .. but if I'm not wrong __dbkey__ is the better and testable way to go.

…wigbigwig

now present at IUC galaxyproject/tools-iuc#5704

fubar2 and others added 30 commits December 24, 2023 13:15

add jbrowse2 - sorta works a bit - bed and reference only

15a045a

Merge branch 'galaxyproject:main' into jbrowse2

d6053b6

got some things working...bed, gff, bw, ..

8e9529c

Merge branch 'jbrowse2' of github.com:fubar2/tools-iuc into jbrowse2

046944a

updated bcbio-gff ;0 to 0.7.0 - bcbiogff stopped at 0.6.6

2523972

blastxml doesn't crash - hard to test..

more or less working - doesn't open default session although it's in …

50ee399

…the config :(

odd. failing planemo test but runs fine.

2f1cfd9

profile 22.05 breaks with boolean instead of select parameters for co…

316d2a2

…nditionals. Such fun was had discovering this.

passes planemo tests and lint now :)

d9aa1cc

minor cleanup

4c9c181

fixed shed.yml :(

7bced94

get rid of some cruft.

dc084d0

and more cruft

66a28d5

Must remove jbrowse-from-dir.xml because it will always fail linting …

40d8d67

…- a test would be needed.

shuffling lines to appease the robot pedants.

7e416e5

robot pedants will always win.

84bf4b9

thought this was done

bb3cce8

whoopsie - "fixed" the wrong shed.yml :(

7395b5b

evil twins

db83b8b

fix readme.rst - didn't see it :(

d7f3624

fix cp

0340aa0

cannot get sample vcf to work - header problem reported - tweaking ad…

2825ddd

…apter...

Update jbrowse2.xml

c8f5d9b

style formatting

2534eb3

add profile version and fix version command

9bf5779

flake8 hanging indents fixed...

214c9c4

Merge branch 'jbrowse2' of github.com:fubar2/tools-iuc into jbrowse2

7303f99

testing maf

2a6a74d

maf now added - runs and seems to make a track of 3.maf from test-dat…

9834e0e

…a but need a proper reference/maf test

adding files to branch

eb8e452

fubar2 added 2 commits January 11, 2024 11:23

change tests - cannot figure out how to make the indexed tests work h…

9b745a7

…ere.

so now with 4 spaces for indenting. Dunno. Prefer 2 personally.

fa51dec

fubar2 requested a review from bgruening January 11, 2024 03:49

bernt-matthias reviewed Jan 11, 2024

View reviewed changes

bgruening added 4 commits January 14, 2024 10:52

move the wigtobigwig to ucsc tools

0c5bace

remove fasta index - not sure if needed

40a149a

add a test for built-in, small cleanups

a727f6c

change ID

d556f75

fubar2 requested a review from nsoranzo as a code owner January 14, 2024 10:49

bgruening and others added 3 commits January 14, 2024 12:27

fix linting

433425b

disable chromInfo test

c4242d1

use data table instead of chromInfo

97a5977

bernt-matthias approved these changes Jan 18, 2024

View reviewed changes

bgruening and others added 5 commits January 18, 2024 19:17

Update wigtobigwig.xml

48c596d

add no_option validator

c652a45

try fix

6ec4eac

typo

38a40ea

Merge branch 'wigbigwig' of https://github.com/fubar2/tools-iuc into …

c3f50b6

…wigbigwig

bgruening approved these changes Jan 18, 2024

View reviewed changes

bgruening merged commit ab36a3c into galaxyproject:main Jan 18, 2024
11 checks passed

bernt-matthias added a commit to bernt-matthias/galaxy that referenced this pull request Jan 18, 2024

hide wig_to_bigWig tool

82f335a

now present at IUC galaxyproject/tools-iuc#5704

bernt-matthias mentioned this pull request Jan 18, 2024

Hide wig_to_bigWig tool galaxyproject/galaxy#17328

Merged

4 tasks

bernt-matthias added a commit to bernt-matthias/galaxy that referenced this pull request Feb 15, 2024

hide wig_to_bigWig tool

e1457c8

now present at IUC galaxyproject/tools-iuc#5704

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Wigbigwig #5704

Wigbigwig #5704

fubar2 commented Jan 10, 2024 •

edited

fubar2 commented Jan 11, 2024 •

edited

bernt-matthias left a comment

bernt-matthias Jan 11, 2024

bernt-matthias Jan 11, 2024

bernt-matthias Jan 11, 2024

mvdbeek Jan 11, 2024

mvdbeek Jan 11, 2024

bernt-matthias Jan 11, 2024

fubar2 Jan 11, 2024

bernt-matthias Jan 11, 2024

bernt-matthias Jan 11, 2024

bernt-matthias Jan 11, 2024

fubar2 Jan 11, 2024

bernt-matthias Jan 11, 2024

fubar2 Jan 11, 2024

bgruening Jan 14, 2024

fubar2 commented Jan 11, 2024

bernt-matthias commented Jan 11, 2024

bgruening commented Jan 13, 2024

fubar2 commented Jan 14, 2024

bgruening commented Jan 14, 2024

bernt-matthias left a comment

		@@ -0,0 +1,133 @@
		<tool id="wigtobigwig" name="Wig-BedGraph-to-bigWig" version="1.1.2" profile="22.05">

		@@ -0,0 +1,133 @@
		<tool id="wigtobigwig" name="Wig-BedGraph-to-bigWig" version="1.1.2" profile="22.05">
		<description>bedGraph or Wig to bigWig converter</description>

Wigbigwig #5704

Wigbigwig #5704

Conversation

fubar2 commented Jan 10, 2024 • edited

fubar2 commented Jan 11, 2024 • edited

bernt-matthias left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fubar2 commented Jan 11, 2024

bernt-matthias commented Jan 11, 2024

bgruening commented Jan 13, 2024

fubar2 commented Jan 14, 2024

bgruening commented Jan 14, 2024

bernt-matthias left a comment

Choose a reason for hiding this comment

fubar2 commented Jan 10, 2024 •

edited

fubar2 commented Jan 11, 2024 •

edited