Unaliased Pango Lineage column gone?

[ryhisner]

I've always found the unaliased Pango lineage column by far the best way to sort lineages on Nextclade, but it now seems to be gone, replaced by what seems to me an entirely useless 'WHO name' column. Is this an intentional change? Is there any way for me to get the unaliased Pango lineage column back?

Also, is there any possibility of making columns so that they can be toggled on or off? I would hide a lot of the columns if I could, and that would be a HUGE improvement and save me a lot of time alternately widening and narrowing my browser window. Thanks.

[ivan-aksamentov]

Hi Ryan @ryhisner,

Looks like it was turned off in the last dataset release:
https://github.com/nextstrain/nextclade_data/releases/tag/2023-04-19--12-45-05--UTC

I'll let @corneliusroemer to comment on that. I can only comment on the technical side of things:

All columns are still available in CSV and TSV downloads of Nextclade Web or outputs of Nextclade CLI. But it's less convenient of course.

For immediate workaround, in order to bring the column back in Nextclade Web UI, you can modify the value of the field

meta.extensions.nextclade.clade_node_attrs[].hideInWeb

in the reference tree JSON file and then use the modified file in Nextclade Web.

For that, download reference tree file tree.json from the latest dataset, for example from here: https://github.com/nextstrain/nextclade_data/blob/master/data/datasets/sars-cov-2/references/MN908947/versions/2023-04-18T12%3A00%3A00Z/files/tree.json

Edit the file to set hideInWeb to false for the column in question:

"meta": {

  "extensions": {
    "nextclade": {
      "clade_node_attrs": [

        {
          "name": "partiallyAliased",
          "displayName": "Unaliased",
          "description": "Partially aliased reconstructed Pango lineage",
          "hideInWeb": false
        }

      ]
    }
  }
}

After dataset is selected in Nextclade Web, open the "Customize dataset files" section and drop the file to the "Reference tree" box. Then run as usual.

Here is the modified file: tree.json.txt
(remove .txt extension; I added it because GitHub does not allow uploading .json files)

You can also configure Nextclade Web URL to fetch a custom tree file by adding a URL pointing to that file into the input-tree URL parameter like this:
https://clades.nextstrain.org/?dataset-name=sars-cov-2&input-tree=https://github.com/nextstrain/nextclade/files/11437245/tree.json.txt
In this case I copied the URL for the modified tree.json file I uploaded above, then pasted it after &input-tree=. You can bookmark this URL. But note that it will be stuck on this tree version forever. So it really is just a temporary workaround. (Nope, the linkg actually does not work, probably because GitHub does not set CORS headers here to avoid abuse. But you can commit this file to a GitHub repo or upload it to another place with CORS enabled, or use a CORS proxy; and use a URL from there).

Regarding making columns configurable in the app , we have thought about it before, as you already discovered in #986, but it hasn't materialized quite yet. We are currently focusing on the algorithms side of things. We might tackle that in the future. Contributions from the community are very welcome!

[ryhisner]

Thank you, @ivan-aksamentov. I've only just started learning how to code (using Julia), so don't think I'll be capable of making any of those alterations any time soon, but I appreciate the time and effort you put in trying to help me do so!

[ivan-aksamentov]

@ryhisner You don't really need to code here. All required modifications are limited to a single field in JSON file (I would not call JSON "code"). And I already provide the modified file for your convenience. You can drop it into the "Reference tree" upload box in Nextclade and it should work.

Happy to help if you have questions.

[ivan-aksamentov]

Peek.2023-05-11.00-01.mp4

[ryhisner]

@ivan-aksamentov, I can't tell you what a life saver this is for me. Having BA.5's, BA.2's, XBB's, and BA.4's all mixed together was driving me crazy. But now all is well with the world. Thank you so much!

[corneliusroemer]

@ryhisner we changed columns only for the main sars-cov-2 build. I actually never use that one, but the sars-cov-2-21L dataset, as almost everything descends from 21L.

You can find it like this, or just select from the dataset selector: https://clades.nextstrain.org/?dataset-name=sars-cov-2-21L

Nightly builds (with the very latest Pango lineages) are available here: https://master.clades.nextstrain.org/?dataset-name=sars-cov-2-21L&input-tree=https://nextstrain.org/charon/getDataset?prefix=staging/nextclade/sars-cov-2/21L

[ryhisner]

I have no idea how to update the tree.json.txt file that includes the unaliased Nextcladepangolineage column, so I'm still using the above one from one from the beginning of May, and I don't think it has any lineage updates (FR.1 is still BN.1.2.3, to give one example). I gather that it's somehow possible to use the links above to use the nightly builds and/or 21L tree, but again, I don't have the slightest idea how to do so. Could someone please help me with this?

If the unaliased Nextcladepango lineage column returns, I won't have to ask annoying questions like these, which no doubt seem very stupid to everyone here. Thanks.

[ivan-aksamentov]

@ryhisner I cannot help with bringing the column by default sadly, but here is how to download the trees and how to modify any JSON file:

You can download the Nightly tree JSON Cornelius linked to from here:
https://nextstrain.org/charon/getDataset?prefix=staging/nextclade/sars-cov-2/21L
That's for 21L dataset. My understanding is that it already has the column enabled. Be careful to only use this tree with the 21L dataset and not the default dataset. Oherwise the results might be wrong.

And for the default dataset the tree JSON is here:
https://nextstrain.org/charon/getDataset?prefix=staging/nextclade/sars-cov-2

You should be able to open this URL in the browser. If download does not start right away and instead you see a bunch of JSON code, then go to menu of your browser and and select "Save as..." or hit Ctrl+S (or Cmd+S) on Mac, and then give it a good name and save to somewhere on your computer.

Then you can open this file in any plain text editor, e.g. VSCode, Sublime text, or even Notepad. There, search for text string "Partially aliased reconstructed Pango lineage". Usually in text editors you can do full-text search in a file with something like "Find..." and it is usually available if you hit something like Ctrl+F. Right after this string there will be something like "hideInWeb":true. And that's what makes the column hidden. You want to change that and to say "hideInWeb":false instead. Then you can save the file (Ctrl+S) and it's ready.

These are exactly the same instructions as in my previous message, but with addition of steps for those who are not familiar with JSON format or with plain text editors.

I think Cornelius is about to release the new datasets:
https://github.com/nextstrain/nextclade_data/blob/master/CHANGELOG.md
The new stable tree is here if you need it:
https://raw.githubusercontent.com/nextstrain/nextclade_data/99636079d5ef010a478122212589d6e61cdb6aa9/data/datasets/sars-cov-2/references/MN908947/versions/2023-06-16T12%3A00%3A00Z/files/tree.json

[ryhisner]

Excellent, thank you @ivan-aksamentov!

[ryhisner]

@ivan-aksamentov, one additional question on this. Is it possible to hide the clade column? It looks in the text as if it should be hidden, but it always shows up anyway. I'd also like to be able to hide the coverage % since it's redundant if the # of NNN's is already listed.

If I have to manually hunt through the whole code to delete or replace stuff, I can do that. I'm used to spending hours doing stuff manually that any computer literate person could do in seconds.

[ivan-aksamentov]

@ryhisner Hi Ryan,

No, the "Clade" column is currently hardcoded into Nextclade software and is not configurable at this time. It is a built-in column that is a result of phylogenetic placement according to the clade_membership attribute on reference tree nodes (according to the input reference tree JSON). You can see this attribute for the first tree node (root node), on your screenshot just below the yellow highlighting.

What you have highlighted in your JSON file is another (custom) configurable column "Nextstrain Clade" (Note the different name) and it is indeed hidden. These custom columns exist due to a column rename @corneliusroemer have done in PR nextstrain/nextclade_data#39 and throughout the Nextstrain ecosystem. And they were toggled on and off during the transition period.

In order to plan our future work, can you tell me more about your motivation regarding hiding columns? Is it to free up space for sequence views? If so, then we have some plans, and Federico have also recently requested this more directly in the issue #1272. I link to related issues and explain the situation in some more details there. We have been focusing on the scientific internals lately and haven't been able to implement this yet. It's a big adventure.

[ryhisner]

In order to plan our future work, can you tell me more about your motivation regarding hiding columns? Is it to free up space for sequence views? If so, then we have some plans, and Federico have also recently requested this more directly in the issue #1272. I link to related issues and explain the situation in some more details there.

Yes, I would like to be able to view the mutation profile and be able to see the QC or Pango lineage at the same time while scrolling through sequences, for example, but it's not really possible to obtain an adequate view of mutations while doing that. Actually, it works fine for the short genes like ORF6 but not for spike, ORF1a, or ORF1b.

Maybe I should try to find a double-wide screen or something that I can plug into my laptop. Do such screens exist? I might have to rearrange my workspace to make room for something like that, but if it works, it would be worth it.

[ivan-aksamentov]

@ryhisner Yes, as a hardware workaround, a wider screen would allow for more space for sequences. For Nextclade we recommend at least a FullHD screen with 1920x1080 resolution, but the higher the better. There are also ultra-widescreen displays which allow more horizontal screen space compared to the usual displays:

As a software workaround, you can also try to scale down the page in your browser: On Windows that can be done by rotating your mouse wheel while holding Ctrl key (probably Cmd key on Mac) or by pressing Ctrl + - (probably Cmd + - on Mac), at the expense of size of all other elements. If you are on Windows and have display scaling greater than 1 in the Windows settings, then making it 1 can also help to fit more stuff on the screen.

But I need to admit that there is a room for improvement in Nextclade table rendering, and I hope we could find some time to address this at some point, so that users don't need additional hardware or scaling tricks.

[ivan-aksamentov]

On this screenshot you can see how Nextclade looks on my 4K screen with 3840x2160 resolution (it's not ultrawide, just a usual 16:9 with 4K resolution) without scaling - around 2/3 of the screen width is sequence views.

[ivan-aksamentov]

I will close the issue for now. Please comment or create a new one if there's still outstanding problems.