WARNING: Unexpected error local variable 'iog' referenced before assignment

[camille-puginier]

Hi ! I've conducted an orthofinder run on 36 algal species
The all-versus-all sequence search seems to be successfully completed
However, once it gets to the reconciliation of gene trees and species trees, an error keeps showing up :

"local variable 'iog' referenced before assignment
WARNING: Unexpected error"

Could you help me ? Thanks !

[davidemms]

Hi

Are you using an old version or OrthoFinder? I think this is an issue that was resolved a year or two ago. If you downlaod the new version then you can restart from you're previous results using "-ot PREVIOUS_RESULTS_DIR" (if the tree inference stage was completed" or "-og PREVIOUS_RESULTS_DIR" if not (this will restart from the groups).

All the best
David

[camille-puginier]

I'm using the 2.5.2 version so that's odd!
Thanks for your quick answer

[davidemms]

Ah, in that case you could post the full output from orthofinder and I will investigate what's causing the issue.

[camille-puginier]

Also, i should mention that the job is still running but it seems to be stuck because nothing has been written after this error showed up

[davidemms]

I've looked into this and it looks like the job won't complete now. I've submitted a fix for this so that it will continue running, the code is in the master branch so you'll need to download the OrthoFinder code using the green 'Code' button on the front page (https://github.com/davidemms/OrthoFinder) and run that. You can restart from your current analysis using

python orthofinder.py -ft /work/cpuginier/db/Lichens/proteomes/OrthoFinder/Results_Mar19_2/

However, I'm still curious to know the errors that caused this problem. In the log you posted it said:

local variable 'iog' referenced before assignment
WARNING: Unexpected error

It should continue running this time but may print a new error message with more info, if it does could you post it here please.

[davidemms]

DIAMOND is also reporting an number of errors, e.g.

Command: diamond blastp -d /work/cpuginier/db/Lichens/proteomes/OrthoFinder/Results_Mar19_2/WorkingDirectory/diamond_ultra_sensDBSpecies6 -q /work/cpuginier/db/Lichens/proteomes/OrthoFinder/Results_Mar19_2/WorkingDirectory/Species6.fa -o /work/cpuginier/db/Lichens/proteomes/OrthoFinder/Results_Mar19_2/WorkingDirectory/Blast6_6.txt --ultra-sensitive -p 1 --quiet -e 0.001 --compress 1

and then later because of this OrthoFinder says

WARNING: Too few hits between species 6 and species 6 to normalise the scores, these hits will be ignored

Do you know why DIAMOND is having a problem with Species 6? You can look in "proteomes/OrthoFinder/Results_Mar19_2/WorkingDirectory/SpeciesIDs.txt" to see which species this is. I can't imagine why a species wouldn't produce any hits when searched against itself. Is "Results_Mar19_2/WorkingDirectory/Blast6_6.txt.gz" empty? What about "WorkingDirectory/Species6.fa"? The error messages also report problems with other species too.

[camille-puginier]

We've fixed the Diamond problem
The only warning we've got is : "WARNING: Unknown error"
However, the run completed !

[davidemms]

Thanks, that's great. The "WARNING: Unknown error" is because one of the worker processes is encountering an error at some point but recovering from it. It shouldn't impact the results but at the moment I'm not sure what's causing it. Was there any text after that message?

[camille-puginier]

Yes !
"Current orthogoup ......
......Done OF Orthologues"

"Writing results files ...."

"Results: path/to/results"

"CITATION:
..."

[davidemms]

Did it print an actual orthogroup number when is said "Current orthogoup ......"? If so it's likely the issue is in the tree for that orthogroup. If you could send me corresponding file in "WorkingDirectory/Trees_ids/" I can take a look.

[camille-puginier]

Yes it does print an orthogroup number !
I just sent you the file
Thanks again for your help

[davidemms]

This has been identified as an issue in the binary package of OrthoFinder and does not occur with the source code version (OrthoFinder_source.tar.gz) or conda. I'll try and track down what goes wrong with the binary package.

[rpelicae]

Dear David,

In my case, the error

local variable 'iog' referenced before assignment WARNING: Unexpected error
occured when Reconciling gene trees and species tree, only with the OrthoFinder version 2.5.2 OrthoFinder_glibc-2.15 version but not with the OrthoFinder.tar.gz version.

[davidemms]

Thanks, that's useful to know.

[davidemms]

The error should only occur with the OrthoFinder_glibc-2.15.tar.gz binary package. The problem looks like it's associated with a problem with too many processes/pipes occuring specifically with older versions. There's some discussion of the issue here:
https://stackoverflow.com/questions/14853712/handle-out-of-range-in-select

If you find this occurring with anything other than the OrthoFinder_glibc-2.15.tar.gz package please let me know.

Fix:
I've put in some error handling code to detect this error and exit gracefully. It also advises on the recommended work-around, which is to use the source code version of OrthoFinder. An alternative is to run it with only a single process for this part of the algorithm using the orthofinder command line option '-a 1'.

The version of GLIBC that was being supported by this package is now very old, from March 2013! Some users had requested this a couple of years ago but I will now retire it and replace it with a newer version, which means the error should never occur anyway.

[Proginski]

Dear David,

I just ran into a similar error with the 2.5.4 version

WARNING: Unknown error
<type 'exceptions.UnboundLocalError'>
local variable 'og_name' referenced before assignment
Current orthogroup OG0005416
WARNING: Unknown error
<type 'exceptions.UnboundLocalError'>
local variable 'og_name' referenced before assignment
Current orthogroup OG0005417
...
Writing results files

2023-03-01 14:55:02 : Writing results files

Results:
/real/path/but/changed/here/manually

CITATION:
When publishing work that uses OrthoFinder please cite:
Emms D.M. & Kelly S. (2019), Genome Biology 20:238

If you use the species tree in your work then please also cite:
Emms D.M. & Kelly S. (2017), MBE 34(12): 3267-3278
Emms D.M. & Kelly S. (2018), bioRxiv https://doi.org/10.1101/267914

I initially ran into this other issue #624
So I followed the 4 steps debug you provided and get to the current bug when running the 4th step/command.

Very first command : orthofinder -f OF -M msa -T iqtree -t 96 (4 species, 250 gb ram availiable)

Any suggestion ?