IndexError: list index out of range

[JensenSpec-Nordic]

My get_news function in main_func.py throws the following error when I try to run through my StockList:

row.xpath("./td")[1].cssselect('div[class="news-link-right"] span')[0].xpath("text()")[0][1:] IndexError: list index out of range

I understand that it has something to do with the XPath-code for locating the "news-link-right"-element, but I have been unable to figure out what I need to change in order to make it work.

Anyone able to help here?

[f-izzat]

    stock_list = Screener(filters=filters, table='Performance', order='price')  # Get the performance table and sort it by price ascending
  File "/Users/faiz/miniconda3/envs/quant/lib/python3.8/site-packages/finviz/screener.py", line 128, in __init__
    self.data = self.__search_screener()
  File "/Users/faiz/miniconda3/envs/quant/lib/python3.8/site-packages/finviz/screener.py", line 436, in __search_screener
    self._rows = self.__check_rows()
  File "/Users/faiz/miniconda3/envs/quant/lib/python3.8/site-packages/finviz/screener.py", line 404, in __check_rows
    self._total_rows = scrape.get_total_rows(self._page_content)
  File "/Users/faiz/miniconda3/envs/quant/lib/python3.8/site-packages/finviz/helper_functions/scraper_functions.py", line 46, in get_total_rows
    etree.tostring(total_element[0]).decode("utf-8").split("</b>")[1].split()[0]
IndexError: list index out of range

Just ran the given example, got the same error

[AGG2017]

It is working for me. My version is different from the last available here with only 3 fixes:

1. in function __get_table_headers(self) was changed as shown in Screener header now includes whitespace characters #166
2. in function get_news(ticker) first line inside for was changed to raw_timestamp = str(row.xpath("./td")[0].xpath("text()")[0][0:-2]).strip() in order to remove the white space to get the proper timestamp
3. the function get_total_rows(page_content) was changed the way shown in Index Error in get_total_rows #167

[AGG2017]

Python 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from finviz.screener import Screener
>>> filters = ['exch_nasd', 'idx_sp500']
>>> stock_list = Screener(filters=filters, table='Performance', order='price')
100%|█████████████████████████████████████████████| 8/8 [00:02<00:00,  2.99it/s]
>>> 

Do the 3 modifications over the latest commit here and everything will be working. Cannot help you more than that.

[tonyz80]

So, in order to get it to work on my system, is there a new release that I need to install or do I need to make all the updates mentioned above myself on my end? It would be great if a guide for the above fix is provided.

[AGG2017]

You can wait for a new release but it may take forever. The updates are far behind the new changed done in finviz.com, so it is better to use the latest github updates plus the fast fixes provided here in the comments.
Depending on how old is your version, it may need to be upgraded to the latest version here first.
Then from the error message it is easy to find where are installed the sources. For the upper example:

  File "/Users/faiz/miniconda3/envs/quant/lib/python3.8/site-packages/finviz/screener.py", line 436, in __search_screener
    self._rows = self.__check_rows()

we can see the sources are in the folder

/Users/faiz/miniconda3/envs/quant/lib/python3.8/site-packages/finviz/

Edit the files screener.py on line 416 and main_func.py on line 110 with the provided fixes. My fix 3 is already done in the latest sources, so if you updated to the latest github commit you don't need to edit get_total_rows, already done. That's all.

[tonyz80]

Thank you. I implemented all 3 fixes above, on my local system, and no issues: everything worked as expected.
Thanks!

[JensenSpec-Nordic]

Thank you for all of the great replies and suggestions!
I have now implemented all of the above fixes, but unfortunately still get the same error.
There must be something I'm missing - guess there's nothing to do but continue to try and find the error ;).

[AGG2017]

It might be from a pretty old version of your finviz installation. It is better to remove it first (pip uninstall finviz) and then to install the latest version from github (pip install -U git+https://github.com/mariostoev/finviz). Then to add first two fixes (3rd one is already implemented). If it still doesn't work you should double check if you fixed properly the latest version. Manual download of the latest version in another folder and folder comparing tool with your installed version can confirm you have the latest version and the difference should be only inside the two fixed functions.