IndexError: list index out of range

[GoogleCodeExporter]

What steps will reproduce the problem?
1. downloaded v 2.0 from site
2. run command: python appdailysales.py -a my_iTC_username -p my_iTC_password -v

What is the expected output? What do you see instead?
Expected: connection to iTC and download of stats file.
Instead:
-- begin script --
Traceback (most recent call last):
  File "_appdailysales.py", line 358, in <module>
    sys.exit(main())
  File "_appdailysales.py", line 351, in main
    downloadFile(options)
  File "_appdailysales.py", line 246, in downloadFile
    dailyName = match[0]
IndexError: list index out of range

What version of the product are you using? On what operating system?
2.0. Mac OS X 10.6.4. Python 2.6.1 (r261:67515, Feb 11 2010, 00:51:29) [GCC 
4.2.1 (Apple Inc. build 5646)] on darwin

Please provide any additional information below.
Also tried it on a rackspace cloud server and gave the same results.

Original issue reported on code.google.com by Erizo...@gmail.com on 13 Sep 2010 at 8:43

[GoogleCodeExporter]

Apple changed terms and conditions, so it was prompting me to accept the new 
terms and conditions by logging into the developers portal right after I logged 
into iTC.
Hope this helps someone else with the same problem!

Original comment by Erizo...@gmail.com on 13 Sep 2010 at 8:53

[GoogleCodeExporter]

I'm getting a related error-- (my contract has been signed already).  

-- begin script --
Traceback (most recent call last):
  File "../appdailysales.py", line 358, in ?
    sys.exit(main())
  File "../appdailysales.py", line 351, in main
    downloadFile(options)
  File "../appdailysales.py", line 244, in downloadFile
    viewState = match[0]
IndexError: list index out of range

Original comment by twigbra...@gmail.com on 14 Sep 2010 at 9:25

[GoogleCodeExporter]

@twigbranch Are you able to view the sales and trends reports using a web 
browser? 

Original comment by whitepea...@gmail.com on 15 Sep 2010 at 7:42

[GoogleCodeExporter]

I updated the script (version 2.0.1) to include additional error reporting when 
the sales report web page is not accessible. This will hopefully clarify why 
the 'list index out of range' exception is thrown.

Whenever the sales report web page is inaccessible this error is raised. 
Reasons for being inaccessible include delayed reporting from Apple and 
unsigned contracts. I'm sure there are other reasons too.

Original comment by whitepea...@gmail.com on 15 Sep 2010 at 8:17

• Changed state: Accepted

[GoogleCodeExporter]

In version 2.0.1, there is a typo causing a "list index out of bounds" on line 
244: "value1" should be just "value".

Original comment by bvander...@gmail.com on 15 Sep 2010 at 3:13

[GoogleCodeExporter]

Oh shit! LOL. I forgot to remove the typo. It was needed to test the error. 
Fixing it right now.

Original comment by kirbyt.w...@gmail.com on 15 Sep 2010 at 3:25

[GoogleCodeExporter]

The typo has been fixed. The latest version, 2.0.2, is good. Sorry about that. 
That's what I get for committing changes made in the middle of the night.

Original comment by kirbyt.w...@gmail.com on 15 Sep 2010 at 3:32

[GoogleCodeExporter]

Still the same issue for me. I updated to 2.0.2 but it didn't help. It seems 
that it goes to the login page again after 
'/WebObjects/iTunesConnect.woa/wo/2.0.9.7.2.9.1.0.0.3'. It succeeds on "first" 
login page and "shows" home page correctly. Then it fails. Btw thanks for that 
script :)

Original comment by goo...@nigdzie.pl on 16 Sep 2010 at 8:39

[GoogleCodeExporter]

@kapelko, are you able to access the sales reports and download using a web 
browser? Do you have multiple vendors? Multiple vendors is not support since I 
don't know what that web page looks like.

Original comment by kirbyt.w...@gmail.com on 16 Sep 2010 at 10:11

[GoogleCodeExporter]

When I login using web browser I can see standard page where I can choose 
"sales and trends", then I go directly to subdashboard.faces (with one 
redirection I think). No multiple vendors.

webFormLoginData = urllib.urlencode({'theAccountName':options.appleId, 
'theAccountPW':options.password, '1.Continue':'0'})                             

html = readHtml(opener, urlActionLogin, webFormLoginData)                       


print html <- gives me standard page sources with correct 'sales and trends' 
link

urlSalesAndTrends = urlITCBase % 
'/WebObjects/iTunesConnect.woa/wo/4.0.9.7.2.9.1.0.0.3'                          

html = readHtml(opener, urlSalesAndTrends)                                      


print html <- gives me login page again

I am not familiar with python and I don't know how to bring more details :(

Additional info:
python-2.4.3-27.el5
python-BeautifulSoup-3.0.7a-3.el5 (without that package also doesn't work)

Original comment by goo...@nigdzie.pl on 16 Sep 2010 at 11:48

[GoogleCodeExporter]

Are you providing your iTC credentials? The command line should look something 
like this:

python appdailysales.py -a MyAppleID -p MyPassword

Or modify the script to include your credentials. The credential variables are 
found on line 54 and 55.

Original comment by kirbyt.w...@gmail.com on 16 Sep 2010 at 12:06

[GoogleCodeExporter]

Yes of course :) I added them to the script. And as I said before it logs in 
successfully - first `print html` shows sources of page after login, with all 
the links like 'sales and trends'. It fails on next request. For me it seems 
like problem with session variables or something like that... but it was 
working correctly before the 10th. 

Original comment by goo...@nigdzie.pl on 16 Sep 2010 at 12:14

[GoogleCodeExporter]

@kapelko I'm not sure why the script is failing with your account, and I'm not 
able to recreate the problem. I'm assuming you added the 'print html' statement 
to the script since there isn't any in version 2.0.2. Of course the login URL 
changed between v1.x and v2.0 of the script so you must be using the new script 
since you said the script is able to log in.

When you look at the raw html, is the sales and trends link this:

https://itunesconnect.apple.co/WebObjects/iTunesConnect.woa/wo/2.0.9.7.2.9.1.0.0
.3


You are correct that the previous script (version 1.10) did work before 
September 10. Actually, it worked up until September 9, which is the day Apple 
rolled out the new web site. The new web site required a new script, hence v2.

Original comment by kirbyt.w...@gmail.com on 16 Sep 2010 at 10:51

[GoogleCodeExporter]

I clearly understand everything. No worries. I'll try to find it by myself. I 
think it must be something specific on my side (python, server settings, 
whatever). The 2.0.2 just doesn't keep the session between calls. I test it on 
2 servers and on 4 separated itc accounts (not connected, just separated 
completely). I am going to try to debug why it loses the session after 
redirection. Thanks for your help.

Original comment by goo...@nigdzie.pl on 17 Sep 2010 at 7:35

[GoogleCodeExporter]

@kapelko Check out version 2.1 posted last night. It now has a feature that 
saves the html to a file called temp.html when an error occurs and the -v 
option is used.

What version of python are you using? I wonder if that could be the problem.

Original comment by kirbyt.w...@gmail.com on 17 Sep 2010 at 11:39

[GoogleCodeExporter]

I got python-2.4.3-27.el5

The 2.1 gives me this error:

File "./appdailysales.py", line 207
    with open(os.path.join(options.outputDirectory, 'temp.html'), 'w') as f:

Original comment by goo...@nigdzie.pl on 17 Sep 2010 at 2:51

[GoogleCodeExporter]

I had the same issue, I changed line 207 from this:

with open(os.path.join(options.outputDirectory, 'temp.html'), 'w') as f:
  f.write(html) 
  f.close()

to this:

f = open(os.path.join(options.outputDirectory, 'temp.html'), 'w')
        try:
            f.write(html)
        finally:
            f.close()

But now I'm getting the "IndexError: list index out of range" error. I can't 
figure out why though.

Original comment by mintdesi...@gmail.com on 21 Sep 2010 at 7:55

[GoogleCodeExporter]

Hi All,

Apple made some minor changes over the weekend. Also, the AppViz guys said 
there are some account specific issues at hand too. This is likely causing the 
random IndexError errors. I received the error on Sunday, once, and have not 
been able to recreate it.

I've unfortunately have not had time to make any changes to the script this 
week. I'm hoping to work on it more next week. I'm still running version 2.0 of 
the script on my server and other than the error on Sunday, it hasn't had any 
problems downloading reports. I have a cron job scheduled to run the script at 
4pm Eastern each day. The later time helps avoid errors caused when Apple 
delays the reports.

Hope the helps. Meanwhile, I find time next week to make improvements on the 
script.

-KIRBY

Original comment by kirbyt.w...@gmail.com on 21 Sep 2010 at 8:05

[GoogleCodeExporter]

The "with" statement has been removed and replaced with what is hopefully more 
compatible code. The latest is in the trunk.

Original comment by kirbyt.w...@gmail.com on 23 Sep 2010 at 12:31

[GoogleCodeExporter]

For those getting the "IndexError: list index out of range" error, I'm guessing 
this might be an account specific issue, which explains why I'm not able to 
recreate the problem. The AppViz app was recently updated to fix account 
specific issues, which reenforces my belief that the problem is account 
specific.

Since I'm not able to recreate the problem, I need your help. If you are 
consistently getting this error, please run the script using the -v parameter. 
A file named 'temp.html' will be stored in the output directory. Please email 
me the file (or post the file here if you are comfortable with that) so I can 
trouble shoot the problem. The email address is support at white peak software 
dot com.

Thanks.

Original comment by kirbyt.w...@gmail.com on 23 Sep 2010 at 12:58

[GoogleCodeExporter]

I have been consistently getting the "list index out of range" error.  With 
version 2.2 the output is:

-- begin script --
Unable to load the sales report web page at this time. A number of reasons can 
cause this including delayed reporting and unsigned contracts. Try again later 
or sign into iTunes Connect and verify access.
Traceback (most recent call last):
  File "/home/********/appdailysales.py", line 393, in ?
    sys.exit(main())
  File "/home/********/appdailysales.py", line 386, in main
    downloadFile(options)
  File "/home/********/appdailysales.py", line 264, in downloadFile
    viewState = match[0]
IndexError: list index out of range

This is while trying to retrieve the report for 09222010, which I know exists 
because I can see it when I log in manually.

The "temp.html" file gets created, but there isn't much point sending it to you 
as it is completely empty (0 bytes)!

Before Apple changed everything, the old script was working perfectly with the 
same parameters.

Original comment by mbro...@mophy.com on 24 Sep 2010 at 10:35

[GoogleCodeExporter]

It seems like origin of the request has something to do with this. While the 
script works flawlessly when I run it on my machine at work or at home I get 
the infamous "list index out of range" error when I run it from the company's 
Rackspace Cloud server.
I'll try to dig a bit deeper and see if something more helpful than this 
message comes up.
Thanks!

Original comment by Erizo...@gmail.com on 27 Sep 2010 at 8:17

[GoogleCodeExporter]

@Erizoman That's interesting. I'm running the script without problem from home 
on Comcast and on a Ubuntu 8.04 server hosted at Slicehost.com.

What version of python are you using on the Rackspace cloud server? I'm also 
wondering if the version of python is part of the problem.

Thanks,
-KIRBY

Original comment by kirbyt.w...@gmail.com on 28 Sep 2010 at 1:43

[GoogleCodeExporter]

I have tested it on 3 servers and all are in OVH. It doesn't work. But from my 
macbook at home it works for the same account! What does it mean?

Regards

Original comment by goo...@nigdzie.pl on 28 Sep 2010 at 8:33

[GoogleCodeExporter]

Additional info: on mac I have python 2.6.1 and on servers there are 
python-2.4.3-27.el5

Original comment by goo...@nigdzie.pl on 28 Sep 2010 at 8:35

[GoogleCodeExporter]

The version of python could be the issue. I've only ever tested using python 
2.5.x and 2.6.x.

Original comment by kirbyt.w...@gmail.com on 28 Sep 2010 at 1:28

[GoogleCodeExporter]

Works on 2.6. Perfect! Thanks!

Original comment by goo...@nigdzie.pl on 28 Sep 2010 at 3:36

[GoogleCodeExporter]

@Kirby
The cloud server is using 2.4.3, and I am running 2.6.1 on my machines.
I tried to do some troubleshooting and the issue seems to be that after 
"clicking" on the Sales and Trends link the next page the script downloads is 
another login page. The link on the sales and trends also has a different "id" 
(0.0.9.7.2.9.1.0.0.3 instead of 2.0.9.7.2.9.1.0.0.3). Attached is the new login 
page received on the 3rd page download.
Please let me know if I can provide anything else to help you guys with this 
issue (I've tried logging in again, but throws me back into this same page).
Thanks!

Original comment by Erizo...@gmail.com on 28 Sep 2010 at 5:29

Attachments:

• new_login.html

[GoogleCodeExporter]

@Erizoman I do not think the first number on the sales and trends link as any 
effect. That number changes as you navigate. Still, I'll look into screen 
scraping the page to determine the sales and trends link.

As for the cloud server, I'm not sure the script works using python 2.4.3.  
I've only tested it using 2.5.x and 2.6.x.  I would not be surprised if the 
error is caused by some incompatibility with python 2.4.x.

-KIRBY

Original comment by kirbyt.w...@gmail.com on 29 Sep 2010 at 8:17

[GoogleCodeExporter]

New version posted, version 2.3. This version will screen scrape the site for 
the Sales and Trends URL. I'm crossing my fingers this will help resolve issues 
for some.

Original comment by kirbyt.w...@gmail.com on 29 Sep 2010 at 8:36

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Apple made more changes.

I updated the script to work with the latest changes from Apple. The latest 
script is in svn.  I'm still testing and would appreciate if others can test it 
too to confirm it works.  Once I'm positive it works, I'll make the "official" 
release and post a .zip download file.

Thanks,
-KIRBY

Original comment by kirbyt.w...@gmail.com on 2 Oct 2010 at 1:13

[GoogleCodeExporter]

Hi All,

My son was nice enough to wake me up in the middle of the night, and since I 
couldn't go back to sleep, I decided to fix the script. Version 2.4 is now 
available in the download area. This updates fixes problems caused by changes 
made by Apple the other day. I also think I found the source of the 
"IndexError: list index out of range" that some have reported and have 
hopefully fixed it once and for all.

Please note: I have only tested the script using Python 2.5 and 2.6. Based on 
previous reports, I doubt the script will run using python 2.4 or earlier. And 
I doubt it will run using Python 3 and greater.

Please get the latest and let me know if there are any new problems.  If there 
are no additional problems then I will close this issue within a few days.

Thanks,
-KIRBY

Original comment by kirbyt.w...@gmail.com on 2 Oct 2010 at 11:21

• Changed state: Started

[GoogleCodeExporter]

Original comment by kirbyt.w...@gmail.com on 31 Oct 2010 at 5:29

• Changed state: Fixed