Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nse f&o data is now available in original nseindia.com today but not in www1.nseindia.com #4

Open
kalyan4512 opened this issue Mar 2, 2020 · 13 comments

Comments

@kalyan4512
Copy link

today observed that f&o still can not be downloaded ,when checking saw that it is not available(2/3/2020 data) in www1.nseindia.com but is available in nseindia.com but here file size is bigger(i guess). sometimes i saw that they upload f&o bhabcopy latter , if that no problem.

@kalyan4512
Copy link
Author

hi virresh ,r u active here? we need this utility very much ,in that case the users can contribute,if u dont mind as per their capacity as i think, so if u check this comment pls give ur view .

@virresh
Copy link
Owner

virresh commented May 1, 2020

Hi @kalyan4512
I'm a bit busy with university work. Though please let me know the exact problems. I'll try to get them solved when time allows, can't make promises though. All contributions are much appreciated 😅

@kalyan4512
Copy link
Author

kalyan4512 commented May 2, 2020 via email

@virresh
Copy link
Owner

virresh commented May 2, 2020

Hi Kalyan.
Thanks for the monetary contribution offer 😅 , I'll set up some project sponsorship programme in future keeping this in mind to help fund this project and keep it free to use for all.

Thanks for being an active user. I'll try to look into the issue of currency downloads and latency and try to work on a new release in a couple of months.

@kalyan4512
Copy link
Author

kalyan4512 commented May 2, 2020 via email

@virresh
Copy link
Owner

virresh commented May 3, 2020

Can you elaborate a bit about the latency issue?
I tried benchmarking the performance and a day's bhavcopy was downloaded and extracted in under a second on a decent internet connection (~2 MBps which is equal to approx 16 Mbps)?

@kalyan4512
Copy link
Author

kalyan4512 commented May 4, 2020 via email

@virresh
Copy link
Owner

virresh commented May 5, 2020

Hi kalyan,
I replicated the issue

Problem seems to be the usage of www1.nseindia.com server
Here is relevant benchmarks with wget on linux:

From the new server:

$ time wget https://archives.nseindia.com/content/historical/EQUITIES/2020/MAY/cm05MAY2020bhav.csv.zip
--2020-05-05 21:53:39--  https://archives.nseindia.com/content/historical/EQUITIES/2020/MAY/cm05MAY2020bhav.csv.zip
Resolving archives.nseindia.com (archives.nseindia.com)... 23.54.90.113, 2600:140f:5:994::2630, 2600:140f:5:9a2::2630
Connecting to archives.nseindia.com (archives.nseindia.com)|23.54.90.113|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 67756 (66K) [application/zip]
Saving to: ‘cm05MAY2020bhav.csv.zip.1’

cm05MAY2020bhav.csv.zip.1                            100%[=====================================================================================================================>]  66.17K  --.-KB/s    in 0.02s   

2020-05-05 21:53:39 (2.82 MB/s) - ‘cm05MAY2020bhav.csv.zip.1’ saved [67756/67756]


real	0m0.192s
user	0m0.003s
sys	0m0.009s

From the old server:

$ time wget https://www1.nseindia.com/content/historical/EQUITIES/2020/MAY/cm04MAY2020bhav.csv.zip
--2020-05-05 21:53:54--  https://www1.nseindia.com/content/historical/EQUITIES/2020/MAY/cm04MAY2020bhav.csv.zip
Resolving www1.nseindia.com (www1.nseindia.com)... 23.54.90.113
Connecting to www1.nseindia.com (www1.nseindia.com)|23.54.90.113|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 68907 (67K) [application/zip]
Saving to: ‘cm04MAY2020bhav.csv.zip.1’

cm04MAY2020bhav.csv.zip.1                            100%[=====================================================================================================================>]  67.29K  --.-KB/s    in 0.03s   

2020-05-05 21:53:59 (2.15 MB/s) - ‘cm04MAY2020bhav.csv.zip.1’ saved [68907/68907]


real	0m4.295s
user	0m0.010s
sys	0m0.021s

I'll try to rectify the latency issue by switching to the newer sources for data
Can't give an estimate on how long will it take me to implement the solution though

@kalyan4512
Copy link
Author

kalyan4512 commented May 9, 2020 via email

@kalyan4512
Copy link
Author

hi virresh ,if u help this time ,it is taking time now to download,mainly indices ,it seems.
i dont know where to write,if u want u can share ur bank credential or paytm etc . if u answer ,then may be i can get it in my email or if u have any site. regard kalyan

@virresh
Copy link
Owner

virresh commented Jun 12, 2020

Hi Kalyan,
I'm trying to re-develop the software for more customizability so that people can load different settings easily.
You can track the progress here: https://github.com/virresh/StockD/
It is far from complete, I hope to be able to finish it by this month. Sorry for the inconvenience and thanks for the support!

@kalyan4512
Copy link
Author

kalyan4512 commented Jun 18, 2020 via email

@virresh
Copy link
Owner

virresh commented Jun 23, 2020

format of data eg ticker,date(yyyymmdd),open,high,low,close,volume,oi(in
case of future). sud keep intact as like recent software, as we have
database according to that format

Sure, I'll try to keep output format such that it's fully compatible with nse-eod-data-downloader.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants