-
-
Notifications
You must be signed in to change notification settings - Fork 7.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How to disable SSL certificate verification in Python? #5394
Comments
|
@synchronizing ↓ does this help? session = cfscrape.create_scraper()
session.verify = False
ex = getattr(ccxt, exchange)(
{
"session": session,
"enableRateLimit": False,
}
) |
Python 3.7.2 and using sync (tested with cfscrape, and without). Will give it a try on async as well, just to be sure, as I know |
@synchronizing let us know if the comment above does not resolve the issue for you: #5394 (comment) |
Tested with the above with the same SSL verification error, unfortunately. |
Tested with However, |
This page says it should have worked with the sync version as well: https://2.python-requests.org/en/master/user/advanced/#ssl-cert-verification @synchronizing can you post a complete short snippet of your code to reproduce it, say, 10-20 lines? We need to make sure that there's no other interference, therefore we need a complete snippet, including the instantiation. |
Sure thing @kroitor -- as of now the code is wrapped in an API server, so give me a few to extract the relevant lines to a separate text file for easy testing on your end. Also: is the |
It should work either way, as a boolean or as a string-path, if the docs are correct. |
Sounds good. Give me a few to compile the problem down to a few lines of code. |
Utilizing the following code: import ccxt
exchange = ccxt.binance()
exchange.session.verify = False # With, or without line.
fetch = exchange.fetch_ohlcv("ETH/BTC", "1m", 1514764800000)
print(fetch) With export to a man-in-the-middle proxy: export http_proxy=http://127.0.0.1:8888
export https_proxy=http://127.0.0.1:8888 I still receive error on the sync version of I just came to realize the reason it might have worked with |
Update: async def fetch_stuff():
exchange = ccxt.binance({"aiohttp_proxy": "http://127.0.0.1:8888", "verify": False})
fetch = await exchange.fetch_ohlcv("ETH/BTC", "1m", 1514764800000)
await exchange.close()
return fetch
print(asyncio.get_event_loop().run_until_complete(asyncio.gather(fetch_stuff()))) With implicit |
async def fetch_stuff():
exchange = ccxt.binance({"aiohttp_proxy": "http://127.0.0.1:8888", "verify": False})
fetch = await exchange.fetch_ohlcv("ETH/BTC", "1m", 1514764800000)
await exchange.close()
return fetch ↑ This is not a correct way of configuring it. You should add an async session and set More about it here: |
It sync-misbehavior might be a bug in the MITM proxy: https://www.google.com/search?q=python+https+proxy+ssl+verify+requests. Looks like you're not the only person having difficulties when using proxies + ssl verify. |
When you say this, would this be the correct format? async def fetch_stuff():
exchange = ccxt.binance({"aiohttp_proxy": "http://127.0.0.1:8888"})
exchange.session.verify = False
fetch = await exchange.fetch_ohlcv("ETH/BTC", "1m", 1514764800000)
await exchange.close()
return fetch
print(asyncio.get_event_loop().run_until_complete(asyncio.gather(fetch_stuff()))) |
Tell me about it -- SLL + proxies is a nightmare, as I've come to find out 😩. However, |
Nope. This would be the correct format: import aiohttp
import asyncio
event_loop = asyncio.get_event_loop()
async def fetch_stuff():
connector = aiohttp.TCPConnector(ssl=False, loop=event_loop)
session = aiohttp.ClientSession(loop=event_loop, connector=connector, trust_env=True)
exchange = ccxt.binance({"aiohttp_proxy": "http://127.0.0.1:8888", 'session': session})
fetch = await exchange.fetch_ohlcv("ETH/BTC", "1m", 1514764800000)
await exchange.close()
return fetch
print(event_loop.run_until_complete(asyncio.gather(fetch_stuff()))) Does that help? |
Yes, it did! I received a request in the |
@synchronizing something like the above should work for the |
Perfectly understandable -- all I was hoping for was a proper |
@synchronizing are you ok if we close this for now? |
Forgive me for the repetitive questioning: but with If this is beyond the project due to Also, completely irrelevant: Don't fall for |
Yes, that should be possible. However, there may be bugs outside of CCXT: I will add the |
Sounds like a plan! Again, I appreciate the help. Feel free to close this issue and re-open when I am needed for testing. |
@synchronizing should not take too long, will let you know when it's there, 30-60 minutes. |
@synchronizing found a couple of minor issues along the way, added the fixes, so, it will arrive shortly (5-10 minutes). Standing by. |
What version should I be on the look out for? |
@synchronizing 1.18.844 (the one upcoming). |
Ok, it has arrived, looking forward to hearing from you. |
import ccxt as ccxt
exchange = ccxt.binance(
{
"proxies": {"http": "http://127.0.0.1:8888", "https": "http://127.0.0.1:8888"},
"verify": False,
}
)
fetch = exchange.fetch_ohlcv("ETH/BTC", "1m", 1514764800000)
print(fetch)
import ccxt.async_support as ccxt
import asyncio
async def fetch_stuff():
exchange = ccxt.binance({"aiohttp_proxy": "http://127.0.0.1:8888", "verify": False})
fetch = await exchange.fetch_ohlcv("ETH/BTC", "1m", 1514764800000)
await exchange.close()
return fetch
print(asyncio.get_event_loop().run_until_complete(asyncio.gather(fetch_stuff()))) SSL error is thrown for the above code. However, it does work when loading |
Ok, I've added one more edit to it, let us know if 1.18.845 does not resolve the issue on the |
Sounds good -- will give it a try, thank you again. |
import ccxt.async_support as ccxt
import asyncio
async def fetch_stuff():
exchange = ccxt.binance({"aiohttp_proxy": "http://127.0.0.1:8888", "verify": False})
print(exchange.verify)
fetch = await exchange.fetch_ohlcv("ETH/BTC", "1m", 1514764800000)
await exchange.close()
return fetch
print(asyncio.get_event_loop().run_until_complete(asyncio.gather(fetch_stuff()))) |
@synchronizing doesn't work with |
Correct, unfortunately. |
@synchronizing are you sure about that? This is strange, because they should be technically equal... |
@synchronizing ah, nvm, found another bug there with the ordering of the calls, will fix it in a moment. |
They seem to be different on further testing: import ccxt.async_support as ccxt
import asyncio
import aiohttp
async def fetch_stuff():
connector = aiohttp.TCPConnector(ssl=False, loop=asyncio.get_event_loop())
exchange = ccxt.binance({"aiohttp_proxy": "http://127.0.0.1:8888", "verify": False})
print(exchange.session._connector._ssl)
print(connector._ssl)
await exchange.close()
asyncio.get_event_loop().run_until_complete(fetch_stuff()) Outputs:
Awesome, lmk! |
@synchronizing ok, let's try again with 1.18.846. Unfortunately, I can't really test it on my side atm, your help with debugging it is very much appreciated! The autobuild takes 10-15 minutes. |
My pleasure man -- I appreciate all the work on your end. I'll be on the lookout for the new release to give it a try. |
Working as expected! Thank you again, cheers. |
) * [bleutrade] started adding v3 endpoints * [bleutrade] ooops * [bleutrade] ledger fixes * [bleutrade] added example order * parse transaction fee correctly for okex3 In `parse_transaction` of okex3, [Line](https://github.com/ccxt/ccxt/blob/master/python/ccxt/okex3.py#L2099) ``` fetchWithdrawals { amount: "4.72100000", withdrawal_id: "1729116", fee: "0.01000000eth", ## here the fee is endwith 'eth' currency: "ETH", .... } ``` the value of fee is end with 'currency'. So the [code](https://github.com/ccxt/ccxt/blob/master/python/ccxt/okex3.py#L2140) cannot parse the real fee, and the feeCost is None. ``` feeCost = self.safe_float(transaction, 'fee') ``` * okex3 parseTransaction fee currency id vs code refix * okex3 typo leftover * okex3 eslint trailing space * fix #5383 * 1.18.839 [ci skip] * 1.18.840 [ci skip] * coingi referral url * 1.18.841 [ci skip] * minor edits in python/setup.py * Huobi Pro: Add back private get order/orders The order/history endpoint is only 48hrs. * 1.18.842 [ci skip] * huobipro fetchOpenOrders is now configurable fix #5376 fix #5392 fix #5388 * 1.18.843 [ci skip] * added self.verify for Python 2/3 fix #5394 * exchange.py async minor edit #5394 * exchange.py async minor edit #5394 * async_support/base/exchange.py minor simplifications #5394 * 1.18.844 [ci skip] * 1.18.845 [ci skip] * async_support/base/exchange.py __init__ ssl verify fix #5394 * 1.18.846 [ci skip] * add info the issue template * 1.18.847 [ci skip] * [coinbase] minor fix to honour limit param in fetchTransactionsWithMethod; and small tidy up in sign method * bleutrade fetchOrders edit * bleutrade parseLedgerEntry minor edits in comments * bleutrade parseLedgerEntry new unified safeCurrencyCode standard * bleutrade parseLedgerEntry edits * bleutrade old doc url is now 404, switched to v3 * bleutrade has['fetchLedger'] = true * bleutrade parseOrder minor edits in comments * bleutrade parseLedgerEntry deduplication rework startsWith → indexOf safe-methods everywhere removed duplication of description parsing for fees, and other data removed fields that do not belong to the ledger * bleutrade fetchLedger respect and propagate params
@kroitor |
Let me just add:
|
@brandsimon in this particular case, the MITM attack is done deliberately by the owner. But in general, yes, I think it would be great to have a warning so that we keep people aware! ) @synchronizing thx for the hints! |
I have read the docs up and down and I can't seem to find a reference for disabling SSL certificate verification. As of right now, I currently have a project where I am doing an intentional man-in-the-middle attack to switch proxies on need-bases.
On the return of anything but a
200
, the proxy switcher automatically switches proxy and try again. A man-in-the-middle is needed to verify that the HTTPS requests are coming back with the proper headers, and this part seems to work fine. However, communicating between the client and proxy switcher seems to be returning back (expected) SSL certificate issues, as proxy switcher automatically generates its own local certificate. This, though, I would like to disable.Without adding the certificate to my local trust env, I receive the error:
Which is warranted, but would be best if could be disabled by some exchange flag. When adding the certificate to my local trust env, I receive the error
Which is also warranted, but a check I would much rather disable. Any help would be appreciated. My idea would be something simple as:
Note: I have tried the
verify
flag with no success.The text was updated successfully, but these errors were encountered: