iOS7 Support and Discussion #542
Comments
|
I really did not want this to turn into a focal point for violations of the developer NDA. Please do not discuss unreleased Apple software on this forum. For now on I'm just going to delete any such posts without warning or modification. |
|
Do not upgrade your devices to iOS 7 GM Seed if SiriProxy is important to you. Please keep in mind that iOS 7 is not yet a public release and we are still bound by an iOS developer NDA. We are investigating any issues for possible solutions. We will provide updates when we have relevant information we can disclose. |
|
Now that iOS 7 has officially been released lets get this working. It's been a few months since i've looked at the source. I don't think iOS7 resolve to guzzoni.apple.com anymore. With my local DNS server handling that domain, siri will still function on iOS7. Perhaps they changed the servers DNS name, or possibly they internalized the ip address for it. They could have also set iOS7 to use a specific name server for Siri. I'll do some packet tracing at work today and see what i can find out. |
|
I know you are trying to be helpful, but please refrain from posting conjecture or speculation. It does not serve anyone any good and dilutes resources from real solutions. Pete is already looking into it. P.S. I'll be sure to let your boss know you are using company resources and time to diagnose the problem. ;) |
|
Guys I came across this while looking for a solution to get siri-proxy working whilst I was in the beta; The server is now daryl.apple.com, which I got also from my own tests. |
|
I know everyone is anxious and trying to be helpful, but once again please refrain from conjecture and speculation. What someone may have discovered while iOS was in beta is not likely to apply to the public release. From now on any information or posts that are known to be inaccurate will automatically be deleted. |
|
The struture of the siri's object have been changed. "Properties" doesn't exist now. It's my observation, i modified the siriproxy code, but the protocol have more changes, because even i modified the structure of the object, siriproxy doesn't work. If someone has more information, let me know and we can work together (@Fab_Lan or html5webapp.blogspot.com). Example for SpeechRecognized (IOS7): |
|
I've noticed 2 different headers with the new iOS 7 (I run the non-beta version). Maybe this can help someone. Also, in another test, I changed some of the SiriProxy code and had what I believe were problems with the SSL connection with the iDevice. (the log below is from the non-modified SiriProxy server and in this test I was able to get a response from Siri) [Info - Server] DNS Server started, tainting 'guzzoni.apple.com' with 10.0.1.25 |
|
Also another issue I discovered was that Siri on my iPhone, when connect to the SiriProxy over wifi, uses cellular data to get the requests. When I have cellular + wifi enabled Siri works and presents 2 different headers. When I disable it (wifi only), it shows only one header and does not work. I tested it on my wifi iPad and I get the same result as the iPhone with cellular off, and in this log it appears to be doing circles, differently from the one I posted above. (the above one was using wifi + 3g) |
|
Siri was known to "leak" over the cellular connection in iOS 6 when the cellular connection was determined to be better, which was often the case for me when connected to LTE. Thus one had to disable the cellular data connection. I came across this quote by Olivier Bonaventure, one of, if not the leading researcher in the field of Multi Path TCP. I am not an expert, but MPTCP seems to offer multiple connections over multiple IP addresses (e.g. Cellular, Wifi) to provide more robust services when one connection quality is poor or is dropped. While these reports suggest Apple is limiting MPTCP between Wifi and cellular and only for Siri, I can see the benefits of providing more robust connection to services when switching cell towers, when for a brief moment your phone has multiple cellular data connections. Also could be a used for persistent MPTCP LTE connections to improve throughput and robustness. Regardless, from what little I have read on MPTCP, it seems turing off the cellular connection to limit Siri to using Wifi only would thwart Siri trying to use both. |
|
@elvisimprsntr Maybe this could represent a problem. I'm looking into MPTCP. Therefore, it is possible that the iPhone prefers wifi over cellular, so if the SiriProxy connection is working it won't be a problem. Another way I though to fix this is to use VPN. While I using iOS 6, I used VPN over cellular and it worked perfectly with SiriProxy. Although, with or without the MPTCP, SiriProxy is yet not able to work with iOS 7. I think the MPTCP problems will be better figured out when it is working. For now we will have to disable cellular while we try to find a fix to SiriProxy. When I have more info of how Siri on iOS 7 works I'll let you know. In the mean time, I'll have a look into MPTCP. |
|
@bgottsch I used a VPN to invoke SiriProxy in iOS 6 with success as well. At the moment I am having trouble getting Siri to route thru the VPN connection. It seems Siri ignores the "route all traffic thru VPN" setting, or at least the symptoms appear that way. |
|
@elvisimprsntr I just tested it using VPN (I run iVPN) over 3G and the SiriProxy identifies a connection from my iPhone. I found 2 headers in the log: [Header - iPhone] ACE /ace HTTP/1.0 and [Header - iPhone] HEAD /ace HTTP/1.1 When I ask Siri something over this VPN connection it does not respond, so I think it is working. On my VPN setup, I point my computer running SiriProxy as the DNS server and run RubyDNS from the SiriProxy on the same address. Maybe this is preventing your iPhone to connect to the server. Also I have found this link http://www.networkworld.com/news/2013/091913-ios7-multipath-273995.html which says "MPTCP is spreading the connection’s data across several subflows, sending it over the least congested paths." |
|
@bgottsch I use DD-WRT for both my VPN server and to perform a transparent guzzoni.apple.com redirect, which worked well under iOS 6. The source of the information all points to Professor Olivier Bonaventure, http://perso.uclouvain.be/olivier.bonaventure/blog/html/2013/09/18/mptcp.html Regardless, it seems like getting SiriProxy working with iOS 7 will be more work than anticipated. I see it as a challenge and opportunity to learn. |
|
@elvisimprsntr It seems that iOS 7 avoids using VPN to connect Siri. I am having problems with the VPN connection and Siri. It appears to be avoiding, as you said before. The DNS + wifi-only solution is working. For now I'll use that to see whats happening to SiriProxy. I agree, it will be a challenge. |
|
@elvisimprsntr @plamoni I could be wrong, but I think the problem is on EventMachine and on the SSL connections. |
|
I'd be kind of surprised if the issue was with the SSL. The SSL is the lowest level thing (from an OSI layer prospective) that gets logged. It tosses bunches of logging when you're having connection issues, even if those are actually TCP or DNS related. If the SSL were failing, we'd be having trouble getting any data at all. I think it's more likely that the problem is related to the multi-path TCP used by Siri in iOS7. It sounds like the VPN use-case might not be viable anymore. Again, I've had just enough free time to follow these threads, I haven't had time to do any real investigation yet. My planned approach is to do basically the opposite of what I did when building the proxy. Essentially, when I started out, I would build the protocol encoders/decoders in one at a time, starting with a simple passthrough that handles the SSL MITM, then building on various elements of the protocol stack. If you look into the connection class, you'll see this behavior and should be able to short-circuit it. By cutting out the higher level decoders (which might be broken), it should be easier to determine at what level things are breaking down. My worry is that the problem occurs within the TCP layer itself, which is actually handled by the kernel of the host OS. This is a sort-of worst-case-scenario. The only real solution would probably be to require people to implement MP-TCP on their servers, which would eliminate support for OSX and Windows hosts. Only Linux hosts would work and you'd probably have to build a customized kernel to add support for MP-TCP. Again, I've done no research at all beyond reading these threads. So don't panic. I just need time. Hope to have some this weekend. Also, I have an iPhone 5S on order. Should get here in a couple weeks. |
|
Here is a good write up on how MPTCP actually works. http://lwn.net/Articles/544399/ |
|
Installation options for various OS distributions: http://multipath-tcp.org/pmwiki.php?n=Users.HowToInstallMPTCP P.S. Sorry native Windows users, you may be out of luck unless you are willing to run Linux on a VM |
|
I wonder if OSX 10.9 Mavericks has MPTCP. I'm also installing the VM so I can test it. |
|
There seems to be a MPTCP virtualization method. https://github.com/multipath-tcp/mptcp-virtual |
|
@elvisimprsntr I've seen your link. Awesome. I'm installing it (since I run on a Mac). I'll see if I can find something new with it. |
|
This might be a solution for testing to see if MP-TCP is at the core of the On Fri, Sep 20, 2013 at 3:41 PM, Beno notifications@github.com wrote:
|
|
@plamoni @elvisimprsntr I agree. Using a VM is too complicated. I think of two solutions to this. One is to disable cellular, which will make Siri use wifi and hence connect to SiriProxy. The second is to fix the server for iOS 7 and see if Siri automatically uses wifi (since it's a more reliable connection it should be automatic). Though, for this options or any other to work, SiriProxy must first be fixed. So in the mean time, while doing the testings, using a VM is not a bad idea. Another work around would be to disable cellular. I'm currently checking the connection.rb file to see if I can find where the problem is. Let you know if I have found something. |
|
@plamoni @elvisimprsntr I think I found the problem. It may be stupid, but I think guzzoni is refusing/disconnecting from the SiriProxy. If analysed the connection.rb and it does almost everything correctly. The first and most important issue is with the new objects, which are a bit different and make the SiriProxy not interpret them. The other (which I think is the key issue) is that the iPhone, even with cellular disabled, creates new headers when it completes the SSL to guzzoni. This makes the SiriProxy/iPhone reconnect resulting in a loop hole where it never connects to guzzoni. I'm not an expert on servers so I might be wrong, but in all my testing I was not able to send/receive objects from guzzoni with iOS 7. In the same server I was able to do so with my iPad running iOS 6 (to verify if it was not another problem). A question I have is the cause of this disconnection, if it is the iPhone or guzzoni who made it. Ps.: sorry to bother on a Saturday. |
|
any updates on the development of this? |
|
Not yet. :-) |
|
Some good news: The important part is here:
This is very good news. Basically, if it's correct, it means that MPTCP is not killing SiriProxy and that it's just a change to the protocol that's breaking stuff. SiriProxy should continue to work as it does today for our purposes as long as it determines it can talk to the Apple servers over the WiFi. Also, consider that since the proxy happens via DNS, it means that the IP address the Siri client connects to is that of the proxy server. So it may try to open multiple paths to that server, but those attempted over 3G/4G/LTE should be unsuccessful (unless your SP server is public -- which is still a bad idea). Therefore, it should limit itself to the SP server and make everyone pretty happy. So just need to figure out why the protocol interpretation is broken (not necessarily easy) and hopefully we'll be back in business. Hopefully. :-) |
|
Hey guys, The Assistant+ app is great. I’ve tied it in to my Home Automation app (Home Remote) for iOS and now can directly control stuff with Siri with no need for a server as long as it’s supported kit. I want to write a plug in for Assistant+ that will read my list of actions in the Home Remote app but can’t work out how to test my extension with Assistant+. I’ve tweeted the author but had no response, do any of you have any experience with this? Ta, G
|
|
@griches no can't help with that I'm afraid, I was looking at making a plugin to interface with any Kodi client, but after looking at I was able to do exactly what I wanted just by using curl commands and the Kodi Json api....infact this (Json, curl) was how I interfaced with my home automation system. |
|
I have started playing with the Amazon Echo. Amazon released a voice API SDK recently, so I wrote the same plugin for it that I had written for SiriProxy: Github project: Second to last post here (mreyn2005): |
|
Assistant+ is awesome, I didn’t even need to write a plug in in the end. You can see a video of the results here: http://youtube.com/watch?v=UAAPpK1ZJb0
|
|
@griches that's very similar to what I've achieved....however I don't see the purpose in the App you've made....is it just for simple integration of official home kit appliances? How mine works is that as long as my home automation system(domoticz) can control a device, so can Siri.....I really like domoticz for the reason that even if the hardware isn't natively supported, as long as you can control the device from the command line, then it's very simple to integrate into domoticz, and as long as it's in domoticz.....it's in Siri! So far I've got my lights, CH and AC, and full Logitech Harmony and Kodi control....all achieved with next to no hassle. |
|
It doesn’t control any HomeKit enabled devices. Milight/LimitlessLED/EasyBulb don’t support any voice control and their apps are awful. Same goes for WeMo currently. The main aim of the app is to provide Today screen access without the need to open any apps, the voice control is just a bonus.
|
|
@griches ahhh....it does look pretty sweet having it all in the today screen. |
|
For example: if domoticz supports calling actions by URL then Home Remote would provide buttons on your Today Screen to call the domoticz actions that would call your other actions, or call them directly. You don’t always want to talk. It also supports Pebble and Apple Watch.
|
|
@griches any chance I can try your app ? I'll purchase it if it works well. |
|
Of course. DM me on Twitter @Gary_BBGames and I'll add you to the beta
|
|
@griches Hi, I'm following u on twitter but can't send u a DM. |
|
Can you just @ me of give me your username and I’ll send you one. Ta, G
|
|
So it turns out siriproxy might work again but only the new appletv. Since there is no mTCP at play on this device. I have left my DNS records on my network and I noticed Siri was not working on the new Apple TV. Turning off that particular dns zone fixed the issue. I'll test and see if I can get it working |
|
Actually the issue will be getting the cert onto the apple tv. |
|
So here’s something interesting I found out today, the Siri remote can drive the focus engine on tvOS/Apple TV. My app, Home Remote, ha been ported to Apple TV (still in review) and this works a treat. This means with the app open, you can control your home from the Siri remote. You can see a video of it here: https://youtu.be/T9_AzlE8Pxs
|
|
I should add that the TV doesn’t need to be on, you can just leave the Apple TV on, with the app open and sleep disabled and it will work.
|
|
So here you guys, here is another update from yesterdays. The Siri remote can drive the Focus engine on the Apple TV, activating any button that has the label you speak…. Now that’s all well and good but can be quite limiting… But what if there was a way to have much more text… There is! My app, Home Remote, allows custom voice commands on the iOS app, so “Study Bright” can have a voice command of “Turn on the study lights”. I tried assigning the custom voice commands to the accessibility label on the button. Here is the result: https://youtu.be/tqT49ItvQ08 This means that with Home Remote for tvOS, you will be able to call any URL command with variations of post/put/json, native WeMo commands, and milight/easybulb all with Siri when the tvOS app is open. Ta, G
|
|
I think with a combination of software I have got a pretty good SiriProxy replacement now. Home Remote for Mac mirrors the iOS app's Today Screen set up. Home Remote can control milight, memo, LIFX & Indigo natively, as well as sending TCP and UDP packets and URL commands with varying payloads. You can see the Echo turning off milight, Hue and LIFX with one action here: https://youtu.be/cSPBfL62XPU This is the end result: https://youtu.be/9GpBUbqndl0 https://youtu.be/9GpBUbqndl0 One server with Home Remote, HomeBridge and Amazon Echo Hue support means I can control absolutely anything, with my voice, internally or externally to the network. G
|
|
Of course with the echo you can hook in to Alexa voice services (avs) it would be trivial to write a Siri clone for ios that's actually utilising avs, essentially providing everyone with the same functionality of siri with a proxy, perhaps re-purposing Siri proxy to support an avs client will bring it back to life ?
|
|
But there is no clone here. It's Siri, calling any URL. Siri > home kit > fake hue > home remote.
|
|
I haven't followed this thread in a while, but is there anything like SiriProxy out there yet? I know there's Assistant+, but it just doesn't have the flexibility and power that SiriProxy had. Is there anything similar to SiriProxy yet that would give me full control over Siri and what it does/say? |
|
I don’t trust tim the crook with my data… I want my own personal cloud and this Siri is the biggest hurtle.. is there a way I can redirect Siri to my own system?? |
|
@graanco on iOS7? |
|
For any iOS version like a man in the middle that redirects the requests. Is that possible to reverse engineer it. That way all devices would use the local server instead of sending them to apple |
|
I used to use something called Siri Proxy back in the early days of Siri. |
|
… that only redirects from the phone and not every device. In order to do what want would be to use a DNS redirect the request to the server. That way everything is contained as mini pod Apple TV all phones iPads. Only issue is when you are outside the internal network but maybe use a vpn into the internal when away… |
|
@graanco what you’re suggesting isn’t possible anymore. That’s what the project on this GitHub repo originally did, back in the iOS 5 and maybe iOS 6 days. Apple broke it by implementing a couple changes, but most notably certificate pinning (IIRC) that makes it completely impossible to MITM Siri unless your iOS device is jailbroken. Even then, I’m sure the protocol has changed so much that this repo wouldn’t be of much help anymore. If you’re concerned with privacy, don’t use Siri. Or only use it for queries that will run entirely on-device, with the support for that coming in iOS 15. Update: looking back through the old posts in this thread, the issue is actually MPTCP. Either way, what you want to do isn’t possible and never really was with this project either (it passed most requests on to Apple unmodified anyway, so in order to do what you want you’d have to basically reimplement all the server side functionality as SiriProxy plug-ins). |
|
Yes host file for Siri but if you run your own dns server you could control that without jb the device. And it would control all home devices. Time to research mptcp |
|
@graanco running your own DNS is what this project did. MPTCP breaks that, because it uses multiple paths for the DNS lookup and the connection to the Siri servers (I’m a little hazy on the details after so long). It isn’t really possible anymore. If it was, this project would be active. |
|
Well I have a will and a big desire to stick it to the man!!! I am going to attack this!! And make if it needs to go to the servers it will save that response into the system so it doesn’t ever do it again hmmm only issue is stuff like weather. Isn’t that what apple was originally about??? |
and others
UPDATE:
Thanks for everyone's continued interest in SiriProxy. I know lots of people are really excited to get it "up and running" with iOS7. Now that the software is public, you're welcome to discuss bugs/issues on on this forum. However, keep two things in mind:
1: Only public releases of iOS are supported by SiriProxy
I don't have a developer account with Apple right now, so I'm not under NDA (I never had access to pre-release iOS7), but my understanding is that the pre-release versions of the software point to a different set of servers from the production versions. Therefore, you'll have a number of issues as
guzzoni.apple.comis hard-coded in in several places (and is an integral part of the SSL MITM cert generation process). This being the case, I highly recommend upgrading to the public release (NOT the GM release from earlier this week) before you file any bugs with SiriProxy.2: SiriProxy has no full time development staff
At this point, I'm the sole developer on SiriProxy. ElvisImpersonator does an AMAZING job at supporting the SiriProxy developer community, but doesn't work on the core development. Both of us have full time jobs that keep us extremely busy and neither of us get paid to work on SiriProxy. It's a labor of love (one that you're free to participate in through pull requests!).
That being said, I am on call this week and Elvis has, um, left the building (he's traveling for work), so it might be a week or more before I even get a chance to do testing on iOS7. So, as with before: UPDATE TO iOS7 AT YOUR OWN RISK. There's no official support for iOS7 in SiriProxy at this time and I can't make any guarantees it will ever work on iOS7. As with any new release of iOS, Apple could (any may already have) put in measures to block SiriProxy from working.
Again, thank you for your continued interest in my project. SiriProxy started out as a fun short-term project that kept me busy while I was between jobs and has turned into much more than I ever would have expected. I will continue to try and keep things working as long as I can and I'm always every excited to see the awesome projects that you, the SiriProxy development community, create! Keep up the great work and thanks again!!!
-Pete
Original:
Okay, so I'm watching the WWDC right now and there's lots of change to Siri coming. This means two things:
1: NEW BETA -- NO DISCUSSION
There's a beta out today for developers. Please remember that as an Apple Developer, you must agree to an NDA regarding unreleased applications (e.g. iOS7). This means that you should refrain from discussion of changes to the Siri protocol or functionality until iOS7 is publicly released. Once it's out for the public we can start working on fixing whatever it might break. Please do not open issues regarding problems when using iOS7 until it is publicly released. Elvis and I will close these tickets.
2: SiriProxy might break with beta software
As mentioned, a new iOS might mean changes to the Siri protocol. The iOS6 changes were minimal and I was able to fix them pretty quickly. No promises for iOS7 or future software. Hopefully it's something I'll be able to work with, but don't go upgrading devices if SiriProxy is important to you.
Thanks everyone for your interest!
-Pete
The text was updated successfully, but these errors were encountered: