Deprecate basic auth and add XAuth capabilities

[stevestreza]

Twitter has repeatedly said that basic auth (via password) will be deprecated in June 2010. MGTwitterEngine should support OAuth login. We should also factor xAuth into any revamping made in this area.

[pocketronic]

Here's some work done to integrate it with Oauth:
http://github.com/yourhead/xAuth_ObjC_Test_App

Here's another which provided some code to the above, but also seems to be its own project:
http://github.com/kimptoc/MGTwitterEngine-1.0.8-OAuth-MyTwitter/tree/master

[justin]

@aral also has a robust implementation using XAuth. It's what I've been using. http://github.com/aral/XAuthTwitterEngine/

[pocketronic]

yourhead doesn't have time to look at integrating his code into the MGTE main branch, said maybe this summer.

But it looks like XAuth is a better solution anyway, as it avoids the web view issue.

[justin]

Suggest renaming this ticket to "Deprecate basic auth and add XAuth capabilities". I can't imagine a scenario where someone would want to use vanilla OAuth here.

[henmue]

Are you aware of Ben Gottlieb's solution?
http://github.com/bengottlieb/Twitter-OAuth-iPhone

Mr. Gottlieb added OAuth to MGTwitterEngine 1.0.8. Maybe you could team up to get this going.

[stevestreza]

In order to add OAuth support, we need to either A) write our own OAuth library (ew), or B) include something like OAuthConsumer as a dependency. I'd prefer B, but that means we need more library dependencies.

[athanhcong]

Twitter is counting down to use OAuth and xAuth in next 9 weeks http://www.countdowntooauth.com/
When the main Engine support xAuth?
Please tell us some information or plan about this.
Thanks

[stevestreza]

Given the imminent deprecation of the basic auth API, I'll take up this ticket. Plan of attack:

1. Create a git branch for XAuth.
2. Add OAuthConsumer as a dependency
3. Add MGTwitterEngine APIs for setting the OAuth consumer key/secret
4. Add MGTwitterEngine APIs for obtaining an OAuth access token from a username/password
5. Add MGTwitterEngine APIs for setting the OAuth access token
6. Remove/deprecate MGTwitterEngine APIs for setting username/password

Existing clients will need to:

1. Add the OAuthConsumer code to their project
2. Obtain OAuth consumer key/secret from Twitter, and contact Twitter to get XAuth permission enabled for that application
3. Make an MGTwitterEngine API call to set the consumer key/secret on the MGTwitterEngine object at construction
4. Make an MGTwitterEngine API call to obtain an OAuth access token from the username/password
5. Replace -[MGTwitterEngine setUsername:password:] with an MGTwitterEngine API to set the OAuth access token

[luciuskwok]

Here is the URL for obtaining OAuth credentials for your app:

http://twitter.com/oauth_clients/new

You will also want to use the xAuth authentication method (which bypasses the usual OAuth practice of requiring users to auth through a web browser), which requires sending an email to Twitter making the request:

http://apiwiki.twitter.com/Twitter-REST-API-Method:-oauth-access_token-for-xAuth

Also, I just put my implementation of Twitter xAuth on github: http://github.com/luciuskwok/HelTweetica

[stevestreza]

All the steps needed to migrate to OAuth/XAuth will be well-documented here (probably on the wiki). Both OAuth and XAuth will be supported.

[stevestreza]

Ok, here's the commits.

1. c5be05d
2. f4a1506

To use this, you will need to include both OAuthConsumer and yajl as subdirectories of the project folder. Once you do this, here's how to enable OAuth/XAuth support:

1. Obtain credentials, and contact Twitter to obtain XAuth access.
2. When you init your MGTwitterEngine, call -[MGTwitterEngine setConsumerKey:secret:] to set the OAuth credentials.
3. Call -[MGTwitterEngine getXAuthTokenForUsername:password:] with the username and password.
4. Implement accessTokenReceved:forRequest: in your MGTwitterEngineDelegate. Call -[MGTwitterEngine setAccessToken:] with the value you get passed.

After this, all calls to the API should work. This needs more testing to make sure all the API calls still work.

[athanhcong]

wow, that's so fast! Thank for your effort :D

[stevestreza]

I'm getting authorization errors when I try to post a tweet, I'll have to look into it more tomorrow.

It is also fairly straightforward to add basic OAuth (which uses in-browser PIN auth to work), so I may add that as well.

[chockenberry]

The problem with posting is because the API is choking on the query parameters in the request. When "status" is just in the HTTP body, the update goes through without a problem.

I'm talking to the Twitter devs about why this is happening: may be a documentation thing or it may be a confusing HTTP response code.

[chrisgummer]

The request URL with query parameters is being used in the signature base string, contrary to: http://oauth.net/core/1.0a/#rfc.section.9.1.2

Additionally the POST body is not being included in the signature base string as the 'prepare' message is being sent to the OAMutableURLRequest object before setting its POST body.

[stevestreza]

I've moved the -prepare calls to the MGTwitterHTTPURLConnection, so it is the last thing called on the OAMutableURLRequest before it gets sent out. See: 38d2f2f

This does not fix the posting issue.

[chrisgummer]

That's because the URL with query parameters is still being used in the signature base string. The query parameters are always added in _baseRequestWithMethod:path:requestType:queryParameters:

See: http://github.com/mattgemmell/MGTwitterEngine/blob/oauth/MGTwitterEngine.m#L560

Obviously a more universal solution needs to be applied, but to quickly test just comment out that line.

[luciuskwok]

I have a working version now, based on 38d2f2f. In "MGTwitterEngine.m", change line 559 to "if (params && ![method isEqualToString:HTTP_POST_METHOD]) {" so that it does not add the query to the URL if it's a HTTP POST method. Then the project will be able to send status updates. (I don't know how to submit changes to the code or what the official way to submit bug fixes is, so sorry if this isn't where it goes!)

[luciuskwok]

Scratch the last post. In order for the code to work both with and without OAuth, you need to remove the query parameters after you create the finalURL. To remove the query parameters from the URL, replace line 621 with:

NSString *urlStringWithoutQuery = [[urlString componentsSeparatedByString:@"?"] objectAtIndex:0]; // Added to exclude query parameters from URL. -LK
theRequest = [[[OAMutableURLRequest alloc] initWithURL:[NSURL URLWithString:urlStringWithoutQuery]

There's way too much legacy code in here for something that's only two years old. Anyway, this code works for both POST, GET, and DELETE, and it doesn't change how Basic Auth and no auth works.

[luciuskwok]

OK, one last post on tonight. Changing line 559 to "if (params && ![method isEqualToString:HTTP_POST_METHOD]) {" was the correct fix. The problem is that for some reason the old code would add the POST method parameters to the URL, which is wrong. POST method parameters go in the HTTP body, not the URL. Only GET and DELETE have parameters in the URL. I don't know how long this bug has been in it, but it seems to have gone undetected until now. I have tested this with both POST and GET Twitter API methods.

[stevestreza]

I've put in the change for line 559, which seems to work for sendUpdate: (yay!). I did not put in the change to line 621.

Thanks, luciuskwok! For future reference, you can fork the project and commit/push changes through there. After that, leave a comment asking for us to pull your changes in.

[stevestreza]

I've put up a preliminary wiki page explaining in detail how to migrate from basic auth to OAuth and XAuth. Please let me know of any problems/improvements. http://wiki.github.com/mattgemmell/MGTwitterEngine/migrating-from-basic-auth-to-oauthxauth

[stevestreza]

OAuth branch has been merged into master.

[henmue]

I just wanted to try the current implementation (2010-05-03) but the project fails to compile. Or rather I fail to compile the project :o). Anyway, I added yajl, oauthconsumer and TouchJSON. Compiling results in the generic error "gcc-4.2 failed with exit code 1". Any hints?

[stevestreza]

henmue: I've forked off your question into a new issue. Please see/follow #23 for updates. Thanks!

[btjones]

FYI, because Security.framework differs between OSX and iPhone OS, OAuthConsumer (specifically OAToken_KeychainExtensions) fails to compile for the iPhone SDK.

[luciuskwok]

I don't think you need to use Keychain and OAToken_KeychainExtensions for OAuth to work. I question the inclusion of Keychain features in the OAuthConsumer framework because it is a case of feature creep. If you remove the files that reference Keychain from the OAuthConsumer and store the token yourself, you should be fine.

[btjones]

My current workaround is to store tokens myself but storing tokens in the keychain would be preferred (but realize not everyone would want to do that so I totally understand if that is not part of MGTwitterEngine). My larger concern is that developers will have to alter OAuthConsumer before using it in their iPhone projects.

[mathieut]

Alright, I followed the instructions:

• My apps have been granted xAuth access by Twitter
• I fetch the token with -[MGTwitterEngine getXAuthAccessTokenForUsername:password:]
• I get the token back in -[id accessTokenReceved:forRequest:]
• I set it on the Twitter engine with -[MGTwitterEngine setAccessToken:]
• I try to send an update with -[MGTwitterEngine sendUpdate:] and I always get a 401 error back
• I'm stuck

I'm not the only one to get this 401 apparently, but the error seems to be from the framework people use. http://groups.google.com/group/twitter-development-talk/search?q=xAuth&start=0&scoring=d&

Is MGTwitterEngine working with xAuth for you guys?

Thanks a lot,

[taylanpince]

I have the same issue, got XAuth access and followed directions for setting up MGTwitterEngine with OAuthConsumer, but getting a 401 when trying to authorize the user.

[taylanpince]

No luck here, I confirmed all credentials and keys. I also confirmed with twitter that our app has XAuth enabled. I tweaked the engine a little bit to be able to get the actual response body that comes from twitter along with the 401 response, and this is what I am seeing: "Failed to validate oauth signature and token"

Were there any reports about the generated nonce or the timestamp being an issue? I tried using a simpler nonce, but it didn't make a difference.

Any insight would be appreciated.

[mathieut]

Same here, everything is confirmed by Twitter. It should work. They did said that:
"It looks like it might be sending the wrong Content-Type as well -- you should be sending application/x-www-url-formencoded as the Content-Type for this request. Also, source is not a valid parameter to send when using OAuth -- source is automatically taken from your application record instead."

[luciuskwok]

You can and probably should omit the "Content-Type" and source parameters in your URL request. The Twitter API does not require them as far as I know. To do this, I have these lines of code:

// Default client token is "source" in the URL parameters, which is not needed with OAuth.
#define DEFAULT_CLIENT_TOKEN nil

[stevestreza]

source parameter fix in b9f4121. If there is an access token attached to the MGTwitterEngine, it will not send the source parameter.

[stevestreza]

mathieut: What is the Content-Type being sent? I'm seeing application/x-www-form-urlencoded when I call sendUpdate: with my token, although it may be inferred by NSURL*, which may be the problem. You can use an app like Charles to find this out pretty easily.

taylanpince: Are you using the demo app to test this? It may behave differently on, say, iPhone, so I want to make sure I'm testing correctly. I never ran into any problems with the nonces or the timestamps, so I'll have to investigate this more.

[taylanpince]

No I was testing using my own app. I am not including TouchJSON or YAJL, using libxml for parsing. I just tried compiling the MGTwitterEngine demo app to test, and I can get it to build without errors, but it just crashes on launch. This probably has something to do with removing TouchJSON and YAJL components?

Not sure if this is of any value, but I also tested with Aral Balkan's XAuthTwitterEngine demo (http://github.com/aral/XAuthTwitterEngine), and I get the exact same 401 error when trying to authenticate.

I'll test with the latest update and see if it changes anything. Will keep fiddling around to see if I can get to the source of this problem. Thanks for your help.

[mathieut]

amazingsyco: Regarding the Content-Type, I used Charles to get it, and it's empty... for all the 2 requests (getting the access token, sending the update). Is there something I have to set myself in the app?

Thanks a lot

[taylanpince]

I decided to start from scratch to make sure I am not causing this issue because of an implementation error. Here is what I am doing, step by step, let me know if you see anything wrong:

• Get fresh checkout of MGTwitterEngine
• Drag and drop group "MATwitterEngine" into my project
• Delete groups "Twitter TouchJSON Parsers", "Twitter NSXML Parsers", "Twitter YAJL Parsers"
• Open project settings, add USE_LIBXML=1 in Preprocessor Macros, and $SDKROOT/usr/include/libxml2 under Header Search Paths (recursive)
• Get fresh checkout of OAuthConsumer from http://github.com/jdg/oauthconsumer
• Add all files within OAuthConsumer to the project
• Add Security framework

After this, I am still seeing the 401 error when I try to fetch an auth token. Any suggestions on where I might be making a mistake, or what I might try differently?

[taylanpince]

I am embarrassed to report that the issue has been resolved, and the problem was the app key/secret provided to me by the client. Following the steps I outlined in my previous comment, iPhone integration works perfectly. Thanks for your help.

[stevestreza]

mathieut: I've spun your issue off into a separate issue, #34.

taylanpince: Glad to hear it, happy to help!

[mathieut]

Unlike Taylan my app key/secret are OK, I confirmed it multiple times by Twitter themselves.

I just tried with the last version of MGTwitterEngine you have here and I still get this error when I do my sendUpdate request with the accessToken I just got back:

2010-05-25 19:09:17.863 testTwitter[11239:207] Request failed for connectionIdentifier = 7642466B-9794-4626-96A0-E93371B31B12, error = Operation could not be completed. (HTTP error 401.) ((null))

I'm not using LibXML unlike taylanpince, but I'm using NSXML. I do have the last version of OAuthConsummer from jdg. Everything seems to be fine in my Project: http://drp.ly/14B451

Still no solution for me.

[henmue]

Hi everybody! How long did Twitter need to grant your clients xAuth access? I made the request few days ago and got a support ticket in reply that I cannot access. I mailed the api team again but got no reply yet...

PS: In answer to my last mail that stated that I cannot access the support ticket right now I got another ticket I cannot access. Great. Any tips?

[chrisgummer]

henmue: I had multiple apps enabled within about 24 hours. My support tickets where also inaccessible.

[henmue]

@chrisgummer Thanks for the answer. I'll wait a little bit longer. Is there any way to check xAuth access apart from using getXAuthAccessTokenForUsername? Up to now I'm getting the 401, but as I get no word from the twitter API team, I don't know if it technically SHOULD work.

[mathieut]

I just tried the same example code with different key and secrets, and it's working for one key/secret pair, not for the other... while Twitter was guaranteeing be that all of the key/secrets had access to xAuth. Hmmm. Just asked them again... wait and see.

[luciuskwok]

I have an idea. Why not plug your consumer key and secret into another app that doesn't use OAuthConsumer or MGTwitterEngine, for example my HelTweetica, and see if it works there? By keeping one variable the same and varying the context, you can determine whether the problem is with the token (key/secret pair) or with the library.

[mathieut]

luciuskwok > Just did that, and it logins fine, but still cannot post to Twitter (I get a 401) while with other key/secret it works fine. It's definitely a problem from Twitter, which apparently they have not figured out yet.

Thanks for the help, I really appreciate!

[luciuskwok]

Sure. If you go to your app's OAuth page on Twitter.com, there is button to "Reset Consumer Key/Secret." Did you try to generate a new key and secret pair? The URL for that page would start with http://twitter.com/oauth_clients/details/.

[mathieut]

I just did, that's what Twitter asked me to do too... no success. Same error.

[stevestreza]

I've closed the issue with Content-Type not being set (#34) which was reported as a possible problem earlier. Try pulling the latest commits and seeing if that causes a successful sendUpdate:?

[mathieut]

Steve, I just did that. Working with other key/secret that I have, not for one. I'm almost 100% sure it's on Twitter's side at this point. I will let you know. Thanks for your help!

[neuhausj]

Hello,
Once I am logged in, how could I share the twitterengine in my application?
Should I have a singleton for the MGTwitterEngine class? or for the access token only?

Thank you for your help.

[stevestreza]

Hi neuhausj,

I don't use singletons for either the MGTwitterEngine or the access token. I use a factory method to create a new MGTwitterEngine, load the access token credentials from the keychain, and set it on the new MGTwitterEngine. I pass that back to my caller, set its delegate, then perform my operations.

There is nothing stopping you from using a singleton, but there is one caveat if you do. The singleton MGTwitterEngine's delegate may be in use by multiple callers at the same time. I originally used singleton MGTwitterEngines, and my solution there was to write another singleton which proxied delegate methods to the desired object. This is kind of gross, and I don't recommend it.

[thomasvsundert]

I'm also unable to perform sendUpdate-requests (error 401)...

[stevestreza]

If you're getting the 401 error when sendUpdate:, please try using Twitter's twurl tool to post a status.

1. Install the gem
2. Run twurl authorize -u <username> -p <password> --consumer-key <consumerKey> --consumer-secret <consumerSecret>, with your xAuth credentials
3. Try posting a status update with twurl -d status=Testing twurl /1/statuses/update.xml.

If the update fails in twurl, then the problem is likely on Twitter's side. If the update succeeds in twurl, but fails in MGTwitterEngine, then the bug is in MGTwitterEngine.

[thomasvsundert]

On line 652 I switched the 2 lines to compose the urlString (SET_AUTHORIZATION_IN_HEADER or not), this fixed the error for me...

[mathieut]

Twitter couldn't figure out why I had the 401s, they litteraly tried themselves with my key/secret and accessToken. So I deleted my app, created it again, got xAuth accees and it finally works now. It was just something weird on Twitter's side...

Thanks for everyone's help here!

[neuhausj]

Thank you for your support.
If I understand correctly, you have one MGTwitterEngine in each class that need access to it, and you instanciate all these with a factory method.
How do you handle when a user logout? or if its permissions are revoked?

[mathieut]

@amazingsyco: Do you use only one instance of MGTwitterEngine in your entire app, or like you seem to say, you have one instance in each class that needs it, and you just pass the info to create a MGTwitterEngine instance from class to class?

I'm also interested in the user logout and permissions revoked as neuhausj asked.

Thanks a lot,

[mathieut]

@neuhausj > Have you been able to make it work since June?

Thanks!