Trouble with Twitter Signup/Login

[jasonmj]

Hi there, I just signed up for a new account using the Twitter option. After authorizing via Twitter's consent screen, I was redirected back to publiclab.org and saw this:

I see the same error when I tried clicking the verification link from the confirmation email and when I tried logging in.

[welcome]

Thanks for opening your first issue! This space is protected by our Code of Conduct - and we're here to help.
Please follow the issue template to help us help you 👍🎉😄
If you have screenshots or a gif to share demonstrating the issue, that's really helpful! 📸
Do join our Gitter channel for some brainstorming discussions.

[cesswairimu]

Thanks @jasonmj for reporting this

[Anurag2012]

hi @jasonmj ,I would like to work on this issue.

[jasonmj]

@Anurag2012 Thanks! Note, that I was able to create an account using email instead. I think my Twitter account still has publiclab.org authorized and in case you want me to try to reproduce the issue.

[jywarren]

Wondering if @SidharthBansal is able to try this out to confirm? Thanks for reporting -- this can be a tough one to maintain as it's hard to run tests on an offsite service integration like Twitter's OAuth. Hope we can figure it out!

[jywarren]

@Anurag2012 if you are still available, would you be able to try creating an account on PublicLab.org using your twitter account?

[tawahpeggy]

i came accross this issue and tried to find out more.when i tried to login to publiclab.org using my twitter account it landed me here:
. same with creating an account.i guess somehing is serriously wrong,i am trying to find out what it is.

[tawahpeggy]

i finally tracked down the issue. its not a problem in the code base its in the registration of the twitter application. a call back url wasnt added and twitter register it as a desktop application and gave it a pin instead of a url and now it is seeing the app as a desktop instead of a web app which makes it misbehave sometimes.@jywarren, @cesswairimu please we will need to change the twitter app settings.
i registered a new twitter app using my twitter account but i havent been able to see where the tokens and security keys are registered in publiclabs. so i can test with mine.
i have spent alot of time on it but i cant seem to find, please i need help.
get more info here:https://stackoverflow.com/questions/1280295/keep-getting-oauthunauthorized-error-when-using-oauth-and-twitter-ruby-gems.

[jywarren]

Ahhh amazing detective work here! Just notifying @SidharthBansal who built this system to stay synced... I will try to look into this tomorrow! Thank you so much @tawahpeggy !

[tawahpeggy]

Thanks @jywarren. looking forward to work with @SidharthBansal so this issue can be clossed once and for all.

[ebarry]

Thanks for reporting this @jasonmj and for working on it @tawahpeggy and @SidharthBansal 🐬

[tawahpeggy]

Thanks @ebarry. the work isn't complete yet though

[jywarren]

Hi all! I think we probably need @SidharthBansal to help us unravel this one. You can find a huge amount of info about the Twitter OmniAuth setup at #2676 -- i'll copy the list here:

• Add omniauth-twitter gem Remaining gems added #2686
• Set up developers app Twitter Provider App Settings #2893
• Docker file changes Docker changes for the Twitter Provider #2947
• Add OmniAuth Configuration to initializer Twitter Provider Initializer changes #2948
• OAuth testing hash Omniauth hash for Twitter #2961
• Write user sessions controller tests for the login through Twitter User Session Controller tests for twitter #2962
• Write integration tests for the login through Twitter Integration tests for twitter #2963
• header OAuth Twitter Button #2986
• /profile/edit page OAuth Twitter Button #2986
• login page OAuth Twitter Button #2986
• sign up page OAuth Twitter Button #2986

@tawahpeggy the tokens are kept secret as environment variables. But if you could screenshot the exact changes that would have to be made on the Twitter settings pages, we can do that!

a call back url wasnt added

thanks for digging so much into this @tawahpeggy -- can you elaborate about that last comment about the callback URL? I could probably help find where it is in our code if I know what to look for!

[jywarren]

Also - is this an intermittent issue or is this affecting ALL twitter login attempts? Thank you!

[tawahpeggy]

hello @jywarren thanks for your timely reply and sorry for reaching you just now,i wanted to first of all study the resources you posted above before replying so we can get a solution as soon as possible.
the issue isn't intermittent as it will persist if not resolved.
i have gone through all the links and resources in your comment above and my thoughts are clear, the call back URL was added successfully and everything looks fine see screen shot.

so what is causing the authentication to fail isnt what i thought.
the only inconsistency i noticed during this research is the fact that you can only login through twitter using a particular URL. I think @SidharthBansal will really need to explain while it has to happen this way before i will be able to continue with the issue.
here is a screen shot of what i am saying.
. i tried all this links and none seem to be working. also changing the secret keys and tokens wont help. i tried that already.

[tawahpeggy]

@SidharthBansal please how did you handle signing up via social links and roles like moderator and admin?
@jywarren please i will wish to have @SidharthBansal's personal email address or something i can contact him directly through if possible.

[cesswairimu]

@tawahpeggy I know @SidharthBansal might be busy, here are a few documentation on the auth system that might give you more context https://github.com/publiclab/plots2/blob/b1c57446d016f8cd0ec149a75298711270e1643e/doc/LOGIN_SYSTEMS.md#how-to-setup-login-modal-on-various-locations and https://publiclab.org/notes/bansal_sidharth2996/08-07-2018/gsoc-multi-party-auth-system

[tawahpeggy]

Okay @cess thanks very much i will look at them ASAP

…

On Thu, Oct 29, 2020 at 12:48 PM Cess ***@***.***> wrote: @tawahpeggy <https://github.com/tawahpeggy> I know @SidharthBansal <https://github.com/SidharthBansal> might be busy, here are a few documentation on the auth system that might give you more context https://github.com/publiclab/plots2/blob/b1c57446d016f8cd0ec149a75298711270e1643e/doc/LOGIN_SYSTEMS.md#how-to-setup-login-modal-on-various-locations and https://publiclab.org/notes/bansal_sidharth2996/08-07-2018/gsoc-multi-party-auth-system — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#8325 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJT4KDSRYNKQAWOL7K5QLDTSNFI7RANCNFSM4QEBTMTQ> .

[jywarren]

Hi @tawahpeggy i believe @SidharthBansal was referring only to local testing and the difference between localhost and 127.0.0.1 - which are usually but not always equivalent. My understanding is that login via twitter begins by visiting /auth/twitter - but it's hard to test because the app is configured primarily for the production server, and not for our testing servers or local hostnames like localhost. I think we'd best start looking into log entries on the server, or even on Twitter's system (if they exist; some companies offer an error log so if you know they do i can probably ask someone with the PL password to make a copy of it for us?).

Shall I look in the logs for /auth/twitter/callback and see what the specific error is?

[jywarren]

Also noting that i was able to log in via Twitter at https://stable.publiclab.org/ as @jywarren (on twitter). I wonder is this error only occurring for /new/ accounts? Or, is it only occurring for accounts which aren't new but have not previously been associated with a Twitter account? That may help us narrow down the specific issue. Thank you!

[jywarren]

We could try by creating a new Twitter account and trying to open a PL account using it...?

[tawahpeggy]

A twitter application you mean. i have already created one using my twitter account. we could use to test that @jywarren .should i email you the secret tokens and keys so you try that since i dont have access to do that.
it is worth noting that this once happened to me during my final year project actually and changing the twitter application didnt solve the problem. lets still try we might get a heads up from there.

[jywarren]

Ah, no i just meant we have at least four scenarios i can think of:

1. person has a twitter account but not PL.org - uses Twitter account to create their PL.org account upon login
2. person has a PL account but has not associated a Twitter account; tries to log in with the Twitter account and it is or isn't matched to their existing PL.org account, using... email address match? (this i think we have an ambiguity around, but it's outside the scope of this issue)
3. person has a PL account but has not associated a Twitter account, is logged in on PL.org and tries to (from profile page) associate their twitter account
4. person has a twitter and PL.org account, already associated (like me) and just logs into PL.org using their Twitter account

I've tested 4) - it works, and I think 2) doesn't involve the login screen (it can only be done from the profile page) so it's not really in question here, but 1) and 3) are what we're worried about. @tawahpeggy which are you -- i would guess 3)?

As the original poster @jasonmj said I just signed up for a new account using the Twitter option i think they are 1) -- so my suggestion was to use a new Twitter account to try to capture a log entry for that scenario of a person with no PL.org account. @tawahpeggy if you are actually doing 3), we may have a deeper issue that affects both 1) and 3), so that'd be good to know.

I'll search logs a bit to see if we can find @jasonmjohnson which is @jasonmj's Twitter handle. But we can also simulate this in GitPod to further investigate.

[jywarren]

I couldn't find the twitter handle, and i think the log is just too old. I recommend doing this in GitPod to try to find the error! https://gitpod.io/#https://github.com/publiclab/plots2/

[jywarren]

Hm. in Gitpod, i see:

Maybe we lack the tokens etc to run this all in GitPod.

I'll try searching the log for /auth/twitter/callback, as well as Sentry.io, our monitoring system.

[sentry-io]

Sentry issue: PLOTS2-Y0

[jywarren]

I just linked a Sentry issue with Faraday::SSLError SSL_connect SYSCALL returned=5 errno=0 state=SSLv3/TLS write finished but it only occurred once, and is for Google OAuth, not twitter. Seems unrelated but that's all I found on Sentry.

[jywarren]

OK, made a new Twitter account:

Huh, i got back here, with "the user has been banned":

[jywarren]

OK, i got logs for that attempt:

[d17146d9-087d-4dd4-a98a-a3d55088c4be] Started GET "/auth/twitter/callback?origin=https://publiclab.org/&oauth_token=ldiYUQAAAAAA62jGAAABdbh7qGc&oauth_verifier=MO6UahlsZ2KnbZalMTrwwsczlAFEZFcj" for 72.92.236.166 at 2020-11-11 18:05:14 +0000
[d17146d9-087d-4dd4-a98a-a3d55088c4be] Processing by UserSessionsController#create as HTML
[d17146d9-087d-4dd4-a98a-a3d55088c4be]   Parameters: {"origin"=>"https://publiclab.org/", "oauth_token"=>"XXXX", "oauth_verifier"=>"XXXX", "provider"=>"twitter"}
[d17146d9-087d-4dd4-a98a-a3d55088c4be] Redirected to https://publiclab.org/?_=1605117915
[d17146d9-087d-4dd4-a98a-a3d55088c4be] Completed 302 Found in 39ms (ActiveRecord: 35.2ms)

Strangely it means the user would seemingly be banned - not sure which though. Here's the test which confirms this behavior:

plots2/test/functional/user_sessions_controller_test.rb

Lines 270 to 280 in c60aaab

	test "logging in with banned user through oauth should fail and redirect correctly" do
	request.env['omniauth.origin'] = "/notes/liked"
	request.env['omniauth.auth'] = OmniAuth.config.mock_auth[:github1]
	post :create
	post :destroy
	# name of omniauth user
	User.find_by(name: "bansal_sidharth309").ban
	post :create
	assert @response.redirect_url.include? "/notes/liked"
	assert_equal flash[:error], I18n.t('user_sessions_controller.user_has_been_banned', username: "bansal_sidharth309").html_safe
	end

[jywarren]

Let me see if the user record was actually created... maybe it's... via line 52 (or 204) of the user_sessions_controller here?

plots2/app/controllers/user_sessions_controller.rb

Lines 52 to 57 in c60aaab

	else # not signed in
	# User U has Provider P linked to U. U has email E1 while P has email E2. So, User table can't find E2 provided
	# from auth hash, hence U is found by the user of identity having E2 as email
	@user = User.where(email: auth["info"]["email"]) ? User.find_by(email: auth["info"]["email"]) : @identity.user
	if @user&.status&.zero?
	flash[:error] = I18n.t('user_sessions_controller.user_has_been_banned', username: @user.username).html_safe

[jywarren]

OK, i think possibly bc I created a Twitter account with no email address (just a phone number) it failed to find someone to associate because it used a nil email to search?

OK! Yes! I added an email to my new twitter account, and now i got an error, hopefully it's the same as the orig. poster!

https://publiclab.org/auth/twitter/callback?origin=https://publiclab.org/&oauth_token=-XXXX&oauth_verifier=XXXX

[jywarren]

TypeError
allocator undefined for Proc

Crashed in non-app: psych/visitors/to_ruby.rb in allocate
app/models/user_tag.rb in create_with_omniauth at line 26

OK! There we go!

[sentry-io]

Sentry issue: PLOTS2-119

[jywarren]

plots2/app/models/user_tag.rb

Lines 25 to 28 in 4fadc45

	def self.create_with_omniauth(auth, uid)
	create(value: "oauth:" + auth['provider'] + ":" + auth['uid'],
	uid: uid, data: auth.to_hash)
	end

is the section of code throwing this allocator undefined for Proc message:

https://stackoverflow.com/questions/9735188/typeerror-allocator-undefined-for-proc-when-working-with-a-model-object-in-a-ra

collectiveidea/delayed_job#478 suggests that an object can not be cleanly deserialized... could it be...

Well OK there seem to be a number of relevant possible scenarios here. Both of these pathways could lead to this same error:

plots2/app/controllers/user_sessions_controller.rb

Lines 75 to 99 in c60aaab

	if User.where(email: auth["info"]["email"]).empty?
	# Create a new user as email provided is not present in PL database
	user = User.create_with_omniauth(auth)
	WelcomeMailer.notify_newcomer(user).deliver_now
	@identity = UserTag.create_with_omniauth(auth, user.id)
	key = user.generate_reset_key
	@user_session = UserSession.create(@identity.user)
	@user = user
	# send key to user email
	PasswordResetMailer.reset_notify(user, key).deliver_now unless user.nil? # respond the same to both successes and failures; security
	if session[:openid_return_to] # for openid login, redirects back to openid auth process
	return_to = session[:openid_return_to]
	session[:openid_return_to] = nil
	redirect_to return_to + hash_params
	elsif params[:return_to] && params[:return_to].split('/')[0..3] == ["", "subscribe", "multiple", "tag"]
	flash[:notice] = "You are now following '#{params[:return_to].split('/')[4]}'."
	subscribe_multiple_tag(params[:return_to].split('/')[4])
	redirect_to '/dashboard', notice: "You have successfully signed in. Please change your password using the link sent to you via e-mail."
	else
	redirect_to "/dashboard", notice: "You have successfully signed in. Please change your password using the link sent to you via e-mail."
	end
	else # email exists so link the identity with existing user and log in the user
	user = User.where(email: auth["info"]["email"])
	# If no identity was found, create a brand new one here
	@identity = UserTag.create_with_omniauth(auth, user.ids.first)

And if there's no email provided, perhaps we should divert away here too:

plots2/app/controllers/user_sessions_controller.rb

Lines 52 to 56 in c60aaab

	else # not signed in
	# User U has Provider P linked to U. U has email E1 while P has email E2. So, User table can't find E2 provided
	# from auth hash, hence U is found by the user of identity having E2 as email
	@user = User.where(email: auth["info"]["email"]) ? User.find_by(email: auth["info"]["email"]) : @identity.user
	if @user&.status&.zero?

I think this section should divert to the final else if there is no email in the provided Twitter response.

[sentry-io]

Sentry issue: PLOTS2-117

[sentry-io]

Sentry issue: PLOTS2-8W

[jywarren]

And this last Sentry error notes a problem if there's no email provided:

    def create_with_omniauth(auth)
      random_chars = [*'A'..'Z', *'a'..'z', *0..9].sample(2).join

      email_prefix = auth["info"]["email"].tr('.', '_').split('@')[0]

OK, so MANY issues related to if Twitter doesn't provide an email. Let's try to write a test for that scenario, AND for the scenario with the allocator undefined for Proc error.

For the second one, if create(value: "oauth:" + auth['provider'] + ":" + auth['uid'], uid: uid, data: auth.to_hash) fails, maybe it's the non-existence of auth['uid'] OR it could be the .to_hash and it's failing to hash something that is too complex to hash? That might be a more subtle one.

[jywarren]

https://github.com/publiclab/plots2/blob/f905c7a01e6824881d55c208c927f008e6fb8bd0/doc/LOGIN_SYSTEMS.md#testing notes that we can modify the test environment to simulate that, in https://github.com/publiclab/plots2/blob/main/config/environments/test.rb:

Attempting these changes in #8734 !!

[sentry-io]

Sentry issue: PLOTS2-12Q

[jywarren]

Hmm, interesting, just saw another Twitter Sentry issue, i just linked it -- Failed to open TCP connection to api.twitter.com:443 (getaddrinfo: Temporary failure in name resolution). Probably unrelated but good to keep in mind.

[tawahpeggy]

Wow you have indeed done alot of research Sir. thats wonderful. let me start from some where and possibly catch up with you 😄

[tawahpeggy]

Sorry @jywarren I haven't been on ma feet for some days cause I wasnt feeling too well. Felt better after visiting a Dr and taking some medications. Will definitely start the work tmr hopefully.

…

On Wed, Nov 11, 2020, 9:55 PM Jeffrey Warren ***@***.***> wrote: Hmm, interesting, just saw another Twitter Sentry issue, i just linked it -- Failed to open TCP connection to api.twitter.com:443 (getaddrinfo: Temporary failure in name resolution). Probably unrelated but good to keep in mind. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#8325 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJT4KDTVZHK6S4WP4QEW6K3SPL25RANCNFSM4QEBTMTQ> .

[jywarren]

@tawahpeggy I had missed this email but wanted to re-iterate my wish for your health and safety. I'm glad you're feeling better.

I'm circling back to say I have created a test and new procedures for when someone tries to use Twitter in scenarios 1-3 (i believe) above, and a test for at least one of those scenarios (i believe they all use the same logic -- identity does not exist so we need to either create a user with identity OR link identity to existing user). So that is resolved, merging it now.

Can folks retry /after/ ensuring their Twitter account has an email address, on https://stable.publiclab.org (our testing server)?

I'm going to close this now but we can re-open if there are remaining scenarios which still cause an error message. Thanks to everyone for their patience and very thorough research and documentation, especially @tawahpeggy and @jasonmj -- 🙌

[jywarren]

If there are further issues here we may want to create a test for a Twitter user with no uid? Though I'm not sure how that could happen.