office365 outlook unexpected data: 2 NO LOGIN failed

[frddbbbl]

Hi, i have just set up fdm on openBSD and have three accounts, the other two work perfectly but my (institution based) outlook account can't fetch mail. fdm fetch returns:

gsa: unexpected data: 2 NO LOGIN failed.
gsa: fetching error. aborted
gsa: 0 messages processed in 1.111 seconds

in .fdm.conf i have:

account "institution"
imaps
server "outlook.office365.com"
user "user@student.institution.ac.uk"
pass "pass"

for my account info. On my previous setup i was using isync and the same details worked.
Thanks for reading,
Cheers,
Freddie

[nicm]

-vvvv output?

[frddbbbl]

Hi, -vvvv output for the account is:

version is: fdm 2.0, started at: Thu Oct 28 21:26:32 2021
running on: OpenBSD 7.0 GENERIC.MP#232 amd64
host is: frdd.lan frdd.lan 192.168.1.1
home is: /home/freddie
loading configuration from /home/freddie/.fdm.conf
added action "INBOX": deliver=0:mbox "~/.mail/mbox" 
added account "countermail": fetch=imaps server "imap1.countermail.com" port imaps user "COUNTERMAILACCOUNT" folders "INBOX"
added account "talktalk": fetch=imaps server "mail.talktalk.net" port imaps user "TALKTALKACCOUNT folders "INBOX"
added account "gsa": fetch=imaps server "outlook.office365.com" port imaps user "OUTLOOKACCOUNT" folders "INBOX"
added cache "/home/freddie/var/mail/.cache": expire -1
added rule 0: matches=all actions="INBOX"
configuration loaded
locking using: flock 
options are: maximum-size=33554432, timeout=900, default-user="freddie", command-user="freddie", file-umask=077, queue-high=2, queue-low=1, strip-characters="\<>$%^&*|{}[]"'`;"
using tmp directory: /tmp
parent: started, pid is 13870
parent: 0 children, 0 dead children
parent: child 52671 (gsa) started
gsa: fetch started, pid 52671
gsa: user is 1000
gsa: started processing
gsa: fetching
gsa: fetch loop start
gsa: calling fetch state (0x6976a178350, flags 0x02)
gsa: fetch, again
gsa: fetch loop start
gsa: calling fetch state (0x6976a17ef00, flags 0x02)
gsa: fetch, block
gsa: queued 0; blocked 0; flags 0x02
gsa: polling: 2, timeout=900000
< * OK The Microsoft Exchange IMAP4 service is ready. [TABPADMAUAAyADYANQBDAEEAMAAwADEAMwAuAEcAQgBSAFAAMgA2ADUALgBQAFIATwBEAC4ATwBVAFQATABPAE8ASwAuAEMATwBNAA==]
gsa: fetch loop start
gsa: calling fetch state (0x6976a17ef60, flags 0x02)
gsa: fetch, block
gsa: queued 0; blocked 0; flags 0x02
gsa: polling: 2, timeout=900000
> 1 CAPABILITY
gsa: fetch loop start
gsa: calling fetch state (0x6976a17f130, flags 0x02)
gsa: fetch, block
gsa: queued 0; blocked 0; flags 0x02
gsa: polling: 2, timeout=900000
< * CAPABILITY IMAP4 IMAP4rev1 AUTH=PLAIN AUTH=XOAUTH2 SASL-IR UIDPLUS ID UNSELECT CHILDREN IDLE NAMESPACE LITERAL+
1 OK CAPABILITY completed.
gsa: fetch loop start
gsa: calling fetch state (0x6976a17f130, flags 0x02)
gsa: fetch, again
gsa: fetch loop start
gsa: calling fetch state (0x6976a17f2f0, flags 0x02)
gsa: fetch, blockgsa: polling: 2, timeout=900000
> 2 LOGIN {27}
gsa: fetch loop start
gsa: calling fetch state (0x6976a17ed80, flags 0x02)
gsa: fetch, block
gsa: queued 0; blocked 0; flags 0x02
gsa: polling: 2, timeout=900000
< + Ready for additional command text.
gsa: fetch loop start
gsa: calling fetch state (0x6976a17ed80, flags 0x02)
gsa: fetch, block
gsa: queued 0; blocked 0; flags 0x02
gsa: polling: 2, timeout=900000
> OUTLOOKACCOUNT {23}
gsa: fetch loop start
gsa: calling fetch state (0x6976a17f6e0, flags 0x02)
gsa: fetch, block
gsa: queued 0; blocked 0; flags 0x02
gsa: polling: 2, timeout=900000
< + Ready for additional command text.
gsa: fetch loop start
gsa: calling fetch state (0x6976a17f6e0, flags 0x02)
gsa: fetch, block
gsa: queued 0; blocked 0; flags 0x02
gsa: polling: 2, timeout=900000
> PASSWORD
gsa: fetch loop start
gsa: calling fetch state (0x6976a17f5e0, flags 0x02)
gsa: fetch, block
gsa: queued 0; blocked 0; flags 0x02
gsa: polling: 2, timeout=900000
< 2 NO LOGIN failed.
gsa: fetch loop start
gsa: calling fetch state (0x6976a17f5e0, flags 0x02)
gsa: unexpected data: 2 NO LOGIN failed.
gsa: fetch, error
gsa: fetching error. aborted
gsa: 0 messages processed in 1.311 seconds
gsa: finished processing. exiting
gsa: sending exit message to parent
gsa: waiting for exit message from parent
parent: got message type 1, id 0 from child 52671
parent: sending exit message to child 52671
parent: 1 children, 0 dead children
parent: 1 children, 0 dead children
parent: child 52671 socket error
parent: 1 children, 0 dead children
parent: waiting for children
parent: child 52671 returned 1
parent: waiting for children
parent: waiting for children
parent: finished, total time 1.323 seconds

[nicm]

Try "no-login" to force AUTH=PLAIN.

…

On Thu, 28 Oct 2021 at 21:33, frddbbbl ***@***.***> wrote: Hi, -vvvv output for the account is: version is: fdm 2.0, started at: Thu Oct 28 21:26:32 2021 running on: OpenBSD 7.0 GENERIC.MP#232 amd64 host is: frdd.lan frdd.lan 192.168.1.1 home is: /home/freddie loading configuration from /home/freddie/.fdm.conf added action "INBOX": deliver=0:mbox "~/.mail/mbox" added account "countermail": fetch=imaps server "imap1.countermail.com" port imaps user "COUNTERMAILACCOUNT" folders "INBOX" added account "talktalk": fetch=imaps server "mail.talktalk.net" port imaps user "TALKTALKACCOUNT folders "INBOX" added account "gsa": fetch=imaps server "outlook.office365.com" port imaps user "OUTLOOKACCOUNT" folders "INBOX" added cache "/home/freddie/var/mail/.cache": expire -1 added rule 0: matches=all actions="INBOX" configuration loaded locking using: flock options are: maximum-size=33554432, timeout=900, default-user="freddie", command-user="freddie", file-umask=077, queue-high=2, queue-low=1, strip-characters="\<>$%^&*|{}[]"'`;" using tmp directory: /tmp parent: started, pid is 13870 parent: 0 children, 0 dead children parent: child 52671 (gsa) started gsa: fetch started, pid 52671 gsa: user is 1000 gsa: started processing gsa: fetching gsa: fetch loop start gsa: calling fetch state (0x6976a178350, flags 0x02) gsa: fetch, again gsa: fetch loop start gsa: calling fetch state (0x6976a17ef00, flags 0x02) gsa: fetch, block gsa: queued 0; blocked 0; flags 0x02 gsa: polling: 2, timeout=900000 < * OK The Microsoft Exchange IMAP4 service is ready. [TABPADMAUAAyADYANQBDAEEAMAAwADEAMwAuAEcAQgBSAFAAMgA2ADUALgBQAFIATwBEAC4ATwBVAFQATABPAE8ASwAuAEMATwBNAA==] gsa: fetch loop start gsa: calling fetch state (0x6976a17ef60, flags 0x02) gsa: fetch, block gsa: queued 0; blocked 0; flags 0x02 gsa: polling: 2, timeout=900000 > 1 CAPABILITY gsa: fetch loop start gsa: calling fetch state (0x6976a17f130, flags 0x02) gsa: fetch, block gsa: queued 0; blocked 0; flags 0x02 gsa: polling: 2, timeout=900000 < * CAPABILITY IMAP4 IMAP4rev1 AUTH=PLAIN AUTH=XOAUTH2 SASL-IR UIDPLUS ID UNSELECT CHILDREN IDLE NAMESPACE LITERAL+ 1 OK CAPABILITY completed. gsa: fetch loop start gsa: calling fetch state (0x6976a17f130, flags 0x02) gsa: fetch, again gsa: fetch loop start gsa: calling fetch state (0x6976a17f2f0, flags 0x02) gsa: fetch, blockgsa: polling: 2, timeout=900000 > 2 LOGIN {27} gsa: fetch loop start gsa: calling fetch state (0x6976a17ed80, flags 0x02) gsa: fetch, block gsa: queued 0; blocked 0; flags 0x02 gsa: polling: 2, timeout=900000 < + Ready for additional command text. gsa: fetch loop start gsa: calling fetch state (0x6976a17ed80, flags 0x02) gsa: fetch, block gsa: queued 0; blocked 0; flags 0x02 gsa: polling: 2, timeout=900000 > OUTLOOKACCOUNT {23} gsa: fetch loop start gsa: calling fetch state (0x6976a17f6e0, flags 0x02) gsa: fetch, block gsa: queued 0; blocked 0; flags 0x02 gsa: polling: 2, timeout=900000 < + Ready for additional command text. gsa: fetch loop start gsa: calling fetch state (0x6976a17f6e0, flags 0x02) gsa: fetch, block gsa: queued 0; blocked 0; flags 0x02 gsa: polling: 2, timeout=900000 > PASSWORD gsa: fetch loop start gsa: calling fetch state (0x6976a17f5e0, flags 0x02) gsa: fetch, block gsa: queued 0; blocked 0; flags 0x02 gsa: polling: 2, timeout=900000 < 2 NO LOGIN failed. gsa: fetch loop start gsa: calling fetch state (0x6976a17f5e0, flags 0x02) gsa: unexpected data: 2 NO LOGIN failed. gsa: fetch, error gsa: fetching error. aborted gsa: 0 messages processed in 1.311 seconds gsa: finished processing. exiting gsa: sending exit message to parent gsa: waiting for exit message from parent parent: got message type 1, id 0 from child 52671 parent: sending exit message to child 52671 parent: 1 children, 0 dead children parent: 1 children, 0 dead children parent: child 52671 socket error parent: 1 children, 0 dead children parent: waiting for children parent: child 52671 returned 1 parent: waiting for children parent: waiting for children parent: finished, total time 1.323 seconds — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#100 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACKI46PPYR6YZHFZLQNQT3UJGXRNANCNFSM5GCQTAWQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[frddbbbl]

Sorry, what should the syntax of this be in my conf?

[nicm]

add it to the end of your account line.

[nicm]

Actually it won't work because fdm can only do LOGIN, CRAM-MD5 and OAUTHBEARER and it looks like your server doesn't have any of those.

[nicm]

It would not be terribly difficult to add AUTH=PLAIN as well.

[frddbbbl]

Hi, when i put no-login at the end of the account line i get a syntax error, no matter which line i put it on i get a syntax error. and adding auth=plain says that auth is an unknown token.
Cheers,
Freddie

[KenMacD]

Actually it won't work because fdm can only do LOGIN, CRAM-MD5 and OAUTHBEARER and it looks like your server doesn't have any of those.

Any plans to add xoauth2 support? Afaik they're pretty similar.

[nicm]

Try this please which implements both: fdm-auth.diff.txt

I have no means to test this so I have not tried it. It will try PLAIN by default, and oauth2 is configured the same way as oauthbearer. If it doesn't work please show me -vvvv logs.

[KenMacD]

@nicm Thank you for the patch. Just tested it and with a couple of patches to the patch it worked. OAUTH2 is XOAUTH, both in the capabilities sent by the server: "AUTH=XOAUTH2" and in the authentication line sent: "%u AUTHENTICATE XOAUTH2 %s".

Looks like there's a copy-paste issue in the patch as well:

 { "no-cram-md5", TOKNOPLAIN },

(when that is fixed the line also has to be moved down for bsearch order).

Finally not sure if it's related or not, but I get the error use either imaps or set starttls if I try to set the options no-cram-md5 no-plain no-login oauth2.

[nicm]

Can you give me a diff of the changes you made?

[KenMacD]

Modified your patch and I think (deleted) covers it. It doesn't fix the parser issue, but that didn't block me.

Where the other option matches the auth line I'm thinking you might want to change the other oauth2 strings to xoauth2, but up to you.

Copy/paste issue in that last one:
fdm-auth.diff.txt

[nicm]

OK, thanks. I have applied this with the SSL error fixed (it is intended so you only use one of imaps or startttls, not both), and after renaming from "oauth2" to "xoauth2" throughout. Please let me know if you see any problems.

Did you try PLAIN as well?

[frddbbbl]

Hi,
Apologies as i assume this is a stupid question, however i am trying to compile from source but in the MANUAL it says to do ./configure && make once you have extracted it from the tar. i downloaded it from github as a zip. once i unzipped it i can't see a configure file, only configure.ac that returns permission denied when i try to run it.
Cheers,
Freddie

[nicm]

If you check out or download a tarball of the repository you need to do sh autogen.sh first (you will also need autoconf, automake, etc installed).

[frddbbbl]

thanks! when running fdm -kv fetch with my conf from the distribution it says/home/freddie

/.fdm.conf: unknown token: pass at line 7 

has the conf formatting been changed?
Cheers.

[KenMacD]

@nicm When I fixed the order of the options in the lexer it worked for me. Without this patch it gives an unknown token error. Thanks a million for adding support.

fdm-sort.patch.txt

[nicm]

Whoops, fixed thanks. @frddbbbl does it work for you now if you update to latest master?

[frddbbbl]

Hi thanks. I've re-installed and the following is outputted by -vvvv now

version is: fdm 2.0, started at: Tue Nov  2 19:23:39 2021
running on: OpenBSD 7.0 GENERIC.MP#232 amd64
host is: frdd.lan frdd.lan 192.168.1.1
home is: /home/freddie
loading configuration from /home/freddie/.fdm.conf
added action "INBOX": deliver=0:mbox "~/.mail/mbox" 
added account "gsa": fetch=imaps server "IMAPSERVER" port imaps user "f.bubble1@student.gsa.ac.uk" folders "INBOX"
added cache "/home/freddie/var/mail/.cache": expire -1
added rule 0: matches=all actions="INBOX"
configuration loaded
locking using: flock 
options are: keep-all, maximum-size=33554432, timeout=900, default-user="freddie", command-user="freddie", file-umask=077, queue-high=2, queue-low=1, strip-characters="\<>$%^&*|{}[]"'`;"
using tmp directory: /tmp
parent: started, pid is 95590
parent: 0 children, 0 dead children
parent: child 42167 (gsa) started
gsa: fetch started, pid 42167
gsa: user is 1000
gsa: started processing
gsa: fetching
gsa: fetch loop start
gsa: calling fetch state (0xf6fd6197dc0, flags 0x02)
gsa: fetch, again
gsa: fetch loop start
gsa: calling fetch state (0xf6fd619f0c0, flags 0x02)
gsa: fetch, block
gsa: queued 0; blocked 0; flags 0x02
< * OK The Microsoft Exchange IMAP4 service is ready. [TABPADQAUAAxADIAMwBDAEEAMAAwADkAMwAuAEcAQgBSAFAAMQAyADMALgBQAFIATwBEAC4ATwBVAFQATABPAE8ASwAuAEMATwBNAA==]
gsa: fetch loop start
gsa: calling fetch state (0xf6fd619f120, flags 0x02)
gsa: fetch, block
gsa: queued 0; blocked 0; flags 0x02
gsa: polling: 2, timeout=900000
> 1 CAPABILITY
gsa: fetch loop start
gsa: calling fetch state (0xf6fd619f340, flags 0x02)
gsa: fetch, block
gsa: queued 0; blocked 0; flags 0x02
gsa: polling: 2, timeout=900000
< * CAPABILITY IMAP4 IMAP4rev1 AUTH=PLAIN AUTH=XOAUTH2 SASL-IR UIDPLUS ID UNSELECT CHILDREN IDLE NAMESPACE LITERAL+
1 OK CAPABILITY completed.
gsa: fetch loop start
gsa: calling fetch state (0xf6fd619f340, flags 0x02)
gsa: fetch, again
gsa: fetch loop start
gsa: calling fetch state (0xf6fd619f550, flags 0x02)
gsa: fetch, block
gsa: queued 0; blocked 0; flags 0x02
gsa: polling: 2, timeout=900000
> 2 AUTHENTICATE PLAIN
gsa: fetch loop start
gsa: calling fetch state (0xf6fd619ec40, flags 0x02)
gsa: fetch, block
gsa: queued 0; blocked 0; flags 0x02
< + 
gsa: fetch loop start
gsa: calling fetch state (0xf6fd619ec40, flags 0x02)
fatal: buffer_add: zero size
parent: 1 children, 0 dead children
parent: child 42167 socket error
parent: 1 children, 0 dead children
parent: waiting for children
parent: child 42167 returned 1
parent: waiting for children
parent: waiting for children
parent: finished, total time 0.786 seconds
parent: exit code 1

[nicm]

Looks like a bug, I'll take a look tomorrow.

[KenMacD]

@frddbbbl Do you get this:

$ git rev-parse --short HEAD
6d5e24d

Tried a make clean all just to be extra sure?

Tried with an account line ending with: no-cram-md5 no-plain no-login xoauth2?

[frddbbbl]

Hi i didn't use git to download the repositry so that command just says it is not a git directory. Doing make clean all doesn't seem to have any affect and adding that to the .fdm.conf just seems to return a syntax error? not sure if i have put it in wrong?

[nicm]

Don't worry about that, I have made a fix, can you try building from git. Do this:

git clone https://github.com/nicm/fdm.git
cd fdm
sh autogen.sh
./configure
make

[nicm]

If it still doesn't work, please show me the -vvvv output again, but you may want to send it to me privately at nicholas.marriott@gmail.com because it will contain your password.

[nicm]

Or remove the encoded password which will come right after the < +, something like:

> 2 AUTHENTICATE PLAIN
< + 
> password-as-base64

[frddbbbl]

Hi, i am still recieving an error, i have emailed you the output.

[KenMacD]

As an extra data point, just tested e74585d and it worked fine for me with O365 using the settings:

account "o365"
  imaps
  server "outlook.office365.com"
  user "..."
  pass $(pass ...)
  no-cram-md5
  no-plain
  no-login
  xoauth2

[nicm]

That fix is for PLAIN though...

[KenMacD]

Tested the latest changes with PLAIN and they worked for me (found the same in my own testing: KenMacD@1824941)

[nicm]

Thanks, yes I think it works with base64 (and some servers also without). @frddbbbl has special characters in his password and they need to be escaped differently for fdm from the program he was using before.

[nicm]

Or more accurately, different characters are special between the two programs. Anyway it is not working yet but I am pretty sure that is the problem.

[nicm]

This is working now, thanks all.