-
-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Outlook/Office365/Microsoft365 and junk mails #2851
Comments
Been there, done that, since 10+ years... My take on this is:
Invite users never to flag spam in their M$-boxes which had been forwarded through your server, especially if it's coming from 3rd party mailingslists: It helps nobody and causes a lot of frustration (especially for your users at M$ services) |
This is indeed frustrating, and so far the only provider causing that amount of issues. In my case, this happened during an ongoing mail conversation. In my opinion that is ridiculous and shouldn't happen at all. I was communicating with my new employer - which are hosting their mail-stuff on Office 365: We wrote about 20 serious, legit mails in both ways. At some point their mail systems suddenly dropped my mails in the junk folders for absolutely no reason. I mean, I was writing with many different mail addresses on their end, which were replying as well. I don't get it, why their systems believe that this must be spam or something harmful. During an ongoing mail conversation. Beside the fact that mails were (detectable) replies, and not entirely new mails. You might imagine how much confusion this has caused in the end... This can't be in the sense of Microsofts' customers. |
My current employer uses Office365 and we have issues like this all the time with smaller companies we work with. The craziest thing is: if i try to see why the email gets flagged (mail flow report via the secruity and compliance center), i just get "Flagged as Spam: No further information available". I myself don't have this problem. I use their SNDS which may help out a bit. |
Did it? Did you ever received a notification from their JMRP/SNDS? I've been in their programs since basically ever, and never got any message. I'm already wondering if this works at all :D |
yes to my SNDS registered abuse@mydomain, also a few weeks ago from staff (at) hotmail dot com w/ subJ "complaint about message from xx.xx.xx.xx" without text and containing as an attachment the original message that a Hotmail user manually marked as spam. No links for followup nor to mark the issue as resolved, |
this is what i stated above: |
"may help " i'm not sure. |
My personal experience is that this problem is mainly on the free tier: outlook.com, hotmail.com, etc. Had this kind of problem a while ago, my mails where not allowed at all (blacklisted, for no reason, new server/ip). However, mails to the Office 365 paid subscription (someone with a own domain) received my mails just fine. After multiple emails asking to at least say why I was blocked, they decided to 'mitigate' my ip for no reason. I never knew exactly it was blocked. My take on this (and this might be a unpopular opinion, but regardless) is... Punishing people who run their own server. By making this thing a PITA, they can subtle convince people that it's not a good idea to run their own server at all (which I totally don't agree with, the more decentralized, the better), and switch to their service instead. Off course, there are other mail companies with a large customer base (and they can't afford to block them), but hey, every customer bringing their mails to them (including the data) is one right? I don't see another good reason why they would do this. |
There might be a possible workaround for some people: Routing all mails to For some unimaginable reason (sarcasm!) there's even a Stackoverflow question with a good reply how to configure this sort of workaround using postfix's |
@patschi, I‘ve been using transport maps to deliver to {outlook,hotmail,live}.{com,de} via sendgrid and that works nicely. Sendgrid has a free tier if you send less than 100 messages per day. Perhaps we could implement |
Yes, that's possible too, however this doesn't work on custom domains or Office365-hosted domains, therefore I personally like the
Yes, I also think that this is a good idea to offer users at least a workaround for this mess. I've already told @andryyy about the idea - it's up to him to decide if and how exactly this should be integrated like. |
How is authentication handled? How does it interfere with transport maps? |
Something like "asd.com FILTER smtp:bla" can also lead to funny behaviors when you bcc an address. It will probably end up duplicated. |
It will be hard to prove that they're doing this on purpose, but if we can, that would mean they're abusing their market power, right? That could lead to some interesting legal consequences... At least that's what I was thinking... |
So apparently Microsoft decides to ignore all the feedback, to kill the conversation and uses an comment* as an excuse to completely lock down the issue: https://github.com/MicrosoftDocs/OfficeDocs-o365seccomp/issues/409#issuecomment-526797089. That's just so ridiculous. Seriously. I'm wondering where's their respect (according their Code of Conduct) for all affected users which are lucky enough not to host their mailservers at Microsoft? Conversation continued here... Wondering how long this issue stays open. *To be fair, the comment was a little bit harsh, but neither wrong nor a reason to lock the conversation IMHO. |
To conclude: No basic change in microsofts policy to silently block mailservers since 2005: If a client does not like to selfhost mail, then they should at least move over to google (they host 3rd party domains too) and provide good spam filtering. |
I want to use that workaround. is the following way the right way ? So i have to add /opt/mailcow-dockerized/data/conf/postfix/main.cf I have to create a file /opt/mailcow-dockerized/data/conf/postfix/finickydestination with this line
It would be great if someone could give me a short "how to" for that workaround |
It will not work. You need to setup default_transport to route through a local smtpd like default_transpor t = [127.0.0.1]:2255. In this new smtpd, you can use check_mx_bla and content_filter through smtp: back into the mailcow system. A lot of functions will break in mailcow. |
And all of a sudden, they blocked my server again. In the middle of a legitimate mail conversation, the mails started to bounce back:
But, why?!? Here we go, again... Edit: Off course.
@andryyy: What functions will break, exactly? Can those be worked around? This is a big issue that will render a mail server useless, if you take into account how much people are using Microsoft's email service. |
I've been using check_recipient_mx_access on my Mailcow server for a long time to route Microsoft bound mail via a relay I haven't seen any major issue, am not 100% sure what functions might get broken using that method. The functions I think that might get messed up or broken would be things like Sender-dependent transports, BCC/Recipient/Transport Maps,Outgoing TLS policy map overrides which I don't use any of them anyway. |
I would be also interested in a small how to in order to fix this problem! :) @Ry3nlNaToR could you maybe help us? Edit update: Solved my issue via the a transport map in the routing menu. |
No @SomeGeek, it does not render your server useless. If your IP reputation sucks, use Amazon SES or any other relay service for Outlook or complain at Outlook. You can implement it and test all kind of routing in mailcow afterwards. A lot of special-case routing will break. If it does not, create a clean and tested PR. I will merge it. This does not mean, that you are unable to relay to Outlook. It is not a bug in mailcow. It is just that Outlook chose the easy path. You can ask people on Outlook.com how they feel about having a bunch of valid mail in junk, because Outlook is unable to filter inbound and forces its senders to scan mail for them. Tell them to whitelist you and tell them why. Ask them how they feel about checking their junk each day. They probably use it like their inbox now. @Ry3nlNaToR you are correct, these are the functions, that will break. If you don't use them, it is fine, I guess. But you probably understand, I cannot implement it like this. Github is NOT a place to get support for mail delivery. Buy a relay service, if you cannot reach the amount of mail per day, to be trusted by Outlook. Or just keep mailing. Tell your rcpts why this is happening. They will probably miss a lot of mail from small businesses. |
Just for reference, this is happening to me too, on my own properly validated domain. It used to work at the beginning, but doesn't anymore for some reason. It looks like Microsoft is getting busy not properly managing spam and yet still blocking legitimate emails from SBEs... Anyway, I'll implement the workaround today with sendgrid. Thanks for the tip! |
Hello, I've tried the proposed fix but couldn't get it to activate using check_recipient_mx_access as @Ry3nlNaToR . I've had to use the transport map and do it manually (and it's not working for some unknown reason). I'm not sure postfix can use both the sql database and the file-based system. Or at least, I couldn't configure it properly. For the sake of it, here are my steps, from the folder
To test the filter, I put a I believe (but haven't tested yet due to lack of time) that if the |
check_recipient_mx_access does break a lot of things. I tried it for a day and gave up, as there were too many configurations that needed to be changed - including routings in sql tables etc. It works, if you don't mind breaking some other things we allow users to manage in mailcow UI. If you don't mind, you can implement it in your cow. :) Otherwise, talk to Outlook users and tell them, why they are missing important mail. |
I found this another check found on google support : https://www.checktls.com/TestReceiver My result : CheckTLS Confidence Factor for "mail@mydomain.tld": 100 I have too send request to google via answer no to all questions (and it's true) : for get this url : |
Also tested that one, all is green for my mail server (as expected). I have submitted a request to this form a couple of days ago. They should provide an answer in two weeks time (what a joke) In my case, I don't use google in any form to send emails so all their tooling to check my "reputation" are mostly useless for me. I just need them to not greylist me for no apparent reason. I'm sorry to read that because of OVH you've been blacklisted :-( Anyway, thanks for your suggestions 👍 |
This works for me as a bandaid. I don't like using it, but I see no other choice. I tried the fix that @EricThi laid out. I swapped to a new IP address, but unfortunately in the middle of implementing @EricThi 's scripts, I got banned on the NEW IP address from Microsoft. My emails were handwritten between my Mailcow installation and my personal MSN account. Like others, I had no luck with Microsoft support. Not eligible for mitigation, and they won't discuss it with me. Edit: I sent an absolutely scathing email to Hotmail support in response to their "sorry but we aren't doing anything" response accusing them of anticompetitive behavior and that I am telling all my clients to shutdown their Microsoft accounts. They responded by reopening my case, investigating, and unblocking me. Please let this whitelist stay. Edit2: I implemented something similar to @EricThi, but used Enron's e-mails instead. I detailed my approach here. |
This is an awesome idea to help boost sender reputation. I was curious why Google and MS postmaster tools never showed anything for my domains, even they are 2 years in prod. Turns out 250 mails a day is not enough. So about 2 weeks ago I implemented your suggestion but as of now postmaster tools still shows no data. Does this technique still work ? I I send to 3 different gmail and 3 different microsoft accounts. Google DMARC report for a typical day: Ipv4 Could it be that because I am dual stacked and each IP is treated separately the postmaster tools won't show any data for me because my ~600 a day are about 50/50 split IPv4 and IPv6 ? Edit: I see in @Clete2 writeup from the comment above
In my case I just have a filter rule to move them to a folder and mark as read. Are they tracking that and disregarding the messages ? I log into my receiving accounts every now and then to reply to bacon ipsum mails to show "interaction" of sorts. |
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. |
On my side, */10 * * * * cp /usr/local/smtp/templates/joke_jod /usr/local/smtp/tmp/joke_jod && curl -X GET "https://api.jokes.one/jod" -H "accept: application/json" -H "content-type: application/json" -H "X-JokesOne-Api- after, i have add a "clean" notification and for that, i use my nextcloud with 3 fake account (with mail 1/mail2/mail3) : 16-30/2 9,16 * * 1-5 sh /usr/local/scripts/reunion-h-r.sh 31-59/2 9,16 * * 1-5 sh /usr/local/scripts/reunion-o-r.sh example, create a file via webdav with notification : and on calendar app, i have create many appointment with notification by email only since mai, i have clean my ip and i have configure dane, mta-sts and all are good now... After, i have test to send mail between mail2 and mail 3 (@Hotmail & @outlook.com) => mail are tagged spam (no bad mail, just a default mail ) @mfld-pub yes, if your send your mail via many ip, your reputation is divided by the number of ip |
I never think, use a bin mail for send many mail for increase mail per day . Example : after create random mail, send many mail (i test it for check if banned with script on cron every minute...) another services with same idea (sorry, in french website, with services french and english) : |
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. |
not stale |
I know this is an ongoing issue with problems on microsoft's side, however contacting Microsoft solved this issue for me:
and not even an hour later the IP has been whitelisted.
So for everybody considering doing a long term mail server, I suggest submitting a ticket. Super fast and E-Mails dont even land in the junk folder. Still doesn't mean microsoft shouldnt fix it on their end 🤷🏻 |
@JustinBack The Hotmail Sender Support told me that they not see any issue, why i can sending mails to Outlook.com Customers, so i argumented "why i'm not a spammer" and after three mails, they escalated my issue to the Microsoft Customer Support. The Microsoft Customer Support replied quickly and after 24h my IP was unblocked from Microsoft. I had the issue, that i received no bounce, but the mails never reached the inbox. I think the Hotmal Sender Support can't debug such issues with the Smartscreen filter, so they have to escalate it to the Microsoft Customer Support. |
Hello, I'm facing the same issue with Postfix, and can't get any support from Microsoft. So I tried at least to detect the Smartscreen abuse using delivery status notifications and sieve. f anyof ( Appears to work, but probably not cover all the cases. |
DMARC protects users against forged email messages and allows you to manage communications that do not pass SPF or DKIM. DMARC protects your email accounts against spam, spoofing, and phishing. |
so what? |
In the meanthime I've had so much trouble with it that I stopped thinking about it and blocking all smtp traffic on all mailservers I get hands on for as8075. Hopefully some day critical mass is reached and they get forced by their customers to provide a proper working email system. Until then, all my mailbox slots stay closed for M$. |
Having to go through Microsoft Support again due to a change of Public IP due to moving from OVH to their SYS (SoYouStart) line but unfortunately can't bring the IP with me. Thought I would share my experiences again since others may get the same issues and this may help. Before I started my journey with Microsoft support, I made sure rDNS, SPF, DKIM, DMARC is all setup correctly. I also signed up to their SNDS and JMRP. SNDS reports the IP as blocked. I also tried the delist request at sender.office.com but that didn't help either. So now we need to use the delist form First response from them is a "please go away", an auto reply it seems, maybe a canned response. Saying the following "...do not qualify for mitigation." and tell you to check your configuration which I've already checked and doubled checked. Gmail can deliver ok, with no warnings or failures in SPF, DKIM and DMARC when checking the headers Gmail adds. They came back to me few hours later and said I need to provide proof of purchase of the IP from my ISP. So I send them PDF copy of the invoice and PDF of the email that says the IP have been delivered. I think this pretty reasonable if they need to be able to tell the difference between legitimate requests and fake ones. So I get another email back from Microsoft hour or so later. Telling me I now need written email from SYS to say I own that IP address. Which is annoying because the invoice and email of delivery proves I own it. So does the reverse DNS record. I logged a support ticket with SYS, which they said the only proof they will send me is the invoice with the IP address in it. If I was a service provider like them, I'd probably say the same, because it has my name, address, and IP on the invoice, no need to do it again, in a less official way through a support ticket. I then reply back to Microsoft and attach their response as an .eml and the invoice again. I go on to say that they will not provide the confirmation via support ticket and that the invoice is the proof of ownership, I also point out that I have DNS control of the IP address as I have the PTR record set to my domain. They get back to me about 3 hours later, saying they are going to escalate it. From past experience this means they are going to put a mitigation in. Another 6 hours later, I get an email telling me the following
Result! Little bit of emailing and issues with their online form I've finally got the block lifted. I can now deliver to Microsoft provided email again. |
I have 10 small businesses I run their emails. All of them go to spam the first msg to gmail, after it's moved to inbox it will never go to spam again. Microsoft is pretty similar, occasionally I need to use their mail tools to whitelist an ip again or msg their postmaster and cry. but then they'll work again. |
It seems like this issue has largely died down. I still run my random email sender and reader script, more out of caution now than anything really. I rewrote the original script due to performance problems. My new script lives here and I wrote a short blog post about why I rewrote it here. I'm providing these for reference in case anyone runs across this in the future. (don't worry, no gain in it for me in these links -- no ads) I hope these scripts can help someone who struggles with reputation. Regardless if it is useful, it was a fun project. Here's to 2024 and no more Microsoft reputation issues! |
Yeah I've had less issues in recent times, guessing they were getting too many false positives. Here's to a good 2024 email delivery! |
I can confirm the issue seems to have improved in 2023 with only three occurrences in April, August and November. |
Still happening just got it the second time this year already and we are not sending anything weird funny even the first time this year we got automatically delisted after 24hours and for the secondtime im waiting to see if it happens again |
What's the volume of emailing that you are sending to Microsoft, just wondering, if it's few or many. |
sorry i somehow didnt saw the reply it just went under somewhere in my emails and notifications well our sending volume is just a few at least to microsoft related emails we have many but most are gmx,gmail,etc. and they work flawlessly not even with getting put in junk its only microsoft that we have trouble with but also not with junk mostly just the blocklist randomly nuking our server once or twice a year |
This issue is more to document and make people aware experiencing similar issues. Discussion, experiences or any tips to come to a solution might be helpful for everyone.
Office 365 / Outlook is quite special when it comes to get mails from your own mailserver delivered to said providers. In a negative aspect, unfortunately.
The problem
There are many reports from users having issues to get serious legit mails delivered to Microsofts' mailing service correctly, even with state-of-the-art non-blacklisted mailservers using latest techniques like DKIM, ARC and strict SPF and being part of their JMRP and SNDS program. In most cases delivered mails are always moved into Junk/Spam folder for absolutely no reason.
Important to notice: This is not limited to mailcow instances overall and is an ongoing issue since a quite long time.
If you have customers at Office 365 or even worse: Outlook.com you should tell them about this issue and migrate them to another service, as they will not be able to receive legit mail from clean mail servers. Business critical mail may never reach their mailbox. This is not the senders problem, this is a serious problem for the recipient and therefore the Microsoft customer.
Microsoft seems not to be able to handle their spam filters and tries to mitigate this problem by blocking whole foreign networks.
Solution
Unfortunately there is no solution available yet. Several users (including me) tried to get more information and support from Microsoft, but without any noticable improvement nor helpful reply. Apparently Microsoft has no interests at all that their users and companies - relying on Office 365 - gets legit mails of any relevance delivered.
Even analysing all headers on the Microsofts' end after delivering just gives you cryptic headers without any sort of explanation why their considered mails as spam. There are several docs around explaining a few details, but so far they are all either outdated or useless.
Following GitHub issues at Microsofts' docs repository are still pending since a longer period of time to hopefully get some more information:
What can you do?
Basically nothing. This might be a workaround.
However you are greatly welcomed to push mentioned GitHub threads above to make Microsoft more aware about this serious issues on their end. If you have more direct connections, use them.
This is going to be continuously updated...
The text was updated successfully, but these errors were encountered: