Long post incoming....
I have a few Smart-UPS RT 6000 RM XL systems on my network that I have configured to send email alerts via a user/mailbox hosted in Office 365. This was working great, however after installing another unit today, my settings no longer work (according to the APC E-mail Test, and the email not reaching my address). I also checked my previously installed units, and they are also receiving the same error now. I'm currently receiving this error:
Oops. Forgot to mention that I have other devices on the same network (copier/scanners), using the same SMTP settings, that are currently functioning properly. These devices use a different o365 user. I even tried using that different account on the UPS, same error.
I am seeing the same issue and will be looking into this and debugging.
I am not quite sure what we'll find yet but just wondering if you're willing to help test or debug if we need it? If so, we can work offline together on this.
P.S. I am wondering if it has to do with this and any changes they made: https://docs.microsoft.com/en-us/office365/securitycompliance/technical-reference-details-about-encryption?redirectSourcePath=%252fen-ie%252farticle%252ftechnical-reference-details-about-encryption-in-office-365-862cbe93-4268-4ef9-ba79-277545ecf221
Thank you for the response and confirmation of the issue. The o365 TLS change definitely came to mind when troubleshooting this, as we did have to make changes to our o365 setup/connectors to fully support TLS 1.2. We went through that process a few months ago, prior to the cut off date stated by Microsoft. I do know, that these alerts were working AFTER we made those changes for TLS 1.2, and this appears to be something that "broke" about 2-3 weeks ago. I initially contacted APC support about the NMC and if it supported TLS 1.2, and from the documentation provided, the AP9631 NMC card supports this encryption/protocol starting with FW v6.5.0. This is why I updated the FW to the latest release(I think) to v6.6.4 during my troubleshooting steps. Unfortunately, the FW update did not solve the issue, and outputs the same error message.
You by chance don't have access to look at connection logs or do a packet capture on this do you? With the cloud setup, I am not sure if that is really a thing anymore depending on if your O365 is cloud or on-premise and what it allows you to see or do.
I don't in my environment either so wasn't sure if you did.
Figured I would post this in hopes of saving someone's troubleshooting efforts.
I have the exact same problem you described as well as the behavior it working previously and then suddenly stopped working. I have been working on this for 4+ days and today a Wireshark captured revealed a "Bad Certificate" on the SMTP transmission.
Here is the packet capture. Bad certificate (In my case) on host 18.104.22.168 (smtp.office365.com). I currently have an open Microsoft O365 case for this as well.
Angela I have a existing case open on this. # 54986873. I haven't updated this ticket with these findings yet because I am waiting to hear Microsoft's response on this.
James, Thank you! I have been swamped the last few days and haven't been able to set up a packet capture to troubleshoot this further. Please post any updates/information that you receive from MS or any steps required to solve this issue(once found).
Hi James (and Jacob),
James - thanks so much for sharing. This is a big help. I will add this to our internal issue logged on the topic as well. We are eager to hear what MS says too - if they recently imposed a requirement to use CA certificate verification or something - which we can do on the NMC as we have a section for email certs. I will also make a note in the case so we are all on the same page.
Also curious why it seemingly works on other devices? Like Jacob, you said it is working on printers and I had it working on another device as well (though I get an authentication error which I am working on with our administrator).
Interesting you mention that.... I have dozens of MFP units and applications using authenticated SMTP in my environment. On the surface, those "things" appears to be working as expected.
The only reason I noticed this problem was finding a NMC in my environment using SMTP without authentication. I went through my normal motions of configuring it to point to smtp.office365.com. When I performed the mail test, it failed. Thinking I did something wrong on the config, I checked another NMC that I knew was working (At least I thought I did). I compared the settings to the original card I was working with and they were identical. On a whim i tested SMTP on the known working NMC and it surprisingly failed too. I looked at three other NMC with various firmware's and they all failed the SMTP tests (If you check the ticket, you will see all of the NMCs I have with the various firmware's). As it stands right now, I have six NMCs failing the SMTP test. The packet capture I ran was for only one of those NMCs and I presume the packet capture results would reveal the same error for the other NMCs.
Let's see what Microsoft has to say.
Angela I have heard nothing from MS on this. If you check the ticket, you should see a set of O365 credentials. Can you test those on an NMC available to you?
Hi James - yes, I just tried your credentials and get stuck at 220 2.0.0 SMTP server ready like with my account too.
220 2.0.0 SMTP server ready
Here is the reply from MS so far
"Unfortunately, we are not certificate providers, if that is something with the certificate being bad, you may have to check with the certificate providing authority."
Angela, Can I message/email you privately?
Hi James - yes, I'll send you an email using the address in your profile.
Can we keep updates to this issue public? Just had an outage at a site and was investigating why no alerts were received. Went through identical steps to James and some additional searching led me to this thread. Curious to find a solution and happy to help troubleshoot as needed.
Yes, I'll do that. I have been working with Jim (James) and he has been helping me debug some clients that do and don't work with packet captures and other observations. Will be sharing my analysis I'll be doing today on Jim's data with our development team so we can figure out what is going on.
Thanks. Let me know if you need another set of data points and I can open an official ticket.
I think I found the issue. We use Meraki firewalls and I discovered the NMC device being flagged with "SMTP_RESPONSE_OVERFLOW" in the threat category. Meraki uses SNOT for Intrusion/Threat Protection. Subsequently, I am also finding that some of our MFP device traffic is also being as well (Not continuously though).
Here is the specific SNOT SIDhttps://snort.org/rule_docs/124-3
I will be placing a call with Meraki to figure out why this this traffic is being flagged and potentially working towards whitelisting rules.
More to follow....
we use Zyxel router, but we have the same problem: we can't connect to O365. Also, you said that it worked before - did you do any changes on Meraki?
We too had been experiencing the same SMTP error using O365, confirmed by doing a wireshark capture as one of the previous posters (also using Meraki firewall appliances). However, noticed since Sunday evening that alerts have resumed. Appears Microsoft has resolved the issue with the certificate being rejected
Can confirm, the problem has been resolved on my systems. SMTP email alerts are now functioning as they were before.
All six of my NMCs are now working as well. The consensus is that this was a Microsoft issue?
Could anyone tell me, whether UPS should be connected to the Internet directly via Ethernet to be able to send Emails?
We have Smart-UPS C 1500I with COM-port only. I configured PowerChute Business Edition to send all types of the notification including any configuration changes, but I still don't receive any emails after any changes.
Choose a location
There are no forums in this space.