Infraly, LLC - R4, R5 and R6 Partial Power Outage – Incident details

Twilio SMS experiencing degraded performance

R4, R5 and R6 Partial Power Outage

Resolved
Partial outage
Started 7 months agoLasted about 10 hours

Affected

Hosturly

Partial outage from 9:38 AM to 10:50 AM, Operational from 10:50 AM to 7:21 PM

Dedicated Servers

Partial outage from 9:38 AM to 10:50 AM, Operational from 10:50 AM to 7:21 PM

Colocation

Partial outage from 9:38 AM to 10:50 AM, Operational from 10:50 AM to 7:21 PM

Updates
  • Resolved
    Resolved

    SBA Edge has provided an additional update at 3:21 PM ET to the partial power interruption that occurred in the early morning hours today. Please refer to it below.

    The building has been transferred to normal source; generators are in now in standby mode. All systems are reporting normal. This is the final update from the SBA Edge Facilities Engineering team until the RCA becomes available.

    Reminder:
    The MV ATS switch will be tested/evaluated/repaired at 8AM tomorrow morning. There should be no impact to clients. Our staff will be readily available to rapidly respond if further issues arise.

  • Update
    Update

    SBA Edge has provided an additional update at 12:58 PM ET to the partial power interruption that occurred in the early morning hours today. Please refer to it below.

    Our utility provider ComEd is reporting utility power has been restored to the CHI603 facility. We are planning a 1PM EST re-transfer to utility. We have additional support personnel onsite to ensure a seamless re-transfer sequence. We expect no interruptions to client services.

    The MV ATS switch will be further evaluated/repaired/tested at 8AM tomorrow morning. The repair and most of the testing can be completed with the switch in a decoupled state. Our staff will be readily available to rapidly respond to further issues until repairs can be made.

    A root cause analysis (RCA) will be provided for this event as soon as possible.

  • Update
    Update

    This is a direct message from SBA Edge:

    Please be advised, at approximately 4:10AM (this is CT), we experienced a utility 2 fault. Emergency generators are running. It appears our medium voltage ATS on utility 2 failed to transfer. The switch was manually cranked and closed restoring power to the B utility system via generators. The site is presently in a stable condition. We will provide more details as soon as possible.

  • Monitoring
    Monitoring

    At 5:38 AM ET, R4, R5, and R6 cabinets experienced a partial power outage across one of their circuits. As of 5:50 AM ET, these cabinets are coming back online; however, some systems may be encountering issues as this was a power loss event. We believe this to be a possible failure of equipment at the datacenter level, and are awaiting communication from the network operations center (NOC) for more information. As soon as we have more information, we will share it. If you are currently experiencing any issues, please open a ticket.