Active Incident

Updated a few seconds ago

Check our system status and planned maintenance events 24/7/365

Locations

Dallas Proxy Brief DisruptionOperational

Incident Status

Operational

Components

SBC Layer

Locations

DP: DALLAS, TX


June 30, 2025 11:17AM EDT
June 30, 2025 3:17PM UTC
[Identified] We are working on preparing the temporary hardware for the v20 proxy workload in Dallas. We plan to migrate the newer Dallas proxy at 11:15PM CST to this new hardware. This may result in a brief downtime of around 5-10 minutes on v20 physical devices, however they should failover to their backup proxy where configured. Devices on v18 systems are unaffected by this maintenance.

June 29, 2025 11:38PM EDT
June 30, 2025 3:38AM UTC
[Identified] We've identified an unexpected memory error in the logs from the issue earlier today. This only occurs under full production load, and not in the smaller scale tests we've run on this specific server. We've narrowed this down to a hardware level issue with the host hypervisor. While we work to diagnose and repair the host, we're setting up a new dedicated server to hold the newer proxy for Dallas v20 workloads. We expect to migrate that in the next day or two once the new hardware is ready. v18 workloads have been moved back to the old Dallas proxy until we have resolved the hardware issue. This will also prevent the issue from occurring for v20 workloads until they can be moved.

June 29, 2025 3:21PM EDT
June 29, 2025 7:21PM UTC
[Investigating] We saw a brief drop in connectivity for the Dallas proxy at 1:47PM CST that lasted about 30 seconds. We are reviewing the logs to determine the root cause, and service is currently operational at this time. We plan to temporarily revert to the old Dallas proxy while we investigate to ensure this disruption does not re-occur during the week before a cause can be determined. That change will occur at 10:35PM CST tonight. No interruption will occur from that change, as devices automatically switch over as they perform re-registration. If we see any further issues, we will perform the reversion immediately.

Cloud UC Platform

Operational

SBC Layer

Operational

IQ Integration Platform

Operational

Control - Multi-Tenant Management Portal

Operational

Device Management & Provisioning Platform

Operational

Microsoft Teams

Operational

DNS Services

Operational

North American Voice & Messaging Carriers

Operational

Europe Voice & Messaging Carriers

Operational

APAC Voice & Messaging Carriers

Operational

Aux Services

Operational

External Services

AWS Route 53

AWS S3

Amazon Web Services

Azure

Bandwidth

ConnectWise

Datto / Autotask

Duo

HubSpot

IT Glue

MS Office365

Okta

OpenAI

Paypal

Salesforce

Status.io

Twilio

Zapier

Zendesk

Zoom