Network Issues. Impacted: API/Voice and Messaging services
Monitoring - Telnyx is observing restored services following packet loss from an earlier DDoS attack. Teams are monitoring all services for signs of further disruption, and continue to mitigate attacks in partnership with the Cloudflare DDoS protection team. Please continue to visit this status page for updates in the event of further disruption.
Nov 23, 17:03 UTC
Update - We are continuing to work on a fix for this issue.
Nov 23, 16:13 UTC
Identified - A DDoS attack is causing packet loss on the Telnyx network. You may experience failed calls, API and portal latency/time outs, and/or delayed or failed messages. Telnyx and Cloudflare engineers are actively engaged in mitigating customer impact from the attack.
Nov 23, 16:06 UTC
Update - Telnyx is observing restored services following an earlier incident. Teams are monitoring all services for signs of further disruption, and continue to mitigate attacks in partnership with the Cloudflare DDoS protection team.
Allow-listing is active. If you have not provided your IPs, please see our instructions for adding allowed IPs: https://telnyx.com/release-notes/allowed-ip-mission-control.
In the event of further disruption, please visit this status page.
Nov 22, 18:08 UTC
Update - Telnyx is currently being targeted by a DDoS attack. Cloudflare DDoS protection is active, however it is possible that you may experience failed calls, API and portal latency/time outs, and/or delayed or failed messages. Telnyx and Cloudflare teams are actively engaged in mitigating the attack.
Nov 22, 17:20 UTC
Update - Telnyx was targeted by a DDoS attack at approximately 15:25 UTC. Cloudflare DDoS protection worked to mitigate the attack, however a small number of calls routing through our Chicago PoP may have experienced momentary disruption between 15:25 and 15:30 UTC. Telnyx and Cloudflare teams are monitoring the situation for signs of further disruption.
Nov 22, 16:02 UTC
Monitoring - Telnyx has observed restored services following an earlier incident. Relevant teams are monitoring all services for signs of further disruption, and continue to successfully mitigate in partnership with the Cloudflare DDoS protection team.
Allow-listing is active. If you have not provided your IPs, please see our instructions for adding allowed IPs: https://telnyx.com/release-notes/allowed-ip-mission-control

In the event of further disruption, please visit this status page.
Nov 17, 22:56 UTC
Update - Telnyx is currently experiencing a renewed DDoS attack. Cloudflare mitigation is enabled, but it is possible that you may experience failed calls, API and portal latency/time outs, and/or delayed or failed messages. Telnyx and Cloudflare teams are actively engaged in mitigating this attack. Please continue to visit this status page for further updates on individual services.
Nov 17, 21:45 UTC
Update - Telnyx continues to observe partially restored services following an earlier incident. Telnyx and Cloudflare engineers continue to monitor all services for signs of further disruption.
At this time, customers who disabled TLS in response to earlier recommendations may now re-enable TLS safely.
We continue to recommend allow-listing IPs. If you have not done so already, please see our instructions for adding allowed IPs: https://telnyx.com/release-notes/allowed-ip-mission-control
In the event of further disruption, please visit this status page.
Nov 17, 18:48 UTC
Identified - Telnyx has observed partially restored services following an earlier incident. Relevant teams are monitoring all services for signs of further disruption, and continue to successfully mitigate in partnership with the Cloudflare DDoS protection team.

Allow-listing (whitelisting) is active. If you have not provided your IPs, please see our instructions for adding allowed IPs: https://telnyx.com/release-notes/allowed-ip-mission-control

Those who whitelisted IPs should have seen resolution at 16:05 UTC.

In the event of further disruption, please visit this status page.
Nov 17, 16:41 UTC
Investigating - Telnyx is currently experiencing a renewed DDoS attack. Cloudflare mitigation is enabled, but it is possible that you may experience failed calls, API and portal latency/time outs, and/or delayed or failed messages. Telnyx and Cloudflare teams are actively engaged in mitigating this attack. If you are using TLS, we recommend you disable immediately, as current mitigation techniques impact signaling using this protocol.
Nov 17, 15:54 UTC
Elastic SIP Trunking | Large UDP messages not reaching Telnyx causing call failures.
Update - All teams from both Telnyx and Cloudflare remain actively engaged in resolving this issue. Additional monitoring has been deployed to assist in the investigation. Telnyx will continue to post updates here as our teams continue to investigate.
Nov 23, 14:37 UTC
Identified - We've identified that some UDP inbound packets are being dropped by our DDoS mitigation provider Cloudflare due to UDP packets being above MTU size. We are able to reproduce the issue and currently working with Cloudflare DDoS engineers to troubleshoot and mitigate the issue
Nov 22, 21:17 UTC
Investigating - We are currently investigating this issue.
Nov 22, 19:36 UTC

About This Site

This page is dedicated to keeping you informed on status updates from Telnyx.

Telephony SIP Regions ? Operational
US ? Operational
Europe ? Operational
Australia ? Operational
Canada ? Operational
Telephony Anchorsites ? Operational
Chicago, IL Operational
Ashburn, VA Operational
San Jose, CA Operational
Vancouver, Canada Operational
London, UK Operational
Amsterdam, Netherlands Operational
Sydney, Australia Operational
Frankfurt, Germany Operational
Toronto, Canada Operational
Other Telephony Systems Operational
Messaging Operational
Call Control Operational
Numbering Operational
Number Searching Operational
Number Ordering Operational
Mission Control API ? Operational
East Region Operational
Central Region Operational
West Region Operational
CNAM API Operational
Central Region ? Operational
West Region Operational
LRN Lookup API ? Operational
East Region Operational
Central Region Operational
Switch Data API Operational
East Region ? Operational
Central Region Operational
Intercom Web Application Operational
TeXML Operational
TeXML Operational
Wireless Core Operational
Programmable Fax Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
System Metrics Month Week Day
Mission Control API Response Time ?
Fetching
CNAM API Response Times
Fetching
LRN Lookup API Response Times
Fetching
Switch Data API Response Times
Fetching
Past Incidents
Nov 29, 2021

No incidents reported today.

Nov 28, 2021

No incidents reported.

Nov 27, 2021

No incidents reported.

Nov 26, 2021
Scheduled Telephony Engine Maintenance - US Region
Completed - Maintenance was completed successfully.
Nov 26, 10:15 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Nov 26, 09:30 UTC
Scheduled - Our Telephony Engineers will be performing maintenance on our SIP proxy servers in the US region.
This maintenance will not disrupt service, but you may notice re-registration of your devices during the maintenance, as our primary and secondary SIP IP addresses are moved between hosts.
Nov 25, 10:35 UTC
Nov 25, 2021
Scheduled Telephony Engine Maintenance - Canada Region
Completed - Maintenaince was completed without issue.
Nov 25, 10:19 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Nov 25, 09:31 UTC
Scheduled - Our Telephony Engineers will be performing maintenance on our SIP proxy servers in the CA region.
This maintenance will not disrupt service, but you may notice re-registration of your devices during the maintenance, as our primary and secondary SIP IP addresses are moved between hosts.
Nov 25, 08:30 UTC
Nov 24, 2021

No incidents reported.

Nov 23, 2021

Unresolved incidents: Elastic SIP Trunking | Large UDP messages not reaching Telnyx causing call failures., Network Issues. Impacted: API/Voice and Messaging services.

Nov 22, 2021
Nov 21, 2021

No incidents reported.

Nov 20, 2021

No incidents reported.

Nov 19, 2021
Scheduled Telephony Engine Maintenance - Europe Region
Completed - Maintenance was completed successfully, without incident.
Nov 19, 18:55 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Nov 19, 18:32 UTC
Scheduled - Our Telephony Engineers will be performing maintenance on our SIP proxy servers in the EU region.
This maintenance will not disrupt service, but you may notice re-registration of your devices during the maintenance, as our primary and secondary SIP IP addresses are moved between hosts.
Nov 19, 15:22 UTC
Prog. Messaging: Enabling SMS on USA/CAN DID's
Resolved - As a follow up to the SMS enabled activation issue with NetNumber, we saw connectivity return at 16:45 UTC 18th November 2021. Engineers have monitored since and do not see further occurrences of timeouts when attempting to assign messaging profiles to DID's or creating 10DLC campaigns. This incident is considered resolved.
Nov 19, 09:24 UTC
Update - We are continuing to work on a fix for this issue with NetNumber.
Nov 18, 15:57 UTC
Identified - Messaging engineers have verified that the activation failures are the result of NetNumber (Central registry for NNID updates) experiencing network connectivity issues in their datacenter.

NetNumber is actively working to restore services and our engineers are in communication with them on an expected resolution time.

Customers when attempting to assign a messaging profile to their DID's may experience a timeout. This timeout may also be observed with campaign creation through our 10DLC product, as the campaign registry (TCR) are also impacted.
Nov 18, 14:49 UTC
Investigating - We are currently investigating this issue.
Nov 18, 14:43 UTC
Voice: Audio Quality Degradation
Resolved - This incident has been resolved.
Nov 19, 09:20 UTC
Monitoring - A network configuration change has been made as of 20:27 UTC, and we are seeing fully restored voice services.
Nov 18, 22:40 UTC
Identified - We've noticed reports of network congestion with traffic egressing from our network that may cause audio quality degradation with some calls.

Our engineers are continuing to investigate and we'll provide further updates on our Status Page
Nov 18, 18:10 UTC
Investigating - We are currently investigating this issue.
Nov 18, 16:57 UTC
Nov 18, 2021
Voice: Inbound call failures over TLS
Resolved - This incident has been resolved since 15:13 PM UTC 17th November 2021.
Nov 18, 15:54 UTC
Monitoring - Telnyx NOC received reports of inbound call failures at 14:50 PM UTC. These calls were associated with SIP Connections over the TLS transport protocol.

The issue was escalated to the telephony team at 15:00 PM UTC and confirmed resolved. An incorrect firewall rule was introduced at 13:38 PM UTC and was accordingly disabled at 15:13 PM UTC.

We sincerely apologise for any inconvenience caused.
Nov 17, 15:23 UTC
Voice Calls: Canadian local call impairment
Resolved - This incident has been resolved.
Nov 18, 14:42 UTC
Update - The upstream provider has confirmed their network is now showing optimal traffic patterns and all systems are functioning per design in normal parameters. Telnyx is observing no further degradation but we'll keep monitoring the situation.
Nov 16, 23:51 UTC
Update - As a further update, while our customers are now able to receive calls some calls may be experiencing audio degradation. The upstream provider is aware and continue to work on restoring their services fully.
Nov 16, 13:58 UTC
Update - The upstream provider has confirmed services are being restored. Telnyx is observing improved connectivity on local Canadian calls at this time. We are continuing to work with them and will monitor this situation closely for further updates.
Nov 16, 13:40 UTC
Monitoring - The upstream provider has confirmed services are being restored. Telnyx is observing improved connectivity on local Canadian calls at this time. We are continuing to work with them and will monitor this situation closely for further updates.
Nov 16, 10:00 UTC
Update - An upstream provider in Canada is being targeted by a DDoS attack. Telnyx customers may see issues with inbound calls to Canadian numbers. We are working directly with this provider and others to mitigate the issue for our customers.

We will endeavor to keep you updated as frequently as possible, however, you can also subscribe to our status page for further information regarding this service impact.
Nov 15, 21:57 UTC
Identified - The issue has been identified, our local carrier is experiencing a network connectivity issue. We are working with them at this time to determine cause and time to restoration.
Nov 15, 17:45 UTC
Investigating - We are currently investigating an issue where local calls within Canada are experiencing connectivity issues.
Nov 15, 17:35 UTC
Nov 17, 2021
Nov 16, 2021
Scheduled Telephony Engine Maintenance - Australia Region
Completed - Maintenance was completed successfully.
Nov 16, 19:49 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Nov 16, 18:17 UTC
Scheduled - Our Telephony Engineers will be performing maintenance on our SIP proxy servers in the AU region.
This maintenance will not disrupt service, but you may notice re-registration of your devices during the maintenance, as our primary and secondary SIP IP addresses are moved between hosts.
Nov 12, 14:35 UTC
Urgent Telnyx Update: Action Required
Completed - The scheduled maintenance has been completed.
Nov 16, 05:16 UTC
Update - Telnyx has reason to believe that sip.telnyx.com will be targeted in a subsequent attack today (Mon 15 Nov). During attacks, it is likely Telnyx will block traffic from non-trusted sources. Telnyx has proactively created a list of trusted user IPs however, we strongly encourage all customers to check their records and add any IPs that they may be using, to their Access Control IP List as per these instructions: https://telnyx.com/release-notes/allowed-ip-mission-control
Nov 15, 15:52 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Nov 15, 05:16 UTC
Scheduled - Telnyx has reason to believe that sip.telnyx.com will be targeted in a subsequent attack tomorrow (Mon 15 Nov). During attacks, it is likely Telnyx will block traffic from non-trusted sources. Telnyx has proactively created a list of trusted user IPs however, we strongly encourage all customers to check their records and add any IPs that they may be using, to their Access Control IP List as per these instructions: https://telnyx.com/release-notes/allowed-ip-mission-control
Nov 15, 05:15 UTC
Nov 15, 2021
← Incident History Powered by Atlassian Statuspage