Network Issues. Impacted: API/Voice and Messaging services
Resolved
This incident has been resolved.
Posted Dec 02, 2021 - 03:15 UTC
Monitoring
Telnyx is observing restored services following packet loss from an earlier DDoS attack. Teams are monitoring all services for signs of further disruption, and continue to mitigate attacks in partnership with the Cloudflare DDoS protection team. Please continue to visit this status page for updates in the event of further disruption.
Posted Nov 23, 2021 - 17:03 UTC
Update
We are continuing to work on a fix for this issue.
Posted Nov 23, 2021 - 16:13 UTC
Identified
A DDoS attack is causing packet loss on the Telnyx network. You may experience failed calls, API and portal latency/time outs, and/or delayed or failed messages. Telnyx and Cloudflare engineers are actively engaged in mitigating customer impact from the attack.
Posted Nov 23, 2021 - 16:06 UTC
Update
Telnyx is observing restored services following an earlier incident. Teams are monitoring all services for signs of further disruption, and continue to mitigate attacks in partnership with the Cloudflare DDoS protection team.
Allow-listing is active. If you have not provided your IPs, please see our instructions for adding allowed IPs: https://telnyx.com/release-notes/allowed-ip-mission-control.
In the event of further disruption, please visit this status page.
Posted Nov 22, 2021 - 18:08 UTC
Update
Telnyx is currently being targeted by a DDoS attack. Cloudflare DDoS protection is active, however it is possible that you may experience failed calls, API and portal latency/time outs, and/or delayed or failed messages. Telnyx and Cloudflare teams are actively engaged in mitigating the attack.
Posted Nov 22, 2021 - 17:20 UTC
Update
Telnyx was targeted by a DDoS attack at approximately 15:25 UTC. Cloudflare DDoS protection worked to mitigate the attack, however a small number of calls routing through our Chicago PoP may have experienced momentary disruption between 15:25 and 15:30 UTC. Telnyx and Cloudflare teams are monitoring the situation for signs of further disruption.
Posted Nov 22, 2021 - 16:02 UTC
Monitoring
Telnyx has observed restored services following an earlier incident. Relevant teams are monitoring all services for signs of further disruption, and continue to successfully mitigate in partnership with the Cloudflare DDoS protection team.
Allow-listing is active. If you have not provided your IPs, please see our instructions for adding allowed IPs: https://telnyx.com/release-notes/allowed-ip-mission-control
In the event of further disruption, please visit this status page.
Posted Nov 17, 2021 - 22:56 UTC
Update
Telnyx is currently experiencing a renewed DDoS attack. Cloudflare mitigation is enabled, but it is possible that you may experience failed calls, API and portal latency/time outs, and/or delayed or failed messages. Telnyx and Cloudflare teams are actively engaged in mitigating this attack. Please continue to visit this status page for further updates on individual services.
Posted Nov 17, 2021 - 21:45 UTC
Update
Telnyx continues to observe partially restored services following an earlier incident. Telnyx and Cloudflare engineers continue to monitor all services for signs of further disruption.
At this time, customers who disabled TLS in response to earlier recommendations may now re-enable TLS safely.
We continue to recommend allow-listing IPs. If you have not done so already, please see our instructions for adding allowed IPs: https://telnyx.com/release-notes/allowed-ip-mission-control
In the event of further disruption, please visit this status page.
Posted Nov 17, 2021 - 18:48 UTC
Identified
Telnyx has observed partially restored services following an earlier incident. Relevant teams are monitoring all services for signs of further disruption, and continue to successfully mitigate in partnership with the Cloudflare DDoS protection team.
Allow-listing (whitelisting) is active. If you have not provided your IPs, please see our instructions for adding allowed IPs: https://telnyx.com/release-notes/allowed-ip-mission-control
Those who whitelisted IPs should have seen resolution at 16:05 UTC.
In the event of further disruption, please visit this status page.
Posted Nov 17, 2021 - 16:41 UTC
Investigating
Telnyx is currently experiencing a renewed DDoS attack. Cloudflare mitigation is enabled, but it is possible that you may experience failed calls, API and portal latency/time outs, and/or delayed or failed messages. Telnyx and Cloudflare teams are actively engaged in mitigating this attack. If you are using TLS, we recommend you disable immediately, as current mitigation techniques impact signaling using this protocol.
Posted Nov 17, 2021 - 15:54 UTC
This incident affected: Programmable Voice - Voice API (US), SIP Trunking - Signaling Regions (US, Europe, Australia, Canada), SIP Trunking - Media Anchorsites (Chicago, IL, Ashburn, VA, San Jose, CA, Vancouver, Canada, London, UK, Amsterdam, Netherlands, Sydney, Australia, Frankfurt, Germany, Toronto, Canada), Programmable Voice - TeXML (US), Numbering (Number Searching, Number Ordering), Programmable Voice - Fax (US), Mission Control API (East Region, Central Region, West Region), and Programmable Wireless, Messaging.