VoIP Troubleshooting

Issue

Like any technology, VoIP can encounter issues that can cause disruptions to phone communications. VoIP issues can vary in nature, ranging from poor call quality, dropped calls, delayed or distorted audio, to network connectivity problems, hardware malfunctions, or compatibility issues between different systems.

Understanding a VoIP System

A traditional VoIP system comprises several elements such as IP phones, SIP Proxy/PBX, and Media Gateway. Several messages are involved in the establishment of a call depending on the signaling protocol used by the VoIP system.

SIP is the most commonly used signaling protocol and it includes messages such as INVITE, 100 Trying, 180 Ringing, 200 OK, etc. SIP uses port UDP/5060 or TCP/5060 for non-encrypted signaling traffic and port TCP/5061 when traffic is encrypted with TLS. SIP will negotiate the IP address, port number, and media attributes each peer uses in the RTP conversation.

RTP (Real Time Protocol) packets carry the voice payload for a VoIP call across the network from transmitter to receiver. It runs over UDP and is used in conjunction with the RTP Control Protocol (RTCP) which provides statistics and control information for an RTP session.

The graph below shows a phone registration process:

The graph below shows a regular flow in a VoIP call:

The functionality of SIP mainly consists of 3 phases.

High-level overview of the 3 phases:

Phase 1: SIP Registration - Phones are registered on the SIP VOIP Server (SIP_Register)

Phase 2: SIP Call Invite - Calls are initiated and phones are ringed/notified (SIP_Invite)

Phase 3: SIP RTP Session - Calls are established and conversations are made before ending the call (RTP)

Troubleshooting VoIP

General Points to note when troubleshooting VoIP:

- Identify the IPs of Phones/endpoints and the SIP Server.

- Verify that access to Phone logs and SIP Server logs is available.

- Allow traffic packet capture (pcap) on Phones/endpoints, SIP Server, and network devices to facilitate analytic/investigation efforts.

1. Troubleshooting SIP Registration

- Check whether the phones are able to connect and register on the SIP server.

- Verify phone logs and SIP Server logs to confirm that the phone is registered successfully.

- Check Monitoring > Events in the CMA for any blocks from the phone's IP address. Identify the firewall rule blocking SIP traffic from the phone and allow the traffic.

- Perform pcap capture to analyze the traffic on the phone, SIP server, and network device to analyze further (look for SIP_REGISTER messages).

- As long as there is a response from the server, you can verify the Cato egress IP in the Via section of the Message Header in packets sent by the server. The Received IP is the Cato egress IP, and RPort is the NAT source port of outbound packets from the PoP.

- If the phone receives repeated 401 Unauthorized messages from the server with no 200 OK, it's likely that the phone account is not configured on the SIP server or the username/password entered on the phone is wrong.

- If necessary, configure DHCP options to define network information needed for phone registration. This may include options 66 & 150 (TFTP server information), or option 43 (TFTP, call server, L2, and DSCP priority). Reach out to the VoIP vendor for more details, if required. 

- Some VoIP systems use the TLS protocol for encrypted phone registration with Cloud providers. Cato TLS inspection may break the TLS connection as the Cato certificate cannot be installed on the phone. Apply a TLSi bypass rule to prevent TLSi from inspecting VoIP traffic.

2. Troubleshooting SIP Call Invite

- Check that phones are registered on the SIP Server and SIP invite logs are seen on the phone and the SIP server.

- Attempt a phone call and Perform pcap to analyze the traffic on the phone, server, and network devices to analyze further (look for SIP_INVITE messages)

- Identify whether SIP UDP packets are being fragmented by the network and whether the fragments are being received by the phones and the SIP server. By default, Cato allows fragmentation at the PoP level which is the desired behavior.

- Consider configuring one Egress IP in the network rule for VoIP traffic when the SIP server is reachable via the Internet. This way, VoIP phones will keep a single public IP when registering with the SIP server.

- If an Egress IP is selected, make sure that the Preferred IP for SIP Traffic advanced setting is enabled. When enabled, SIP traffic always egresses from the same IP. Packets are temporarily dropped if the egress IP is unavailable. This prevents SIP flows from being temporarily NATed to a different egress IP which can cause issues. 

- If the VoIP protocol is different than SIP, please Contact Support to apply backend changes to force traffic to stay on the egress IP.

- We may consider bypassing the Cato Cloud as a test to rule out any issues with Cato.

3. Troubleshooting SIP RTP Calls (Call Dropped, One-way sound, Jittering, delay...)

- Verify the issue seen (Call Drop, One-way sound, Jittering, delay...)

- For Call Drop, perform a pcap of the Call and identify the time and events before the Call was dropped. Match the events on Cato devices and identify the possible cause (e.g. session timeout, keepalives not seen). Reach out to the VoIP vendor for more details, if required. 

- For one-way sound encountered. perform a pcap of the Call and identify the IP/ports used. SIP ALG may be required to be enabled or disabled depending on the VoIP vendor's requirements. SIP ALG translates the phone/PBX private IP address to the Cato public IP address (and vice-versa) in the plaintext SDP message after Cato performs NAT. SIP ALG is disabled by default at Cato but can be enabled in Advanced Configuration. Reach out to the VoIP vendor for more details, if required. 

- For Jitterring and Delay, check Network Analytics to identify any high latency or jitter over the Cato tunnel. Perform a pcap as required. ITU-T G.114 recommends less than 150ms of one-way latency for VoIP. QoS and BW Management may be considered/reviewed to resolve this issue.

- For voice quality issues, make sure that a Network Rule is in place to give VoIP traffic the highest priority and with packet loss mitigation enabled. If there are no network rules with this parameter, create a new one for VoIP traffic. Additionally, identify and troubleshoot any packet loss that may be affecting VoIP calls.