Cloud Interconnect for Azure Public Cloud

This guide explains how to connect Azure to the Cato Cloud via Cloud Interconnect.

For more information about Cloud Interconnect sites, see Getting Started with Cloud Interconnect Sites.

Overview of Cloud Interconnect for Azure

Cato supports Active-Passive model only for  Cloud Interconnect site with 2 circuits. BGP is used to exchange the routing information between Cato PoPs and Azure edge routers and also to determine the active acting circuit for the site.

Best Practice: Cato recommends connecting two circuits in Azure for resiliency and redundancy scenarios. Single circuit configurations are also supported.

Preparing to Create a Cloud Interconnect Site

When you create an Azure Cloud Interconnect site it's necessary to configure settings for your Azure tenant and the Cato Management Application.

Note

Note: To deploy the Cloud Interconnect site, Cato provides a seamless configuration using some basic credentials, to create the connection with the fabric provider (e.g Equinix). If your use case is not supported through the automated configuration, contact your account representative (PS/SE/CSM) to deploy the connection manually.

For issues you encounter after deploying your Cloud Interconnect site, contact Support.

High Level Overview of Configuring a Cloud Interconnect Site

image2.png

This is a high-level overview of the process to configure Cloud Interconnect for an Azure Cato site:

  1. Create two ExpressRoute circuits in Azure.

  2. Send the Service Keys for the created circuits to Cato for provisioning. (For EA - this is a manual step)

  3. In the Cato Management Application create a new site and choose site type as Cloud Interconnect.

    1. Configure /30 IP subnets for the primary and secondary circuits for the site.

    2. Configure the Bandwidth per circuit.

  4. Configure BGP between Cato and Azure:

    1. In Azure - Once provisioned, configure the ExpressRoute circuit peering and configure the same IP subnets as added to the Cato Management Application.

    2. In the Cato Management Application - Configure BGP peer settings for the primary and secondary circuits. (Cato automatically prefers the primary peer’s metrics)

  5. Connect your Azure Virtual Network Gateway to the primary and secondary ExpressRoute circuits.

  6. Test connectivity with your new site.

Below is a low-level example topology of an Azure cloud environment connected to Cato via Cloud Interconnect.

AzureHighLevel.png

Creating the Cloud Interconnect Site

In the Cato Management Application, create a new site for the Cloud Interconnect provider.

This article assumes that you are creating an HA Active-Passive Cloud Interconnect site. If you are creating a single circuit Cloud Interconnect site, please create only one primary ExpressRoute Circuit. (Only recommended for testing purposes)

For an Azure Cloud Interconnect site, we recommend that you create the site at the same time that you send the initial request for provisioning. This lets you set up the Cato configuration for this site more quickly.

To create the Cloud Interconnect site:

  1. In the Cato Management Application, under Network > Sites, click New to create a new site.

  2. Select the Connection Type as Cloud Interconnect and define the settings for the site.

    image8.png

  3. Click Apply, and then click Save.

  4. Select the new site and go to Site Configuration > Cloud Interconnect and click New Connection.

    Note

    Note: For HA deployments, make sure to create the Primary connection first.

    azure-cross-connect.png

Configuring the Settings in the Azure Account

This section describes how to configure the settings for an Azure data center so it can connect to the Cloud Interconnect site in your Cato account.

To configure Azure settings for the Cloud Interconnect site:

  1. In Azure, create a new ExpressRoute circuit under ExpressRoute Circuits > Create.

  2. Under Basics, select the Resource Group and region of instance.

    CreateExpressRoute.png

  3. In the Configuration tab, configure these settings:

    1. Select the Port Type as provider, and select a Cloud Interconnect Provider from the Provider dropdown list.

      Note: The currently supported provider is EquinixCloud Exchange (ECX). Additional providers will be supported in the future.

    2. In Peering Location, select the same location as the PoP to which you are connecting.

    3. Bandwidth: Select the bandwidth aligned with your Cato license.

    4. SKU: Select Standard or Premium. 

      Premium allows connecting a high number of VNets to the circuit while Standard is limited to 10 Virtual Networks.

    5. Billing Model Metered: is a per-consumption basis cost analysis while unlimited is a fixed monthly fee for this circuit.

      Note: Changing the billing model from Unlimited to Metered is NOT supported by Azure. However, you can change from Metered to Unlimited at any time.

      CreateExpressRoute_config.png

    6. Review and click Create the ExpressRoute circuit.

  4. Once the ExpressRoute circuit is deployed successfully, it will appear as Unprovisioned. At this stage it is not possible to configure the circuits peering until they are completely provisioned by the Service Provider. (I.e. Equinix)

    unprovisioned.png

    To complete the provisioning process the Cloud Interconnect datacenter service provider requires the unique Service Key generated by Azure for each ExpressRoute circuit. (Seen in the Overview page)

    1. Copy the Service Key. You will need it when using the automated connection wizard in the Cato Management Application. If you are creating the connection manually, send the Service Key to your Cato representative for provisioning with the service provider.

    2. You cannot continue to the next steps of this guide until the provisioning is complete.

  5. Once provisioned, the circuit configuration becomes editable and the circuit status on Azure portal will change from Unprovisioned to Provisioned.

    The next step is to associate the ExpressRoute circuit and BGP peering (The following configuration will be completely replicated on the Cato Management Application in the next steps)

    1. Under Peerings, these˛ are the options - Azure private, Azure public and Microsoft.

      Azure public is deprecated for new circuits and Microsoft is used for Azure PaaS services. For this circuit we will select Azure private (More information is available here)

    2. Peer ASN – This is the ASN which will be used to represent the Cato side in the Azure configuration. You can use any private ASN for the Cato side (The ‘Peer’ ASN).

    3. Subnets – IPv4.

    4. Primary and secondary subnets – Unique address spaces are required for the peering, each subnet has to be defined by a /30 notation. As a Microsoft Azure standard, the first usable IP serves as the peer IP for the customer’s router while the second IP serves as Microsoft’s router IP. (For example, a primary subnet of 172.16.0.0/30 uses 172.16.0.1 as the Cato router and 172.16.0.2 as the MS router)

      Azure requires both a primary and secondary subnet to be configured but Cato uses only the primary subnet for each ExpressRoute circuit. (In case of HA configuration)

      Therefore, a dummy “fake” secondary subnet is required to satisfy Azure’s requirements but will NOT be used in the Cloud Interconnect peering configurations.

      The primary subnet configured here will be used in the Cato Management Application configuration steps as well.

    5. VLAN ID – Mandatory field: The same settings as the VLAN ID configured in the Cato Management Application by Cato’s team. This VLAN ID is denoted in all parties for the connection – The cloud provider, Equinix colocation and the Cato PoP. This VLAN ID will be decided by Cato and has to be configured the same for your cloud providers peering, if required.

Linking a VNet in Azure to the ExpressRoute Circuit

Once the circuit is provisioned, what remains is connecting resources to the ExpressRoute circuit and establishing connectivity between Azure and the Cato Management Application.

This section covers the former, if you wish to establish connectivity with the Cato Management Application first please skip to Completing the Cloud Interconnect Configuration.

Limitations for Linking an Azure VNet with an ExpressRoute Circuit

  • According to Azure, deploying a VNet Gateway may take up to ~45 minutes.

  • Make sure to avoid associating network security groups with the Gateway subnet, this may cause the VNet Gateway to stop functioning.

  • VNets connected to an ExpressRoute can communicate with one another by default. Microsoft recommends to use VNet Peering.

  • The number of address spaces advertised from the local or peered virtual networks needs to be equal to or less than 1,000.

  • With a Standard ExpressRoute circuit SKU, you can link up to 10 virtual networks. (VNets) All virtual networks must be in the same cloud region while a Premium ExpressRoute circuit allows more than 10 VNets in multiple cloud regions. (The SKU per circuit can be edited in the circuit’s Configuration page)

    SKU_standard.png

Before the ExpressRoute circuit can be linked you must create a Subnet Gateway and a Virtual Network Gateway or have pre-existing ones available. More on how to do this can be found here in Azure’s official documentation.

To link a Virtual Network Gateway:

  1. On the ExpressRoute circuit page go to Connections > Add.

  2. In the Create connection page, select Connection type as ExpressRoute along with a given name and region and select Next.

    CreateConnection.png

  3. Select the Gateway to link under Virtual Network Gateway and select your provisioned circuit under ExpressRoute Circuit.

  4. The connection now appears under the circuit information.

    Connection_done.png

Completing the Cloud Interconnect Configuration

After creating the Express Route, continue with the Cloud Interconnect site configuration you started above.

azure-cross-connect.png

  1. Under Azure Express Route Configuration, enter your Azure Express Route Service Key that you created.

  2. Under Validate Your Connection, click Validate. The Bandwidth and Azure Peering Location are automatically populated.

  3. Click Apply.

    After the connection is established, proceed to configure BGP.

Defining the BGP Settings for the Site

This section explains how to define the BGP session on top of the existing private peering connection.

The cross-connect site is shown in the Connected state when at least one BGP peer is reachable, regardless of the provisioning status with your cloud provider.

BGP establishment is the sole indicator of site connectivity. BGP also allows for determining routing exchange and tunnel failover.

To configure the BGP settings for the Cloud Interconnect site:

  1. Configure the same settings in the BGP tab as in the  Cloud Interconnect site configuration. For each circuit, go to Site Configuration > BGP and click New.

  2. Under ASN Settings configure the Cato ASN as the value of your choice (make sure it matches the Peer ASN previously configured on the Azure ExpressRoute configuration page)

  3. Configure the Peer ASN as as 12076. This is a reserved ASN notation used by Azure for ExpressRoute circuits.

  4. Under IP settings configure the Peer IP. This is the same IP as configured in the Cloud Interconnect settings as Site. (This IP represents the cloud provider peer IP.)

    The Cato IP is automatically selected based on the Cloud Interconnect settings.

    image10.png

  5. Define the BGP routing policy - It is possible to manipulate the routes advertisement policy for each peer.

    We recommend that there are equal policies on both peers in a Cloud Interconnect site to avoid routing discrepancies.

  6. Define if BGP should advertise Cato as a Default Route, advertise All routes from the Cato Cloud to your cloud provider and if BGP should accept Dynamic routes received from the peer.

    image11.png

  7. Configure the Additional Settings for the BGP routes:

    1. MD5 – An additional layer of security.  This field is mandatory. (In Azure this is the Shared Key attribute)

    2. Metric – Peer priority may be changed.

      The expected configuration profile is to have a better metric for the primary peer than the secondary peer.

    3. Hold time and Keepalive interval values.

    4. Track > Email Notification – Optional alerts for BGP connectivity changes.

    image12.png

  8. Click Apply, and then click Save.

    The Cloud Interconnect site is configured. Verify that the site is working correctly, see below Monitoring and Testing Connectivity for the Cloud Interconnect Site.

Sample BGP Policy for Cloud Interconnect Site

See the following example of two peers in the supported Active-Passive configuration with Cato. (Primary peer metric is 90, secondary peer metric is 100)

image13.png

Monitoring and Testing Connectivity for the Cloud Interconnect Site

Now that the site setup is complete, let's review how the connection can be tested and monitored.

Cato provides multiple tools to help you monitor your Cloud Interconnect site and troubleshoot any potential issues, including:

Test Connectivity Tool

You can test the point-to-point Cloud Interconnect IP reachability of each circuit by using the Test Connectivity tool.

The Test Connectivity tool in Site Configuration > Cloud Interconnect sends ICMP probes from the Cato PoP IP to the site remote IP of either the primary or secondary connection.

These are the results of the ICMP probes:

  • Success - Test executed successfully

  • Error - Test was not executed. (Test timed out by the Cato PoP or unable to be executed)

  • Failed - Test performed successfully with no response from the remote peer IP

LAN Monitoring

You can use the LAN Monitoring feature to:

  • Perform continuous ICMP probing from the Cato PoP to the primary circuit remote IP.

    Note: LAN Monitoring only monitors the primary circuit for the  Cloud Interconnect site.

  • Hosts liveliness state changes for instances within the Cloud Provider network.

It is possible to set custom thresholds and ICMP Intervals and define email notifications to a mailing list upon meeting these thresholds.

image16.png

This is an example of an email notification for LAN Monitoring:

image.png

BGP Status

In Site Configuration > BGP, the Show BGP Status confirms connectivity of each circuit.

The status outputs granular information on the subnets learned, advertised and additional data on the BGP peers.

Sample of BGP status output:

image15.png
BGP Email Notifications

For each peer, we recommend configuring BGP neighbor status change notifications. Email Notifications are sent directly to an admin mailing list upon a BGP peer connection state change.

Configure email notifications in Site Configuration > BGP > BGP Neighbor > Additional Settings > Track.

Select the Frequency of alerting and Mailing List.

Cato allows for the following Frequency configuration:

  • Immediate - an email is sent to the recipients for every occurrence.

  • Hourly - an email is sent to the recipients with the first occurrence, and if there are any additional occurrences, then the next email for any additional occurrences follows after 1 hour.

  • Daily - an email is sent to the recipients with the first occurrence, and the next email follows after 24 hours (including all additional occurrences).

  • Weekly - an email is sent to the recipients with the first occurrence, and if there are any additional occurrences, then the next email for any additional occurrences follows after 1 week.

Sample BGP email notification:

BGPemail.png

Routing Table

The Monitoring > Routing Table screen shows all routes of your account including dynamic routes.

The routing table can be used to determine which tunnel, primary or secondary, is responsible for adverting these routes based on the Next Hop, PoP and Tunnel Metric.

Routes originating from BGP appear as a Dynamic routing type. Routes from the Passive circuit’s peer appear as greyed out. The point-to-point subnets of both circuits appear as a Static routing type on the routing table.

For example, the following dynamic route 172.29.0.0/24 is advertised from the New York PoP and has a metric of 5 (highest) which is the primary and currently active tunnel.

The same route is advertised by the secondary tunnel on Ashburn PoP as well with a lower metric of 10.

In case the secondary tunnel on Ashburn PoP would become the active tunnel, the routing table would adjust accordingly for this route.

image18.png

The BGP peers are Static and have their own routing table entry. This peers server as the Next Hop for the Dynamic BGP routes advertised behind them. Similarly as other routes, Metric information can be discerned to understand which peer is currently active with a higher metric and through which Cato PoP location.

image.png

Events

In the Site Monitoring > Events screen, Cato aggregates all logged events related to the site.

Key events can be used to analyze a timeline of events such as. You can filter the relevant Events using the following event sub-types:

  • BGP Session – Notify of BGP session establishment or disconnection. An identified reason for disconnection can be inspected in the expanded event log. (Under the ‘+’ icon)

  • BGP Routing – BGP route changes such as addition or removal of new routes from the BGP peer.

  • LAN Monitoring – These events are logged as part of the LAN Monitoring setup you configured. If LAN Monitoring is not set these events would not be logged.

BGPevent.png

Site Network Analytics

Site analytics lets you monitor the site’s traffic and throughput and includes these dashboards:

  • Network Analytics – Analyze connectivity state changes, number of Flows, Hosts and Throughput.

    It is important to remember the Cloud Interconnect site connectivity is based on the BGP peers. If both BGP peers are unreachable, the site is considered Disconnected.

    xconn-mbps.png

  • Events - Site events feed.

  • Application Analytics - This dashboard dissects hosts throughput and application usage. It is possible to add filters such as IP/host, Application, Category, etc...

    AppAnalytics.png

  • Priority Analyzer - This dashboard allows analyzing QoS distribution over time. (read more about Priority Analyzer)

  • Known Hosts – A real-time dashboard for hosts behind the site. IP, OS type and Host Activity are among the data points available per host.

    knownHosts.png

  • Real Time - This dashboard allows for real-time monitoring of active hosts, throughput, top applications, active QoS and more.

    RealTime.png

Cloud Interconnect Limitations with Azure

The following is a highlight list of limitations to consider before setting up an Azure Cloud Interconnect site:

  • The peering IPs are predetermined – The first usable IP represents the “On-Prem” peer and the consequent IP represents the MS Azure router.

  • Each tunnel VLAN ID is assigned by Cato and has to be configured in the Azure settings. Without the Cato assigned VLAN ID the setup would not work.

  • When advertising routes, Azure does not have an option to exclude/include routes from the VNet. The whole VNet is advertised.

  • ExpressRoute allows advertising up to 1,000 IPv4 prefixes and 100 IPv6 prefixes.

  • ExpressRoute allows receiving up to 4,000 prefixes from Cato. (Cato has an option for custom route summarization. If you would like to configure route summarization with BGP, please contact Cato Support)

  • ExpressRoute Premium allows for more than 10 VNets attachments in multiple Azure regions and up to 10,000 received prefixes.

    • If the prefix limit is met, Azure tears down the BGP connection until limit capacity is restored.

You may read more on Azure’s official ExpressRoute documentation.

Was this article helpful?

0 out of 0 found this helpful

0 comments

Add your comment