Cross Connect for AWS Public Cloud

This guide explains how to connect Amazon Web Services (AWS) to the Cato Cloud via Cross Connect.

For more information about Cross Connect sites, see Getting Started with Cato Cross Connect Sites.

Overview of Cross Connect for AWS

Cato supports Active-Passive model only for Cross Connect site with 2 circuits. BGP is used to exchange the routing information between Cato PoPs and AWS edge routers and also to determine the active acting circuit for the site.

Best Practice: Cato recommends connecting two circuits in AWS for resiliency and redundancy scenarios. Single circuit configurations are also supported.

 

Sample AWS Cross Connect Use Case

This article explains how to connect your AWS cloud environment with Direct Connect to Cato.

Preparing to Create a Cross Connect Site

When you create an AWS Cross Connect site it's necessary to configure settings for your AWS tenant and the Cato Management Application.

This article assumes that you are creating an HA Active-Passive Cross Connect site. If you are creating a single circuit Cross Connect site, please create only one primary Direct Connect Circuit. (Only recommended for testing purposes)

At first, it is necessary to provision Direct Connect circuits by the Service Provider (such as, Equinix). To complete the provisioning process the Cross Connect datacenter provider requires the Account ID of your AWS account.

  • Send the AWS Account ID to your Cato representative with the desired BW and PoP locations. After the circuits are provisioned – continue to the next steps.

  • For an AWS Cross Connect site we recommend that you create the site in the Cato Management Application during the initial request for provisioning for a faster setup with Cato (see below Creating the Cross Connect Site).

Note

Note: The provisioning process with service provider may take up to 24 hours to be reflected in the AWS portal. Once provisioned, the circuits will appear in the AWS console under AWS Direct Connect > Connections.

High Level Overview or Configuring a Cross Connect Site

image2.png

This is a high-level overview of the process to configure Cross Connect for an AWS Cato site:

  1. Send your AWS Account ID to Cato for provisioning. 

  2. In the Cato Management Application create a new site and choose site type as Cross Connect.

    1. Configure /30 IP subnets for the primary and secondary circuits for the site.

    2. Configure the Bandwidth per circuit.

  3. Configure BGP between Cato and AWS:

    1. In AWS - Once provisioned, Accept the Direct Connect circuit peering and configure the same IP subnets as added to the Cato Management Application.

    2. In the Cato Management Application - Configure BGP peer settings for the primary and secondary circuits. (Cato automatically prefers the primary peer’s metrics)

  4. Connect your Direct Connect Gateway to the primary and secondary Direct Connect circuits.

  5. Test connectivity with your new site.

Below is a low-level example topology of an Amazon Web Services cloud environment connected to Cato via Cross Connect.

image3.png

Configuring the Settings in the AWS Account

This section describes how to configure the settings for an AWS data center so it can connect to the Cross Connect site in your Cato account.

The diagram below shows the AWS configuration for a Cross Connect site.

image4.png

  • Each VPC has a Virtual Private Gateway that connects to the Direct Connect Gateway using a Virtual Private Gateway Association.

  • The Direct Connect Gateway uses a private Virtual Interface for the connection to the AWS Direct Connect location.

  • There is an AWS Direct Connect Connection from the location to the customer data center partner. (such as Equinix)

Depending on the requirements for the AWS data center, it's possible to connect a Transit Gateway instead of a Virtual Private Gateway to the Direct Connect Gateway. For more information, see the AWS documentation about Transit Gateway Associations and Virtual Private Gateway Associations.

To configure AWS Direct Connect for the Cross Connect site:

  1. In AWS, accept the ordered circuits under AWS Direct Connect > Connections. (Once accepted, the connection State appears as Available)

    AWS Connections

  2. In AWS Direct Connect > Direct Connect Gateways create a new Direct Connect Gateway.

    1. Give it a Name and a private ASN value.

      This ASN will be referenced later in the Cato Management Application BGP setup as the AWS side.

      Note: This ASN has to be different than the ASN defined for your Virtual Private Gateway or Transit Gateway.

    2. Select the newly created Direct Connect Gateway and under Gateway Associations click on Associate Gateway.

      Here you can select between Virtual Private Gateway(s) or Transit Gateway(s). (Multiple associations are possible but must be of the same gateway type) Allowed Prefixes – If you’d like to customize the prefixes allowed to be advertised to Cato, list them here. Otherwise, leave this field empty.

      image6.png

  3. Attach VLAN Interfaces to your Direct Connect gateway: AWS Direct Connect > Virtual Interfaces > Create Virtual Interface.

    Here we attach the two provisioned circuits to the Direct Connect gateway.

    1. Select Private or Transit, according to your gateway type.

    2. Give a Name identifier to each Virtual Interface.

    3. In Connection, select each provisioned circuit.

    4. Direct Connect gateway - Select the DCG defined in step #2.

    5. VLAN – Define a unique VLAN for each interface.

    6. BGP ASN – This is the BGP for the Cato side. (One ASN for both circuits)

    7. Your Router Peer IP – A /30 IP representing the Cato peer per circuit

    8. Amazon Router Peer IP – An IP in the same /30 subnet as above per each circuit.

    9. BGP Authentication Key – MD5 authentication layer for each interface. AWS allows you to define the key on your own OR allow AWS to randomly generate a key for you.

    10. Jumbo MTU – Not supported.

    Note: It may take several minutes for the virtual interfaces to change from Down state to Available.

  4. Once configured, you should have a Direct Connect gateway in the following state with two Virtual Interfaces

    image7.png

Creating the Cross Connect Site

In the Cato Management Application, create a new site for the Cross Connect provider.

This article assumes that you are creating an HA Active-Passive Cross Connect site. If you are creating a single circuit Cross Connect site, please create only one primary Direct Connect Circuit. (Only recommended for testing purposes)

For an AWS Cross Connect site, we recommend that you create the site at the same time that you send the initial request for provisioning. This lets you set up the Cato configuration for this site more quickly.

To create the Cross Connect site:

  1. In the Cato Management Application, under Network > Sites, click New to create a new site.

  2. Select the Connection Type as Cross Connect and define the settings for the site.

    image8.png

  3. Click Apply, and then click Save.

  4. Select the new site and go to Site Configuration > Cross Connect and configure the Subnet and peering private IPs similarly to the previous steps in AWS for each Cross Connectcircuit.

    The Cato IP is the Your Router Peer IP as configured in AWS and Site IP is the Amazon Router Peer IP.

    image9.png

Defining the BGP Settings for the Site

This section explains how to define the BGP session on top of the existing private peering connection.

The cross-connect site is shown in the Connected state when at least one BGP peer is reachable, regardless of the provisioning status with your cloud provider.

BGP establishment is the sole indicator of site connectivity. BGP also allows for determining routing exchange and tunnel failover.

To configure the BGP settings for the Cross Connect site:

  1. Configure the same settings in the BGP tab as in the Cross Connect site configuration. For each circuit, go to Site Configuration > BGP and click New.

  2. Under ASN Settings configure the Cato ASN as the value of your choice (make sure it matches the Peer ASN previously configured on the AWS Virtual Interface configuration page)

  3. Configure the Peer ASN as Your preconfigured private ASN for AWS Direct Connect Gateway.

  4. Under IP settings configure the Peer IP. This is the same IP as configured in the Cross Connect settings as Site. (This IP represents the cloud provider peer IP.)

    The Cato IP is automatically selected based on the Cross Connect settings.

    image10.png

  5. Define the BGP routing policy - It is possible to manipulate the routes advertisement policy for each peer.

    We recommend that there are equal policies on both peers in a Cross Connect site to avoid routing discrepancies.

  6. Define if BGP should advertise Cato as a Default Route, advertise All routes from the Cato Cloud to your cloud provider and if BGP should accept Dynamic routes received from the peer.

    image11.png

  7. Configure the Additional Settings for the BGP routes:

    1. MD5 – An additional layer of security. This field is mandatory for AWS Direct Connect.

      AWS gives you an option to generate your own MD5 value or use their autogenerated MD5 value upon creation of each Virtual Interface.

    2. Metric – Peer priority may be changed.

      The expected configuration profile is to have a better metric for the primary peer than the secondary peer.

    3. Hold time and Keepalive interval values.

    4. Track > Email Notification – Optional alerts for BGP connectivity changes.

    image12.png

  8. Click Apply, and then click Save.

    The Cross Connect site is configured. Verify that the site is working correctly, see below Monitoring and Testing Connectivity for the Cross Connect Site.

Sample BGP Policy for Cross Connect Site

See the following example of two peers in the supported Active-Passive configuration with Cato. (Primary peer metric is 90, secondary peer metric is 100)

image13.png

Monitoring and Testing Connectivity for the Cross Connect Site

Now that the site setup is complete, let's review how the connection can be tested and monitored.

Cato provides multiple tools to help you monitor your Cross Connect site and troubleshoot any potential issues, including:

Test Connectivity Tool

You can test the point-to-point Cross Connect IP reachability of each circuit by using the Test Connectivity tool.

The Test Connectivity tool in Site Configuration > Cross Connect sends ICMP probes from the Cato PoP IP to the site remote IP of either the primary or secondary connection.

These are the results of the ICMP probes:

  • Success - Test executed successfully

  • Error - Test was not executed. (Test timed out by the Cato PoP or unable to be executed)

  • Failed - Test performed successfully with no response from the remote peer IP.

LAN Monitoring

You can use the LAN Monitoring feature to:

  • Perform continuous ICMP probing from the Cato PoP to the primary circuit remote IP.

    Note: LAN Monitoring only monitors the primary circuit for the Cross Connect site.

  • Hosts liveliness state changes for instances within the Cloud Provider network.

It is possible to set custom thresholds and ICMP Intervals and define email notifications to a mailing list upon meeting these thresholds.

image16.png

This is an example of an email notification for LAN Monitoring:

image.png

BGP Status

In Site Configuration > BGP, the Show BGP Status confirms connectivity of each circuit.

The status outputs granular information on the subnets learned, advertised and additional data on the BGP peers.

image14.png

Sample of BGP status output:

image15.png

BGP Email Notifications

For each peer, we recommend configuring BGP neighbor status change notifications. Email Notifications are sent directly to an admin mailing list upon a BGP peer connection state change.

Configure email notifications in Site Configuration > BGP > BGP Neighbor > Additional Settings > Track.

Select the Frequency of alerting and Mailing List.

Cato allows for the following Frequency configuration:

  • Immediate - an email is sent to the recipients for every occurrence.

  • Hourly - an email is sent to the recipients with the first occurrence, and the next email follows after 1 hour (including all additional occurrences).

  • Daily - an email is sent to the recipients with the first occurrence, and the next email follows after 24 hours (including all additional occurrences).

  • Weekly - an email is sent to the recipients with the first occurrence, and the next email follows after 1 week (including all additional occurrences).

Sample BGP email notification:

BGPemail.png

Routing Table

The Monitoring > Routing Table screen shows all routes of your account including dynamic routes.

The routing table can be used to determine which tunnel, primary or secondary, is responsible for adverting these routes based on the Next Hop, PoP and Tunnel Metric.

Routes originating from BGP appear as a Dynamic routing type. Routes from the Passive circuit’s peer appear as greyed out. The point-to-point subnets of both circuits appear as a Static routing type on the routing table.

For example, the following dynamic route 172.29.0.0/24 is advertised from the New York PoP and has a metric of 5 (highest) which is the primary and currently active tunnel.

The same route is advertised by the secondary tunnel on Ashburn PoP as well with a lower metric of 10.

In case the secondary tunnel on Ashburn PoP would become the active tunnel, the routing table would adjust accordingly for this route.

 

image18.png

The BGP peers are Static and have their own routing table entry. This peers server as the Next Hop for the Dynamic BGP routes advertised behind them. Similarly as other routes, Metric information can be discerned to understand which peer is currently active with a higher metric and through which Cato PoP location.

image.png

Events

In the Site Monitoring > Events screen, Cato aggregates all logged events related to the site.

Key events can be used to analyze a timeline of events such as. You can filter the relevant Events using the following event sub-types:

  • BGP Session – Notify of BGP session establishment or disconnection. An identified reason for disconnection can be inspected in the expanded event log. (Under the ‘+’ icon)

  • BGP Routing – BGP route changes such as addition or removal of new routes from the BGP peer.

  • LAN Monitoring – These events are logged as part of the LAN Monitoring setup you configured. If LAN Monitoring is not set these events would not be logged.

BGPevent.png

Site Network Analytics

Site analytics lets you monitor the site’s traffic and throughput and includes these dashboards:

  • Network Analytics – Analyze connectivity state changes, number of Flows, Hosts and Throughput.

    It is important to remember the Cross Connect site connectivity is based on the BGP peers. If both BGP peers are unreachable, the site is considered Disconnected.

  • Events - Site events feed.

  • Application Analytics - This dashboard dissects hosts throughput and application usage. It is possible to add filters such as IP/host, Application, Category, etc...

    AppAnalytics.png

  • Priority Analyzer - This dashboard allows analyzing QoS distribution over time. (Read more on Priority Analyzer)

  • Known Hosts – A real-time dashboard for hosts behind the site. IP, OS type and Host Activity are among the data points available per host.

    knownHosts.png

  • Real Time - This dashboard allows for real-time monitoring of active hosts, throughput, top applications, active QoS and more.

    RealTime.png

Cross Connect Limitations with AWS

The following is a highlight list of limitations to consider before setting up an AWS Cross Connect site:

  • AWS assigns AWS router IP, Cato peer IP and MD5 key automatically. (You may opt to customize these values manually)

  • Direct Connect interfaces may receive up to 100 route advertisements over BGP. (Cato has an option for custom route summarization. If you would like to configure route summarization with BGP, please contact Cato Support)

    You may read more on AWS’s official Direct Connect documentation.

Was this article helpful?

0 comments

Add your comment