Different applications have different needs in terms of delay, jitter and packet loss so it makes sense that routing decisions should be linked to each application. However, legacy routing protocols are not able to provide the routing calculation(s) considering the metrics at such level. The new Software-Defined WAN (SD-WAN) solutions have arisen on the market to more intelligently pass traffic through the WAN. One of them is a Cisco Intelligent WAN (IWAN), which has PfRv3 and DMVPN as its core. In this article, I’m going to describe the design considerations related to these technologies, as valid in October 2016.
It is worth stating that the PfRv3 is not just an updated version of previous releases - originally named OER - Optimized Edge Routing, but PfRv3 is in fact a new code written from scratch, whose biggest improvement as compared to its predecessors is that it is simpler to configure.
IWAN DMVPN-related design considerations
PfRv3 uses the GRE header to insert its own information to the packet and therefore requires the DMVPN to be used as an overlay. As result, when designing the IWAN solution for any customer, we need to consider the aspects of DMVPN.
- Each physical WAN circuit at the branch sites should be a member of a separate DMVPN cloud, which is known as Path in IWAN. But the DMVPN overlays split the common underlay network into different independent islands. Let’s imagine that each branch has two redundant circuits on the same MPLS Layer 3 VPN provider. What happens if the primary circuit at the HUB fails? With conventional routing and no overlays, the branches are not impacted and continue using their local primary circuit for the traffic destined to the HUB. But with IWAN, the branches will start using only the second circuit. It happens because circuits are now decoupled from a common underlay network into different DMVPN overlays.
- Customers who have branches in countries with local regulations impacting IPsec encryption need to look for other alternatives depending on the services they're using. This could include such services as MPLS without IPsec, Internet-based VPNs with an allowed encryption level for the entire network, or services which can offload the encryption function to the applications themselves, for example.
- Multicast traffic inside DMVPN may cause capacity issues on the HUB site as each multicast stream is replicated separately to each tunnel towards the spokes with listeners. If underlay transport was used for multicast routing, it is advised to continue sending multicast traffic natively bypassing the tunnels.
IWAN PfRv3-related design considerations
Now let’s review the PfRv3 part of the solution.
- With current IWAN2.1, terminating multiple DMVPN clouds (Paths) on the same border router (BR) at the HUB is not allowed. This behavior doesn’t affect the branch sites. So network designers should plan for a separate router per Path at the HUB as shown in Figure 2.
- The branches cannot have multiple BRs connected to the same DMVPN cloud. This is not the case for the HUB sites, however. Please refer to Figure 3.
- PfRv3 allows only one main HUB site. Other HUB sites are called PfRv3 Transit sites. Keep in mind that the traffic between the HUB and Transit sites or between different Transit sites is not controlled by PfRv3 and therefore follows the conventional routing.
- As PfRv3 cannot guarantee symmetrical routing, some applications may be incorrectly discovered by NBAR2 at sites with multiple routers. It is advised to design the PfRv3 policy based on the DCSP marking rather than the application name.
- When designing HUB Master Controller (MC) resiliency, keep in mind that the failover is stateless and therefore all sites will reestablish the connections to the backup HUB MC and will re-learn all PfRv3-related info. There is no dynamic synchronization between Primary and Backup HUB MCs so their configurations should be kept the same manually.
- The PfRv3 connections between BRs and their MCs cannot be formed via the DMVPN tunnels used by IWAN, so when designing the HUB MC resiliency ensure that HUB BRs can reach the redundant MC bypassing IWAN tunnels.
- The branch MC has to be Layer 3 adjacent with its BRs. In other words, branch MCs multiple hops away from the BRs are not allowed as shown on the Figure 4. That is not the case for the HUB sites, however.
These are the most important design considerations for PfRv3 and are enough for you to start designing IWAN solutions. IWAN deployments require additional considerations for the routing design which I will cover in the next article.
About the Author
Dmytro (Dima) Muzychko is a principal design engineer at Verizon and mainly deals with non-standard customer requirements. His experience is built around service provider networking and is strongly focused on routing and switching, virtualization and lastly the SD-WAN. Dima's professional quote is "Intelligence is needed for solving problems; wisdom for avoiding them. Network design is all about the latter."
Dima currently holds a CCDE and a CCIE in Routing & Switching and a Masters in Computer Technologies.
Here are a few additional ways for us to engage and keep the conversation going:
- Cisco Learning Network CCDE Study Group
- Connect on LinkedIn: Dmytro Muzychko
- Connect on Twitter too
- CCDE study materials for the Written and Practical exams
- Related Unleashing CCDE blogs: IWAN Part 2: Routing Rules by Dmytro Muzychko, IWAN Part 3: Microloops on IWAN networks by Dmytro Muzychko, IWAN Part 4: Load Sharing and Load Balancing on IWAN Networks with Dmytro Muzychko, Commercial Solutions for Classified (CSfC) with Joe Galimi, A Network Designer’s Thought Process Part 1 by Cary Chen, Network Function Virtualization in Enterprise by Stephen Lynn, All Smoke and Mirrors by Michael Kowal – Part 1