12 Replies Latest reply: Mar 30, 2015 7:57 AM by -Ypsilon- RSS

    Branch Network Design 1...What did I overlook, and what would you do differently?


      Ok. So a few months ago, I did a design on the fly for one of our sites (I had only been here for about 6 weeks). I miss having someone to bounce this stuff off of, but at this company I am currently the only Network Engineer. Its not too bad, but I miss having another persons opinion.  So the design is below, somethings could have possibly been done differently, but I only had a short-time to get this figured out and implemented. There was an oversight on my part, and I just want to see if someone here can point it out. I had to figure it out the hard way lol.


      I would like to get some others involved, so if you know the answer, that's fine, but also explain how you would implement this type of network with the equipment below.


      The routers are 2911s. The ASA 5520 w/ IDS/IPS module. Two separate ISPs, no BGP, private non-routed network between the ASA and the two routers. There is some PBR going on here to isolate the private network from the public. GLBP between the two routers for Internet Access, load-balancing weighted, 60/40. The core is a 3750 stack running OSPF between itself and both routers. MPLS network on the left router, and DMVPN network on the right.


      This has already been implemented. I could only use the equipment below.






      Design Thoughts.png

        • 1. Re: What did I overlook? Just take a look...What would you do?

          So what I'm trying to do is generate some interest in this study group by posting examples of Design requests I get from Management. This however, doesn't seem to be working. So maybe I should re-phrase the question. Given the requirements below, and the equipment listed above, how would you implement this network.


          For me, I did this fairly quickly, and although I have few network people, none of them are network engineers. So I basically bounced this around in my head while working on other sites, trouble tickets, and other day-to-day activities. I wrote out my configs for each and every device, and the implementation went fairly well.


          Bringing up and configuring the ASA and IPS module took the longest because it was my first time doing it right out of the box.



          Hardware Available (Due to budget constraints, security requirements, and getting equipment into the country):


          2 x Cisco 2911

          2 x 3750v2 (Core 48-Port)

          1 x ASA 5520 w/ AIP SSM-10

          1 x 3750X (INET Switch)




          Implement a redundant/resilient network solution for a new site in Kiev. (Along with the equipment list, this was the only information I was given.)




               Internal Network:


               Connect one WAN router to our network OSPF Area 0 through our global MPLS provider.


               Connect the 2nd router to OSPF Area 0 using DMVPN, and a 2nd global provider.


               Connect Core Switch to both Router via Point-to-Point connections, OSPF Area 136.




               Force all traffic destined for and coming from the Internet through the Firewall for inspection.


               Connect Core switch to ASA inside interface, default route pointing to ASA.


               Connect ASA outside interface and both routers to INET switch. Non-routed network for GLBP.


               ASA default route to VIP of the GLBP group of two routers. <----- THIS IS WHERE I WENT WRONG.

          • 2. Re: What did I overlook? Just take a look...What would you do?

            Although that is a simple list, and it doesn't cover vlan/ip addressing scheme, qos, wan security, and other things that I implemented and would be part of a normal design, this is enough to maybe get some interested.


            So what I overlooked is what happens when you point a firewall to a VIP for a GLBP group. It was a quick oversight on my part because I tried to fulfill a request and was given a list of equipment, pre-determined limitations and requirements.


            The access circuits had been ordered already, the equipment purchased, and I didn't have much say. I came in on the backend of this project, and overlooked a minor but important detail. The firewall only has one mac-address, and so there will be no load-balancing.


            The symptom after a week in production, it seemed the Internet traffic was flipping over from one router to the other at the same interval. It took me a little while to figure out what was going on. The ARP would timeout after four hours, and the Internet traffic would flow over the other router. One mac, no load-balancing. Lesson learned.


            I guess I could have posted this in the CCNP study group.


            I think the best thing given the limitations, would have been to not use the ASA and implement the firewall and inspection on each of the routers. Then I could have used the same setup and pointed the core to the VIP, and achieved the load-balance, but the equipment was already there.


            So now I will have all Internal traffic going over the MPLS connection and all Internet traffic going over the 2nd router, with each acting as the backup for the other. This will achieve what's needed.


            Just wanted to bring some life to this group, not really sure how though.





            • 3. Re: Branch Network Design 1...What did I overlook, and what would you do differently?

              Hello Del,


              i have same kind of network but with different devices.


              2 routers Cisco 1941

              1 cisco ASA 5505 firewall

              1 cisco SF300 Switch


              can you please tell me how i can cnfigure GLBP on both routers and can use the 5505 firewall for incoming and outgoing trafic inspection ?


              i have 2 dadicated ISP, 20 Mbps and 8 Mbps both are T1 conections.




              • 4. Re: Branch Network Design 1...What did I overlook, and what would you do differently?

                I do not believe a 1941 router supports GLBP. Probably only HSRP or maybe VRRP.

                • 5. Re: Branch Network Design 1...What did I overlook, and what would you do differently?

                  I just googled and found that it support High availability, supporting Hot Standby Router Protocol (HSRP), Virtual Router Redundancy Protocol (VRRP), and Gateway Load-Balancing Protocol (GLBP)


                  actually my both routers are connected to different ISP right now. i am confused that how i will connect both routers inside interface to switch and then switch to ASA 5505 for firewall inspection.


                  1 router have 20 Mbps link and 2nd router have 8 Mbps link so i want to utilize both links with load sharing. anf want failover at the same time.


                  can you please suggest the best cofiguration based on my requirement.

                  • 6. Re: Branch Network Design 1...What did I overlook, and what would you do differently?

                    I LOVE this idea! Too bad this thread has been resurrected only now. But better late than never. I shall respectfully steal this idea and use it to describe real-world scenarios (as I encounter them) that others can learn from. I urge others to do it too.

                    • 7. Re: Branch Network Design 1...What did I overlook, and what would you do differently?
                      Darby Weaver CCDP/CCNP x8

                      Where was there a need for GLBP?


                      Why not VRRP for interoperability or just use HSRP - you would not have had the flipping you experienced...?


                      Less you want me to put in my lab and try it out see the results first hand...


                      Otherwise it doesn't look too different than most similar deployments.  Did you want to add more features to the design?




                      Darby Weaver



                      • 8. Re: Branch Network Design 1...What did I overlook, and what would you do differently?

                        Hi Sandy,


                        GLBP did not work in this case because of the reasons I stated above. Knowing far more now than I did then, I would scrap this design, and in fact, I have already for this office.


                        I can give you the reasons if you like, but instead I will just address your question.

                        First off, you can't use GLBP like I mentioned, the reason is GLBP is as its name implies, a load-balancing protocol. It is meant to be used in a Distribution layer, or more accurately in a first-hop scenario. In this scenario, the way I set it up, the firewall routes the traffic from the internal LANs to the VIP in the network of its external interface. Which in this case was in a private subnet shared between the two routers and the firewall.

                        The firewall pointed to the VIP of the two routers for failover, then NAT was setup on both routers for Internet Access.

                        I will post later to give your more details.



                        • 9. Re: Branch Network Design 1...What did I overlook, and what would you do differently?

                          This is an old post Darby, and yes GLBP was not neccesary in this case, HSRP would work great, and no I would not use VRRP in this case because all the gear was Cisco, and then and now, and for the forseeable future, I only use Cisco gear at my branch offices of this size. Anyhow, this was one of my first experiences designing a Branch, and I actually wanted to use GLBP for load-balancing. What I didn't realize at the time was the limitation of my design which I pointed out above.


                          In addition to that, there were other limitations of this design which I quickly found out later on. This design has been scrapped, and I created a customized architecture for all my branch offices based on size and function, and now designs are far simpler and definitely more consistent when I have to open a new office.


                          However, this design will work in the right circumstances.





                          • 10. Re: Branch Network Design 1...What did I overlook, and what would you do differently?

                            So to continue....I think the following is how I set it up, but its been a while, and I know longer use this design.



                            The ASA was the default route for the Internal LAN. It was directly connected to the internal switch, and also running OSPF with the switch that was handling inter-vlan routing. The diagram that I posted does not show the entire LAN, just a portion of it. There were actually about 10 switches at this site, and a core stack of 3750s handling the interVLAN routing.


                            The complexity lies in the fact that I used PBR on the routers to force all traffic from and to the Internet to take the path through the firewall, while also allowing OSPF relationships between the core stack and the two routers for the MPLS and DMVPN WAN connections on the additional interfaces (don't know how many interfaces are on the 1941, I believe 2, so this may not work for you). The connections the routers shared with the firewall through the switch in the diagram was a non-routable vlan, that did not participate in OSPF. It was strictly alive on those three devices (Firewall, router1, and router2). In addition I used ACLs to keep any traffic from "leaking" from Internal to this quasi-external network, and vice-versa, although I doubt that would have occurred anyway.


                            This could also be done with VRFs...but I didn't know any better then.


                            In order to get the NAT to work, I did not put NAT on the Inside interface, only on the quasi-external interface of the routers connected to the switch. This allowed only the traffic from the firewall to be NAT'ed.


                            This got very complex. Especially for a novice with design such as myself. At the time I really did not know what I was doing, designing even a small network like this is complicated. Not to mention, I was trying to appease the masses by giving them everything they requested, because I was new on the job. In the end I scrapped the complexity, and learned more about what was needed and adjusted the design. Then I finally scrapped the design altogether, and changed how I dealt with Internet traffic at this type of site.


                            The key here is knowing what can be done, distinguishing that from what should be done, and also understanding what the true business needs are and the limitations that are being placed on you (budget/time/etc.).


                            These things tend to come with time, because engineering is not necesarily design, and putting the best stuff together doesn't always mean you will get the best results.


                            Let me know exactly what you need, if this doesn't make it clear.





                            • 11. Re: Branch Network Design 1...What did I overlook, and what would you do differently?

                              On another note, I couldn't recommend you use my design above as is, even if you could (I do think there are ways to make it work).


                              So ignoring my hodgepodge of stuff above, and based solely on what you posted, I would suggest you use the ASA with no NAT or in Transparent mode (a layer 2 device only), but since I am not familiar with ASAs at this point I couldn't tell you exactly how to do it. Depending on what type of WAN connections you have, and what they are used for (Internet, MPLS, VPN, etc.) your requirements may be different. However, it will be difficult, or maybe not possible to use both connections at the same time unless the routers have dot1q sub-interfaces on the Internal interface and are the default gateways for all of your VLANs.


                              However, you can still achieve failover using HSRP (or VRRP) and pointing the firewall to the VIP. Like I said, I am not that familiar with ASAs, haven't used one in a couple years actually, so I couldn't tell you exactly how to do it.


                              Old thread, but glad someone finally replied to it lol.

                              • 12. Re: Branch Network Design 1...What did I overlook, and what would you do differently?

                                It seems the double internet made things a little complex above. In a similar hardware environment I have my 3750x leading through a checkpoint to the internet and through a riverbed to my wan router, so my firewall is not connected to the wan router. The difference is that I don't get internet from the wan unless I need to, which means unless there is an outage. In this case the traffic, including internet is sent through the mpls wan and at the same time the tracking object checks the down link in order to be ready to bring the internet traffic back through the firewall.


                                By the way this is a great quote!: "putting the best stuff together doesn't always mean you will get the best results."


                                Still I am curious. You said you changed the design but I don't think you mention above what you did change later.