12 Replies Latest reply: Aug 8, 2016 1:45 PM by alejo-VIRL Support RSS

    Having trouble to connect my physical lab to VIRL

    chanochm

      Hi,

      I have VIRL installed on ESXi with all the necessary NIC's.

      i wanted to create a simple simulation where i can reach a router through a FLAT network.

      i changed the IP's in the FLAT network to match my external environment (172.31.252.0/22).

      when building a simple simulation of a flat network and a router connected to it, i cant seem to ping the router or the opposite.

      im able to reach the LXC with no problem

      what is also weird, is when i ping My PC for example i see in wireshark the request reaching, but no answer is going out.

      on arp tables on both sides (my pc and the router) i see each other IP->MAC.

      that means that layer 2 working correctly.

      this is the routing table of the router:

      Gateway of last resort is not set

       

       

            172.31.0.0/16 is variably subnetted, 2 subnets, 2 masks

      C        172.31.252.0/22 is directly connected, GigabitEthernet0/1

      L        172.31.254.175/32 is directly connected, GigabitEthernet0/1

            192.168.0.0/32 is subnetted, 1 subnets

      C        192.168.0.1 is directly connected, Loopback0

       

      this is part of the arp table of my PC after doing ping from the router:

      172.31.254.101        00-0c-29-c3-0c-6e     dynamic

      172.31.254.105        00-0c-29-c7-07-b2     dynamic

      172.31.254.109        00-0c-29-c8-23-dc     dynamic

      172.31.254.169        00-0c-29-60-05-79     dynamic

      172.31.254.175        fa-16-3e-9e-2d-7d     dynamic

      172.31.254.244        00-0c-29-75-10-22     dynamic

      172.31.255.1          00-14-5e-0a-02-81     dynamic

       

      you can see that the arp request from the router (172.31.254.175) reached BackBox.

       

      you can also see the PC ip in the router arp:

       

      iosv-1#sh ip arp

      Protocol  Address          Age (min)  Hardware Addr   Type   Interface

      Internet  172.31.254.120          0   4c72.b956.7fc0  ARPA   GigabitEthernet0/1

      Internet  172.31.254.175          -   fa16.3e9e.2d7d  ARPA   GigabitEthernet0/1

      iosv-1#

       

       

      any idea what im doing wrong?

        • 1. Re: Having trouble to connect my physical lab to VIRL
          Ralph

          Make sure that your vSwitch security settings are set like shown below. Also, if you're having multiple physical network adapters connected to the vSwitch for redundancy or bundling then that was misbehaving as well. Try to get the FLAT networks on a vSwitch with a single interface only.

          Screen Shot 2016-07-06 at 10.56.57 AM.png

          • 2. Re: Having trouble to connect my physical lab to VIRL
            chanochm

            flat,flat1,snat and int was all set as Promiscuous mode during the installation.

            there is only one physical NIC on the ESX.

            • 3. Re: Having trouble to connect my physical lab to VIRL
              gmojo0210

              chanochm wrote:

               

              there is only one physical NIC on the ESX.

              Ideally you would have 4 to 6 physical Nic's on your ESXi host . If the Nic card support 802.1q you may be able to configure trunk link and tag vlans at the port group.

               

              Virl-Ntworks-1.PNGVirl-Networks.PNG

              • 4. Re: Having trouble to connect my physical lab to VIRL
                Ralph

                I've seen issues with the very setup that you describe. In particular the situation where you have multiple network adapters connected to ESXi in a port channel or bundling configuration. Maybe (ESXi) version dependent, though... apparently, it seems to work OK for you.

                 

                That's the reason why I said that it might help to pinpoint the issue by reducing the FLAT connection to a separate vSwitch with a single network adapter connected to that vSwitch. Also would not put the FLAT port group into a VLAN as you would not be able to have FLAT send dot1q tagged packets itself (e.g. use it as a trunk).

                 

                If you go 1:1 (e.g. FLAT maps to 1 port group maps to 1 network adapter) then you can have a trunk from a virtual switch in VIRL all the way to a physical switch behind ESXi.

                • 5. Re: Having trouble to connect my physical lab to VIRL
                  gmojo0210

                  Ralph wrote:

                   

                  I've seen issues with the very setup that you describe.

                   

                  It has worked flawlessly although I may try your recommendation and give FLAT a dedicated NIC.

                   

                  In particular the situation where you have multiple network adapters connected to ESXi in a port channel or bundling configuration. Maybe (ESXi) version dependent, though... apparently, it seems to work OK for you.

                  I have had nothing but good luck teaming NICs and port aggregation on both Cisco and Juniper switches.  VST is the preferred method, most practical and listed as best practice for ESXi anyway.

                    

                  Vlan-Tagging.PNG

                   

                  BTW - Nice work on this product. I've just updated and having a look at the Annotations --  OWL!!!

                  • 6. Re: Having trouble to connect my physical lab to VIRL
                    chanochm

                    we have a flat network without any VLAN's, so im not sure any VLAN tagging is the issue. also there is only one NIC without grouping.

                    i can try to create another Vswitch and assign the ports to it, but im not sure why it should affect anything if i dont have any VLAN's.

                    • 7. Re: Having trouble to connect my physical lab to VIRL
                      alejo-VIRL Support

                      To better understand what you have please correct me if the following is wrong.

                       

                      ESXi Server

                      NIC-1 ==> vSwitch1

                                          VMKernel Port ==> VIRL Server IP

                                          Port-Groups

                                          Flat; Flat1; SNAT; INT

                       

                      No VLAN IDs assigned to any of the port-groups; please verify this with show mac address vlan1

                      Since you are pinging a router and we do not know how it is configured, try to telnet to the router from you PC. Does that work? If not, please post the MAC address table from the command and the route table of the IOSv router.

                      • 8. Re: Having trouble to connect my physical lab to VIRL
                        chanochm

                        i have all this as you mentioned:

                        ESXi Server

                        NIC-1 ==> vSwitch1

                                            VMKernel Port ==> VIRL Server IP

                                            Port-Groups

                                            Flat; Flat1; SNAT; INT

                         

                        \telnet to the router is not working.

                        you can see the ARP table and the routing table of the router in my first post.

                         

                        not sure where im suppose to execute the show mac address vlan1

                        • 9. Re: Having trouble to connect my physical lab to VIRL
                          chanochm

                          i separated everything.

                          only VIRL now is on NIC one of the ESX. flat network without VLAN's. still not working.

                          • 10. Re: Having trouble to connect my physical lab to VIRL
                            alejo-VIRL Support

                            Ok. So lets back up...

                            You are setting this up on a flat network, meaning there are no VLANs defined and there is no need for a router. Only devices configured for mgmt will talk to mgmt network and likewise for Flat. So, since there is no router in place between your VIRL server and your laptop learning the MAC address is expected because each device is on the same broadcast domain. What is not happening is unicast packets making a round-trip. For this, we need to make sure that your laptop is not sending the packets the wrong way and likewise for the simulation nodes. Make sure that if a gateway is defined, it is not for the Flat network. If you laptop is Windows, make sure you have not defined a gateway for each interface.

                            On your ESXi server, in case I misunderstood, ensure that you have a single NIC defined for the vSwitch that contains the Flat port-group. If you have a physical managed switch connecting your ESXi server to your network that is where you need to look at the mac address table. This is under the assumption that you have already ensured that promiscuous mode has already been enabled as noted earlier in this thread.

                            • 11. Re: Having trouble to connect my physical lab to VIRL
                              chanochm

                              Ok.

                               

                              1) Topology:

                              1.png

                              2) this is the vSwitch:

                              esxnic.png

                              3) NIC's on ESXi:

                              nicdef.png

                              4) interfaces on router:

                              iosv-1#sh ip int b

                              Interface                  IP-Address      OK? Method Status                Protocol

                              GigabitEthernet0/0         10.255.0.10     YES NVRAM  up                    up     

                              GigabitEthernet0/1         172.31.254.177  YES NVRAM  up                    up     

                              Loopback0                  192.168.0.1     YES NVRAM  up                    up

                               

                              5) route table on router:

                                    172.31.0.0/16 is variably subnetted, 2 subnets, 2 masks

                              C        172.31.252.0/22 is directly connected, GigabitEthernet0/1

                              L        172.31.254.177/32 is directly connected, GigabitEthernet0/1

                                    192.168.0.0/32 is subnetted, 1 subnets

                              C        192.168.0.1 is directly connected, Loopback0

                              6) MAC on router interface :

                              iosv-1#sh int gi0/1

                              GigabitEthernet0/1 is up, line protocol is up

                                Hardware is iGbE, address is fa16.3e41.6c21 (bia fa16.3e41.6c21)

                                Description: to flat-1

                               

                              7) pinging my pc dosent give an answer, but here is a capture on the request arriving:

                              wireshark.png

                              8)i dont have a managed switch between then to show you mac address table.

                               

                              if you need anything else, please tell me

                              thanks!

                              • 12. Re: Having trouble to connect my physical lab to VIRL
                                alejo-VIRL Support

                                I know this is a very late reply...

                                You have configured eth0 and eth1 on the same network. So your VIRL server is on 172.31.252.0/22 and so is your Flat network. Also, not really sure where you grabbed the capture from; meaning packet being received on your laptop or packets being sent from your ESXi server?

                                My recommendation is if you do not have any layer-3 devices between your laptop and your ESXi server, then set Flat network to a separate subnet and try again.