5 Replies Latest reply: Mar 26, 2019 2:54 AM by Vasily RSS

    ACI Shared L3Out... BD not able to associate to L3Outs

    Micheline

      Hello CLN and a happy Monday morning--I'm trying to configure ACI so that EPGs in one tenant (Onefish) can use a L3Out from another tenant (Bluefish) and endpoints in the Onefish EPG can ping the external subnet coming in on Bluefish's L3Out.  Here's where I'm running into a problem.  When I go to the BD in Onefish to associate it to the Bluefish L3Out, I'm not given the option of selecting the L3Out I want.

       

      ACI BD not finding L3Out.png

       

       

      Has anyone seen this problem?  Thanks in advance, MM

        • 1. Re: ACI Shared L3Out... BD not able to associate to L3Outs
          Vasily

          Hi Micheline,

           

          Per my understanding you don't need to attach this "shared" L3Out to both Tenants. L3Out can only be attached to the Tenant Bluefish. And then, if there is a proper global contract in place between EPGs of Tenant Onefish and external EPGs of Tenant Bluefish (the one with shared L3Out), you should have your connectivity.

           

          In other words the contract should allow EPGs of one tenant to use the L3Out of another tenant.

           

          I might be wrong, but I will validate this in the lab later this week as I wanted to try this scenario anyhow for my exam preparation.

          • 2. Re: ACI Shared L3Out... BD not able to associate to L3Outs
            Micheline

            Hello Vasily--I understand that in order for an EPG to use any L3Out--without regard to where the L3Out is located--there are a few things that need to have happen.  Like, the EPG scope has to be adjusted.  There needs to be a contract between the internal and external EPG to whitelist traffic.  And the BD that the internal EPG uses needs to be associated with the L3Out.

             

            Nevertheless, I tried what you suggested, and I configured everything but the BD.  That is, I configured the external EPG subnet with all the shared flags.  The subnet for the internal EPG in Onefish was configured for "advertise externally" and "shared between VRFs."  And I configured a contract in Bluefish, imported it to Onefish, and connected the external EPG in Bluefish to the internal EPG in Onefish.  The external router has a route to an aggregate route representing all VMs attached to the ACI fabric.

             

            [cisco@aci1-onefish-app-1-epg1-vm4 ~]$ ping 172.50.1.1 -c 5

            PING 172.50.1.1 (172.50.1.1) 56(84) bytes of data.

             

            --- 172.50.1.1 ping statistics ---

            5 packets transmitted, 0 received, 100% packet loss, time 3999ms

             

             

            If I look at the routing table for Onefish, you can see that it's not getting a route for the external subnet from Bluefish.

            apic1# fab 102 show ip route 172.50.1.0 vrf Onefish:VRF1

            ----------------------------------------------------------------

            Node 102 (aci1-leaf-102)

            ----------------------------------------------------------------

            IP Route Table for VRF "Onefish:VRF1"

            '*' denotes best ucast next-hop

            '**' denotes best mcast next-hop

            '[x/y]' denotes [preference/metric]

            '%<string>' in via output denotes VRF <string>

             

            Route not found

             

            I think you need to associate the BD with the L3Out, even if the L3Out is in a different tenant.

             

            MM

            • 3. Re: ACI Shared L3Out... BD not able to associate to L3Outs
              Vasily

              You were fast to put it on test

               

              My physical ACI pod is going to be available later this week, I couldn't book it for today, though I badly wanted to...

               

              Still not sure if the L3Out needs to be referenced under the consuming BD of Onefish, which is also indirectly confirmed here:

              https://community.cisco.com/t5/application-centric/shared-l3out-in-common/m-p/2974211/highlight/true#M2048

               

              But I tend to believe your practical exercise more than something written online.

               

              If you figure it out before I test it, let us know!

              • 4. Re: ACI Shared L3Out... BD not able to associate to L3Outs
                Micheline

                Hi Vasily--I read that document before, and it's worked before.  Even this author explains that you need to associate the BD to the L3Out.  Will keep everyone posted!  MM

                • 5. Re: ACI Shared L3Out... BD not able to associate to L3Outs
                  Vasily

                  Micheline,

                   

                  As promised, I've tested it in the lab and it appears to be working.

                   

                  Here's my setup.

                   

                  Tenant POD6-ISOLATED

                      - Doesn't have an L3Out

                      - BD has a Subnet 10.82.3.1/24 configured

                          - Subnet is marked to be "Advertised Externally"

                          - Subnet is marked to be "Shared between VRFs"

                      - BD is NOT attached to any L3Out

                   

                  Tenant POD6-OSPF-TRANSIT

                      - Has an L3Out configured

                      - BD is attached to the L3Out

                      - Subnets learned via L3Out are marked to be shared between VRFs: "Shared Route Control Subnet"

                          - Subnet 10.82.4.0/24

                          - Subnet 10.82.5.0/24

                   

                  Important:

                  - There is a "global" contract between L3Out in POD6-OSPF-TRANSIT and EPG in POD6-ISOLATED.

                  - EPG in Tenant POD6-ISOLATED is statically deployed on a port (otherwise you wouldn't be able to ping the Subnet's IP)

                   

                  After everything is configured as above, Subnets learned via L3Out appear under the Tenant POD6-ISOLATED:

                   

                  1.png

                   

                  And the other way around, the Subnet advertised by the Tenant POD6-ISOLATED appears in the routing table of external router connected to the L3Out:

                   

                  2.png

                   

                  You can ping from external router to the Tenant POD6-ISOLATED:

                   

                  3.png

                   

                  Hope I didn't miss anything. Follow the same pattern and it should work!

                   

                  Good luck!


                  / Vasily