Traffic engineering requirement though I cannot achieve. The traffic from R4 to R5 follows this path:
R4#traceroute 22.214.171.124 so lo 10
Type escape sequence to abort.
Tracing the route to 126.96.36.199
VRF info: (vrf in name/id, vrf out name/id)
1 188.8.131.52 72 msec 8 msec 24 msec
2 10.1.109.10 [MPLS: Labels 19/81 Exp 0] 92 msec 116 msec 104 msec
3 184.108.40.206 [MPLS: Label 81 Exp 0] 132 msec 104 msec 92 msec
4 220.127.116.11 132 msec 96 msec 132 msec (Router PE4)
5 18.104.22.168 80 msec 84 msec 96 msec
6 10.1.67.7 [MPLS: Labels 19/61 Exp 0] 208 msec 220 msec 212 msec
7 22.214.171.124 [MPLS: Label 61 Exp 0] 176 msec 180 msec 148 msec
8 126.96.36.199 176 msec 236 msec 236 msec
It chooses PE4 as the best exit because of the lower Router-ID in the bgp path selection process. The same thing happens on R5 to R4 path.
What I want to achieve is that PE1 should be the preferred exit for spoke to spoke traffic and PE4 should be the backup/alternate exit. The reason I want to do this is that I want to implement BGP PIC EDGE on PE1 and PE4 for the spoke prefixes. I have made this work in a full mesh topology. You will have PE1 generate the BGP updates for the spokes with higher local preference than PE4. This will make the route-reflectors and PE2 and PE3 choose PE1 as the best exit to get to the spokes. But by doing this, you will hide the alternate exit that PE4 has through its EBGP connection; it looses in BGP path selection because of lower local preference and starts to believe that the best route is through PE1, and hence it withdraws the update about the spokes that it was sending to PE1.
To allow the alternate exit to be advertised by PE4, you have to configure PE4 with bgp advertise-best-external command under the vrf C1_2SPOKES. So now PE4 will advertise its EBGP route as the best route even though it is using PE1 as the best route. Now the route-reflectors will know about the alternate exit and because both paths (through PE1 and PE2) have unique RDs, the reflectors will reflect both routes and so PE2 and PE3 will know about both exits but will prefer PE1 as the best exit.
Once this is achieved you can turn on BGP PIC EDGE on all the PEs for the respective VRFs or you can turn it on for the entire VPNv4 address-family.
Also I do not want to use MED values on R3 to influence the traffic. This should be done using Local Preference only.
So if you lab it up using the corrected configs of the separate VRFs on the spoke PEs, you will see that R4 has 2 routes to reach R5, through 188.8.131.52 (PE2) or 184.108.40.206 (PE3). R5 has 2 routes also to reach R4, through 220.127.116.11 (PE3) and 18.104.22.168 (PE2).
Now PE2 will have 2 routes also to reach R5's loopback and R4's loopback. So will PE3. PE2 routes will point to BGP next hop 10.100.11.11 (PE1) and 10.100.6.6 (PE4). When R4 tries to send traffic to R5's loopback, it must follow this path R4->PE2->P1->PE1->R3->PE1->P1->PE3->R5. It does not matter how IGP recurses to the BGP next hop in the MPLS core, but it should arrive at PE1. This is the primary route when R4 wants to send traffic to R5.
When R5 wants to send traffic to R4, R5 has 2 routes PE2 and PE3. It should choose PE3 as the best next-hop. Now PE3 will have 2 BGP routes to reach R4; 10.100.11.11 (PE1) and 10.100.6.6 (PE4). PE3 should also choose 10.100.11.11 as the best next hop. Again it does not matter how IGP recurses to the BGP next hop in the VPN core, so long as it ends up at PE1. So the primary path when R5 wants to send traffic to R4 is, R5->PE3->P1->PE1->R3->PE1->P1->PE2->R4.
Now the alternate/backup route (if the R4-PE2 link goes down and either PE1 goes down or PE1-R3 link goes down) for the case when R4 wants to send traffic to R5 should use PE4 as the best BGP next hop. So the path should be something like R4->PE3->P2->PE4->R3->PE4-P1->PE3->R5. Again IGP recursion to BGP next hop in the VPN core does not matter as long as it arrives on PE4.
Now to the part where you will need BGP advertise-best-external. When you use local preference to make PE1 the preferred exit, you will loose the visibility of the alternate path through PE4, because all routers will select PE1 as the best route based on higher local preference. So PE4 will not be able to advertise its route and it will install only one route through PE1. So in order to allow PE4 to advertise its best external route on top of the IBGP learned route, you need to configure bgp advertise-best-external on PE4.
tried the following on PE1 and PE4 in hopes to raise the local preference of the routes. Idea being the PE1 should be used as the primary exit router for the spoke-to-spoke and spoke-to-hub traffic.
On PE 1
ip prefix-list as220 permit 22.214.171.124/24
ip prefix-list as220 permit 126.96.36.199/24
ip prefix-list as220 permit 188.8.131.52/24
ip prefix-list as220 permit 184.108.40.206/24
ip prefix-list as220 permit 220.127.116.11/24
ip prefix-list as220 permit 18.104.22.168/24
route-map set-local-pref permit 10
match ip address prefix-list as220
set local-preference 200
router bgp 65000
address-family ipv4 vrf C1_2SPOKES
neighbor 22.214.171.124 route-map set-local-pref in
router bgp 65000
address-family ipv4 vrf C1_2SPOKES
So you would think that PE2 and PE3 will now see PE1 as the preferred exit and PE4 as the backup. But applying this route-map causes a control plane loop and routes appear and disappear in the bgp table. The debugs show the routes being looped around.
Don't know of a way how to make this work.