MPLS PART7 - TROUBLESHOOTING MPLS VPN

MPLS VPN Troubleshooting

MPLS VPN troubleshooting sometimes involves in-depth troubleshooting of the routing

protocols deployed between the PE routers and CE routers

Basic checks in the MPLS VPN case:

-          Can you ping between CE routers?

-          Is there an end-to-end LSP between ingress and egress PE routers?

-          Is CEF enabled on PE-CE interfaces?

1)    Can you ping between CE routers?

If you think the PE-CE subnets should be part of VPN routing but they are not propagated between PE routers, you have probably forgotten to configure redistribute connected in the VRF address family configuration within the BGP routing process

The proper way of using ping in the MPLS VPN environment is the extended ping using the

IP address configured on the LAN interface of the CE router as the source IP address. If this

ping works, the MPLS VPN network is probably working correctly. However, to ensure that

the user applications do not encounter unexpected problems, you should always perform the

extended ping with various packet sizes to verify that there are no hidden problems related

to IP packet fragmentation within the provider backbone. The label imposition process increases the size of an IP packet. The resulting packet might be too large for the physical media it needs to traverse, resulting in a need for packet fragmentation, which might fail for various reasons (including incorrect operation of Path MTU discovery due to misconfigured access-lists).

The Don't Fragment bit should be set (to prevent fragmentation in the MPLS VPN network to

interfere with the test).Packet sizes vary from MTU size minus 32 bytes (8 labels in label stack—a condition you should almost never encounter) to the MTU size (1500 bytes in most networks).

If the extended ping starts losing packets as you near the MTU size, you have to check the

MTU sizes in your MPLS VPN network and the giant frame support on your LAN switches,

steps that an IP prefix undertakes as it is propagated from the egress CE router to the ingress CE router

The following steps are needed to transport an IP prefix across the MPLS VPN backbone:

Step 1. The egress CE router sends the prefix to the PE router through a CE-PE routing

protocol. Alternatively, the prefix is configured statically on the PE router.

Step 2. The PE router installs the prefix into one or more VRF routing tables.

Step 3. The IP prefix is redistributed from the routing table into the MP-BGP table. In

this process, the IP prefix is prepended with the route distinguisher. Various BGP

attributes, including extended community route target (RT), are attached to the MP-BGP

route.

Step 4. The MP-BGP prefix is propagated across the MPLS VPN network.

Step 5. Best MP-BGP prefixes received by Ingress PE router are installed in VRF routing

tables configured on the PE router based on the RT attached to the MP-BGP prefix and

the RT configured for import in the VRF.

Step 6. Prefixes from the VRF routing table are redistributed into the PE-CE routing

protocol and propagated to the ingress CE router.

Based on the process of propagating IP prefixes between customer sites, the MPLS VPN

troubleshooting should follow these major steps:

Step 1. Check the CE-PE routing exchange.

Step 2. Check the route export functionality.

Step 3. Check the propagation of MPLS VPN routes.

Step 4. Check the route import functionality.

Step 5. Check the PE-CE routing exchange.

Egress CE-PE Routing Exchange

The egress CE-PE routing exchange is verified by using the show ip route vrf protocol

Verify that the route to the customer LAN is present in the VRF routing table and that it points to the expected outgoing interface and next-hop.

integration, security, and troubleshooting features essential to providing the advanced

protocol. Alternatively, the prefix is configured statically on the PE router.

Step 2. The PE router installs the prefix into one or more VRF routing tables.

Step 3. The IP prefix is redistributed from the routing table into the MP-BGP table. In

this process, the IP prefix is prepended with the route distinguisher. Various BGP

attributes, including extended community route target (RT), are attached to the MP-BGP

route.

Step 4. The MP-BGP prefix is propagated across the MPLS VPN network.

Step 5. Best MP-BGP prefixes received by Ingress PE router are installed in VRF routing

tables configured on the PE router based on the RT attached to the MP-BGP prefix and

the RT configured for import in the VRF.

Step 6. Prefixes from the VRF routing table are redistributed into the PE-CE routing

protocol and propagated to the ingress CE router.

Based on the process of propagating IP prefixes between customer sites, the MPLS VPN

troubleshooting should follow these major steps:

Step 1. Check the CE-PE routing exchange.

Step 2. Check the route export functionality.

Step 3. Check the propagation of MPLS VPN routes.

Step 4. Check the route import functionality.

Step 5. Check the PE-CE routing exchange.

Egress CE-PE Routing Exchange

The egress CE-PE routing exchange is verified by using the show ip route vrf protocol

command on the egress PE router, where protocol stands for the PE-CE routing protocol used

in the VRF under observation. (A sample printout is included in Example 9-18.) Verify that the

route to the customer LAN is present in the VRF routing table and that it points to the expected

outgoing interface and next-hop.

If you do not see the customer route in the VRF routing table, check the PE-CE routing protocol

The best command to use is the show ip protocols [vrf name] command that displays the actual settings of the routing protocols in a VRF.

Some of the most common mistakes apart from routing protocol errors

-          The PE-CE routing protocol is not configured. (The operator configured the global routing

protocol, not the VRF routing protocol.)

-          RIP version 1 is running in the VRF. RIP version 1 is not supported in MPLS VPN

environments.

-          Auto-summary is configured in the routing protocol. Customers who have migrated to

MPLS VPN environments usually have discontiguous subnets that require routing protocol

auto-summarization to be turned off.

-          OSPF routes are received from the CE router with the down bit set because another PE

router redistributed them into OSPF.

-          Customer BGP routes are ignored because they are propagated between customer sites

that use the same BGP autonomous system number.

Route Export

When you have verified that the egress PE router has received the route from the CE router

(and that you have fixed potential errors), verify that the route received from the CE router is

redistributed into BGP by using the show ip route vrf name prefix command, as shown in

Example 9-20. The most important line in the printout is the Advertised by bgp line indicating

that the customer's IP prefix is indeed advertised by MP-BGP.

NOTE

One of the most common MPLS VPN configuration mistakes is the omission of route

redistribution into BGP, without which the connectivity between PE routers cannot be

established.

Example

Egress#show ip route vrf vpna 203.1.4.0

Routing entry for 203.1.4.0 255.255.255.0

Known via "rip", distance 120, metric 1

Redistributing via rip, bgp 3

Advertised by bgp 3

Last update from 150.1.31.18 on Serial0/0.100, 00:00:20 ago

Routing Descriptor Blocks:

* 150.1.31.18, from 150.1.31.18, 00:00:20 ago, via Serial0/0.100

Route metric is 1, traffic share count is 1

After you verify that the customer route is redistributed into MP-BGP, verify the route

distinguisher and RT attached to the customer route with the show ip bgp vpnv4 vrf name

prefix command

show ip bgp vpnv4 vrf vpna 203.1.4.0

BGP routing table entry for 3:10:203.1.4.0/24, version 9

Paths: (1 available, best #1, table vpna)

Advertised to non peer-group peers:

192.168.3.3

Local

150.1.31.18 from 0.0.0.0 (192.168.3.4)

Origin incomplete, metric 1, localpref 100, weight 32768, valid, sourced,

best

Extended Community: RT:3:10

If you get unexpected route distinguisher or RT values, check the VRF configuration with the

show ip vrf detail command

Egress#show ip vrf detail

VRF vpna; default RD 3:10

Interfaces:

Serial0/0.100

Connected addresses are not in global routing table

Export VPN route-target communities

RT:3:10

Import VPN route-target communities

RT:3:10

No import route-map

Export route-map: SetRT

Another common source of unexpected RT values is the export route maps, where users tend

to forget the additive keyword in the set extcommunity command.

Propagation of MPLS VPN Routes

The routes that the egress PE router redistributes into MP-BGP need to be propagated to the

ingress PE router. Verify that the ingress PE router receives the BGP route with the show ip

bgp vpnv4 rd target prefix command

Ingress#show ip bgp vpnv4 rd 3:10 203.1.4.0

BGP routing table entry for 3:10:203.1.4.0/24, version 21

Paths: (1 available, best #1, table vpna)

Not advertised to any peer

Local

192.168.3.2 (metric 10) from 192.168.3.2 (192.168.3.2)

Origin incomplete, metric 3, localpref 100, valid, internal, best

Extended Community: RT:3:10

An MP-BGP route might not be propagated from the egress to ingress PE router for several

reasons:

A VPNv4 session between the PE routers has not been activated. Verify that the VPNv4

sessions are active with the show ip bgp neighbor command

Ingress#show ip bgp neighbor 192.168.3.2

BGP neighbor is 192.168.3.2, remote AS 3, internal link

BGP version 4, remote router ID 192.168.3.2

BGP state = Established, up for 00:41:40

Last read 00:00:40, hold time is 180, keepalive interval is 60 seconds

Neighbor capabilities:

Route refresh: advertised and received(new)

Address family IPv4 Unicast: advertised and received

Address family VPNv4 Unicast: advertised and received

Received 60 messages, 0 notifications, 0 in queue

Sent 51 messages, 0 notifications, 0 in queue

Route refresh request: received 0, sent 0

Default minimum time between advertisement runs is 5 seconds

VPNv4 sessions might be active, but the route reflector clients have not been configured

on the VPNv4 sessions. (The route reflector clients must be configured separately for each

address family.) Verify the proper route reflector client configuration on the route

reflector with the show ip bgp neighbor command.

RR#show ip bgp neighbor 192.168.3.1

BGP neighbor is 192.168.3.1, remote AS 3, internal link

BGP version 4, remote router ID 192.168.3.1

BGP state = Established, up for 00:44:11

Last read 00:00:10, hold time is 180, keepalive interval is 60 seconds

Neighbor capabilities:

Route refresh: advertised and received(new)

Address family IPv4 Unicast: advertised and received

Address family VPNv4 Unicast: advertised and received

Received 54 messages, 0 notifications, 0 in queue

Sent 63 messages, 0 notifications, 0 in queue

Route refresh request: received 0, sent 0

Default minimum time between advertisement runs is 5 seconds

[... part of the printout deleted ...]

For address family: VPNv4 Unicast

BGP table version 41, neighbor version 41

Index 1, Offset 0, Mask 0x2

Route-Reflector Client

3 accepted prefixes consume 180 bytes

Prefix advertised 24, suppressed 0, withdrawn 0

Number of NLRIs in the update sent: max 4, min 0

Connections established 1; dropped 0

Last reset never

The PE router might drop an MP-BGP route if it has no matching route target. (None of

the route targets attached to the route are configured in a VRF.) In this case, verify the

VRF settings with the show ip vrf detail command.

Route Import

The best MP-BGP route that a remote PE router receives is imported into the VRF routing table

(assuming there is a match in route targets configured in the VRF and route targets attached

to the route). Verify that the route is inserted into the VRF routing table with the show ip

route vrf name prefix command

NOTE

In complex environments that have multihomed customer sites, a PE router might

receive more than one MP-BGP route toward the same destination. In these cases,

standard BGP route selection rules determine which route is the best route

(regardless of route targets attached to the route). You can use any BGP attribute to

influence the route selection process; however, you are advised not to use the MED

attribute if you redistribute BGP routes into RIP or OSPF because the RIP/OSPF

metric of the redistributed route is taken from the MED attribute.

Ingress#show ip route vrf vpna 203.1.4.0

Routing entry for 203.1.4.0 255.255.255.0

Known via "bgp 3", distance 200, metric 3, type internal

Redistributing via rip

Advertised by rip metric transparent

Last update from 192.168.3.2 00:14:42 ago

Routing Descriptor Blocks:

* 192.168.3.2 (Default-IP-Routing-Table), from 192.168.3.2, 00:14:42 ago

Route metric is 3, traffic share count is 1

AS Hops 0

There are three main reasons that a route the ingress PE router receives is not imported into

the VRF:

·         Mismatch in route targets, which can be verified easily with the show ip vrf detail

command.

·         Misconfigured import map. The printout shown in Example 9-29 includes a sample

configuration error where the VRF refers a nonexistent route map. As you can see in the

printout, the MP-BGP route is received by the ingress PE router but not inserted into the

VRF routing table. Further investigation reveals that the route-map that the import

map command refers to does not exist.

·         Maximum VRF route limit has been exceeded.

Redistribution of MPLS VPN Routes and Ingress PE-CE Routing

Exchange

The last, but often misconfigured, part of the MPLS VPN route propagation process is the

sending of the routes received via MP-BGP from remote customer sites to the CE router.

Common configuration errors in this part include the following:

-          Routes not being redistributed from MP-BGP into the PE-CE routing protocol

-          Routes being redistributed with illegal metric

You can verify whether a route is redistributed from MP-BGP into the PE-CE routing protocol

with the show ip route vrf name prefix