Speaker 1: Welcome back to the deep dive. We are really getting
Speaker 1: into the weeds today. Oh yeah, yeah, we're tackling vxland
Speaker 1: BGP EVPN fabrics. I mean, this is the stuff running
Speaker 1: modern data centers, the real backbone.
Speaker 2: It definitely is. And you know, it's a huge shift
Speaker 2: from how things used to be done totally.
Speaker 1: We're talking about moving away from those older kind of
Speaker 1: clunky three tier designs.
Speaker 2: Right exactly. This is the fast track guide basically to
Speaker 2: understanding that jump to well high speed, really scalable software
Speaker 2: defined networks.
Speaker 1: So what problem does VXLANT actually solve? Why do we
Speaker 1: need this shift?
Speaker 2: Well, the old way just hit a wall. Traditional campus designs,
Speaker 2: heavy on layer two spanning tree, they just couldn't scale,
Speaker 2: not for the kind of density and speed modern applications demand.
Speaker 1: Yeah, you think about virtualization, multi gig traffic everywhere. The
Speaker 1: old models just choked, didn't.
Speaker 2: They They really did. You need a predictable performance, active
Speaker 2: paths everywhere, not just stand by links, and that's what
Speaker 2: this fabric is built for.
Speaker 1: Okay, So let's unpack this, starting right the bottom, the
Speaker 1: physical setup, the spine and leaf architecture.
Speaker 2: Right. The keyword here is symmetry. Every leaf switch connects
Speaker 2: to every spine switch.
Speaker 1: Bull mesh between layers basically exactly.
Speaker 2: And what that do you is a super predictable traffic pattern.
Speaker 2: Any device connected to a leaf is always just two
Speaker 2: hawks away from any other device source leaf.
Speaker 1: To a spine to the destination leave always always.
Speaker 2: That predictability is golden for performance.
Speaker 1: Makes sense, Yeah, So let's define those roles a bit
Speaker 1: more clearly. The spine layer it connects the leaves, aggregates traffic.
Speaker 2: It connects the leaves, yes, but it's doing much more
Speaker 2: than just layer two aggregation. Now, in these layer three fabrics,
Speaker 2: the spines are pure routers. They are the high speed
Speaker 2: core and crucially they act as the BGP EVPN route reflectors.
Speaker 1: AH, reflecting the roots down to the.
Speaker 2: Leaves precisely, and they often handle multicasts two acting as
Speaker 2: rendezvous points or oreps for the underlay network and.
Speaker 1: The leaf layer. This is where things really change. The
Speaker 1: traditional core functions move down here.
Speaker 2: That's the fundamental shift. The leaves are where your servers,
Speaker 2: your device is connect but they're also making the main
Speaker 2: routing decisions. Now. They are the layer three cores distributed
Speaker 2: across the whole fabric.
Speaker 1: So Instead of one big core, you have lots of
Speaker 1: smaller active.
Speaker 2: Course exactly, all working together.
Speaker 1: Okay, quick question. Then, if the spines are central for
Speaker 1: commutation like those route reflectors, what happens if one fails?
Speaker 1: Doesn't that break things?
Speaker 2: That's where the design shines. You always have at least
Speaker 2: two spines, since every leaf connects to all of them.
Speaker 2: If one spine goes down, traffic just instantly moves over
Speaker 2: the links to the other active spine or spines.
Speaker 1: Okay, built in redundancy totally.
Speaker 2: Built in, and you get redundancy at the leaf level
Speaker 2: too for servers connecting to multiple leaves using things like VPC.
Speaker 2: The physical resilience is just inherent.
Speaker 1: Okay, physical structure makes sense. Now, This is where my
Speaker 1: brain starts to hurt a little. The underlay and the overlay.
Speaker 2: Ha. Yeah, let's use that roller coaster analogy. It actually
Speaker 2: works pretty well.
Speaker 1: Okay, hit me with it.
Speaker 2: So the underlay, think of it as the physical roller
Speaker 2: coaster track, the motors, the breaks. It's the foundation the
Speaker 2: physical links between spines and leaves exactly. Its only job
Speaker 2: is basic IP reachability, making sure all the key interfaces
Speaker 2: on the leaves and spines can talk to each other.
Speaker 2: It usually runs a simple routing protocol like OSPF or
Speaker 2: isis just to.
Speaker 1: Build that basic connectivity.
Speaker 2: Map, right, And because of that spine and leaf connection pattern,
Speaker 2: we get to use equal cost multipath routing ECMP. All
Speaker 2: those links between a leaf and the multiple spines, they're
Speaker 2: all active, all forwarding traffic at the same time.
Speaker 1: So it's like layer three link aggregation, super fast.
Speaker 2: Super fast, and super resilient. That's the underlay, the solid
Speaker 2: fast foundation.
Speaker 1: Okay, foundation lay. Now the overlay the roller coaster cars
Speaker 1: and the riders.
Speaker 2: That's it. This is where the vxcellent magic happens. The
Speaker 2: overlay runs on top of that physical underlay. Vxcel An
Speaker 2: takes your normal layer two frame like an Ethernet frame
Speaker 2: from a server, and wraps it up inside a layer
Speaker 2: three UDP packet. It tunnels L two over L three.
Speaker 1: Okay, and it uses BGP EVPN to manage all this exactly.
Speaker 2: EVPN is the control plane for this overlay. But the
Speaker 2: real game changer vx land brings is multi tenancy AH.
Speaker 1: Separating different customers or departments. So in the analogy, each
Speaker 1: roller coaster car is a tenant like a VRF.
Speaker 2: Perfect Each car is a VRS a separate routing world,
Speaker 2: and the riders in that car are the vlands belonging
Speaker 2: to that tenant.
Speaker 1: And riders in different cars can't just talk to each other.
Speaker 2: Nope, completely isolated by default unless you specifically build bridges
Speaker 2: can figure route leaking between them.
Speaker 1: Okay, that isolation is huge now practical point. Yeah that wrapping,
Speaker 1: that encapsulation adds overhead, right, yeah.
Speaker 2: Big time, about fifty bytes or so. So so you
Speaker 2: absolutely must enable jembo frames on all those underlay links.
Speaker 2: And to you nine thousand maybe.
Speaker 1: Higher, right, because if you don't, what happens.
Speaker 2: Fragmentation city small pings might work, but try moving a
Speaker 2: real file. Packets get chopped up performance tanks. It's like
Speaker 2: the number one deployment.
Speaker 1: Gotcha, good tip, don't forget the mtumm. Okay, So moving
Speaker 1: up to the control plane. This is really the heart
Speaker 1: of EVPN, isn't it. Shifting from data plane learning.
Speaker 2: Like flooding and praying with spanning tree.
Speaker 1: Yeah, that mess, shifting that intelligence to the control plane
Speaker 1: with BGPEVPN. What's the big win there? Operationally?
Speaker 2: Oh, operationally it's night and day. Think about troubleshooting layer
Speaker 2: two loops before it.
Speaker 1: Was awful, tell me about it.
Speaker 2: With EVPN, the local leaf sees m address, learns its
Speaker 2: IP and bang, it advertises that MSEP pair as a
Speaker 2: BGP route to.
Speaker 1: The route reflectors the spines right.
Speaker 2: The spines reflected to other leaves that need it. No
Speaker 2: more network wide flooding to learn max CS.
Speaker 1: So finding where a device is becomes a routing lookup,
Speaker 1: not a frantic MP table search across dozens.
Speaker 2: Of switches exactly. Troubleshooting L two problems becomes essentially L
Speaker 2: three troubleshooting, much much easier.
Speaker 1: Okay, so we've killed most broadcast issues, but what about
Speaker 1: necessary evils like you know, ARP broadcast, unknown unicast, multicast
Speaker 1: the bum traffic.
Speaker 2: Right, you still need some way to handle that. VX
Speaker 2: lane uses multicast in the underlay for this, Basically bum
Speaker 2: traffic for a specific VX line segment of V and
Speaker 2: I gets mapped to a specific multicast group in the underlay, and.
Speaker 1: The spines act as the rps for those multicast groups.
Speaker 2: Usually, yeah, you'd configure the spines as the rendezvous points,
Speaker 2: and you'd want redundancy there too, using something like any
Speaker 2: cast RP. So both spines can handle it actively makes sense.
Speaker 1: Let's nail down a couple of interface terms, NVE and VTT.
Speaker 1: They sound similar, they work together.
Speaker 2: The NVE Network Virtual Interface is kind of the logical
Speaker 2: engine on the leaf switch that does the actual vx
Speaker 2: land encapsulation and decapsulation.
Speaker 1: Thing, doing the wrapping and n wrapping right.
Speaker 2: And the VTT Virtual Tunnel end point is the IP
Speaker 2: address used as the source and destination for those vxland tunnels.
Speaker 1: And that VTTPIP. It's usually a special loopback interface.
Speaker 2: Always typically loop back one, and this is important. It's
Speaker 2: different from the loopback you might use for the underlay
Speaker 2: BGP peering, which is often loopback zero.
Speaker 1: Why the separation why two loopbacks stability?
Speaker 2: Mainly loopback zero is for the underlay riding protocol itself
Speaker 2: rock solid reachability loop back one. The VTT address is
Speaker 2: what the overlay tunnels use. Keeping them separate means if
Speaker 2: something weird happens with your BGP peering, it doesn't necessarily
Speaker 2: break your established vx land tunnels. It isolates the planes.
Speaker 1: H good design practice. Okay, and we mentioned the spines
Speaker 1: are rote reflekers. Why is that so vital for scaling?
Speaker 2: Imagine if they weren't, every leaf switch would need a
Speaker 2: direct BGP peering with every other leaf switch.
Speaker 1: The full mesh Nightmare.
Speaker 2: Total nightmare ten leaves forty five BGP sessions, twenty leaves,
Speaker 2: one hundred and ninety sessions with route reflectors on the spines.
Speaker 2: You add a new leaf, it just peers with the
Speaker 2: say two spines, two new sessions.
Speaker 1: That's it. That makes adding capacity way.
Speaker 2: Way easier, massively easier. It's built for scale.
Speaker 1: All right. Let's dig into that multi tenancy piece more
Speaker 1: separation and routing between tenants. You mentioned. Roade targets are ts.
Speaker 2: Yeah. Root targets are basically tags we attached to the
Speaker 2: EVPN routes. Think of them like labels. They usually look
Speaker 2: like as number, dot, I D maybe six five five
Speaker 2: five zero one point one zero zero one or something.
Speaker 1: And these tags control who gets witch routes exactly.
Speaker 2: Each VRF each tenant has specific root targets. It exports
Speaker 2: tags its routes with and imports accepts routes tagged with.
Speaker 2: It's the BGP way of controlling visibility between virtual networks.
Speaker 1: Okay, and we have two kinds of v and ies involved,
Speaker 1: L two v and I and L three V and
Speaker 1: I back to the roller custom let's do it.
Speaker 2: The L two V and I. Think of that as
Speaker 2: the identifier for a specific group of riders within one car.
Speaker 2: It maps directly to a traditional VLAN So.
Speaker 1: VLAN ten might become L two v and I one
Speaker 1: zero ten something like that.
Speaker 2: Yeah, it carries that layer two traffic across the fabric
Speaker 2: and the big plus you can have over sixteen million
Speaker 2: v and ees, way past the old four thousand VLAN.
Speaker 1: Limit, huge scale increase. Okay, So that's L two V
Speaker 1: and I for L two traffic within a tenant. What
Speaker 1: about the L three V and I.
Speaker 2: The L three V and I is different. It's not
Speaker 2: for carrying VLAN traffic directly. It's the dedicated routing interface
Speaker 2: for the entire VRF, the whole tenant car.
Speaker 1: Okay, so what's its job?
Speaker 2: Its only job is Layer three routing interviewland routing for
Speaker 2: devices within that tenant, especially when they live on different
Speaker 2: leaf switches.
Speaker 1: Ah. So if a server on leaf one in VLAN
Speaker 1: ten needs to talk to a server on Leaf five
Speaker 1: in VLAN twenty, but they're in the same tenet VRF.
Speaker 2: The traffic goes from the source server to its leaf.
Speaker 2: Leaf one gets routed using the shared gateway, encapsulated using
Speaker 2: the L three V and I tunnel, sent across the
Speaker 2: underlay to Leaf five, decapsulated, and then routed to the
Speaker 2: destination server in Vland twenty.
Speaker 1: Got it. The L three V and I is the
Speaker 1: dedicated interview land highway for that tenant across the fabric.
Speaker 2: Perfect analogy, and.
Speaker 1: This ties into that fabric any cast gateway feature right,
Speaker 1: making every leaf an active router.
Speaker 2: Absolutely Forget old HSRP or VA, where one router was
Speaker 2: active and the other just sat there waiting to fail.
Speaker 1: Over wasting half your capacity.
Speaker 2: Right with any cast gateway, all the leaf switches share
Speaker 2: the exact same virtual MAC address and the same default
Speaker 2: gateway IP address for a given v Land.
Speaker 1: So a server just sends traffic to its gateway.
Speaker 2: IP and whichever leaf receives it can immediately route it,
Speaker 2: no tromboning traffic to a specific active core switch. Every
Speaker 2: leaf is an active Layer three core for the v
Speaker 2: lands it serves. It distributes the routing load beautifully.
Speaker 1: Very cool. Okay, final big section, getting traffic in and
Speaker 1: out of this fancy fabric external connectivity. We need a
Speaker 1: border leaf for that YEP.
Speaker 2: One or more leaves need to be designated as border leafs.
Speaker 2: They're the ones physically connected to the outside world, the wan,
Speaker 2: maybe a firewall cluster, the rest of the campus network.
Speaker 1: And they need some extra configuration.
Speaker 2: Obviously for sure, because now you're bridging the BGPEVPN world
Speaker 2: with you know, potentially OSPF static routes whatever is running externally.
Speaker 1: So when that border leaf learns an external route, say
Speaker 1: from OSTF, how does it tell the rest of the
Speaker 1: fabric about it.
Speaker 2: Inside EVPN, it advertises those external prefixes as EVPN Type
Speaker 2: five routes. That's the key type for external.
Speaker 1: Reachability, okay, type five. And what is that rout Perry.
Speaker 2: It carries the external prefix like your WHAN subnet. And critically,
Speaker 2: the next hop for that route is set to the
Speaker 2: vtep ip address of the border leaf itself.
Speaker 1: Ah. So if leaf seven needs to send traffic to
Speaker 1: the one it.
Speaker 2: Sees the type five route sees, the next hop is
Speaker 2: the border leaf's VTEP encapsulates the packet using the L three,
Speaker 2: V and I tunnel pointed at the border leaf and
Speaker 2: fires it off slick.
Speaker 1: Okay, But now the tricky part integrating OSPF for something externally.
Speaker 1: Isn't there a big risk of asymmetric routing?
Speaker 2: Huge risk. This is probably the second biggest deployment headache
Speaker 2: after MTU.
Speaker 1: Traffic goes out border leaf A tries to come back
Speaker 1: in be a border leaf B, and the firewall freaks
Speaker 1: out because it didn't see the outbound.
Speaker 2: Flow exactly that scenario state mismatch, broken connections. So the
Speaker 2: goal has to be symmetric routing.
Speaker 1: How do you force that?
Speaker 2: You have to play with the routing protocol metrics or
Speaker 2: administrative distances. You need to make sure the path back
Speaker 2: into the fabric from the external network prefers the same
Speaker 2: border leaf the traffic went out on.
Speaker 1: So maybe making the EVPN route learn via IBGP more
Speaker 1: preferred than the OSPF route.
Speaker 2: That's a very common way. Default IBGP eight is two hundred,
Speaker 2: but you can lower it, say below OSPFS one ten.
Speaker 2: That usually ensures return traffic prefers the EVPN path back
Speaker 2: through the correct border leaf. You got to make sure
Speaker 2: forward and return paths match.
Speaker 1: Gotcha last detail, the default route getting zero points zero
Speaker 1: point zero zero zero advertised across the fabric, so everything
Speaker 1: can reach the Internet, presumably via the border leaf you
Speaker 1: mentioned needing two BGP commands for this.
Speaker 2: Yeah, people trip over this. Sometimes you need default information
Speaker 2: originate under the BGP VRF canfig that tells BGP, hey,
Speaker 2: I want to advertise a default route.
Speaker 1: Okay, step one.
Speaker 2: But BGP won't advertise a route unless it's actually in
Speaker 2: its BGP table. So you also need the network point
Speaker 2: zero point zero point zero or a zero zero command
Speaker 2: in that same VRF address family configuration.
Speaker 1: To actually inject the route into BGP so it can
Speaker 1: be originated exactly.
Speaker 2: You need both default information originate enables it, networks point
Speaker 2: zero point zero point zero zero zero, provides the route itself,
Speaker 2: then the border LAFE advertises it as a Type five
Speaker 2: and the whole fabric knows how to get out.
Speaker 1: Makes perfect sense. Wow, Okay, we covered a lot of ground,
Speaker 1: we really did, but the summary feels clear now. VX
Speaker 1: land BGPEDPN. It smashes the old vland limits from massive scale,
Speaker 1: gives you amazing performance with ECMP, and crucially uses that
Speaker 1: smart control plane to dish the nightmare of layer two
Speaker 1: loops and flooding.
Speaker 2: Yeah, the big picture is this blend of layer two
Speaker 2: VPN technology running over a solid layer three foundation. It's
Speaker 2: all about flexibility, speed and making network deployments, especially using templates,
Speaker 2: much faster and more reliable.
Speaker 1: And for you listening, if you want to go even deeper.
Speaker 1: The source material mentioned things like multipod and multi site,
Speaker 1: extending this fabric idea across different data centers.
Speaker 2: Right, making multiple physical sites look like one giant logical
Speaker 2: fabric multipod more connecting separate fabrics together multi.
Speaker 1: Site, which leads to our final thought.
Speaker 2: If you're stretching one logical fabric across multiple physical locations,
Speaker 2: maybe even miles apart, what's the absolute most critical routing
Speaker 2: principle you have to maintain everywhere across all sites to
Speaker 2: keep things working smoothly.
Speaker 1: It's not just about can packets get there?
Speaker 2: No, it comes back to symmetry and filtering, ensuring traffic
Speaker 2: flows follow the same path out and back globally and
Speaker 2: making sure your route filtering is consistent everywhere. That becomes
Speaker 2: absolutely paramount. Get that wrong in a multi site setup
Speaker 2: and you're in for a world of pain.
Speaker 1: Something to definitely keep in mind. Okay, that was a
Speaker 1: fantastic deep dive. Thanks for bringing this complex topic.
Speaker 2: Glad we can unpack it.
Speaker 1: And thank you for joining us. We'll Catch you next
Speaker 1: time on the Deep Dive