Speaker 1: Welcome back to the deep dive. Today, we are wrestling with, uh,
Speaker 1: probably the defining architectural challenge of the last few years,
Speaker 1: maybe the.
Speaker 2: Decade definitely feels like it.
Speaker 1: How do you reliably route traffic when the ground beneath
Speaker 1: your feet is constantly shifting. We're talking about the shift
Speaker 1: from stable, predictable monoliths.
Speaker 2: Right, the quarterly release cycle kind of.
Speaker 1: Thing exactly, to this well sometimes chaotic world of micro
Speaker 1: services that scale up and down constantly.
Speaker 2: Yeah, it's like comparing I don't know, a printed map
Speaker 2: from the nineties to Google Maps in a city where
Speaker 2: roads just appear and disappear every few minutes and buildings
Speaker 2: resize themselves.
Speaker 1: That's a great analogy. And those older load balancers, the
Speaker 1: ones built for the static map, they just can't cope.
Speaker 2: They're stuck in that static config mindset. They pretty much
Speaker 2: melt down when faced with how dynamic a modern cloud
Speaker 2: environment really.
Speaker 1: Is and that operational headache. That's why we're diving deep
Speaker 1: into Treevik today. It's an open source API gateway and
Speaker 1: it's build specific to handle that dynamic complexity.
Speaker 2: Right. The idea is to simplify deploy micro services, especially
Speaker 2: if you're in the kuber eddies world, which, let's face it, many.
Speaker 1: Are so our mission today.
Speaker 2: Our mission is to unpack how trific acts as this
Speaker 2: crucial link. Think of it as the intelligent gateway tier.
Speaker 2: It connects that volatile ecosystem of services to the outside world.
Speaker 1: And by the end you listening should have a pretty
Speaker 1: good handle on the cutting edge of network routing.
Speaker 2: A shortcut maybe, yeah, a shortcut to being well informed
Speaker 2: about this stuff. Resilience patterns too.
Speaker 1: Okay, let's start with the big picture, the monolith problem.
Speaker 1: We probably all remember it, right, tight coupling.
Speaker 2: Slow releases, Oh the.
Speaker 1: Pain, and that really expensive all or nothing scaling. Need
Speaker 1: more horsepower for login scale.
Speaker 2: The whole thing, huge waste of resources. That worked. Okay,
Speaker 2: I guess with the classic three tier model presentation, application data, simple.
Speaker 1: Enough, but micro services break that model completely.
Speaker 2: When you shatter that happened to I don't know, dozens
Speaker 2: hundreds of tiny services. You need a different architecture.
Speaker 1: You have to evolve the four tier model.
Speaker 2: Exactly, build for distributed systems.
Speaker 1: And that fourth tier is where Trafiic lives, right, that's.
Speaker 2: His home turf. Right, So the four tiers you really
Speaker 2: need are first, content delivery, the UI, the client stuff.
Speaker 1: Okay.
Speaker 2: Second, the gateway tier, that's STRAFIK, discovery, routing, correlating requests,
Speaker 2: aggregating responses sometimes all that happens here the traffic hap
Speaker 2: sort of. Yeah. Then third is the services tier, your
Speaker 2: actual decoupled business logic units, high cohesion, loose coupling, all
Speaker 2: that good stuff. And finally, the data tier databases, message queues,
Speaker 2: but now ideally exclusive to the services that own that data.
Speaker 1: Okay, So the gateway tier is critical, it's the front door.
Speaker 1: What does a modern gateway like trafic absolutely have to
Speaker 1: do to handle that chaos in tier three?
Speaker 2: Right, It's got to be more than just a simple
Speaker 2: port forwarder. Layer seven routing is non negotiable.
Speaker 1: Where seven meaning application layer.
Speaker 2: Exactly routing based on HTDP headers, host names, paths, maybe
Speaker 2: even stuff in the request body, not just layer four
Speaker 2: like TCP or UDP ports. And it needs to speak
Speaker 2: different languages essentially HDTP one, HGDP two, gRPC rest it
Speaker 2: shouldn't care.
Speaker 1: And security that feels like a huge piece, especially with
Speaker 1: all those services chattering away behind the gateway.
Speaker 2: Oh massive, Absolutely, the gateway must handle TLS termination, you know,
Speaker 2: decrypting the incoming.
Speaker 1: Public traffic standard stuff, right, But.
Speaker 2: Then inside the cluster for service to service chat you
Speaker 2: need mutual tls MTLs.
Speaker 1: So both sides prove who they are precisely.
Speaker 2: It's not just the client showing ID. The server demands
Speaker 2: ID back, show me your papers too. It's essential for
Speaker 2: locking things down inside your perimeter if something goes wrong,
Speaker 2: limits the blast radius.
Speaker 1: Okay, that makes sense, which leads us right to maybe
Speaker 1: the killer feature, autoconfiguration. Because, like you said, hundreds of services,
Speaker 1: maybe thousands of instances, updating config files byhand impossible right there, It.
Speaker 2: Just doesn't scale. That's where Treyfik fundamentally solves the service
Speaker 2: discovery problem. Instead of a human editing.
Speaker 1: A file, which is always error prone.
Speaker 2: Always, instead, Trephi talks directly to a service registry. Think Console,
Speaker 2: etca a Kubernetes itself. These things are like near real
Speaker 2: time databases of where every active service instance lives on
Speaker 2: the network.
Speaker 1: Ah, so Triffic doesn't need its own map. It just
Speaker 1: asks the map maker constantly.
Speaker 2: Exactly perfect analogy. Treyfik calls these map makers providers. It
Speaker 2: has first class support baked in, just sits there and
Speaker 2: watches the provider. A new service instance spins up. Treyfix
Speaker 2: C is it yep, an old one dies, treyfix E
Speaker 2: is that too, and it automatically reconfigures its own routing
Speaker 2: tables crucially without needing a restart or dropping existing connections.
Speaker 1: Hot reloads zero downtime. That's the dream.
Speaker 2: That's critical. Dynamic configuration and hot reloads are absolutely key.
Speaker 1: How tricky is it if you're running say, Docker and
Speaker 1: Kubernetes and maybe console, can one Trafiic instance watch all
Speaker 1: of them?
Speaker 2: Yeah? Surprisingly easily. That's the beauty of the provider concept.
Speaker 2: Trayfik kind of abstracts away the specific details of talking
Speaker 2: to Kubernetes versus talking to Console, so you can centralize
Speaker 2: routing even in a mixed environment.
Speaker 1: So developers just deploy to whatever platform.
Speaker 2: They use, and Treyfik figures out how to find it
Speaker 2: and send traffic there. Developers focus on code. Treefiic handles
Speaker 2: the routing complexity.
Speaker 1: Okay, so Treyfik knows where everything is. Now let's talk
Speaker 1: about actually sending the traffic efficiently. We all know basic
Speaker 1: round robin, right, just deal them out equally fine for
Speaker 1: stateless stuff.
Speaker 2: Yeah, simple, effective, if all your servers are identical, but.
Speaker 1: They rarely are so weighted Round Robin WRR. How does
Speaker 1: that work?
Speaker 2: Right? WRR is about being smarter with resources. Maybe you
Speaker 2: have an older, cheaper server with less CPU. You don't
Speaker 2: want to getting the same traffic as your brand new
Speaker 2: beat cloud instance makes sense, So WRR lets you assign weights.
Speaker 2: You could say, send three requests to the powerful guests
Speaker 2: B one group for every one request you send to
Speaker 2: the older guest D two, a three point one ratio
Speaker 2: for example.
Speaker 1: So it's not just load balancing, it's cost optimization too.
Speaker 2: Definitely in the cloud especially, WRR helps you squeeze maximum
Speaker 2: value out of cheaper or older instances alongside the new ones.
Speaker 2: Keeps everything utilized efficiently, saves money, no resource just sitting
Speaker 2: idle or getting totally slammed.
Speaker 1: Okay, let's flip that. What about apps where the user's
Speaker 1: state matters, like a shopping cart stored in memory on
Speaker 1: one specific server instance, Round Robin would break that.
Speaker 2: Yeah, that needs sticky sessions. If a user's second request
Speaker 2: hits a different server, poof their cart is gone or
Speaker 2: they get logged out. Bad experience.
Speaker 1: So how does trophy candle that it uses cookies.
Speaker 2: Typically when the first request hits a back end instance,
Speaker 2: treefix sets a cookie in the response for subsequent requests
Speaker 2: from that same user, trific reads the cookie and make
Speaker 2: sure to send the request back to that same original instance.
Speaker 2: Keeps a session alive.
Speaker 1: Okay, sticky sessions makes sense, But underlying all this balancing,
Speaker 1: you need health right. Making sure you're not sending traffic
Speaker 1: to a dead.
Speaker 2: Server absolutely fundamental. You only want to route traffic to
Speaker 2: instances that are actually healthy, usually meaning they return a
Speaker 2: two XX or a three X HTTP status code. Anything
Speaker 2: else is an error.
Speaker 1: Doesn't constantly poking every instance ad overhead though a performance tax.
Speaker 2: That's a fair question. It's a trade off. Trefik does
Speaker 2: use active checks where it sends a probe and passive
Speaker 2: checks watching responses. But you can figure the interval. You
Speaker 2: tune it so you find a balance, right, you said it, so.
Speaker 2: The monitoring overhead isn't painful, but it's frequent enough to
Speaker 2: pull an unhealthy instance out of the pool quickly when
Speaker 2: it does fail. It's crucial for the stability of things
Speaker 2: like round robin.
Speaker 1: Let's shift gears a bit to more advanced resilience patterns.
Speaker 1: Traffic mirroring sometimes called shadowing, sounds useful for testing.
Speaker 2: Oh, it's fantastic for canary deployments, really safe testing. The
Speaker 2: idea is you take your live production traffic, the real stuff,
Speaker 2: the real stuff, and you copy a small percentage of it,
Speaker 2: say ten percent, and send that copy asynchronously to a
Speaker 2: new test environment, maybe your guess V two.
Speaker 1: Version, asynchronously, so the original user isn't waiting exactly.
Speaker 2: And critically trafiic ignores the response from that mirror request.
Speaker 2: It just fires it off and forgets about it unless
Speaker 2: you see how your new code behaves under real load stability,
Speaker 2: resource use without any risk to the actual user experience.
Speaker 1: That's clever. Okay, so we've handled load and safe testing.
Speaker 1: But what about when things actually fail, not just one instance,
Speaker 1: but maybe a whole downstream database or API becomes slow
Speaker 1: or unresponsive in a micro services world. That seems like
Speaker 1: it could cause chaos.
Speaker 2: It absolutely can. That's the dreaded cascading failure scenario. One
Speaker 2: slow dependency makes its callers wait, they.
Speaker 1: Run out of threads or connections.
Speaker 2: Exactly, and then they fail, taking down the services to
Speaker 2: call them. It ripples outwards.
Speaker 1: So how does trific act as a ble kid prevent
Speaker 1: that ripple.
Speaker 2: That's the job of the circuit breaker pattern. Trafic middleware
Speaker 2: can implement this. It watches for failures going to a
Speaker 2: particular back end service.
Speaker 1: Failure is meaning errors or timeouts.
Speaker 2: Both typically yeah, if the failure rate or maybe latency
Speaker 2: crosses a threshold you.
Speaker 1: Define like too many errors in the last minute.
Speaker 2: Right, or responses are taking too long. If that happens,
Speaker 2: Trefix trips the breaker. It stops sending requests to that
Speaker 2: struggling service altogether for.
Speaker 1: A period and just returns an error immediately.
Speaker 2: Yep, usually a five zero three service unavailable. It does
Speaker 2: this instantly without even trying the failing service. This protects
Speaker 2: the calling services from getting bogged down and saves resources
Speaker 2: across the system. It's like the system saying nope, that
Speaker 2: are closed for now, try again later.
Speaker 1: And the conditions for tripping. It can be quite sophisticated.
Speaker 2: I saw yeah. Treefix implementation is pretty powerful. It's not
Speaker 2: just simple failure counts. You could use expressions like trip
Speaker 2: if latency at quantil ms fifty point zero hundred meaning
Speaker 2: the meeting response time is over one hundred.
Speaker 1: Milliseconds, or based on error ratio exactly.
Speaker 2: Response cut a ratio five hundred, six hundred point twenty
Speaker 2: five trip if more than twenty five percent of recent
Speaker 2: responses were five xx errors gives you fine grain control.
Speaker 1: Okay, circuit breakers handle the big failures. What about those
Speaker 1: little annoying transient glitches like a brief network kickup that
Speaker 1: just needs a quick retry.
Speaker 2: Perfect use case for retries middleware, Just like getting refresh
Speaker 2: in your browser when it page times out right, TRIFIC
Speaker 2: could be configured to automatically retry a request, maybe once
Speaker 2: or twice if it fails with specific errors like a
Speaker 2: connection timeout or maybe a five h two bad gateway.
Speaker 2: It provides a basic level of self healing for those
Speaker 2: intermitt network blips.
Speaker 1: Makes sense. So we've got routing balancing resilience. But when
Speaker 1: things do go wrong despite all this, we need to
Speaker 1: figure out why. Let's talk observability.
Speaker 2: Crucial observability isn't just knowing that something is wrong, but
Speaker 2: having the data to understand why. And TRIFIC, sitting at
Speaker 2: the entry point, is perfectly placed to collect that data.
Speaker 1: Across the three pillars right, logs, traces, metrics exactly.
Speaker 2: Let's start with logs.
Speaker 1: Now people off and say application logs alone aren't enough
Speaker 1: in micro services. What makes trifix logs actually useful here?
Speaker 2: Well, it generates standard error logs, of course, but the
Speaker 2: real value is often in the access logs. The trick
Speaker 2: is logging everything for every request can be really resource intensive.
Speaker 1: Yeah, generates huge amounts of data.
Speaker 2: So trific lets you filter them intelligently. You might say,
Speaker 2: only lawged requests that resulted in a redirect status codes
Speaker 2: three hundred to three h two, or only log requests
Speaker 2: that took longer than say, five seconds to complete using
Speaker 2: a mind duration filter.
Speaker 1: Ah, so you capture the interesting or problematic events without
Speaker 1: drowning and routine data.
Speaker 2: Precisely optimizes performance, gets you the diagnostic data you actually need.
Speaker 1: Okay, logs tell us what happened at the edge, But
Speaker 1: to follow a request through multiple services, we need tracing.
Speaker 2: Right request tracing stitches the whole journey together. Each piece
Speaker 2: of work done by a service is a span. All
Speaker 2: the spans for one user request combine into a single trace, like.
Speaker 1: A timeline of the request's life.
Speaker 2: Exactly, and Trafik being the first point of contact, can
Speaker 2: generate standardized trace headers, often B three propagation headers, things
Speaker 2: like XB three trace seed. Think of them like a digital.
Speaker 1: Passport, and it passes that passport along.
Speaker 2: It injects those headers into the request before forwarding it
Speaker 2: to the first back end service. That service, if it's
Speaker 2: trace aware, adds its own span and passes the headers on.
Speaker 2: So even if the request hits five different micro services,
Speaker 2: you can.
Speaker 1: See the whole chain in a system like Zipkin or
Speaker 1: Jaeger exactly.
Speaker 2: End to end visibility invaluable for debugging distributed systems.
Speaker 1: And the third pillar metrics the numbers yep.
Speaker 2: Treyfix exposes key application level metrics, things like total request counts,
Speaker 2: request latencies, average quantiles, error rates, information about the.
Speaker 1: Back end servers, and you feed that into.
Speaker 2: Standard monitoring systems, typically Prometheus. Prometheus scrapes these metrics from
Speaker 2: Treyfi periodically. Then you can use tools like Rafona to
Speaker 2: visualize trends, plan capacity, and set up automated alerts if say,
Speaker 2: aer rates spike or latency degrades.
Speaker 1: Got it? Okay, let's bring this home to the place
Speaker 1: where treefix seems most popular. Kubernetes. You mentioned earlier that
Speaker 1: the original Kubernetes ingress API wasn't great.
Speaker 2: Yeah, it was. Let's say a bit under specified vague,
Speaker 2: which forced vendors like treyfick in Jinks and others to
Speaker 2: rely heavily on custom.
Speaker 1: Annotations, annotations being those kind of messy tech strings. In
Speaker 1: the Yamo.
Speaker 2: Exactly, you'd have dozens of vendor specific annotations to configure
Speaker 2: basic things like timeouts or retries or sticky sessions. It
Speaker 2: wasn't clean, wasn't standardized.
Speaker 1: So how did trefik improve on that? They gave up
Speaker 1: on Ingress in treyfiic v two.
Speaker 2: They shifted strategy. They embraced Kubernetes's custom resource definitions or crds.
Speaker 2: They introduced their own resources like ingress, root middleware TLS.
Speaker 1: Option, So instead of annotations, you define routing rules using
Speaker 1: these custom but still native feeling Kubernetes's objects.
Speaker 2: Precisely, it's a much nicer experience. As they say, configuration
Speaker 2: becomes structured, version controllable Kubernetes YAML, just like your deployment services.
Speaker 2: Any Kubernetes engineer can understand it. It follows familiar patterns,
Speaker 2: no more digging through annotation documentation for different vendors.
Speaker 1: That sounds like a huge improvement. And you also touched
Speaker 1: on TLS simplification getting certificates is often a real pain.
Speaker 2: Oh historically it was awful manual requests, validation hoops, remembering
Speaker 2: to new high chance of error, high risk.
Speaker 1: So how does trifick fix that.
Speaker 2: It integrates directly with the ACME protocol, which is the
Speaker 2: standard let's encrypt uses for automating certificate issuance for public domains.
Speaker 1: Let's encrypt the free certificate authority right.
Speaker 2: When in trifick you basically just configure a cert resolver
Speaker 2: pointing to let's encrypt. Than when you define an ingress
Speaker 2: route for a public host.
Speaker 1: Name, trifiic just handles it.
Speaker 2: It handles the entire life cycle automatically. It requests the certificate,
Speaker 2: handles the domain validation challenge, often using something called the
Speaker 2: TLS ALPN zero one challenge. It's quite neat, retrieves the certificate,
Speaker 2: installs it, and even handles renew before it expires.
Speaker 1: Wow. So the developer just defines the route asks for
Speaker 1: TLS and trefick and let's encrypt do the rest.
Speaker 2: Pretty much focus on the application logic. The complicated, error
Speaker 2: prone task of certificate management just happens.
Speaker 1: So wrapping it up, trefix core value seems to be
Speaker 1: replacing that old, rigid manual configuration world.
Speaker 2: Which just breaks under micro service dynamism.
Speaker 1: With the dynamic self configuring system built for that reality.
Speaker 1: It's the traffic cop that learns the roads automatically as
Speaker 1: they get built or torn down well put.
Speaker 2: And there's a final thought, maybe a provocative one, tied
Speaker 2: to that certificate automation. We just discussed why traditionally certificate
Speaker 2: management was so painful and manual. People did it infrequently,
Speaker 2: maybe once a year. This meant certificates were valued for
Speaker 2: a long time. If one got compromise somehow, an attacker
Speaker 2: had a year long window.
Speaker 1: Right. Long lived credentials are risky.
Speaker 2: Very yeah. But because Trefix integration with let's encrypt automates
Speaker 2: the renewal process, certificates typically only live for ninety days now,
Speaker 2: and the renewals automatic, often no human touch needed.
Speaker 1: So it drastically shrinks the window of opportunity for an
Speaker 1: attacker using a compromise certificate.
Speaker 2: Exactly here removes a tedious, error prone operational task and
Speaker 2: significantly improves your security posture. By enforcing short certificate lifetimes.
Speaker 2: That whole category of operational security risk just kind of
Speaker 2: melts away thanks to automation.
Speaker 1: That's a really powerful side effect of adopting modern tooling.
Speaker 1: A fantastic insight to end on, Thank you for taking
Speaker 1: us through this deep dive into trafit.
Speaker 2: My pleasure is fascinating technology.
Speaker 1: Then thank you our listeners for joining us. We'll catch
Speaker 1: you on the next deep dive.