Speaker 1: Hey, folks, welcome back to another episode of the Ruby
Speaker 1: Rogues podcast. This week, on our panel we have Valentino Stole.
Speaker 2: Hey.
Speaker 1: Now, I'm Charles Maxwood from top End Devs and this
Speaker 1: week we're here with Brian Valalunga. Now, you are the
Speaker 1: CEO and co founder of Doppler, and that's an app
Speaker 1: that helps manage secrets. You probably have a better pitch
Speaker 1: than that, so I'll let you explain what you do.
Speaker 3: Hey all, it's great to be here. Yeah, Doppler is
Speaker 3: a secrets manager designed to be your single source treat
Speaker 3: for secrets across all your projects, environments, team members, and infrastructure.
Speaker 3: So you kind of think about it like as get
Speaker 3: up for secrets.
Speaker 1: Right, And I think I don't know. In the Ruby
Speaker 1: community at least where I deal with people, and especially
Speaker 1: in Rails, right, we kind of understand what the secrets are.
Speaker 1: It's your password, it's maybe your you know, you have
Speaker 1: your Rails master key, which gives you access to the
Speaker 1: secrets in Rails, but you also have like your what
Speaker 1: is that it's a token for your sessions. I can't
Speaker 1: remember what it's called anyway, So you need that secret key,
Speaker 1: and yeah, people can compromise your security if you don't
Speaker 1: have that secured. Are there things besides like passwords and
Speaker 1: API keys that you consider secrets?
Speaker 3: Yeah, outside like API keys, like a stripe token, yeahse
Speaker 3: rls encryption keys are typically the ones that we see
Speaker 3: the most. I also just like have like a general
Speaker 3: thought process around this of anything that's configured by the
Speaker 3: environment should be treated but as a secret, so it
Speaker 3: should be treated like the most sensitive thing. So we
Speaker 3: kind of tell all of or we recommend our audience
Speaker 3: that you should kind of treat your environment variables and
Speaker 3: your secrets all the same. Right, your poored in future
Speaker 3: flag should be treated securely as your secrets. That way,
Speaker 3: developers don't need to make the choice of is this
Speaker 3: secure or is this not? They're always just doing the
Speaker 3: secure thing by defaults. And then the trick is just
Speaker 3: make that whole workflow and tooling as developer friendly as possible.
Speaker 3: So they wanted to go down that.
Speaker 1: Path, right, So I'm just going to kind of cover
Speaker 1: the basics of how I do this, and then maybe
Speaker 1: you can tell us if there are other ways that
Speaker 1: you're seeing people do it. But typically these kinds of
Speaker 1: things where I see them and I'm sure Valentino is
Speaker 1: in kind of the same boat. Is they either show
Speaker 1: up in environment variables or they show up in the
Speaker 1: rail secrets or and effectively, what it is is it's
Speaker 1: a it's an encrypted file and so then the encryption
Speaker 1: key is usually put into an environment variable or you know,
Speaker 1: passed into the system that you're running it on on
Speaker 1: the other end in some way. And lately I've been
Speaker 1: doing my deployments with Kamal, which uses stocker, and so
Speaker 1: it kind of sets that all up when it sets
Speaker 1: up the container. But I've always kind of wondered because
Speaker 1: you can ask the system if you can get into it,
Speaker 1: what the environment variables are, So I don't even know
Speaker 1: if that's the best practice anymore.
Speaker 3: Yeah, that's pretty much the two paths that we see.
Speaker 3: Either some encrypted file somewhere or it's just struggling available
Speaker 3: in the environment. I'm a pretty big fan of if
Speaker 3: you're going to do the environment path or injected through
Speaker 3: the environment, immediately do a cleanup of the environment before
Speaker 3: all your other dependencies get loaded in. So like basically
Speaker 3: load those secrets out of the environment, then clean them
Speaker 3: up from the environment, then load everything else. That way,
Speaker 3: you can't have like a rogue dependency, go and capture
Speaker 3: all your secrets and then send it somewhere. The other
Speaker 3: path of like using a file that's encrypted, is also
Speaker 3: a really good path too, So I think both are
Speaker 3: pretty good.
Speaker 1: What do you do in Valentino? Is it kind of
Speaker 1: the same or Uh?
Speaker 2: We use a lot of AWS stuff to handle a
Speaker 2: lot of the.
Speaker 1: Lot a lot of secrets management.
Speaker 2: We have a secret there. Yeah, and then I think
Speaker 2: that it may even be isolated from the main environments
Speaker 2: too and handled that way in an isolated environment. We
Speaker 2: use soccer as well, so it could be like an
Speaker 2: isolated container that then feeds back into the other container.
Speaker 3: So are you I'm guessing you have your secrets stored
Speaker 3: on like a secrets manager, and then those secrets are
Speaker 3: then pushed into the container. H.
Speaker 2: I think so something like that. Uh, it may even
Speaker 2: be one step further where uh it's in a w
Speaker 2: S and then at like run time, it's like the
Speaker 2: secrets are loaded into the apps memory. Uh for the
Speaker 2: particular use case. I don't really know all of the details,
Speaker 2: to be honest, I'm not on the infrastructure team. Everyone's
Speaker 2: a while. We have to rotate keys and that's definitely, uh,
Speaker 2: you know, an easy process for us. But yeah, we
Speaker 2: have some command line that does it. It's not just
Speaker 2: like changing the environment variable.
Speaker 3: Sounds like you guys are pretty well set up there.
Speaker 3: I mean that's that's like the dream is to make
Speaker 3: secret trans super easy and simple but also very secure
Speaker 3: at the same time.
Speaker 2: Yeah. That makes me, uh curious because like I think
Speaker 2: at one time we were using forget what hash rockets
Speaker 2: product is, we ended up moving away from that. Yeah,
Speaker 2: I don't know, we were using some vendor for doing
Speaker 2: the secret management, but yeah, having a command line or
Speaker 2: some kind of central place. Uh, the command line doesn't
Speaker 2: even need to be your you know, your ux of choice,
Speaker 2: but just having the central lives place where all of
Speaker 2: that access points happen. It definitely helps like isolate and
Speaker 2: track to be honest, who's touching what and changing it.
Speaker 2: I don't know if Doppler has those kind of features
Speaker 2: as far as like tracking changes and getting the full
Speaker 2: logs and things like that, but that's definitely been super valuable.
Speaker 3: Yeah, So I think Doppler kind of comes from a
Speaker 3: first principal standpoint about like what is a good secrets
Speaker 3: manager look like? And for us, we have a couple
Speaker 3: of things that we deeply care about. First is that
Speaker 3: all your secrets are centralized in one place across your
Speaker 3: projects and environments. And once they're all in one place,
Speaker 3: we can set up pretty robust access controls, either by
Speaker 3: users or by groups that are policy driven. And then
Speaker 3: from there you get a complete audit story, so you
Speaker 3: know every time a son's read a secret, written a secret,
Speaker 3: change some settings on every device. With the ri ip address,
Speaker 3: you get a whole picture about everything that's going around
Speaker 3: or going on around your secrets. And then the other
Speaker 3: key part is that it's deeply connected to their workflows
Speaker 3: and their infrastructure. So on the workflow side, and like
Speaker 3: local developments, we want you to be able to edit
Speaker 3: your secrets side by side with your code. You have
Speaker 3: a vis code extension, and so those secrets actually kind
Speaker 3: of look like a Yami file, but they're actually not.
Speaker 3: It's like a something called a virtual file system, so
Speaker 3: it behaves like a file, but it's actually not as
Speaker 3: connecting a dowtware in the back end, so it's super secure.
Speaker 3: And then on the orchestration side for production is we
Speaker 3: will write into a toa secrets manager into Kubernetes or
Speaker 3: wherever it may be, so that your infrastructure immediately gets
Speaker 3: those secrets when they change and restart your deployment seed,
Speaker 3: you don't have any downtime.
Speaker 1: Yeah, I remember using a system like this way back
Speaker 1: in the day. I think you had to log into
Speaker 1: their interface or use their command line interface in order
Speaker 1: to edit your secrets. And I can't, for the life
Speaker 1: of me remember what it was called. But I played
Speaker 1: with it and it was cool. It was just there
Speaker 1: were steps to setting it up, and I'm lazy, and
Speaker 1: so I just went back to what I was doing.
Speaker 3: I for that matter, you got to make it Like
Speaker 3: we have this theme at Doppler, and it's it's quite simple.
Speaker 3: It's make vegetables taste like candy, vegetables being the security
Speaker 3: candy be in the developer productivity. If you do not
Speaker 3: make it, if the tool does not make the developers'
Speaker 3: lives better, Like you can't just say, hey, use this
Speaker 3: tool because it's more secure, like that doesn't fly, because
Speaker 3: developers then they care about moving fast and shipping product,
Speaker 3: and so you have to aid them in that you have,
Speaker 3: Like in our case, we try to give them two
Speaker 3: hours a day or sorry, two hours a week. Back
Speaker 3: in productivity. And so if you if you do that,
Speaker 3: if you give them good candy, then you'll get the
Speaker 3: vegetables too.
Speaker 1: Yeah, that makes sense. I kind of want to dive
Speaker 1: in a little bit more on just why we protect
Speaker 1: the secrets because I think we all kind of intellectually know,
Speaker 1: but I don't know. When I get into stuff and
Speaker 1: I start, you know, dealing with the secrets, it's like, oh,
Speaker 1: somebody got access to my like my stripe tokens. I
Speaker 1: mean that could be really really bad.
Speaker 3: It's really bad. So I like to kind of bucket
Speaker 3: the risk in sorry, yeah, bucket the risk into two buckets.
Speaker 3: The first is like the risk of the company and
Speaker 3: the second is the risk to people. And I think
Speaker 3: the company side is like pretty well understood, right, You're
Speaker 3: gonna have a pretty if you have a data breach,
Speaker 3: you're gonna have a pretty big brand hits and trust hits.
Speaker 3: You're gonna probably have some unexpected spend because now you're
Speaker 3: investing in like legal pr insurance, you may get sued
Speaker 3: as well, because if you're not managing your secrets, and
Speaker 3: the secrets are things that get leaked that leads to
Speaker 3: a data breach. That's just straight up negligence, and you're
Speaker 3: the customers are right or in their rights to sue.
Speaker 3: But I think the thing that's most of the time
Speaker 3: not talked about is like the cost to real people
Speaker 3: at the end day, all these companies that we use
Speaker 3: were trusting them to store our data and keep our
Speaker 3: private data private. Imagine if like every YouTube video that
Speaker 3: you ever saw got out, or every ride you ever
Speaker 3: took on an uber, every transaction you ever made on
Speaker 3: a credit card all got out, and actually have a
Speaker 3: real personal story of like how this has impacted me. Sadly,
Speaker 3: I've been part of a couple of data breaches now
Speaker 3: from some of the services I've used, and that came
Speaker 3: back and really bit me in an interesting way. I
Speaker 3: was so I just moved to Austin, Texas a couple
Speaker 3: of months ago and from California, and I convinced my
Speaker 3: mom to fly out with me and see Austin for itself.
Speaker 3: When we were at this barbecue joint and while we're
Speaker 3: eating food, I get a call from the Texas Customs
Speaker 3: and Borders and they say, hey, we found a packaging
Speaker 3: your name that has illegal money and drugs in it,
Speaker 3: and we're federally investigating you now. And this is like
Speaker 3: the phone call for me where I was like, oh shit,
Speaker 3: my life has just ended like from the high really
Speaker 3: really quickly, and I was really scared, but like also
Speaker 3: kind of like this is odd, like never bought anything
Speaker 3: from Mexico before, so like okay, and the back of
Speaker 3: my head, I was like, this is a scam, probably,
Speaker 3: But then they started rattling off all this information about
Speaker 3: me that they knew all the places I'd lived in
Speaker 3: the past, locations I had been to, like just so
Speaker 3: like a wealth of information about me that I was like,
Speaker 3: no one could possibly know this but the government, and
Speaker 3: so like I trusted them. I was like, okay, I'm
Speaker 3: actually being investigated here. And obviously we get lawyers on
Speaker 3: the call as well, like this becomes a really scary
Speaker 3: ordeal for all of us. And it wasn't until about
Speaker 3: like an hour in that our lawyers picked up that
Speaker 3: it wasn't that it was a scam. And but during
Speaker 3: that hour we also gave them a whole lot more
Speaker 3: information about me, right because we thought we would be
Speaker 3: investigated and they were asking questions and so like this
Speaker 3: is and like again I'm a CEO of a cybersecurity company.
Speaker 3: I'm trained on this, our employees are trained on this,
Speaker 3: our legal team is trained on this, and they still
Speaker 3: got us for an hour, Right. I can imagine every
Speaker 3: day Joe who's not trained could really get it attacked here.
Speaker 3: And like, I was lucky that all they do is
Speaker 3: get a little bit more information about us. But they
Speaker 3: were able to get that information because of other information
Speaker 3: that got breached, right, that bought them the credibility to
Speaker 3: get new information. But imagine like my mom, my sister,
Speaker 3: or anyone all like that who's not gone through any
Speaker 3: of that training, does not have legal counsel immediately readily
Speaker 3: available to them that can jump on the call for this.
Speaker 3: Oh my gosh, I cannot imagine all the information that
Speaker 3: I've been given up and all of a sudden, their
Speaker 3: bank counts are drained or the credit score is completely ruined,
Speaker 3: or whatever it may be. It can get really terrifying,
Speaker 3: really fast. And that's why I come back to Like
Speaker 3: whenever I see like a data region, it's like X
Speaker 3: millions of people's data was just got out. It's like wow,
Speaker 3: Like scammers are now like parading in the streets going like,
Speaker 3: oh my gosh, I've just got a ton of data
Speaker 3: that's going to get me so much more data that
Speaker 3: can eventually get me to like the jackput of all
Speaker 3: this money or whatever they're trying to go after. And
Speaker 3: so there's real people's lives, including mine, that have been
Speaker 3: impacted by this, and that is something that's like often
Speaker 3: not talked about at all. And so when you're a
Speaker 3: developer and you're like, ah, should I put this in
Speaker 3: a new follow or not? Or should I secure my
Speaker 3: secrets just one token, it's it's just a string. It's like, well,
Speaker 3: that string may impact a million people's lives someday, so
Speaker 3: to treat.
Speaker 1: Right right, Well, it reminds me your story a little
Speaker 1: bit of if you're a ghost the Wires, is it?
Speaker 1: Kevin Mitnick, Yeah, Kevin Minnick.
Speaker 3: Yeah, I've watched the movie about him. I haven't read the.
Speaker 1: Book, and see I didn't know there was a movie,
Speaker 1: but I read the book. But yeah, he talks about
Speaker 1: how effectively, yeah, he gets just a little bit of
Speaker 1: information about the next person in the chain, right, and
Speaker 1: so he doesn't start with the analyst or whoever, right,
Speaker 1: he starts with you know, the low down person on
Speaker 1: the totem pole. And then you know, gets enough information
Speaker 1: to know who the next person is and then works
Speaker 1: his way up and so it's yeah, it was a
Speaker 1: fascinating read just from the sense of yeah, you know,
Speaker 1: just just anyway. So yeah, so I can definitely see that.
Speaker 1: I think. The other thing that I worry about is
Speaker 1: like if I have a key to I don't know,
Speaker 1: like chat, GPT or something like that, somebody could run
Speaker 1: up my bill yep, or you know, somebody gets in
Speaker 1: my hosting account and you know, all of a sudden,
Speaker 1: I've got you know, they they breach my other stuff
Speaker 1: that way, and anyway.
Speaker 3: No, I think you're totally right. I mean, there's two
Speaker 3: common things that we see all the time. One is
Speaker 3: like some source code goes public or some email gets
Speaker 3: leaked and that has a secret in itto like an
Speaker 3: AWS account, and you'll see one of two things happen immediately.
Speaker 3: The first is their bill just like skyrockets in seconds
Speaker 3: because now they're using AWS for some other bought attack
Speaker 3: or for mining crypto or whatever may. That happens very
Speaker 3: very commonly, and it's surprisingly like how fast that that
Speaker 3: like that those secrets get found, Like I think there
Speaker 3: was like an average set that it's like within three
Speaker 3: to five seconds of like a rebook going public if
Speaker 3: it has a secret, and even in the history, it
Speaker 3: will get found and start getting exploited. So it's like
Speaker 3: very like it's it's it's it's insanely quick. And then
Speaker 3: the other side of it is now that they have
Speaker 3: access to AABS, they can start scanning all your resources. Okay,
Speaker 3: what execution environments are set up, what databases are set up?
Speaker 3: Can I get access to those database? How privileged to
Speaker 3: is this credential? And you start having this ripple fact
Speaker 3: where it's like, oh my gosh. Now, especially if it
Speaker 3: as like like strong super admin permissions, it can start
Speaker 3: creating other credentials. And now you have this like canon
Speaker 3: mouse game of like how do I get out of
Speaker 3: the system? Could have locked me out of the system?
Speaker 3: Are their back doors it's placing in so even if
Speaker 3: we remove it's still actually there. Like it gets scared
Speaker 3: really quickly. Once they get into the system, it's very
Speaker 3: very hard to get them out.
Speaker 2: Yeah, this always makes me wonder like because there's two
Speaker 2: parts of this, right, Like we have the secrets and
Speaker 2: then there's like the whole encryption aspect of it, which
Speaker 2: is on separate piece of the.
Speaker 1: Discussion when you say the encryption aspect of it, is
Speaker 1: that like how you're securing your secrets or is that
Speaker 1: like encryption keys.
Speaker 2: Well, you're I mean, ultimately, what you're using a lot
Speaker 2: of the secrets for I would imagine is to encrypt
Speaker 2: something or even to Yeah, I mean to encrypt stuff.
Speaker 1: You're either hiding stuff or accessing stuff is usually what
Speaker 1: you're doing.
Speaker 3: Based on the data that we have from Doppler, we
Speaker 3: can't see customer secrets at all, but we can get
Speaker 3: an auto report of the names of the secrets they
Speaker 3: use if they give a set permission, and from the
Speaker 3: data we have, it's like ninety percent of the of
Speaker 3: the sensitive secrets that we put in Doppler and customers
Speaker 3: evan ability say this is sensitive or this is not
Speaker 3: is actually like access token, so it's not encryption tokens.
Speaker 3: So it's like if you get that stripe key, you
Speaker 3: have it all, like you have every customer's CARDIT card
Speaker 3: and bank counter. Yeah in there, there's no extra layer
Speaker 3: of protection.
Speaker 2: Yeah, that's all. I always want to One of the
Speaker 2: the biggest things that I always hope for when I'm
Speaker 2: like getting a new vendor, like access token or whatever
Speaker 2: it may be, is that they like have some kind
Speaker 2: of like you know, white list stability with like either
Speaker 2: their domain like a domain key, or some kind of
Speaker 2: identify that can be like uh, you know, securely matched
Speaker 2: when the handshake is made using their service from whatever
Speaker 2: server that you're end up using. And it's in my
Speaker 2: experience like that it's very honestly very rare, yes, that
Speaker 2: that actually happens. And so I under that makes a
Speaker 2: lot of sense to you saying that your customer is
Speaker 2: like mostly store like access tokens, because most vendors don't
Speaker 2: give you that like white listability and like handshake process,
Speaker 2: which is like very important, right, I think causes a
Speaker 2: lot of the breaches when it's not there.
Speaker 3: Agreed. I do think that the world is changing, it's
Speaker 3: just gonna be very very slow. So like you can
Speaker 3: see this in passwords today, where we're going from passwords
Speaker 3: to pass keys, so anyone familiar past caise. It's basically
Speaker 3: like you can sign into a website with like your
Speaker 3: face or touch id and you never had to put
Speaker 3: in a password. It's a password passwordless world. And the
Speaker 3: way that's working is for cryptography, you have like a
Speaker 3: fing a print that leads to a cryptographic signature that
Speaker 3: can be used authenticate yourself. And that thing that's happening
Speaker 3: in passwords is now starting to happen in secrets with
Speaker 3: identity based authentication. And so like the kind of picture
Speaker 3: that we can start to see happening very very slowly
Speaker 3: in like Doppler's job is to help move this along,
Speaker 3: help move the industry along, is to have like this
Speaker 3: bridge of connection. So like on one side, you have
Speaker 3: like your AWS GCP execution environment and that has an
Speaker 3: identity already today, and then you compare that with Doppler,
Speaker 3: and now you're authentic by the fact that you're in
Speaker 3: that easy two instance or you are now automatically authenticated
Speaker 3: to Doppler, and then eventually Doppler will have the connection
Speaker 3: of now that you're authenticated with Doppler, you're automatically authenticated
Speaker 3: to Stripe. There's no more apike needed. We are all
Speaker 3: cryptographically authorizing each other process chain or this high way
Speaker 3: of authentication. And that's kind of like the dream state
Speaker 3: that we're going to be getting to over the next
Speaker 3: couple of years. It's going to take a lot of
Speaker 3: work of working with partners because now they have to
Speaker 3: commit strike Twilio and so many other providers to enable this,
Speaker 3: and we're gonna try to make diseases possible, but it's
Speaker 3: gonna be it's gonna be an alcohol battle. But if
Speaker 3: we can do that, we're a secretless work.
Speaker 2: Is there an open standard that is following this process
Speaker 2: or no?
Speaker 3: There is a little bit of an open standards mostly
Speaker 3: between like a WUS and X like Doppler or whatever
Speaker 3: may be, but there's no open standard on or even
Speaker 3: a standard altogether on the like secrets Manager to third
Speaker 3: party service, x Stripe, Petulio. So I think our goal
Speaker 3: is we're going to probably try to create an open
Speaker 3: standard that both sides adhere to so that if you're
Speaker 3: anywhere in this chain you can directly you can just
Speaker 3: plug in very quickly. And I think we'll probably have
Speaker 3: to build SDKs for like every primary language and Kubernetes
Speaker 3: and a bunch of other infrastructure toolings who also just
Speaker 3: like work set up the box.
Speaker 1: So, I mean, I I find this all really fascinating,
Speaker 1: But I guess my question is what kinds of mistakes
Speaker 1: might I be now or you know, what kinds of
Speaker 1: things should I be considering doing now to protect these
Speaker 1: secrets because yeah, you know, what you're talking about sounds brilliant,
Speaker 1: but it doesn't exist yet. Yeah, I'm just I'm a
Speaker 1: little worried that. Okay, we're going to talk the rest
Speaker 1: of this time about futuristic stuff, and then I'm gonna
Speaker 1: be Yeah, I'm gonna miss something.
Speaker 3: All right. So I have four questions that I think
Speaker 3: every person listening should ask themselves. And it's like basically
Speaker 3: a very simple sniff test to see if I have
Speaker 3: a problem. So to highlight what the problem is, it's
Speaker 3: called secret sprawl, and that is basically, all your secrets
Speaker 3: are scattered across a bunch of different places, and because
Speaker 3: of that, you have no way of controlling and or
Speaker 3: governing it, which is like one of the most important
Speaker 3: parts about protecting your secrets is being one of govern
Speaker 3: those secrets. And so the four questions are, can you
Speaker 3: confidently answer where are all my secrets? Right? Not just
Speaker 3: the ones that you know about him and secrets manager somewhere,
Speaker 3: but all the ones on developers, laptops, all the ones
Speaker 3: in slack and email, in your code somewhere, Like you
Speaker 3: gotta have a complete picture where all your secrets are.
Speaker 3: And you got to be able to make that guarantee
Speaker 3: that you're not going to miss one by not knowing
Speaker 3: about it, And I'll talk about how to solve that
Speaker 3: in a second. The second is do you know who
Speaker 3: has access to all of those secrets? And this gets
Speaker 3: really scary, especially if you don't have like a centralized system,
Speaker 3: because like, if I have a file, I can to
Speaker 3: share it to another person and no one on the
Speaker 3: scary team is going to know about that. Can I
Speaker 3: control who has accessed those secrets, when they were accessed
Speaker 3: and by who? Right? So, do I have access controls?
Speaker 3: And do I have an autolog associated with those access events?
Speaker 3: And then, most importantly, when a data breach does happen,
Speaker 3: can I stop it very very quickly? When I say quickly,
Speaker 3: I mean in the orders of seconds and minutes, not
Speaker 3: in the order of days, in weeks or months. And
Speaker 3: so if you cannot answer those four questions confidently, like
Speaker 3: to the point where your job would be on the
Speaker 3: line for it, because it essentially is on the line
Speaker 3: for it, then you have a problem. And I can
Speaker 3: talk about it in a second what it looks like
Speaker 3: to have a solution there. I don't have to go
Speaker 3: into like Dopper's features, but I can talk about like
Speaker 3: the fundamental properties you'd want to look for in any
Speaker 3: secrets manager, Doppler or someone else.
Speaker 1: Yeah, I think that's important. It's funny because so I
Speaker 1: kind of have two levels of this, one where I'm
Speaker 1: working for a client and the other one where I'm
Speaker 1: just working on my own stuff. And on my own stuff.
Speaker 1: I'm fairly confident that I can answer these questions because
Speaker 1: I'm basically the only person accessing or using them. But
Speaker 1: then the question for me becomes, Okay, well, how do
Speaker 1: I you know, let's say I bring in another developer,
Speaker 1: how do I make sure that I know the answers
Speaker 1: to those questions? Is I bring them on board?
Speaker 3: Yep? And I think once your surface area grows of
Speaker 3: team members, infrastructure devices, it gets pretty hard to answer
Speaker 3: those questions confidently unless you have a CAD system for it.
Speaker 3: So maybe valuable to go into like what actually does
Speaker 3: a dedicated solution look like? The first thing is you
Speaker 3: can centralize all your secrets in it. So once you
Speaker 3: know where all your secrets are and you can confidently
Speaker 3: say this is the place we're doing, We're not going
Speaker 3: to start our secrets to code we're not going to
Speaker 3: send it over email or Slack or Microsoft teams or whatever.
Speaker 3: Maybe this is the one place you can get all
Speaker 3: your secrets. Now you have a complete landscape of your secrets.
Speaker 3: The next thing you should be able to do in
Speaker 3: this system is you should be able to can set
Speaker 3: up access controls and audit logs around them, so you
Speaker 3: can say you have this developed full stack. Developers have
Speaker 3: access to the development environment for these sets of projects
Speaker 3: for the teams are on, but the DevOps team has
Speaker 3: access to production, so you can start gating access and
Speaker 3: you have a full audit picture of it. And obviously
Speaker 3: that audit picture then gets pushed into your auditing tools
Speaker 3: like data Dogs, Simal, Logic, Amazon wherever. Maybe you should
Speaker 3: be able to have this infrastructure natively integrated with their
Speaker 3: developer workflows for local development. That way, developers aren't working
Speaker 3: around the system and then recreating that trying to avoid
Speaker 3: and it should be natively integrated with with your production
Speaker 3: and staging infrastructure. So that's when it is time to
Speaker 3: roll out a secret. You don't have this race condition
Speaker 3: of the code arriving before the secrets. Do the secrets
Speaker 3: have to arrive first? If the code is going to
Speaker 3: rely on those secrets, and so then this kind of
Speaker 3: gives you this whole picture. If I know that if
Speaker 3: I can see the secrets in this project in this
Speaker 3: environment on the solution like the Secrets Manager, then it's
Speaker 3: guaranteed in my production infrastructure, or it's guaranteed on my
Speaker 3: developer's laptops in a secure way. And that basically gives
Speaker 3: you the ability to remediate secrets pretty quickly. And so
Speaker 3: the last part of this puzzle is being able to
Speaker 3: rotate secrets. Rotation is the fancy way of saying, I'm
Speaker 3: going to swap out the locks on the door, and
Speaker 3: you got to do it in a way because if
Speaker 3: you just like immediately destroy the lock in this case,
Speaker 3: you'll just bring yourself down in production. So you have
Speaker 3: to do it in a way that does not create
Speaker 3: any downtime. And it's usually requiring a two key system.
Speaker 3: So like in a database, for example, you don't ever
Speaker 3: want to not be able to connect to the database.
Speaker 3: So you'd have two credentials to the database and you
Speaker 3: swap them and then you rotate one and then you
Speaker 3: swap again. And so you want a system that does
Speaker 3: all that that should not be done by humans that
Speaker 3: should be done by a system that's running on a schedule,
Speaker 3: where you have an immediate like I'm fucked button, I
Speaker 3: need to click this thing to get me out of
Speaker 3: a data breach. And then obviously it orchestrates to the
Speaker 3: rest of your infrastructure. And so those are the things
Speaker 3: that you should be looking for in a developer, for
Speaker 3: a secrets manager.
Speaker 2: So I love this. You have this great secret sprawl
Speaker 2: page on Doppler I link to and it's really great.
Speaker 2: Kind of helps you quickly assess your risk exposure based
Speaker 2: on your team size. You go through kind of like
Speaker 2: a quick calculation, Oh, we have like this number of projects,
Speaker 2: estimate number of secrets per project. Then you have a
Speaker 2: certain number of environments, right, so it's a multiplier, and
Speaker 2: then you have a multiplayer over the number of team
Speaker 2: members too, and so those quickly add up and you
Speaker 2: make a great point like you know, the malicious actor,
Speaker 2: they only need they need one right of the that
Speaker 2: that adds up to uh, And so I'm curious, like, uh,
Speaker 2: you know from the from the like the whole point
Speaker 2: kind of is like risk assessment and remediation, right, and
Speaker 2: like do you find that uh, focusing on that risk
Speaker 2: aspect is just as important as like kind of the
Speaker 2: secret storage management mechanisms.
Speaker 3: Yeah, they go in hand in hand. For sure. You
Speaker 3: need something that developers are gonna want to use, that
Speaker 3: is secure when they do use it, and you want
Speaker 3: to think about that risk like a good a good
Speaker 3: way that I kind of like talk about it when
Speaker 3: I'm talking Turnal with their employees. The game of offense
Speaker 3: and defense are very different games. The game of defense
Speaker 3: is build walls around your perimeter as high as possible,
Speaker 3: as strong as possible, no uh no cracks in the foundation, right,
Speaker 3: And then the goal of offense of a of an attacker,
Speaker 3: of a hack of a hacker is to find one
Speaker 3: crack in the foundation and hit it as hard as
Speaker 3: fricking possible. That brings brings down the entire wall. Right,
Speaker 3: they only need one week link, and if secrets having
Speaker 3: to be that week link, well then you're really giving
Speaker 3: them the keys to your digital kingdom. That's extra bad.
Speaker 3: I'd much rather be the weak link be something far
Speaker 3: outer in the perimeter, like I don't know, some authentication
Speaker 3: bug or something like that that that impacts like one
Speaker 3: or two users, not your entire users user base, And
Speaker 3: so Uh, yeah, managing risk is incredibly more near. That's
Speaker 3: that's the end goal that you're trying to accomplish is
Speaker 3: like make sure you can stay up and orchestrate like that,
Speaker 3: get get all the DevOps and platform tooling you're looking for,
Speaker 3: and then also manage the risk associated with that.
Speaker 2: So I'm curious, like if you have any tools or like,
Speaker 2: what would you do to kind of like visualize the
Speaker 2: like you know, organizational structure in the in combination, Like
Speaker 2: it's more than just like secrets management, it seems right,
Speaker 2: Like how do you how do you personally visualize like
Speaker 2: the security aspects surrounding the secrets and who accesses them
Speaker 2: and things like that to understand kind of what is
Speaker 2: happening and where the access points are.
Speaker 3: Yeah, great question, truly a great question. We've tried really
Speaker 3: hard to build a model that maps very closely to
Speaker 3: what you'd have with code so with GitHub or get lab.
Speaker 3: So we have projects just like you have repos and
Speaker 3: get it and get lab, and we have environments which
Speaker 3: are very similar to branches, and then we have user
Speaker 3: groups that mapp in with access control policies, and then
Speaker 3: we have integrations that connect to those environments and so
Speaker 3: you can have these like this bare bones higerical structure
Speaker 3: of projects and environments. From there, you can assign users
Speaker 3: to them, and you can assign infrastructure that connects with them,
Speaker 3: and it's very easy within dppler to kind of see
Speaker 3: all that connect together in an extremely visual way. We
Speaker 3: even have to show kind of how far we've taken it.
Speaker 3: When you go into a project, you actually have a
Speaker 3: conbind board style user experience that shows all your different
Speaker 3: environments from development to staging to production everything in between.
Speaker 2: So what inspired you to start Droppler? Like where where
Speaker 2: does this story start? Like I'm curious, like where your
Speaker 2: background is, and you're like, oh, we need a better
Speaker 2: you know, secrets manager.
Speaker 3: Yeah, I had nothing to do with secrets. From the started.
Speaker 3: I was working on a crypto machine learning marketplace, so
Speaker 3: I'll kind of all the buzzwords in one on the side.
Speaker 3: It was literally every buzz where you can find jam
Speaker 3: together and man, that was a hard project. I worked
Speaker 3: on it for about eight months straight and it felt
Speaker 3: like pushing a massive boulder up a hill. Every foul
Speaker 3: fower I'd slipped five six five to six feet back
Speaker 3: from exhaustion. It was just hard and At one point,
Speaker 3: I like really got to my breaking more. I was like,
Speaker 3: I don't know if I can keep working on this thing,
Speaker 3: and so I decided to take a trip to Cancun, Mexico.
Speaker 3: And the whole goal was like, don't think about this
Speaker 3: project actively, do not And so I failed at that miserably.
Speaker 3: I was thinking about it the entire time I was there,
Speaker 3: And what I realized is that this problem I was
Speaker 3: never gonna be able to get this project off the ground.
Speaker 3: But that doesn't mean that I had to stop there.
Speaker 3: I think like there's a lot of people that like,
Speaker 3: like follow like sports players, and like they know all
Speaker 3: the stats of all these players. I'm very much the
Speaker 3: same way with founders. I like, I love learning about
Speaker 3: founder stories, and I think store Butterfield from Slack is
Speaker 3: like the best at failing off boards I've seen creates
Speaker 3: a video game out of that video game. The video
Speaker 3: game fails was born out of that was Flicker Comes
Speaker 3: Back a Second Times, Like I'm going to create a
Speaker 3: video game again. Born out of that was Slack, which
Speaker 3: we all use today. And so I was like, what
Speaker 3: can I learn from from my experience with this crypto
Speaker 3: machine Leary Marketplace and managing secrets was just one of
Speaker 3: them that was like super painful in all these weird,
Speaker 3: interesting ways. It was weird from a developer productivity side
Speaker 3: because I always felt like I was copying pasting things in.
Speaker 3: But sometimes the code would get into production faster than
Speaker 3: the secret sources have an outage. I had a couple
Speaker 3: of developers at was working with on this project, and
Speaker 3: I was like really painful to sink all these secrets
Speaker 3: with them, Like I would literally slack them and email
Speaker 3: them the secrets, and I, like my heart would cringe
Speaker 3: every time i'd do that, like, oh, that didn't feel right,
Speaker 3: But that's the only thing I have at my disposal
Speaker 3: to do. And there was actually one time where we
Speaker 3: dropped the crypto part of the marketplace because it wasn't working.
Speaker 3: We just did straight up machine learning marketplace using Stripe,
Speaker 3: and I had the Stripe production key in staging, and
Speaker 3: the staging went and prod and it took us a
Speaker 3: month to figure out why we weren't doing any transactions.
Speaker 3: And it was because of that because the Stripe testing
Speaker 3: key rejects all live credit cards, it only uses test
Speaker 3: credit cards, and we didn't realize that for a long time.
Speaker 3: And so I was like, Okay, this is a real problem.
Speaker 3: And so I come back from Cancun and I go
Speaker 3: to this dinner that Stripe was hosting that had a
Speaker 3: bunch of founders and developers at it, and we were
Speaker 3: all going around the room talking about what we were
Speaker 3: thinking about, and I just asked them openly, am I
Speaker 3: a shitty developer as the world broke? And I just
Speaker 3: can't tell anymore? Are you guys having this problem too?
Speaker 3: And like sixty percent of them were like, yes, this
Speaker 3: is a very big problem for us. And one woman
Speaker 3: in particular, like literally comes running up to me right afterwards.
Speaker 3: I thought she was going to trample me. And she says,
Speaker 3: I've had three outages this week, having a solution by Sunday.
Speaker 3: And I was like no, no, no, no, no, no, Like
Speaker 3: I'm not even started yet and this is a Wednesday.
Speaker 3: There's no way that's happening. And she goes, I don't
Speaker 3: give a fuck happen by Sunday. And I was like, okay,
Speaker 3: there are some interesting if people want it, but we
Speaker 3: didn't obviously have it by Sunday. That never happened. But
Speaker 3: what we did is we started doing a lot of
Speaker 3: research and we found that individual developers are struggling because
Speaker 3: there wasn't tools made for developers. Medium sized companies are
Speaker 3: struggling building their own things, and large companies were buying
Speaker 3: hash Core Vault, which really isn't meant to solve this problem,
Speaker 3: and still struggling. So like everyone's struggling, and I was like, Okay,
Speaker 3: we can build something that's developer first instead of instead
Speaker 3: of just purely secure first. And we got a first
Speaker 3: version working in three weeks. And we got a set
Speaker 3: of customers right after that by just dragging them in
Speaker 3: Chipola and saying hey, I'll get you anything on the
Speaker 3: menu as long as I get a rant at you
Speaker 3: for about two hours, and that got us our first
Speaker 3: set of customers. And after the races, we were.
Speaker 1: Yeah, it's funny. How Yeah, you wind up scratching an itch.
Speaker 1: That's not how do I put it, it's not like
Speaker 1: the core thing you were trying to build. Yeah, anyways,
Speaker 1: but sured out to be a good business. Yeah, it's
Speaker 1: It's interesting because I've been kind of in that same
Speaker 1: space where I've been looking for a solution that does
Speaker 1: a whole bunch of things that I want for the podcast,
Speaker 1: and I've just you know, so I built little pieces
Speaker 1: of it here and there, and so I finally just
Speaker 1: started bolting it together so that I have the full solution.
Speaker 1: And yeah, it's it's interesting how many how many times
Speaker 1: you have yeah things come out of that.
Speaker 3: Yeah, I would probably index and be like, Okay, are
Speaker 3: other people that I know that also do podcast recording
Speaker 3: asking for.
Speaker 1: This pool for the people?
Speaker 3: Does it continues to get better and better the more
Speaker 3: I mount onto this like hoshpodge of a product? And
Speaker 3: is that value going up exponentially not linearly? And if
Speaker 3: the both those answers are corra are yes, and you
Speaker 3: still feel like there's nothing in the market as anything
Speaker 3: close to you have a product, the question is is
Speaker 3: that products product tenets better than whatever's in the market,
Speaker 3: Because if it isn't ten x better or ten x different, uh, people,
Speaker 3: there won't be enough of a differentiator to create a
Speaker 3: buying event.
Speaker 1: I found.
Speaker 2: Right, So I'm curious, like what is some common like
Speaker 2: preventable breaches that you've encountered, and like how can developers
Speaker 2: emploid them?
Speaker 1: Right?
Speaker 3: Yeah, there's a couple that really come to mind. I mean,
Speaker 3: let's look at he.
Speaker 1: Left, like he's got some doozies of some stories.
Speaker 3: They're all public, Okay, So I'll share one that like
Speaker 3: isn't public, but I won't say the name of the company,
Speaker 3: and then I'll share a couple that are public. So, Twitch, Toyota,
Speaker 3: and Twitter all kind of had the exact same breach happen,
Speaker 3: and they had secrets in code. The code got leaked somehow,
Speaker 3: either by them leaking them opening up the repos themselves,
Speaker 3: or by hackers being able to get access to the
Speaker 3: get up account from an employee that got compromised. So
Speaker 3: like someone got called and said, hey, I'm your boss,
Speaker 3: and give me access to get up, and then the
Speaker 3: hacker got access to get up, and then from get
Speaker 3: from get up based on the code, and the code
Speaker 3: had the secrets in it, and boom, now the hackers
Speaker 3: are really off to the races. And that's like that
Speaker 3: happened multiple times and keeps happening because there wasn't a
Speaker 3: system in place to make sure your secrets never entered code.
Speaker 3: They should never be in code.
Speaker 1: Yeah, So I just want to just jump in here
Speaker 1: because essentially what you're telling us is just because your
Speaker 1: GitHub repo is private does not mean that you can
Speaker 1: check your secrets into it and expect them to stay safe.
Speaker 3: Absolutely. I mean that is like the number one thing
Speaker 3: most attackers are going after is like get Hub because
Speaker 3: they're assuming you're going to be making this exact mistake.
Speaker 3: And it's very easy to impersonate a boss, especially in
Speaker 3: the days of AI, So these attacks are going to
Speaker 3: get more and more sophisticated and feel more and more real,
Speaker 3: like you're actually talking with your boss and all of
Speaker 3: a sudden, boom, you have access or they have access,
Speaker 3: and then I'll tell a story. I can't the name
Speaker 3: of the company. I can promise you this. You have
Speaker 3: absolutely heard of the name of the company. You probably
Speaker 3: have used their product at some point in time, and
Speaker 3: they're very big. And this is before they were a
Speaker 3: customer of ours. They had a data breach where they
Speaker 3: hired a malicious actor in the company, so there was
Speaker 3: a rogue agent that they had hired. That rogue agent
Speaker 3: stole all their credentials or all their secrets, about two
Speaker 3: thousand of them. They detected a bit late in the
Speaker 3: game obviously exited that employee. But then the security team
Speaker 3: was tasked with rotating these secrets, so again swapping out
Speaker 3: the locks, making sure that all those secrets they stole
Speaker 3: aren't valid keys the locks anymore. And it took their
Speaker 3: security team six months across three engineers to rotate all
Speaker 3: of these credentials. So six months about that. That's not
Speaker 3: just like six months of only doing this and not
Speaker 3: doing any other projects and a major distraction, but also
Speaker 3: six months that the attackers got to steal data. Well,
Speaker 3: if data moves at gigabytes per second through AWS, they
Speaker 3: have an enormous amount of time to drain all the data, right,
Speaker 3: like they had all the time in the world. Basically,
Speaker 3: like when you're dealing in data breaches, you're dealing in
Speaker 3: seconds and minutes, not months. So months is an absolute
Speaker 3: insane amount of time that the attackers had to basically
Speaker 3: use all those stolen credentials to get anything they wanted
Speaker 3: out of the system or compromise system in any way.
Speaker 3: And it was from what we heard, it was quite
Speaker 3: a bit of a battle between the hackers and like
Speaker 3: just because you revoke the key, well if they use
Speaker 3: that key to create new keys, now you're like you
Speaker 3: have this cat and mouse game, right, And so it
Speaker 3: was quite a big challenge for them. Post oppler, they
Speaker 3: have a secret radition set up a whole level of
Speaker 3: orchestration with their infrastructure, and that same thing would have
Speaker 3: taken about like thirty seconds to minute. They clicked a
Speaker 3: button and it had been done. But yeah, it was quite
Speaker 3: painful for them, and you can imagine all the users
Speaker 3: that were deeply impacted by that along the way. It's
Speaker 3: not fun. I think that the surprising thing is like
Speaker 3: that I hear a lot is oh, this won't happen
Speaker 3: to me because my company is really big and we
Speaker 3: had the best security team. Or I also hear the
Speaker 3: quite the other in versual chapters all the time of
Speaker 3: like I'm a small startup, packers won't care about my company, right,
Speaker 3: and like you hear this from both from the opposite sides,
Speaker 3: like the stars are saying one thing, that the big
Speaker 3: companies are saying the other. And the truth is hackers
Speaker 3: are just looking for any company that they can exploit,
Speaker 3: does not matter if big and small. They're looking for prices.
Speaker 3: Sometimes they hit a jackpot, million dollar payouts, sometimes they
Speaker 3: get a ten k payout right, and like are small
Speaker 3: a small attack or a big attack, But they're just
Speaker 3: looking to make an exploit and so really every and
Speaker 3: that's why the number of data breaches are just rising
Speaker 3: exponentially every single year. Like if you look at the graphics, terrifying,
Speaker 3: and it's getting and it's getting cheaper and cheaper for
Speaker 3: them to do that because the techniques are getting more
Speaker 3: and more understood, the AI is getting better and better
Speaker 3: at doing this, So it's it's a really big growing problem. So,
Speaker 3: like everyone is vulnerable. It's not just if you're startup
Speaker 3: you're insulated because of your scale, or if you're a
Speaker 3: large company you have your invulnerable because of your scale.
Speaker 3: Like Microsoft got hacked, and Video got hacked, Samsung got hacked,
Speaker 3: Like these are really really large companies that you think
Speaker 3: would have their stuff together and still they got hacked.
Speaker 3: So it is it's on all of us, every single
Speaker 3: every all of us are responsible for keeping these credentials
Speaker 3: safe so we keep our users safe.
Speaker 2: Yeah, I think you make a great point. I mean
Speaker 2: being able to RESIGND quickly is definitely like high up
Speaker 2: on that list and being able to remediate that as
Speaker 2: fast as possible because it will happen, like you kind
Speaker 2: of just like have to go with Eventually something will
Speaker 2: happen and you have to like just stop it as
Speaker 2: soon as possible, and in my experience, just having that
Speaker 2: ability to move quickly to rotate things around has been
Speaker 2: like the best defense in the long run, at least
Speaker 2: from minimizing damage.
Speaker 3: Right, agree, Like when you're in a firefight, you shouldn't
Speaker 3: be strategizing how to fight the firefight. You should just
Speaker 3: be doing the strategizing should have happened before the firefighting
Speaker 3: happ right. You just want to be an execution mode
Speaker 3: when that happens, and you want the execution mode to
Speaker 3: be as quick as possible, Like you should click a
Speaker 3: button and should be done thirty seconds later and that's it.
Speaker 2: Yeah. This it just reminds me to a story. I
Speaker 2: forget what the company was, but some some like you know,
Speaker 2: software engineer at a big company had a home plex
Speaker 2: server and the plexnerver was outdated and it gave like
Speaker 2: you know, access to their home network, and on their
Speaker 2: home network was a work laptop that they then were
Speaker 2: able to gain access to because it was on a
Speaker 2: shared network. And then from there they then hopped through
Speaker 2: remote systems and so it's like you know, the you're right,
Speaker 2: it's getting much more like evolved.
Speaker 3: Oh yeah, Like there's some interesting ones that I've heard
Speaker 3: of recently of like printers like old printers that are
Speaker 3: connected to Wi Fi getting exploited, or like everyone has
Speaker 3: all these IoT devices in their home that are connected
Speaker 3: to like the home hub system for Apple or Alexa
Speaker 3: or whatever, and those are Internet connected to so like
Speaker 3: a very small device can then basically get access to
Speaker 3: the Wi Fi network, which then gets access to every
Speaker 3: device on that network, and thus the attack starts. So
Speaker 3: there's a lot of exploits outside that that don't have
Speaker 3: to start with just your computer or just AWS.
Speaker 1: So let's say that I don't know I'm cheap and
Speaker 1: I don't want to use something like Doppler. I don't
Speaker 1: even know what you cost. Maybe it's like ridiculously inexpensive.
Speaker 1: But let's just say that, you know, I decide, for
Speaker 1: whatever reason, I want to do this by myself. What
Speaker 1: does that look like.
Speaker 3: Yeah, well, first off, Doppler are super cheap for free
Speaker 3: actually to start, so it doesn't get keeper than that. Yeah,
Speaker 3: it's free from the first three years users, and then
Speaker 3: it's basically get hub priced from there. So if you're
Speaker 3: comfortable paying for get Up, you should be comfortable playing
Speaker 3: for Doppler. But let's say you don't want to do
Speaker 3: that for whatever reason. I think the most important thing
Speaker 3: you can do if you're not going to use any
Speaker 3: tooling at all, is or bare minimum tooling is used
Speaker 3: like a cloud secrets provider like AWS Secrets Manager, GCP
Speaker 3: Secrets Manager and so on, and then build some tooling
Speaker 3: around it so that you can fetch secrets locally in
Speaker 3: development without them touching disc So what you need to
Speaker 3: do is have some wrapper around your application that starts
Speaker 3: your application. So I'd imagine the workflow would be like
Speaker 3: fetch secrets from a to BA secrets manager, inject them
Speaker 3: into the environment or write them to an encrypted file,
Speaker 3: and then have the decryption key in the environment. Either pathworks,
Speaker 3: and then once the application starts, you clean up the
Speaker 3: environment and then load all your other dependencies. And if
Speaker 3: you did that plus having year secrets and a to
Speaker 3: B A secrets manager and that being the same process
Speaker 3: happening for production workloads, at least you solve most of
Speaker 3: the risk. In my mind. Also, when you're doing this
Speaker 3: and this where things get a little bit more of
Speaker 3: a burden because eight of US credit or managing privileges
Speaker 3: like I am roles is quite complicated, in my opinion,
Speaker 3: is You're going to want to make sure that every
Speaker 3: individual developer and every set of infrastructure has their own identity,
Speaker 3: and that way you have robust audit logs of like, Okay,
Speaker 3: the secret is taken by X developer at this time
Speaker 3: or read by this time, and we can we have
Speaker 3: this auditor of what happened. So you don't want to
Speaker 3: basically create one master i AM credential and just share
Speaker 3: it with your entire team, because then you lose a
Speaker 3: lot of your auditing story along with it. And you
Speaker 3: also probably want to set up if you have a
Speaker 3: little bit extra time to spend an auditing story around reads.
Speaker 3: So every time a secret is downloaded from this endpoint,
Speaker 3: you're going to want to make sure that you have
Speaker 3: the right logging and auditing set up in a WS
Speaker 3: for that, and that you're actually looking at those logs
Speaker 3: and have analysis over those logs absolute bare minimum.
Speaker 1: One other thing that I've seen and I think that
Speaker 1: I saw this on the camal, which is a deployment
Speaker 1: system that was written by David Heinemeier Hansen, and anyway
Speaker 1: it can pull secrets from like one password. Yeah, you've
Speaker 1: seen people using one password for their secrets management.
Speaker 3: Less and less, so they they added something called one
Speaker 3: Password Secrets Automation. Everything that I've heard from my community
Speaker 3: is that they are not actively investing in that product anymore.
Speaker 3: And it's kind of like shelfware now, so she never
Speaker 3: heard that it's there, it's available, you can use it,
Speaker 3: but it's not actively being invested in. It's on the shelf.
Speaker 3: But the amount of people every year that I hear
Speaker 3: using is less and less because a I think there's
Speaker 3: like first class products now that that really work, and
Speaker 3: the amount of things that you want out of secret
Speaker 3: Manager is growing far faster than what they're adding to,
Speaker 3: which is at this pace for my are staying at zero.
Speaker 3: But you can't do it. I mean like secrets Manage
Speaker 3: or one Pastor is engen encrypted, so like you're getting
Speaker 3: a lot of security benefits from that. So I'm not
Speaker 3: gonna say, hey, don't ever use it. If if it
Speaker 3: push came to shove, use it. I just think there's
Speaker 3: like far better things that are also easier to use
Speaker 3: in far better priced out there that are really available.
Speaker 1: Valentino, look steep and fun is.
Speaker 2: Is uh Doppler part of the GitHub secret Scanning? Yeah, yep,
Speaker 2: that's all.
Speaker 3: Sark if you if there is a Doppler token, a
Speaker 3: Doppler issue token found on GitHub gettabile, contact us immediately,
Speaker 3: will immediately revoke the token, notify you and work with
Speaker 3: you generated your token, and we're going to be investing
Speaker 3: next year pretty aggressively in the secret scanning space as
Speaker 3: well of like really having a full auto picture there,
Speaker 3: Like I'd love to be able to say, hey, we
Speaker 3: found some secrets in your code, We've automackally revoked them,
Speaker 3: your new ones, and those new secrets are now and
Speaker 3: not in your code anymore. Like some really powerful workflows
Speaker 3: like that can just make sure that even though the
Speaker 3: tooling is set up so that you'd want to use it,
Speaker 3: if you actually don't use it and you fall back
Speaker 3: to bad habits, that the system catches you.
Speaker 2: Yeah, I'm curious on like as far as the observability goes,
Speaker 2: Like what are the latest like friends happening now for
Speaker 2: like observing like secret use and like maybe detecting bad actors,
Speaker 2: you know, ahead of the breach.
Speaker 3: Detection ahead of the breach at least in the secret
Speaker 3: space is almost zero from my understanding right now, it's there.
Speaker 3: There isn't a classic tool yet for that. But on
Speaker 3: the scanning side, there there's Trufflehog, there's get Guardian. Get
Speaker 3: ub is adding some stuff too. There is a space
Speaker 3: developing right now where basically you can scan. The goal
Speaker 3: is to scan the perimeter of your infrastructure, right, so
Speaker 3: like Slack, Microsoft teams, email, your code base, all of
Speaker 3: that infrastructure basically be scann in real time for secrets.
Speaker 3: That's basically the extent of what we have today. The
Speaker 3: problem is like you have to connect those dots, right, So, like,
Speaker 3: you found a secret, what do you do next? Right,
Speaker 3: especially if you're like an individual developer, you don't control
Speaker 3: the secret manager that's being used. You don't control or
Speaker 3: if they're not using a secrets manager, what the flow is.
Speaker 3: And so like if you're in if you like a class,
Speaker 3: example that I hear all the time is, oh, we
Speaker 3: added a secret scanner, and the secret scanner found all
Speaker 3: of our INBI files, but the companies decided we use
Speaker 3: eb files. So like now it's a mood point because
Speaker 3: we know we've always known secrets we're in code because
Speaker 3: they're in eb files, but there's nothing to get us
Speaker 3: to the next step of like getting it out of code.
Speaker 3: So now all you're doing is surfacing a problem that
Speaker 3: we can't solve. So I think that's really where like
Speaker 3: we're trying to help move the industry is connecting those dots.
Speaker 3: You found a secret, and you can immediately take an
Speaker 3: action or an action is taken on your behalf based
Speaker 3: on a policy to clean up the mess.
Speaker 1: I'm gonna have to go back and listen to this
Speaker 1: and then go through all my stuff.
Speaker 2: Run your checklist. Yeah, do you have a checklist, Brian
Speaker 2: of like you know at least run through this for
Speaker 2: your projects or your team or do you publish anything
Speaker 2: around that?
Speaker 3: Yeah. We have a couple of blog posts said that
Speaker 3: highlight like basic secrets management practices outside of the stuff
Speaker 3: I already shared today around working with like his cloud provider,
Speaker 3: I would strongly recommend having a secret scanner, so Truffle
Speaker 3: ogs free use it it's open source, or get Guardian
Speaker 3: both the partners of ours. The other big thing is
Speaker 3: outside of secrets, I would have a vulnerability scanner really
Speaker 3: important on all your major dependency streams, so like Docker, images, Ruby, gems,
Speaker 3: if you're using Node, NPM, but basically any dependency you have,
Speaker 3: you should have vulnerability scanning on it. Super freaking important
Speaker 3: because it's so easy these days for like one dependency
Speaker 3: that nobody's cared about for ten years to just get
Speaker 3: bought for like a couple hundred bucks, and then someone
Speaker 3: pushes an update for it, and all the other dependencies
Speaker 3: having frozen on that older version, and so now all
Speaker 3: these dependencies get that update, which means you get the update,
Speaker 3: and that update has motious code in it, and that's
Speaker 3: like a very common paradigm for attack these days. So
Speaker 3: a freezer, you're right your dependencies, meaning like I'm fixed
Speaker 3: at this version and no amount of them pushing a
Speaker 3: new update will change that. I'm on this version and
Speaker 3: this version has been like guaranteed good, no vulnerabilities in it,
Speaker 3: and that at least can help prevent any issues where
Speaker 3: like a vulnerable code gets pushed up and you guys
Speaker 3: immediately get attacked from it. What you really want is
Speaker 3: you want all your dependencies frozen, and then when a
Speaker 3: new exploit happens, you go, oh, that was kind of
Speaker 3: cool where you are using that package, but our version
Speaker 3: doesn't have that problem, so we're good, And then you
Speaker 3: just wait for the fix to come out and then
Speaker 3: you can do another refresher another update.
Speaker 1: Cool, anything else that you guys want to dive into.
Speaker 1: So give us kind of the the what two minute
Speaker 1: pitch on on Doppler. Just still let people know what
Speaker 1: it does and how makes your life better.
Speaker 3: Yeah. So at the end of day, Doppler makes your
Speaker 3: life better in a couple of ways. One, it's gonna
Speaker 3: add two hours of productivity per developer per week, so
Speaker 3: in a way it kind of pays for itself. On
Speaker 3: top of that, it's going to make you far more
Speaker 3: secure from an organization standpoint, managing your secrets so you
Speaker 3: can get You can sign up today by creating account
Speaker 3: for free, and then if you're much much larger org
Speaker 3: you can talk to our solutions entering team who can
Speaker 3: help figure out how this integrates into your complex infrastructure.
Speaker 3: We have a whole host of great companies that are
Speaker 3: using us, so you're not alone from a center Krumble Cookies,
Speaker 3: Puma Children's Cancer Institute, stock x Travel, trip Advisor, so
Speaker 3: there's a number of g companies you won't be alone
Speaker 3: on this side. And we manage about thirty billion secrets
Speaker 3: a month right now, so like at quite large scale.
Speaker 3: And I think the key thing here if you decide
Speaker 3: not to use Doppler is at least manage your secrets.
Speaker 3: Like at the end day, I think we are all
Speaker 3: tired of data breaches happening because they keep impacting us
Speaker 3: and they impact your customers. And so if you're not
Speaker 3: going to use Doppler, that's totally fine by me. You
Speaker 3: could even use a competitor. Just use a secrets manager
Speaker 3: protect those secrets.
Speaker 1: I'm a little gratified you mentioned Grumble Cookies, their local company. Anyway,
Speaker 1: let's go ahead and do our picks and then we'll
Speaker 1: wrap up now. I don't know if you had a
Speaker 1: chance to listen to other episodes of the show, but
Speaker 1: we just shout out about stuff that we're playing doing
Speaker 1: or that we like. Valentino, do you want to start
Speaker 1: us off?
Speaker 2: Sure? Yeah, I mean the first thing that comes to
Speaker 2: mind is the Dark Knight Diaries podcast, which is just
Speaker 2: like an incredible resource of deep diving exploits and just
Speaker 2: like people, people are just so creative out there and
Speaker 2: uh nefarious or not. It's just like wild to see
Speaker 2: what uh what kind of attack vectors there are and
Speaker 2: how people are using and exploding your behavior, which it
Speaker 2: can be patterned, right, So I'd recommend if you're interested
Speaker 2: in any of that kind of stuff, It's it's an
Speaker 2: awesome show. Next to that, I've been following a lot
Speaker 2: of the Rails World fallout, which is just like, uh,
Speaker 2: I've definitely missed out. I wanted to go to conference. Yeah,
Speaker 2: there's so much, so much great stuff that I'm just
Speaker 2: watching from Afar come out of that and it's really exciting.
Speaker 2: Uh So, I guess just watch the YouTube channel for
Speaker 2: updates of new talks that are coming out of there.
Speaker 2: And then the last pick I have is Landing Gray.
Speaker 2: He's in the Ruby AI space and he basically made
Speaker 2: stream lit for Ruby, which I'm really excited to mess
Speaker 2: around with. And if you're not familiar with stream lit,
Speaker 2: it's kind of like the uh you know, I don't
Speaker 2: know if it's JavaScript or it's some other language, but
Speaker 2: it's what a lot of the you know, machine learning
Speaker 2: enthusiasts will use to just quickly get up a chat
Speaker 2: and uh, you know, fine tune or use lms in
Speaker 2: creative ways. And lang Chain is kind of like promoted
Speaker 2: it for a long time for just quickly iterating through
Speaker 2: AI products development, and it's been really incredible just like
Speaker 2: quickly get up and running if you on something new,
Speaker 2: and so I'm excited to see that come to Ruby
Speaker 2: in a very like Ruby way. So it looks very
Speaker 2: much like you know, a rack app or or something
Speaker 2: like that where you can just run it and it
Speaker 2: has a chat that you can use, and I'm really
Speaker 2: excited to see that develop more.
Speaker 1: Cool. I'm gonna throw in my picks, So the first
Speaker 1: one I always do a board game pick. I'm gonna
Speaker 1: pick keep the Heroes out. So the kind of the
Speaker 1: design of the game and stuff is kind of like
Speaker 1: a dungeon for like D and D, except simplified, and
Speaker 1: you're not playing the heroes, you're playing the monsters. And
Speaker 1: so each player gets to play a different set of
Speaker 1: monsters and they have different abilities. And so the way
Speaker 1: that you play is you take your turn by you know,
Speaker 1: playing your cards. So you have a deck of cards
Speaker 1: and so it's a little bit of a deck building
Speaker 1: game and you can do all kinds of stuff. You
Speaker 1: can move, you can create artifacts, you can do stuff
Speaker 1: with the artifacts depending on which room you're in and
Speaker 1: what actions are available to you and anyway, you can
Speaker 1: also spawn new monsters, you know, again depending on what
Speaker 1: actions you can take, and so then after you do
Speaker 1: all your stuff, then what happens is then you spawn heroes,
Speaker 1: and so you draw the card and it tells you
Speaker 1: what rooms to put, what kinds of heroes in. The
Speaker 1: heroes do their special abilities, and then they'll either attack
Speaker 1: monsters or try to open the chests in the room,
Speaker 1: or if they can't do either of those two things,
Speaker 1: then they'll move one step closer to the vault and so.
Speaker 1: And the chests are numbered, so if it's a one,
Speaker 1: then it takes one activated hero to open it. If
Speaker 1: it's a two, it takes two. You kind of get
Speaker 1: the idea. So anyway, it that's essentially the game. There
Speaker 1: are a couple of other you know, nuances to it,
Speaker 1: and of course it has a series of dungeons that
Speaker 1: you can play through, and so you know, they're different
Speaker 1: rules for the different dungeons as far as how they work.
Speaker 1: The way you win is you exhaust the hero pile twice,
Speaker 1: and yeah, the way you lose is if the heroes
Speaker 1: opened the chest in the vault, which is the four chest.
Speaker 1: So anyway, really fun game takes about what forty five
Speaker 1: minutes ish to play. The artwork on it is awesome.
Speaker 1: The different types of Monsters are fun and so, yeah,
Speaker 1: I've played it. It plays one to four players. I've
Speaker 1: played it with three and four players, and anyway, it's
Speaker 1: a fun game. Yeah, so I'm gonna pick that. And
Speaker 1: then one movie I've been wanting to watch for a while,
Speaker 1: mostly because I just heard it was really good, is
Speaker 1: He'll Billy Ellogy. Now. I recognize that this is the
Speaker 1: memoir by jd Vance, who's currently Donald Trump's running mate.
Speaker 1: So politics aside, right, and I try not to get
Speaker 1: into it too much on this show. Longtime listeners know
Speaker 1: where I stand on all this stuff, but I don't
Speaker 1: want to make the show about politics. But the story
Speaker 1: is amazing, and you know the fact that you know,
Speaker 1: this kid can come from basically the kind of place
Speaker 1: and lifestyle that he did and then become you know,
Speaker 1: wind up going to Yale Law School and then you know,
Speaker 1: becoming a US senator. I mean, I just find that
Speaker 1: incredibly inspiring. And you know, at the end of the day,
Speaker 1: then you know how you feel about him. Now whatever,
Speaker 1: it's really well done. It has Glenn Close and Amy
Speaker 1: Adams in it. I haven't read the book, so I
Speaker 1: don't know how closely they stuck to it. But anyway,
Speaker 1: there were just several I think life lessons that anybody
Speaker 1: can benefit from from the standpoint of, hey, look, if
Speaker 1: you really want these things out of your life, then
Speaker 1: you can go have them if you're willing to do
Speaker 1: what it takes. And so you know, from that standpoint, Yeah,
Speaker 1: I really liked the movie. There was something else I
Speaker 1: was going to pick and I just oh, yeah, it
Speaker 1: was the micro film And this one is on the
Speaker 1: Angel Studios app. I think it's called something Something to
Speaker 1: Stand For. And this is more it's almost a documentary.
Speaker 1: So if you've listened to the rest of the story
Speaker 1: by Paul Harvey, it's kind of based on that. That's
Speaker 1: also the premise of Mike's podcast, at least when he
Speaker 1: started it. Now he just interviews people that are interesting
Speaker 1: to him. But what he would do is he would
Speaker 1: tell a story without telling you who it was about, right,
Speaker 1: And so at the end then he were, you know,
Speaker 1: there's the big reveal and so this person was, you know,
Speaker 1: Neil Armstrong or whatever, right, and you know, and so
Speaker 1: that's the rest of the story is kind of the idea.
Speaker 1: And so he tells a bunch of those from US history,
Speaker 1: from moments that in some way shaped the direction of
Speaker 1: the country And anyway, I really really enjoyed that too,
Speaker 1: so I'll put a link to that in the comments
Speaker 1: as well. But yeah, and then the last pick I have,
Speaker 1: so Valentino mentioned rails World and the stuff that came
Speaker 1: out of it. I haven't watched any of the talks.
Speaker 1: I haven't really dived into what came out of it.
Speaker 1: But I've been using Kamal to deploy my stuff, and
Speaker 1: I've gotten really tired of fighting the traffic reverse proxy
Speaker 1: load balancer, and so I wanted to upgrade to Kamal
Speaker 1: too so I could get the Kmal proxy put in
Speaker 1: and make my life a little simpler. Because it looked
Speaker 1: like it made it would make my life a little simpler.
Speaker 1: And so I did the upgrade, and I kind of
Speaker 1: screwed it up and then fixed it, so it took
Speaker 1: a little longer and a little more work than I
Speaker 1: thought it would. Essentially when I messed up was the
Speaker 1: directions are a little bit The documentation on Camal is
Speaker 1: just not awesome. I'll just put it that way. But anyway,
Speaker 1: what what it does is it uh, it tells you
Speaker 1: to upgrade by upgrading to Comal one nine, one dot nine.
Speaker 1: There's one dot nine dot one now, but I think
Speaker 1: it tells you to do one dot nine dot zero. Anyway,
Speaker 1: you want to make sure that you can deploy with
Speaker 1: one dot nine and then you can upgrade to two
Speaker 1: dot one and do the Camal upgrade. And then it
Speaker 1: goes in and it stops traffic, and it starts the
Speaker 1: Camal proxy, and it, you know, does the let's encrypt
Speaker 1: and all of that stuff for you without having to
Speaker 1: know all of the arcane magic that you need to
Speaker 1: know in order to get traffic to do all that
Speaker 1: for you. And so I'm anyway, I'm pretty excited about it.
Speaker 1: It's it's been pretty awesome so far. So I'm going
Speaker 1: to pick that I got a couple of things I
Speaker 1: still need to figure out on it, because I want
Speaker 1: to do multi tenancy and allow people to pick their
Speaker 1: own domains, and so then I want it to do
Speaker 1: the let's encrypt stuff, and so I'm trying to figure
Speaker 1: out if I can make Kamal do that for me
Speaker 1: or Kamal proxy do that for me without having to
Speaker 1: have it in my configuration. But for now, it's it's
Speaker 1: employing great. So I've been pretty happy. With that, Brian,
Speaker 1: what are your picks?
Speaker 3: Yeah, those are some good picks. Hard to compete with that.
Speaker 3: I think a huge Star Wars fan, so I'll just
Speaker 3: shout out Star Wars clone Wars. I feel like not
Speaker 3: a lot of people know about it, but it's between
Speaker 3: episode two and three, and it's so freaking good. It's like,
Speaker 3: I'm a guy that usually doesn't like like animated stuff,
Speaker 3: and it's just blown me away. So if you're a
Speaker 3: big Star Wars fan, there's a lot of seasons and
Speaker 3: it's insanely good, and it's basically like Dave Filonia at.
Speaker 1: His best.
Speaker 3: Outs of that really big fan of the Perplexity app.
Speaker 3: It's like, basically we're placed any Google Search, as I
Speaker 3: do now for anyone who doesn't know what Perplexity is,
Speaker 3: it's Google such Google Search plus chat GBT is like,
Speaker 3: so ask it a question and it just gives you
Speaker 3: the knowledge back. It doesn't just give you a bunch
Speaker 3: of links and it uses uh and just like a
Speaker 3: Google search, its update in real time, so if something
Speaker 3: comes out yesterday or earlier today, it'll be informed on that.
Speaker 3: And it is just an incredible job at answering, like
Speaker 3: really in depth questions like you can ask it very
Speaker 3: very complicated questions that have a lot of nuance, that
Speaker 3: have a lot of requirements on real time data, like
Speaker 3: I don't know what's the compare the market cap of
Speaker 3: Nvidia to its revenue and then compare that to Microsoft's
Speaker 3: and figure out which one is a better stock to investment.
Speaker 3: Like you can ask that question, I'll give you a
Speaker 3: pretty concrete answer with like charts and graphs included. So
Speaker 3: very impressed with, Like, I think that's like a pretty
Speaker 3: big step up and the ability to like get knowledge
Speaker 3: very very quickly. And then the other one is John Coogan.
Speaker 3: I think he does a it's a YouTube channel he
Speaker 3: that's out there, and he just does an incredible job
Speaker 3: of like talking about I think tricky topics, so like
Speaker 3: what's happening with Taiwan and chips or and or old
Speaker 3: Prime defense. Like he just does a good job of
Speaker 3: telling the story of some event or some company really
Speaker 3: really well, and I feel like it's extremely fact based
Speaker 3: and not like bias based. So I generally go to
Speaker 3: him when I'm looking for like really high impact news breakdowns.
Speaker 3: And then if you're also a Star Wars fan, I
Speaker 3: just done with the skin because again huge Star Wars nerd.
Speaker 3: Here there's a board game called Star Wars Rebellion. It's
Speaker 3: played with two players. One guy gets to play as
Speaker 3: the Empire or the other as the Rebellion and you
Speaker 3: guys battle it off and it's it's quite fun. It's
Speaker 3: a very asymmetric game, but I think that's what makes
Speaker 3: it fun, because like Star Wars is very asymmetric. Uh,
Speaker 3: the Empire had a lot more resources than the rebels.
Speaker 3: But if you like Star Wars but fun wars and that,
Speaker 3: that board game will do you a lot of good awesome.
Speaker 1: I don't think i've played this one. It's it's pretty
Speaker 1: highly ranked on board game Geek. It has a weight
Speaker 1: of three point seventy four, which means it's a pretty
Speaker 1: complicated game.
Speaker 3: Oh yeah, it took us eight hours to figure out.
Speaker 1: Yeah, it says two to four players and the playtime
Speaker 1: is like three to four hours, And I'm assuming that's
Speaker 1: if you know how to play.
Speaker 3: Yeah, I'd say me and my brother we play. It
Speaker 3: took us eight hours the first game to get it.
Speaker 3: The second game, we actually did it across two days
Speaker 3: just because we were kind of like very slow with
Speaker 3: every decision and be really methodical and now we'll play
Speaker 3: like in an hour and a half. So like it,
Speaker 3: once you get comfortable, rams up pretty quickly.
Speaker 1: Yeah. But yeah, so the rank on it overall is nine,
Speaker 1: So it's the ninth most popular game on board game geek.
Speaker 3: Hey there we go.
Speaker 1: Yeah, and it's it's the fifth thematic, which means that
Speaker 1: it has some you know, some theme to it, like
Speaker 1: Star Wars. So yeah, so next time I'm at a
Speaker 1: board game convention maybe I'll see if they have it,
Speaker 1: if I can play it. Anyway, I'm a board game
Speaker 1: that I really enjoy that stuff.
Speaker 3: So it's all right a lot of fun.
Speaker 1: Yeah. So, Brian, if people want to connect with you online,
Speaker 1: how do they find you?
Speaker 3: Brian at dopper dot com for email that will long
Speaker 3: get Brian for LinkedIn and twin Cool.
Speaker 1: All right, well thanks for coming, yeah, thank you. All right,
Speaker 1: until next time, folks, max out