Speaker 1: Hello, everybody, this is just Reach's Adventure and Devil. Will
Speaker 1: you already laughing at me? You can't put me on
Speaker 1: the spot? Do the intro? Now they're laughing at me?
Speaker 1: All right, Well, I'm Jillian Rowe and with me this
Speaker 1: week is Will Button and Jonathan Hall Highouse.
Speaker 2: And technically I'm laughing with you, not at you, because
Speaker 2: you were laughing too.
Speaker 1: It's okay. I only felt a little bit put on
Speaker 1: the spot.
Speaker 3: I'll forgive you some day to be fair.
Speaker 2: I did say okay, let's go and then hit record.
Speaker 3: I'll stand with Billian that she was put on the spot,
Speaker 3: but I fully support putting Billian on the spot. So, yeah,
Speaker 3: what are we talking about today, guys?
Speaker 1: All right, we're going to talk about c i CD tools.
Speaker 1: What do you use, what do you hate? Why did
Speaker 1: you use it? Do you get a choice? Does your
Speaker 1: job just like dictate it for you? Or do you
Speaker 1: have like some kind of some kind of input into
Speaker 1: this decision? Why would you make your decisions? Who wants
Speaker 1: to hit us up first? And I think you suggested
Speaker 1: the topics, so she gets put you on the spot.
Speaker 3: Now, okay, yeah, so it's a big topic. So the
Speaker 3: reason I thought of it is earlier this week was
Speaker 3: last week on LinkedIn, one of my contacts asked the
Speaker 3: public which CICD tool would you like me to use?
Speaker 3: In a demo about how to get started with CICD,
Speaker 3: which led into a bigger conversation of which ones are people?
Speaker 3: And she did a poll which gave four options, and
Speaker 3: none of the four options were things I'd even used before,
Speaker 3: and I've used probably a dozen of them. So that
Speaker 3: just goes to show that there's a huge selection of
Speaker 3: tools to pick from. And I don't know how she
Speaker 3: chose her for but I think it was Jenkins and
Speaker 3: whatever AWS provides, and maybe Azure DevOps and I don't
Speaker 3: know what else. You know, it's a good conversation to have.
Speaker 3: I think, how do you choose one of these tools
Speaker 3: to use? Because there's so many and they all do
Speaker 3: roughly the same thing. They all automate the running of tasks,
Speaker 3: which can be running tests, building software, tagging releases, ordering
Speaker 3: pizza perhaps whatever they whatever they do, they can they
Speaker 3: can automate these things. So yeah, I don't know how
Speaker 3: we want to tackle this. I mean, if somebody asked
Speaker 3: me quickly, Jonathan, what CSD S will do you prefer.
Speaker 3: I have a quick answer to that, and that is
Speaker 3: get laud ci. But that's not a very useful answer
Speaker 3: without a lot of context, because that's not the answer
Speaker 3: I would get to everybody. And if ask me what
Speaker 3: should they do, that's a different question than what do
Speaker 3: I prefer? So, yeah, I don't know how we want
Speaker 3: to tackle this today.
Speaker 2: I think that I think there's like two different ways
Speaker 2: to answer this. You know, there's I'm starting out choosing
Speaker 2: a CICD tool, and we have a CICD tool in
Speaker 2: place and we want to replace it. And for that
Speaker 2: latter question, my go to response is always don't. I mean,
Speaker 2: they all have things that they're good at and bad at,
Speaker 2: and the cost of transitioning from one to the other.
Speaker 2: There's just not enough value in it because you might
Speaker 2: solve one specific problem, but you're introducing other problems that
Speaker 2: you don't know you have problems with.
Speaker 3: Yet I tend to agree it's.
Speaker 1: Always a trade off. Yeah, everything like that is a
Speaker 1: trade off.
Speaker 2: The best CICD is the one that you already have installed.
Speaker 3: Usually. Yeah, So there are times when it's appropriate to change,
Speaker 3: but not for that reason, like if you're changing from
Speaker 3: at lastly and to GitHub for example, you maybe need
Speaker 3: to change from Bamboo to get hub actions at the
Speaker 3: same time, but not because you want to change your
Speaker 3: CSD tool, just because you're changing everything else simultaneously, for sure,
Speaker 3: So I would almost I mean, there have been times
Speaker 3: when I migrated off of a CSP tool, but there
Speaker 3: were cases where it was not being used properly or
Speaker 3: being used very little, so that there was very little
Speaker 3: invested and it wasn't doing much for us. So you know,
Speaker 3: it was basically a proof of concept that we decided
Speaker 3: not to continue with, if you want to make up
Speaker 3: that way. So that's a different scenario than I think
Speaker 3: what you're talking about. If you have a CICD configured
Speaker 3: and already doing some thing valuable, I think really hard
Speaker 3: before changing, really really hard. When you when you're done
Speaker 3: thinking hard, go think hard again.
Speaker 1: I think there are some good stories behind these opinions.
Speaker 1: You guys have a like emotional trauma to share around
Speaker 1: switching up c I S peoples and then realizing it
Speaker 1: was it was a terrible, terrible mistake. Or clearly you
Speaker 1: guys have never made such a mistake like that, right.
Speaker 3: No, No, I thought really hard and decided not to
Speaker 3: do it.
Speaker 2: I've I think my experience forming that opinion has been
Speaker 2: from not from switching, but from having exposure to several
Speaker 2: different ones, Like I've used git Lab, I've used Jenkins,
Speaker 2: I've used Circle CI, Travis CI and all of them. Like,
Speaker 2: I can't see the value. There's a fe use cases
Speaker 2: for it, but I can't see on the surface the
Speaker 2: value in switching from one to the other because I've
Speaker 2: I've used a lot of them, and each has their
Speaker 2: own strengths and weaknesses, but enough to justify rebuilding all
Speaker 2: that stuff, especially if you have like a lot have
Speaker 2: it doing a lot. You know, if you have a
Speaker 2: lot of different pipelines, that's going to be some pain.
Speaker 2: And at the at the end, you know, what did
Speaker 2: you really gain? What value did the customers who are
Speaker 2: buying or using your product to get from you switching
Speaker 2: the CICD solution.
Speaker 3: I want to add to that. And there might be
Speaker 3: times when it's appropriate to change, if you're using an
Speaker 3: anti antiquated tool, for example, or it's just too slow
Speaker 3: or it doesn't integrate with things you're needing to do.
Speaker 3: But if you go that route, don't change everything at once,
Speaker 3: just implement the new things you need. I know that
Speaker 3: there's the engineer and every one of us hates this
Speaker 3: idea of maintaining two systems, but it's worth it versus
Speaker 3: trying to do turn to ram everything of the new
Speaker 3: system and breaking stuff along the way, just to admit
Speaker 3: you're going to maintain two systems, one for the new
Speaker 3: stuff that immigrates with the Raspberry pies or whatever weird
Speaker 3: thing you're trying to do, and the old one that
Speaker 3: manages the old stuff.
Speaker 1: Cool. Well, what about somebody who's switching to or not
Speaker 1: switching but starting clean slink projects CICD tool. What are
Speaker 1: kind of the scenarios where we would choose one or
Speaker 1: the other. I have one right off the bat. If
Speaker 1: you have data or some kind of data or software
Speaker 1: that have to be private. There are scenarios where you
Speaker 1: can use the open ones like GitHub, GitHub, circle, all
Speaker 1: that kind of thing. Most of them now, I think,
Speaker 1: do allow for at least some actions or like CICD
Speaker 1: minutes or I forget exactly how it's metered. I think
Speaker 1: on GitHub it's tide of the number of actions or
Speaker 1: the number of minutes. I forget how much for private repositories,
Speaker 1: but it's often quite limited. Unless you're on a public repository,
Speaker 1: so that could be a reason to change. You're like, Okay,
Speaker 1: we have this heavy duty workload. It has to be
Speaker 1: private for some reason, so either we need to take
Speaker 1: get hub a bunch of money, or we need to
Speaker 1: move it onto something in house like Jenkins or get
Speaker 1: lab that's managed internally. But really that's it. That's the
Speaker 1: only differentiation that I make because all my stuff is
Speaker 1: in make files anyways. So I can be on GitHub,
Speaker 1: I can be on Jenkins, be anywhere because it's called
Speaker 1: colin make, you know, like make tests or whatever. Then
Speaker 1: there we are.
Speaker 3: So my simple answer is use whatever's easiest to get
Speaker 3: started with. And if you're using GitHub, that probably means
Speaker 3: to get hub actions. Pre using git lab it probably
Speaker 3: means to get lab CI. If you're using Bitbucket, it
Speaker 3: probably means Bamboo. If you're I don't know when it
Speaker 3: would ever mean Jenkins. I can't think of a reason
Speaker 3: to start using Jenkins unless you're already using it. So
Speaker 3: do you? Uh, Maybe you just want to become a
Speaker 3: Maybe you like pain and like to learn old systems
Speaker 3: because it feels fun, like, you know, get a Commodore
Speaker 3: sixty four at the same time, why don't you.
Speaker 2: You can on a Commodore sixty four?
Speaker 3: Can you? I'm sure you can't, but it sounds like
Speaker 3: a good project. I think I might have to learn
Speaker 3: Jenkins and Commodore sixty fours now. So that's my short answer. However,
Speaker 3: So that's what that usually means. If somebody takes that
Speaker 3: at face value, that probably means using get hub actions
Speaker 3: because it's it's easy. Most people use get hub their
Speaker 3: coporate actions. However, I think get how Actions is actually
Speaker 3: a really bad tool. It feels like a version zero
Speaker 3: point six of get lad ci, you know, it feels
Speaker 3: like it has a lot of rough edges. Still, it's
Speaker 3: not really intuitive to use yet. The one killer feature
Speaker 3: it has over get lad Ci is is this sort
Speaker 3: of pre bundled ability to just PLoP in other people's code,
Speaker 3: what do they call it their marketplace. You can just
Speaker 3: you know, choose an action from some random person off
Speaker 3: the street and start running it. It also strikes me
Speaker 3: as really dangerous. It's like it's like n PM for
Speaker 3: for get houb actions and who wants that? Really? Right?
Speaker 1: I just picked up the Helm release from GitHub Actions.
Speaker 1: There's I don't know what the doctor doctor builds push
Speaker 1: like that's the that's all I'm doing.
Speaker 3: So if you don't care about security.
Speaker 1: Well see no, if you care about security, I still
Speaker 1: think you should be hosting it in house. That's I
Speaker 1: think that's well.
Speaker 3: There's different aspects of security though, I mean, yeah, I
Speaker 3: mean it depends how much you care about security, right,
Speaker 3: But I mean if you don't care about security and
Speaker 3: you want easy then maybe get have actions is fine
Speaker 3: if you don't have to edit them yourself. If you
Speaker 3: need to edit them yourself, then the learning curve goes
Speaker 3: up really quickly compared to just picking something from the
Speaker 3: at least and Yammel and and more than that. I mean,
Speaker 3: JavaScript isn't the hard part. It's it's the format and
Speaker 3: the way the different actions interact with each other and
Speaker 3: trying to tie puzz them together. So if you need
Speaker 3: something more more customized, I think get lab CI is
Speaker 3: far easier to use and get actions for simple things,
Speaker 3: either any of them will work. I mean, if you're
Speaker 3: just writing a cell script, you can do that anywhere now.
Speaker 3: But if you need something more complicated, I feel like
Speaker 3: get lab is easier. That part of that's probably because
Speaker 3: I've been using it. Longer, so I have the mental
Speaker 3: model better in my head. But I've heard many other
Speaker 3: people say the same that they feel like get lab
Speaker 3: CI is more mature product and get have actions that
Speaker 3: probably won't be true forever.
Speaker 1: I explicated, like you have to build for a bunch
Speaker 1: of different hardware, so you have like really complic matrix
Speaker 1: builds because I'm just running make tests like the time,
Speaker 1: like make tests the doctor build and that's.
Speaker 3: Yeah, if that's all you're doing, you're you're pretty good
Speaker 3: to go. If you're trying to deploy to Kubernetes at
Speaker 3: the same time, you know, if if you want to
Speaker 3: automate your deployments to Kubernetes or to Heroku or to whatever,
Speaker 3: then to Firebase, then you have other things that need
Speaker 3: to be done. You need to integrate your your authentication
Speaker 3: with those things. So you know, there's a certain number
Speaker 3: of complex steps that need to happen. If you start
Speaker 3: to do interstep dependencies, things get complicated very quickly. In
Speaker 3: other words, say you have five repositories and they each
Speaker 3: film an artifact, and then whatever one of those is updated,
Speaker 3: you rebuild a master artifact that takes you know, those
Speaker 3: libraries and compiles of together the single thing and then
Speaker 3: deploys it. So things like that they can start to
Speaker 3: get complicated quickly. And no doubt you can do any
Speaker 3: of these things with any of the tools. It's a
Speaker 3: question of how difficult. And this is why, going back
Speaker 3: to Will's earning point, if you already have this working,
Speaker 3: for God's sake, don't redo it.
Speaker 1: Yeah, I think about these like really specialized CICD tools
Speaker 1: that I'm starting to see pop up. So we were
Speaker 1: talking a bit before the show, as we do before
Speaker 1: we press record, about this tool that I'm using called Spacelift,
Speaker 1: which is I think it's kind of supposed to be
Speaker 1: like terraform Cloud, except hopefully hopefully less expensive for me
Speaker 1: and my wallet. But no, no, I'm kidding, Like it
Speaker 1: looks good, but it has these sort of like managed
Speaker 1: terraform environments, and it's very specifically for people doing terraforms
Speaker 1: with care about security and policies and maybe building different
Speaker 1: terraform modules and also protecting your like your stacks as well,
Speaker 1: so you can insert these policies to you know, say
Speaker 1: don't kill it if this happens, or kill it if
Speaker 1: this happens. Like these kind of things that are very
Speaker 1: specific to deploying infrastructure, and I think now GitHub has
Speaker 1: one maybe for JavaScript or for CI like you know,
Speaker 1: like I don't, I don't know, but I do feel
Speaker 1: like it's kind of a trend to see these more
Speaker 1: very specialized CICD services pop up for particular frameworks and
Speaker 1: code tools and things. Are you guys seeing that too?
Speaker 2: I haven't, but I'm the work I'm I've been on
Speaker 2: stainly has been pretty generic work, you know, not anything
Speaker 2: specific that sounds like it's pretty pretty siloed to address
Speaker 2: a specific use case. And I can imagine if you're
Speaker 2: in that use case, you're feeling that pain. But if
Speaker 2: you're not in that use case, then you probably don't
Speaker 2: know that it exists.
Speaker 3: So I have not used any specific tools like that,
Speaker 3: but I've definitely used things that are intended to plug
Speaker 3: into an existing CICD tool or pipeline for specific needs,
Speaker 3: you know, simple things. Sonarchy was an example, or depending
Speaker 3: on the Textec you're using, you might have specific linters
Speaker 3: for your for your language or your tool set. You know.
Speaker 3: We had a guest on if you want to go
Speaker 3: talking about doing this sort of analysis for Kubernet's manifests
Speaker 3: and you know, so that would be a tool that
Speaker 3: you could plug into any CICD pipeline, whether it's get
Speaker 3: hub or get lab or genkins or whatever. So I've
Speaker 3: used a lot of those, and you know, so you
Speaker 3: sort of build up your pipeline with all these different tools.
Speaker 3: Some of those sometimes are are more sasas than like
Speaker 3: a tool one that I've used is called coke co
Speaker 3: or other co covered tools. Cover alls is not the
Speaker 3: one where they just do like reporting of your test
Speaker 3: coverage stats, but they usually have a thin hook into
Speaker 3: your CICD tool that reports the stats with a mainline
Speaker 3: tool or over a STAPI or something like that, and
Speaker 3: then you log into their web interface to see all
Speaker 3: the fancy reporting you do. So that's kind of a
Speaker 3: hybrid solution there. You know, it's a stable one tool,
Speaker 3: but it hasn't hooked into your existing thing. I think
Speaker 3: what you're talking about is more of a holistic thing
Speaker 3: for a specific use case. So I'm not mistaken, And
Speaker 3: it sounds like it more maybe applies more to like
Speaker 3: a low code or no code type of product, or
Speaker 3: maybe not product, but like problem solving. I don't know how,
Speaker 3: I don't know if I understand exactly what you're saying.
Speaker 1: What I'm seeing is a lot of enterprise companies that
Speaker 1: are having trouble hiring people or maybe just don't want
Speaker 1: to hire, you know, like as many people as they need,
Speaker 1: and are starting to outsource to these different agencies and
Speaker 1: different services. So instead of hiring their own Terrorform expert,
Speaker 1: they're going a Terraform cloud and signing up for whatever
Speaker 1: the highest SLA plan is and getting some kind of
Speaker 1: support through that. So I'm not even sure that it's
Speaker 1: so much the software as it is about to support,
Speaker 1: to be honest, you know, like when I'm actually looking
Speaker 1: at it, I'm like, ah, which way is that? So
Speaker 1: I don't know. I don't know because I'm still pretty
Speaker 1: new to these kind of things myself. I am checking
Speaker 1: out Spacelift for some of my like for some of
Speaker 1: my stacks and things, and it does look like it
Speaker 1: will be good, especially after I had a very expensive
Speaker 1: AWS bill a couple months ago last month that wasn't
Speaker 1: even a couple months ago. I've just got like, you know,
Speaker 1: I'm still upset about that. So that's kind of what
Speaker 1: made me start. That's what made start looking at this
Speaker 1: kind of thing, was that I had this really expensive AWS bill,
Speaker 1: and I was like, oh, maybe I should make sure
Speaker 1: my stacks are actually getting destroyed when I think they
Speaker 1: should be getting destroyed. And since I just had a
Speaker 1: two thousand dollars AWS bill, maybe it would be worth
Speaker 1: me throughout a couple hundred bucks a months at it,
Speaker 1: because that's I mean two thousand that's ten months of
Speaker 1: spacelift while I'm looking at it. So I don't know,
Speaker 1: I'm still figuring these things out too, Okay, anybody thought
Speaker 1: I had any answers? Answer is no, I don't. I'll
Speaker 1: just figuring along.
Speaker 3: I'm curious to hear what's CICD tools each of us
Speaker 3: have used, and maybe just brief impressions, you know, if
Speaker 3: you liked incoins or not likedcons and why you know,
Speaker 3: in a sentence or.
Speaker 2: Two, I love and hate Jenkins. You know, Yeah, I've
Speaker 2: used it quite a bit. And the thing I love
Speaker 2: about it is you can do absolutely anything with it.
Speaker 2: And the thing I hate about it is you can
Speaker 2: do just absolutely anything with it. So I like, yah, yeah, yeah,
Speaker 2: I had a scenario. I know this is longer than
Speaker 2: my two sentences, but I'm gonna do it anyway. That
Speaker 2: a few years back, I was working with a company,
Speaker 2: and I think this ties into your statement about which
Speaker 2: c ICD you know do the simple one. We had
Speaker 2: one We were moving this application from Heroku to AWS
Speaker 2: and Heroku has their own pipeline tool that works really,
Speaker 2: really slick, and I wanted to in moving to ABS,
Speaker 2: I wanted to make sure that we didn't lose any features,
Speaker 2: and so we went with Jenkins. And the scenario was,
Speaker 2: you know, you would write your code, you would push
Speaker 2: it up, you would run the tests, all pretty straightforward stuff.
Speaker 2: But then when you opened up a poor request, we
Speaker 2: would create a poll request environment, so we would spin
Speaker 2: up the Docker containers, provision the databases, create the load balancers,
Speaker 2: create the route fifty three DNS entries, all of that
Speaker 2: stuff so that anyone who was reviewing your code had
Speaker 2: a working environment to see what the actual end product
Speaker 2: looked like. And then when that poor request was either
Speaker 2: merged or closed, it tore down that environment, and that
Speaker 2: kicked off the build process to build the Docker images,
Speaker 2: pushed those up into the ABS Docker registry, deployed to
Speaker 2: the staging environment, and all of that kind of stuff.
Speaker 2: So it was pretty There was a lot of moving
Speaker 2: pieces and nothing really had the ability to do that
Speaker 2: without writing your own Python accessing the photo three library.
Speaker 2: And so that's where Jenkins came in handy, because you
Speaker 2: could just give it this application or this code that
Speaker 2: accessed Boto three and it would do it, and then
Speaker 2: you could secure that with im permission, so that Jenkins
Speaker 2: only had access to what it needed to, which was
Speaker 2: still a lot, because it had access to S three
Speaker 2: and rds and firegate and Route fifty three and the
Speaker 2: load balancers and all that kind of stuff.
Speaker 1: That is cool, though Human in the loop, well, I
Speaker 1: like it, yeah, And.
Speaker 2: The part that I hate about that is when you
Speaker 2: get to that level of Jenkins, you know, you write
Speaker 2: your Boto three stuff, and then you have a Jenkins
Speaker 2: DSL that knows how to call and access that, and
Speaker 2: when there's an error in it, you get a Java
Speaker 2: stack stack trace, so a Java stack trace that refers
Speaker 2: to a DSL that refers to a Python Vota three library,
Speaker 2: so that any error messages you got were just adsbsolutely
Speaker 2: worthless fascinating.
Speaker 3: Have you used any others or is that really the
Speaker 3: main one you used?
Speaker 2: Will That's the only one I've really gone deep with.
Speaker 2: I've used a Circle, CI and Travis and get lab,
Speaker 2: but all for just really a high level basic CiCe stuff,
Speaker 2: nothing that extensive.
Speaker 3: My first introduction to CSD was a Travis. I was
Speaker 3: working at a company several years ago where we actually
Speaker 3: hosted our own on prem Travis Travis Enterprise. That was
Speaker 3: the first time I used CI, and I really, I
Speaker 3: really liked it. I later came to prefer the get
Speaker 3: lab model, which maybe Travis has changed since then, but
Speaker 3: get lab has the concept of pipelines and Travis didn't
Speaker 3: really have that concept. So I prefer the pipeline approach.
Speaker 3: In fact, if somebody was an asked me, how would
Speaker 3: you choose a good CI tool, I would say that
Speaker 3: could be the first thing. Look, so does it have
Speaker 3: pipeline support because it doesn't have to. I don't know
Speaker 3: if Jenkins does. I don't know if he uses that model.
Speaker 3: Okay it does. Yeah, well that that model is available.
Speaker 3: Like you just log into Jenkins and point and click
Speaker 3: your way tout success. You don't get the pipeline model,
Speaker 3: but there is if you write your own pipelines, and
Speaker 3: that's what we did there. There was a we created
Speaker 3: a Jenkins library that executed the pipeline and then each
Speaker 3: development team had a YAML file in their repo where
Speaker 3: they could specify how many running containers they wanted, the
Speaker 3: memory and CPU allocated to each doctor container, and for
Speaker 3: auto scaling, could specify your metrics of whether you scaled
Speaker 3: on memory, CPU, or latency and what that value was.
Speaker 3: I've also used Circle a little bit on one of
Speaker 3: the open first projects I help with, so I don't
Speaker 3: have extensive experience. My main experience with Circle is that
Speaker 3: I find the dashboard confusing, but that's probably just because
Speaker 3: I haven't used it enough. I've also used Codeeship at
Speaker 3: a company several years ago. I didn't really like that
Speaker 3: one very much. We actually migrated from Codehip to Travis.
Speaker 3: We broke all the rules that we'll talk about it. Wait.
Speaker 3: Actually at that company, at that company we had we
Speaker 3: were using like three different SI tools, like each team
Speaker 3: would pick their own, and so it was more of
Speaker 3: a consolidation thing than it was a real migration. And
Speaker 3: it wasn't doing anything complicated. It was doing what what
Speaker 3: Dillian talked about, basically, make test and make build, and
Speaker 3: that's kind of all it did. So I also used,
Speaker 3: of course get hub actions and get lab. I talked
Speaker 3: about both of those. There's probably few others I've dabbled with,
Speaker 3: but those are the ones I had the most experienced
Speaker 3: with and and the main reason I like get lab
Speaker 3: is because, well two reasons really. The first is I
Speaker 3: like the pipeline model, and second is I like to
Speaker 3: get lab interface in general. I feel like it's a
Speaker 3: more complete solution in gethub, although GitHub is catching up
Speaker 3: since la Kasoft's acquisition, they've been adding a lot of
Speaker 3: new features. But yeah, so that's that's my experience.
Speaker 1: I think. Yeah, I've used all the kind of all
Speaker 1: the big names. I've used drinkings when I've used something
Speaker 1: in house. Travis, I think Travis was the first one
Speaker 1: that I used for like a big open source project
Speaker 1: because we were using Travis and that's it was there
Speaker 1: and we're I think it was it was probably one
Speaker 1: of the first, like I mean, I kind of think
Speaker 1: it was one of the first like publicly available ones
Speaker 1: right that was free, that was available for open source projects.
Speaker 1: So we were on that, and then I think we
Speaker 1: switched to Start because the Circle guys were throwing some
Speaker 1: cash at open source projects, and you know open source projects,
Speaker 1: we always need the cash. So we're there. And then
Speaker 1: actually I've been kind of, i suppose breaking all the
Speaker 1: rules by kind of rolling a lot of my own
Speaker 1: solutions lately, which is that I do a lot of this.
Speaker 1: I do a lot of like human in the loop
Speaker 1: and things that specifically need a really specific, almost editorial
Speaker 1: workflow where there has to be at some point a
Speaker 1: person who approves something, and they can't like they're scientists usually,
Speaker 1: so they can't you can't give them like a stack trace,
Speaker 1: all right, Like that's not okay, we can't give normal
Speaker 1: people stack traces, all right, Like we're we're not doing
Speaker 1: that over here. So you know, the one thing that
Speaker 1: I've really been liking lately is to use I'll actually
Speaker 1: use airflow Patchy Airflow, which is it's a workflow orchestration engine,
Speaker 1: so it's I mean, it's there. It builds basically data pipelines.
Speaker 1: But then because it's built on Python and flask, which
Speaker 1: is a like a web framework that I'm familiar with,
Speaker 1: it's really easy to build like plug ins and additional
Speaker 1: interfaces and things like this. So let's say I have, like,
Speaker 1: you know, one of these pipelines that's generating tools, and
Speaker 1: then one of those tools and then another tool will
Speaker 1: generate some kind of QC metrics on this pool. At
Speaker 1: some point a person has to make a decision on this,
Speaker 1: and sometimes it's a part of the data engineering workflow,
Speaker 1: and sometimes it's a part of a software workflow that is,
Speaker 1: you know, still informed by a data engineering workflow because
Speaker 1: the software has to produce correct data. So it's all
Speaker 1: kind of there's a lot of gray area between the
Speaker 1: data engineering and the softlow the software engineering, is what
Speaker 1: I'm saying. So I've really been liking that lately because yeah,
Speaker 1: because again I can give like people who need actual
Speaker 1: like web interfaces, I can give them pretty easily web interfases.
Speaker 1: But at the same time, it's all just really integrated
Speaker 1: into the workflow, and I can automate as much as
Speaker 1: can be automated. So normally it will look like, you know,
Speaker 1: we have like three steps, and then there will be
Speaker 1: some point where human has to approve something. The human
Speaker 1: will get like a yes no kind of form, and
Speaker 1: then that feeds back to airflow and we'll either pick
Speaker 1: off the rest of the workflow or cancel it, depending
Speaker 1: on how that how that ends up going. If I'm
Speaker 1: not doing anything that's complicated, though, I tend to just
Speaker 1: use githab actions because it usually is just like Make builds,
Speaker 1: Make tests.
Speaker 3: And the problem with the problem with starting with whatever
Speaker 3: it's easy because it's not complicated, is six months later
Speaker 3: it's complicated, and then you're married to that tool. But
Speaker 3: that's the problem with like make files versus basscripts versus
Speaker 3: Pearl or Python or something like that, like, oh, it's
Speaker 3: just simple, I'll just use Make tests, and then six
Speaker 3: months later you have the six thousand line make file
Speaker 3: and nobody can understand it. And I should have done
Speaker 3: this in Python.
Speaker 1: I don't think I'm quite ready to give up my
Speaker 1: Make files, Jonathan.
Speaker 3: I don't think that's I like me, although I like Make,
Speaker 3: but I have done some really nasty make stuff before,
Speaker 3: stuff I would these days never do again. I think,
Speaker 3: I mean, I have no problem. Yeah, I'm pretty sure
Speaker 3: Make is touring complete and it should not be.
Speaker 1: Files from like cold Dead Hands.
Speaker 3: No.
Speaker 2: No, I think there's a really good lesson hidden in there. Though.
Speaker 2: One of the keys I think to making successful pipelines
Speaker 2: is to wrap all of your individual steps up into
Speaker 2: a smaller command and then just have the pipeline call
Speaker 2: that command. So when you look at the pipeline, it's
Speaker 2: like it does these five things, and now those five
Speaker 2: things might represent a bunch of other smaller tasks, but
Speaker 2: I think that's still more easily understood by someone coming
Speaker 2: in to work on it. Then if you have a
Speaker 2: CICD pipeline that has six hundred individual steps in.
Speaker 1: It, well, that's what I like about air flows that
Speaker 1: right out the gate you get a picture of the
Speaker 1: directed a syclick graph, so you know step one, step
Speaker 1: two depends upon step one, which if that's successful, it
Speaker 1: goes here. If it's not successful, it goes here. Like
Speaker 1: it gives you a picture of that right in the
Speaker 1: dashboard so you can see it, and then you can
Speaker 1: actually click on each individual part and it like colors
Speaker 1: it based on whether or not it was successful and
Speaker 1: this kind of stuff. So I just find it much
Speaker 1: more intuitive for like a person to deal with rather
Speaker 1: than a any of the other CICD work cloths. I mean,
Speaker 1: I'll use them because they're there and they're free, but like,
Speaker 1: if I'm going to build anything complicated, most of the
Speaker 1: time these days, I just go build with an airpluck.
Speaker 2: I think get lab does a fairly good job that
Speaker 2: of visualizing the pipeline and the you know how how
Speaker 2: to build or how the pipeline is going to run,
Speaker 2: which steps are dependent and which one failed?
Speaker 1: Oh my goal trial. Okay, I don't actually think your
Speaker 1: lab is probably one of the ones I haven't only used.
Speaker 3: One thing I really like about get lab is that
Speaker 3: it so easy to start your own runners. I run
Speaker 3: some private projects that aren't eligible for their free, open
Speaker 3: source unlimited minutes or whatever, and so every month I
Speaker 3: get this thing. If you here, I don't know how
Speaker 3: many minutes or four hundred minutes or whatever, upgrade now
Speaker 3: for like twelve dollars or whatever it is. I'm like, no,
Speaker 3: I just I have a Hetzner dot d account for
Speaker 3: I think three dollars a month, and if run a
Speaker 3: get lab runnerund there and I connect to that, and
Speaker 3: so for three bucks a month, I have unlimited minutes.
Speaker 3: It's super easy to do that. I haven't logged into
Speaker 3: that server for probably two years's probably been hacked by now,
Speaker 3: I don't know it's mining block chain. I think get
Speaker 3: hub Actions is going to be adding support for the
Speaker 3: maybe they have already added the ability to run your
Speaker 3: own runners too, but I mean that's also really nice
Speaker 3: if you have special hardware you need to be using,
Speaker 3: if you need a GPU to run your tests or something,
Speaker 3: or you want to run on an ARM system or
Speaker 3: Raspberry Pie or something like that, that's something that that's
Speaker 3: the reason we didn't discuss at the beginning. If you
Speaker 3: have special needs like that, you're going to be limited
Speaker 3: in which CIC tools you can use. Like try to
Speaker 3: see I believe offers Linux and MAAC. Maybe they do
Speaker 3: Windows now I'm not sure. But if you need to
Speaker 3: run your code against Windows on Windows servers or a
Speaker 3: Raspberry Pie or whatever you're gonna have, you're gonna be
Speaker 3: limited and the choices you have. So that's one nice
Speaker 3: thing about being able to run your own runners that
Speaker 3: get love gives you, and I think ge hub actions
Speaker 3: either does or soon will. You can set up your
Speaker 3: own runners to access your hardware encryption devices, or your
Speaker 3: MPEG encoders or whatever random kind of custom stuff you
Speaker 3: might need, So that that's something to think about too.
Speaker 2: Yeah, specifically, if you're building iOS apps, I've dealt with
Speaker 2: that quite a bit, and that's a huge pain because
Speaker 2: it's got to be done on a Mac. So you're
Speaker 2: either using a Mac that you have physical access to
Speaker 2: as a runner, or there are some third party services
Speaker 2: that will integrate with your CICD and do the build
Speaker 2: for you, but all the ones I'm aware of are
Speaker 2: are relatively expensive.
Speaker 3: Yeah.
Speaker 1: I got into like three D modeling a little bit
Speaker 1: is my quarantine hobby, and that immediately made me think, like,
Speaker 1: how do video games like when people make video games, right,
Speaker 1: because they're the point is is that you're supposed to
Speaker 1: be able to release them on like all the platforms
Speaker 1: now right, Like you see a video game, it's not
Speaker 1: just like on PlayStation or Nintendo or whatever. It's it's
Speaker 1: on like everything. It's on. It's for games, it's for
Speaker 1: the switch, it's for all of them. So I always
Speaker 1: kind of wonder, like, well, how do they do their testing?
Speaker 1: I kind of find a whole lot on it. It
Speaker 1: seems like every it's still like a very closely guarded,
Speaker 1: secreted video games. I did see a little bit that
Speaker 1: like Unity uses AWS because AWS have actual Mac like instances,
Speaker 1: Like you can run Windows instances and you can run
Speaker 1: Mac instances. But for the rest of them, I don't know,
Speaker 1: just build and hope it runs and people can install it.
Speaker 2: If you listen to the comments from the gamers. Then, yeah,
Speaker 2: that's the exact strategy. Build it and let your customers
Speaker 2: figure it out.
Speaker 1: Yes, your customers need the acceptance testing. That's my business
Speaker 1: model too. You know what.
Speaker 2: I haven't done. That's a really good point. I haven't
Speaker 2: done any work in the game industry. But yeah, there's
Speaker 2: a lot of compiling for different platforms that happens there.
Speaker 2: And I don't know what to compile time or to
Speaker 2: build time for your average game is.
Speaker 1: I don't know. Let's get pretty involved, Like I feel like,
Speaker 1: just like so really small scenes and like small little
Speaker 1: games and unreal and that's it's backed by C plus plus,
Speaker 1: so you actually have to go through and compile everything,
Speaker 1: and even for just like a you know, something really small,
Speaker 1: it can take a while.
Speaker 2: Yeah, any game you download these days is it seems
Speaker 2: like it's a minimum of five gigs. A lot of
Speaker 2: that's video assets, I would assume, So I don't know
Speaker 2: if those are involved in the compiling process or if
Speaker 2: you can compile your video assets once and then just
Speaker 2: compile the run time for each specific platform. I don't
Speaker 2: know how that works.
Speaker 3: Next week on adventures and game development.
Speaker 2: Right, let's see who we know that works in that
Speaker 2: industry that we can get on the show.
Speaker 1: No, there was a couple of plug ins and like
Speaker 1: they weren't they weren't very well developed for different platforms,
Speaker 1: and I know, like an f C plus plus that
Speaker 1: I was like, huh, maybe this should be my new calling.
Speaker 1: Just go and then fix all the platforms and set
Speaker 1: up the CICD and make plugins that actually compile to
Speaker 1: the different the different platforms and sell them on the marketplace.
Speaker 1: And what am I doing all this data science stuff for? Anyways?
Speaker 1: And then I figured out, you know, very quickly, that
Speaker 1: it wasn't that simple hopes and dreams, most as most
Speaker 1: things tend to be, It was not that simple. Yeah,
Speaker 1: I think if you have like specific HERD, if you
Speaker 1: have specific hardware or specific security considerations, you should really
Speaker 1: like look into your choice and not be nearly as
Speaker 1: asual is I tend to be with like, well this
Speaker 1: one looks good today. You know, it's not the action
Speaker 1: that I need this morning, So we'll use that, you know,
Speaker 1: maybe actually go through and do your research and think
Speaker 1: about it a little bit and see which one kind
Speaker 1: of or if any of them support the different platforms
Speaker 1: that you need.
Speaker 2: Unless you have a coupon because we know if you
Speaker 2: have a coupone, you're using it.
Speaker 1: Oh so you know that's one thing with thought. I
Speaker 1: signed up for the free prile of Spacelift and they
Speaker 1: immediately like they contacted me on like day, They got
Speaker 1: me in a Slack connect group with them. They will
Speaker 1: like call us if you're not call us, but you know,
Speaker 1: like message us if we need anything. And I was like, oh, well,
Speaker 1: this is almost as good as a coupon. So do
Speaker 1: we have any like good projects for people starting off
Speaker 1: and see ICD? I think you were talking about that
Speaker 1: a little bit early, Terronic.
Speaker 3: Yeah, I'm actually gonna do a course probably starting on
Speaker 3: April about this. I'm going to walk people through, so
Speaker 3: stay tuned. I'll give them more details as it approaches.
Speaker 3: But the concept for my course is I intend to
Speaker 3: walk people through publishing an app. Maybe it maybe you
Speaker 3: build a PHP app or or something, or if you
Speaker 3: haven't built one, we get to see this work press
Speaker 3: or react some react to do NBC or something like that. Yeah. Yes,
Speaker 3: to take a simple app that uses the database and
Speaker 3: deploy it to Kubernetes using CICD with the sort of
Speaker 3: review environment that will talk about, so you can every
Speaker 3: time you create a ProQuest, it'll pop up this temporary
Speaker 3: environment you can play around with, do your testing on,
Speaker 3: and then when you merge it will teare it down
Speaker 3: to it away. So I think that's that's a really
Speaker 3: good way to start. Pick a simple app if you've
Speaker 3: written one. If you haven't written one, you can just download,
Speaker 3: like I said, to do NBC or something like that,
Speaker 3: some sort of test app or work press or whatever, PHP,
Speaker 3: my ad Man, anything like that, anything that's an app
Speaker 3: that at a web or basis of a place to start,
Speaker 3: and just deploy it somewhere and deploy automatically. I don't
Speaker 3: mean like type cupit, cuddle whatever, you'll do that in
Speaker 3: the process, but get it automated so that when whenever
Speaker 3: you commit, you run your tests, and when you hit
Speaker 3: merge at automatical ploys. Which tool you use is less
Speaker 3: important than just getting that process done. So I'll probably
Speaker 3: do the course of get lab to CEI, but I'm
Speaker 3: going to try to keep it as generic as possible
Speaker 3: and simple Bash commands so that anybody who doesn't want
Speaker 3: to use a good lab can apply the same knowledge
Speaker 3: to GitHub actions or to genkins or whatever other tool
Speaker 3: they want.
Speaker 2: Yeah, I would agree that's. Yeah, the act of taking
Speaker 2: something from code to deployed is a huge learning experience
Speaker 2: for somebody interested in getting getting into DevOps. And another
Speaker 2: key point that you mentioned there is if you don't
Speaker 2: know how to write code, or if you haven't built
Speaker 2: an application, that's not a blocker because in the software
Speaker 2: engineering world, like one of the most common things for
Speaker 2: learning a new language is to create a to do apps.
Speaker 2: So if you go to GitHub and search for to
Speaker 2: do apps, you're going to find thousands of them that
Speaker 2: you can download that meet those criteria of having a
Speaker 2: web user interface and a database back end and just
Speaker 2: use that exactly.
Speaker 1: Yeah. I would also add try to get at least
Speaker 1: familiar with the idea of matrix builds, which is where
Speaker 1: you want to kind of build things with different options,
Speaker 1: even if it's kind of you know, you're sort of
Speaker 1: artificially putting it in there. So an example of that
Speaker 1: would be you take your you take your to do
Speaker 1: app and you install it sent hosts, Ubuntu an Alpine,
Speaker 1: and you do that through doctor containers or however you're
Speaker 1: going to do that. But that would be three variants
Speaker 1: of the same package. And I'm sure that you know
Speaker 1: like things will come off along the way there and
Speaker 1: then you'll kind of you'll kind of learn about that
Speaker 1: process as well.
Speaker 3: That's a really good thing. If you're building a shared
Speaker 3: library and you don't know who's going to be using it,
Speaker 3: you know you want to make sure it works on Mac,
Speaker 3: Windows and Linux, for example, and you want to make
Speaker 3: sure it works on three different versions of Linux, then
Speaker 3: that's exactly the time when those matrixes are matrices are
Speaker 3: are called for.
Speaker 1: Oh well, does anybody else have anything to add or
Speaker 1: should we just go to picks?
Speaker 3: Just do some pics every week?
Speaker 1: One of us, all right, Donald, who wants to go
Speaker 1: for astronomers? Not me?
Speaker 3: I'm not.
Speaker 2: I've got I'm ready this week. So I'll stall for
Speaker 2: time and share way more details about my picks than
Speaker 2: what I should to give Jonathan an extra few minutes.
Speaker 2: So I've got two picks this week. First one is
Speaker 2: a movie called The Blacksmith and the Devil. It's on Netflix,
Speaker 2: and its super cool movie about a surprise no school
Speaker 2: over alert here. It's about a blacksmith and he captures
Speaker 2: this demon that he thinks is responsible for all the
Speaker 2: trauma and turmoil in his life. And he's got this
Speaker 2: demon captured and he tortures him. But then there's this
Speaker 2: really cool plot twist at the end that has you going, oh, dude,
Speaker 2: right on, let's go. And so I won't tell you
Speaker 2: any more than that to keep from ruining the suspense.
Speaker 2: But it's a really cool movie. The Blacksmith and the
Speaker 2: Devil really well done. Only on Netflix as far as
Speaker 2: I know, but I only watch about maybe two or
Speaker 2: three movies a year, and this was one of them.
Speaker 2: So it was it was cool. It was a good
Speaker 2: couple of are spent. The other thing for my pick
Speaker 2: this week is these eye drops called Acuity. So like
Speaker 2: as the as the old man in the group, I
Speaker 2: feel obligated to tell you all that as you get older,
Speaker 2: your eyesight's going to go bad and you need like
Speaker 2: a close yeah, Like you need a close circle of
Speaker 2: friends who will call you out so when you're heading
Speaker 2: into the store with your sunglasses on top of your
Speaker 2: head and your eyeglasses on your face, that they stop
Speaker 2: you and go, no, no, no, we don't do that
Speaker 2: around here. So you need that group of friends in
Speaker 2: your life. But otherwise you can get these eye drops
Speaker 2: that just came out in fall of twenty twenty one.
Speaker 2: If you're like me, whenever you got old and your
Speaker 2: eye your eye drop, eyesight for reading went to hell.
Speaker 2: These eye drops are specifically made for that. So you
Speaker 2: put these eye drops in and then for like the
Speaker 2: next four hours, my vision is like it was ten
Speaker 2: years ago. And so it beats wearing glasses, or more importantly,
Speaker 2: it beats trying to figure out where I left my glasses.
Speaker 2: So those are my two picks for the week. Do
Speaker 2: you ever wonder where you left the eye drops so far? No,
Speaker 2: I've been using the eye drops for about a week now,
Speaker 2: so we'll see what the long term impact of that is.
Speaker 2: But right now, I just keep them in the bathroom
Speaker 2: and whenever I get up and get ready for work,
Speaker 2: I'll drop them in, and then around lunchtime I'll go
Speaker 2: and put the next set in because they only last
Speaker 2: for about four hours.
Speaker 3: I haven't pick ready, now go for it. So I
Speaker 3: just finished reading a book. It's an old book, but
Speaker 3: it's a classic book. It's a good book. It's been
Speaker 3: made into two movies. That's how old it is. It's
Speaker 3: been around long enough to make two movies out of it.
Speaker 3: That's all quiet on the Western Front and It's a
Speaker 3: fictionalized story of a German soldier during World War One,
Speaker 3: and it really just tells the story of more the
Speaker 3: feeling of combat in World War One and the trenches
Speaker 3: more than like a military story like this. There's no
Speaker 3: story conquest or whatever. It's here's what it was like
Speaker 3: to live in the trenches and deal with death and
Speaker 3: with the fear of death and not enough food and
Speaker 3: that sort of stuff. So it's a really good book. Obviously,
Speaker 3: it's I mean, probably everybody's heard of it. You don't
Speaker 3: want to read or listen to the book. Wat's the movie.
Speaker 3: There was one that came out in the twenties, I
Speaker 3: think black and white, and then it was remade again
Speaker 3: in the nineteen seventies. So I me recommended nice cool.
Speaker 1: Well, I'm going to pick The Wheel of Time. It
Speaker 1: is one of my favorite book series. I reread it
Speaker 1: every couple of years, and I wanted rereading it again
Speaker 1: because I tried to watch the Amazon series of The
Speaker 1: Wheel of Time and I couldn't quite get through that.
Speaker 1: But I just decided to go back to the source
Speaker 1: material and go back to the books reread those. It
Speaker 1: really is like one of my favorite series. There's a
Speaker 1: couple there in the middle that you could probably skip
Speaker 1: and like, you know, you would be fine in terms
Speaker 1: of plotline and things happening and all that kind of stuff.
Speaker 1: Just get the cliff notes. But I don't know, they're
Speaker 1: really great. They're one of my favorite series. Go for It.
Speaker 1: So there's that, and then I think on tech pics,
Speaker 1: I've been playing with this program called Hissora, which is
Speaker 1: like as it's one of these web frameworks, you like
Speaker 1: throw it on top of the database and then it
Speaker 1: just kind of spins up all your APIs. But one
Speaker 1: kind of thing is that it has subscriptions and real
Speaker 1: time events, and another nice thing is you can also
Speaker 1: pull in like additional APIs, so if you have if
Speaker 1: you have like a series of micro services, you could
Speaker 1: use to have to kind of tie them all together
Speaker 1: and then it could handle, you know, throwing your authentication
Speaker 1: tokens and stuff like that around. So I've just been
Speaker 1: playing a little bit with that. It's been fun. So's
Speaker 1: it's a good program. I recommend it. Go see it.
Speaker 1: I think it's got lots of stars on GitHub. And yeah,
Speaker 1: that's it for me. All right, guys, Well that's a wrap.
Speaker 1: Thanks and talk to you next week, all right, See
Speaker 1: you guys,