How'd you like to listen to dot NetRocks with no ads? Easy? Become
a patron For just five dollars a month, you get access to a private
RSS feed where all the shows have no ads. Twenty dollars a month will
get you that and a special dot NetRocks patron mug. Sign up now at
Patreon dot dot NetRocks dot com. Hey, Carl and Richard, here with
your twenty twenty four NDC schedule. We'll be at as many NDC conferences as
possible this year, and you should consider attending no matter what. Ndcoslo is
happening June tenth through the fourteenth. Get your tickets at ndcoslo dot com.
The Copenhagen Developers Festival happens August twenty sixth through the thirtieth. Early bird discount
ends April twenty sixth. Tickets at Cphdevfest dot com. Ndcporto is happening October
fourteenth through the eighteenth. The early bird discount ends June fourteenth. Tickets at
Ndcporto dot com. We'll see you there, we hope. Hey, guess
what it's dot net rocks. Well you knew that because you pushed the play
button. I'm Carl Franklin and I'm Richard Campbell, and we're gonna have a
good show today. Barry O'Reilly is here. We're going to be talking about
fragility, or anti fragility as he talks about it. But how are you,
sir, up on the coast back home? I didn't power out it
yesterday for a couple hours of my new ups rig. You know, you
find every mistake. Yeah, right, So the network stayed up perfectly,
but it's like, oh, you plug that monitor into the surge protected outlet,
not the power backed up. So over the course of the two hours,
I had my head lamp on and I was running around making fixes on
stuff so that when it all came back, everything would be better. That's
so so like you, It's so yes, it's like, oh, powers
out now, I'm busy. I was busy last week trying to come up
with a solution to a problem that I couldn't believe was a problem. But
that's my better no framework, So let's roll the music. Also, apartment,
what do you got all? Right? So everybody does authentication authorization in
a myriad of ways in twenty twenty four, not one right way. That's
for sure. Oh no. But you know, if you just select when
you're when you're creating a Blazer application, if you select individual accounts, you
know you're basically buying into the Microsoft sp net Core identity system and it and
it wants a database, right, and you can create the template is really
good now. And I got to say thanks to Jeremy Lecknis and he's not
working on that now, but he's one of the guys behind all the new
UI in the Blazer template that all Razor pages now and it's all in there.
You don't have to scaffold anything, well most of it. But you
know, roles is something that seems like an antiquated thing, right because everybody's
using claims and building their own claims and we got claims and servers and two
tokens and all that stuff. But there's still a case to be made for
simple roles like I want to know if you know, Joe is in the
admin role so that he can change things and somebody else isn't simple things like
that, And when it turns out that those are just strings, they're just
tags, and so it's very easy for you to they've really turned into claims
effectively a role in a claim is not that different. Yeah, and in
a role is a type of claim. Right. But here's the deal.
If you're doing a Blazer server application, you just add one line of code
to that template in your program cs, you know, ad roles of identity
role, then you're fine. You do that in a web assembly applications a
little bit more complicated. Yeah, now, but there's but but in the
template, there's support for roles in the you know, in the in the
server side and the client side, things that talk to each other. It's
just that Oops. They took that out right before dot net eight went to
you know, in October, right before dot net eight went live, because
there were situations where you have, if you have multiple claims with the same
name, they were interfering with each other, and instead of fixing that,
they punted on it. And it's still an open issue. So essentially you
don't have roles. Wow. So basically I figured out a workaround for it.
Wrote a component that's free, and it's an MIT license component called Blazer
Authorized Roleview And it's a replacement for the authorized view component that you can use
to you know, authorize markup based on a role, except that it works
in a web assembly. Uh, and it's probably a workaround. They'll probably
fix it or they'll come up with some other solution. Something will change for
nine. Yeah, maybe hopefully. So that is it. This is show
eighteen ninety six. So if you go to eighteen ninety six dot pop,
dot me, that will take you to my repo. And there's also a
Blazer train about it. So that's what I got. Excellent solve problems.
Cool. Who's talking to us, Richard crowded commentator Show eighteen ninety four,
the one which is just recently with Carl Geittz talking about programming and speech,
and this comment comes from Rob who says, Hey, I really appreciate your
coverage of artificial intelligence. It's positive and tamps down fear. As an instructor,
I hear a lot of concerns by other instructors about how AI will affect
students, mostly from the perspective that a students are using AI to cheat,
but from a programmer's perspective. To me, it's just another tool, and
it's a really cool tool, but a tool nonetheless, And the sooner as
instructors that we know how to use this new tool in the context of whatever
job. We are teaching students to do the better. So I loaded up
gethub copilot. I use it, and that has made me realize that,
yes, it makes me faster, and yes the student could still figure out
syntax more easily, but what's wrong with that? They still need to learn
the syntax to complete the code. They still need to learn how to read
the code, and they still need to understand how complex frameworks are structured and
how to properly and effectively use them. And most importantly, they can still
look forward to hours and hours of WTF research the figure out how to solve
complex problems that I give them. Here's zer kind of instructor. It seems
like most of my day is WTF. Yes, a lot of WTF research,
right, just turn on the light. That's got to be a hashtag
man, and AI is likely to never be able to figure everything out.
That's our job as human beings. You know. I was recently on a
different podcast. I was on Jeff Plermo's podcast, and I said, listen,
you know your job's not to write code. Your job is to find
solutions, right, and if there's a tool that helps you make they get
to that solution faster, you should use it. That's the responsible thing to
do exactly. So, Rob, thank you so much for your comment,
and a copy of music Coby is on its way to you. And if
you'd like a copy of music Cobe, I write a comment on the website
at dot at Rocks dot com or on the facebooks you publish every show there
and even comment there at e reading on the show, we'll send you copy
of music. Go bye music to code by still going strong. Stuff in
use by thousands, I would say thousands of people based on my sales.
Dude, it's in this room every day, like don't don't be crazy,
Like that's the truth. When I'm home and I got to work mostly writing,
to be honest. Yeah, that's what's on Cool Blue. I love
Blue. Blue puts me Blue puts me in the zone. Like click,
that's the first one. Yeah, okay, Well, you can of course
follow us on Twitter if you want. But the cool kids are hanging out.
I'm mastered on these days. I'm at Carl Franklin at tech Hub dot
social, and I'm Rich Campbell at masterd dot social. And you can find
all the ways to get in touch in the at Carl Franklin dot com All
right, let's get to our guest who hasn't been on in sickx years Berry
six years. Barry O'Reilly is the founder of Black Tulip Technology and creator of
residuality theory. He has held chief architect roles at Microsoft's Western Europe Consultancy practice.
Among others. He's been Microsoft's IoT Tap lead for Western Europe and worldwide
lead for Microsoft's Solution Architecture Community. He has also been a startup CTO and
was founder of Sweden's Azure user Group. He's currently finishing up a PhD in
complexity science and software design. I really want to know about complexity science and
in particular how it helps me oncomplexity five. Yeah, welcome Berry, Welcome
back. Yeah, heeers. It's been a long time. A lot,
a lot has happened. Last time I was on, I was talking about
the con accept of anti fragility and software and how we can build a software
that appeared to survive on its own even though it wasn't designed for things,
and that all of those ideas came out of work you did at Microsoft trying
to teach new architects. So how do we turn good developers into into great
architects kind of thing, and that I wrote a paper, and that paper
led to a lot of things, a lot of attention, and I decided
that I wanted to do things a little bit differently. So I went back
to school and the things that I had noticed, the things that i'd written
about, I said, you know, if you don't gather requirements, if
you don't try to predict the future, if you just randomly mix stuff up
and stress and application, you actually end up with an architecture that's better than
what we normally do. And that's a very big statement, you know,
and it's it's fun and it's and I realized that this is the way I've
been working for a very long time. I've given up on process and templates
and all of these things, and and you know, we're we're all just
making stuff up until it starts to spin, until the wheel starts to spin,
and the hamster looks happy, right, And I decide, I decided,
you know, this is a wild claim. And while it's entertaining for
me to go to a conference, getting a stage and talk to people about
these wild claims, wouldn't it be great if if they actually had some depth?
So I went back to university, and I said, I want to
I want to do a PhD. I want to produce academic work that says
this is real. This is a scientific this is a scientific fact, not
just not just someone you know, because if you get up on stage in
I conference, this an amazing thing happens. People believe you. Yeah,
And and I wanted to, you know, make it a little bit more.
I wanted to answer the hard questions. And it turns out that that
that's actually was an incredibly difficult thing to do. And I'm approaching the end
of that process now. I've just released a little book on Lean pub that's
going to be given away as a gift at some conferences across Europe in the
next couple of weeks that talks about the ideas. And so for the last
six years I've been burrowed away writing frantically and rewriting and testing and teaching people
how to do these things. And the basic idea that instead of using requirements,
instead of trying to predict what's going to change and using patterns and encapsulation
and all these methods we've learned over decades, you can simply start with an
architecture that's really simple that you know is wrong. One example I use that,
I say to architects, just what you're allowed one component and all of
your data has to be handled in memory. You're not allowed to do anything
else. There's one way in, there's one way out. This is how
we're going to solve the problem. And if they want to make a change
to that architecture, they have to suggest something that will cause it to fall
apart. And they say, wow, what happens if we read but then
we lose the in memory data stores? Okay, well you can have a
data store that's you know, find you deserve state, you've qualified. Yes,
you've justified persistence. So let's do that, and and the same with
cues and everything else. And you build your architecture up in that way.
And the funny thing is when you do this, when you stress your way
to an architecture, you eventually reach a point, a magical point, where
you can't seem to find a way to break it anymore. And everything that
you come up with actually is survived by the architecture. And that's what I
talked about last time. And so as they've gone away and read an entire
library of books, and complexity, sciences and philosophy and all sorts of weird
and wonderful stuff. I've managed to boil this down to a simple theory that
says, this is why this works. This is why there's an actual scientific
reason why you can produce an architecture just by being incredibly negative. You know.
I just want to point out that writers, you know, writers of
fiction do this all the time. Okay, they just have to sit down
and start writing, even nonfiction, right, Richard. Yeah, sure,
you sit down, you start writing, even if it sucks, even if
you know it sucks, and you throw lots of it away, throw lots
of it away, but it gets your brain going. You have at least
somewhere to start. Yeah. And if you and if you've got somebody watching
over your shoulder saying that's not right, you just want to say, shut
up, you go away. I'm working it out here, man. Yeah.
And that's that's one of the things in these ideas that's a huge,
a huge culture shock for I think for a lot of software developers. Because
we come from computer science backgrounds and mathematics and stam we're used to having a
formula, We're used to being able to say this is the correct way to
do this, This is the structured way, this is this is how you
rationally arrive at an answer. And this, this way of working is a
little bit wild. You're wrong for a very long time until suddenly you're right.
And that's a very very different way of working. And I've shown when
i've shown these ideas, for example, too very senior architects, people who've
built big stuff in azure, and they've said, yes, this is what
we do. We don't have these fancy words or this academic waffle, but
we we this is how we think we're we're you know, negative. And
so if you're an architect, you've ever been called negative, that's that's a
very good thing. It means you're doing right. So it's and so it's
starting to I've started to tour these ideas. I've been at a lot of
conferences. There's a lot of a lot of views coming on YouTube, kind
of growing interest for the ideas, and so I've started to publish books on
the subject, and so they're starting to gather p s and it's it's a
it's a pretty exciting time for these ideas. I mean, I think the
key point you're making here is that you're not afraid to constantly revise the architecture.
But I also appreciate, you know, the insight here, which is,
don't start with an architecture like go minimal so that you're not tearing much
down, you're only adding, yeah, exactly making changes. I tend to
find I go into companies and help them when they're having some troubles with their
architecture. And sometimes you'll find people locked in combat over and oh, this
should be an event driven architecture, or this should be an API driven architecture,
and this is an exting I know, Yeah, how do you know?
Yeah, it's a lot of gut feeling, and not that there's anything
wrong with gut feeling exact how most of us do our jobs, but it's
it doesn't become a very concrete discussion when you're fighting about the differences between these
two different patterns, and so boiling it down to something simple and saying,
tell me, it's something in the environment that's going to push this architecture forces
it to behave in this particular way, and and people they can come up
with stories about what might happen in the environment and what's that happening is that
we're the way that we traditionally build architectures as we try to gather information from
stakeholders by running around and saying, tell me your requirements, and they run
away from us. Most of the time. They don't want to tell us,
and they won't tell us because they don't really know what the future looks
like. And some people say that you can capture risk and you can use
probability, or that some people believe they can predict what's going to change in
an architecture and write all that down. That that's not something I've seen in
my career that's possible for us to do in a complex kind of business environment.
And so the solution of this problem is to just randomly simulate the environment.
And so even if you're just making stuff up and you say, what's
what's going to happen in this in this system, and you make something up,
you can still find a set of conditions in which your architecture breaks,
and once you know where breaks, you can start to make it better.
And it's so I was doing an example of this. I was talking to
my professor yesterday and he said, well, you know, you're calling it
a random simulation. If I just opened a dictionary and just pick a random
word, can you use this to find out something about the architecture? And
I said, so, we were talking about building a system for the National
Health Service in Great Britain and he opened the dictionary and he just pointed a
random word and he said the word is shed. Can you use this to
make this architecture better? And so I said, okay, let's just say
that some doctors and nurses start offering health services in their garden sheds at the
bottom of their gardens. That's absolutely ridiculous, right, that's probably not going
to happen in the near future. I'd never say never. And I said,
I'm talking about medical services. So the question is what impact would that
have in our architecture. Well, suddenly we're distributed. Our workforce is distributed
in a way that it hasn't been distributed before, and that puts challenges on
our architecture with updates, with sequences, with all of those things you know
that get nasty when you distribute your workforce. And I said, okay,
and let's ask what's the path in our architecture. Can we move to that?
Or is our the architecture that we've decided on from the start. Is
it so stuck in it's in its assumption that it can't move to this?
Would it? Would we have to scrap it and start again. And then
as you talk about it, you realize actually there there is a movement in
the United Kingdom where people believe that I guess what you guys call the emergency
room, what we would call A and E. There are people who want
to have lots and lots and lots of small emergency rooms in town centers,
you know, especially on a Saturday night in the UK, where people like
to beat each other up and they need they need, they need quick access
to healthcare. Then and so, while the idea that doctors and nurses will
start practicing in their garden sheds is clearly ridiculous. It's a random data point,
it's a random story, there is actually what we call in complexity science
and attractor there which is a distributed workforce that you start to talk about.
And if you solve for the shed, you solve for this potential change in
policy. You don't have to implement it in your architecture. Now, what
you're looking for is an analogy that can be triggered by any kind of word
yeah, and any kind of random now. And so the theory that I've
worked on is that doing this in this random way on your architecture will actually
give you a better architecture than asking for requirements or risks, or saying what's
going to change and what's not going to change, and what I've done,
then what I'll do when I release, if I pass my PhD after the
summer is showed that this actually happens in a statistically significant way that when when
you do there's a way of measuring with success of these of these programs,
and you can show that this happens almost every single time you use this method,
you get an architecture that shows itself to be able to survive in conditions
which it hasn't been designed for. I mean, I would argue that even
when you do go in with an architectural vision initially, there's a halfway point
in almost every project where that gets substantially revised where you just like, wait,
we now understand this problem much more deeply. We've done a bunch of
testing. Maybe we've got to it may even be a V one, but
you're a few features and you're like, we've thought about this wrong, and
if you're lucky, you're able to make enough changes to be able to deliver
on what the customer actually needed. Yeah, and that's one of the things
that I talk about, is, you know, is that the way that
software architecture is being done today at large companies is kind of in some kind
of gray zone or twilight zone in terms of what is architecture. And there's
a younger generation of our of developers who don't want architects in the room.
They'll know, they'll they'll light their torches and they'll chase you out, and
they'll say go away with your lines and your boxes and your rules, and
chase you back up into your ivory tar. And especially within the agile movement,
there's a there's a distrust of architecture because people have been burned by these
big upfront plans. And and that means that a lot of times we'll will
architecture is seen as something that's just going to emerge over time. And and
and like you said, as you move through a project, you learn and
you find things out, and suddenly one day the team gets to a point
and this can be very far down the line where the architecture won't move anymore.
So we'll we've just discovered that this is all wrong and we can't save
it with a simple refactory. They have to tear it all down. And
then project management will come in and say, no, we're not going to
tear it all down. We'll just keep building on top of this very shaky
platform and we'll call it technical debt. So it sounds good, and that's
where that's where we're going. And what I'm trying what what I'm trying to
get architects to understand is that there are three major challenges to software architecture that
that we really don't know how to handle yet. And those three things are
time, because your architecture we dross and if you look at someone's architecture,
it's two dimensional diagram with lines and boxes, and that will not stay to
the saame ever over time, and there's an idea that it should do.
And then there's change. Over time, things are going to change, and
that's the big problem. That's why time is nasty. And the problem with
that change is that we don't know what it is. You can't sit down
and figure out what's going to change because it's a complex business environment and everything
is moving all at once. So the market and the customers and the employees
and your competitors, everyone's moving. You don't know what's in the future.
You can definitely, if you go back to partner US, you can definitely
say the database connection strengths, we know that's going to change. We can
encaptulate that will be okay if we do that. But you don't know what's
going to happen in your market. And so what I find was that when
we do architecture, architects talk to each other through two dimensional diagrams, but
in their heads they carry around the picture. Everyone has different pictures of many
potential futures, many fears and diets and things that could go wrong. And
so what I wanted to do is to develop a way of designing architecture that
actually made it easy to capture. This architecture has or a life cycle over
time with many changes, and we don't know what those are. And the
concept then becomes that this random simulation that we do where we randomly stress the
thing from any perspective from talking about sheds, then we can arrive at an
architecture that has an expression of uncertainty and time and change in it. And
then we can show in a very simple mathematical way that this architecture is more
likely to survive in conditions which it hasn't been built for. It's more likely
to survive those requirements or new features that pop up out of nowhere, those
changes in the market. And there's a body of evidence for this in the
complexity sciences. So we go back to the nineteen sixties. A biologist called
Kaufman developed a model for how life evolved out of a bunch of amino acids,
how they connect to each other, how they control each other, and
at the end that you can get these very simple amino acids, and at
the end of the day you have something insanely complex like humanity. And we
think, oh, it must have a master designer. Somebody say, must
have architected all of this at the outset, Yes, And herein we're engaging
in these practices where that's see, it is like very difficult to do.
Yes, And but the key to it is the same as the way life
evolved is to randomly stress things until they take on a structure. And if
the structure is wrong, and you keep randomly stressing your architecture, it will
die and you have but luckily we haven't built anything yet, so we can
change it. And what we end up with is we have this concept of
a residue, and a residue is an architecture in one particular timeline we have
this is our residue, and it's whatever is left over in that timeline,
and we change our original structure so that in that timeline things look a little
bit better for us, maybe not entirely solve everything, but just just movable,
just survivable. And what we end up with is describing an architecture as
you start off with your original naive architecture, which is the first thing you
came up with, which if we're honest, you know we all do,
and and a whole bunch of these residues that represent the architecture in different timelines.
And the trick then is to integrate all of those different architectures into one
single set of components. And we do that using some tools that we steal
from the complexity sciences called incidence matrices, and we use these matrices to find
out where where is there hidden coupling in what I've done that's going to stop
this architecture from moving through its different timelines if it needs to. And those
matrices. In the book, I describe seven trigger seven refactoring triggers that you
can see just by drawing a simple matrix and put filled in it with ones
and zeros, and it can actually guide you to component boundaries. And we're
way outside of object orientation or nouns and verbs. Now we're building our building,
our component structure, our service structure based on the different timelines that might
exist in the future of this architecture. And it's a completely different way of
thinking. It sounds very very alien. Compare too. We gather requirements,
we measure all the risks, we follow some object oriented principles or gang of
four or something that then we build the system. This is a much wilder
way of doing architecture, and it takes people a few days of training to
get into the idea and let go of all all the old stuff. And
I'm thinking about things in this very different way, but it does produce fascinatingly,
produces really robust, resilient architectures, and in a way that's empirically verifiable.
You can show that I've done something good with a little mathematical formula at
the end of your work, and that changes everything because as an architect,
for the first time, you can actually show I've done something useful, which
is hard for architects. I can see that if you approach the subject with
a word like anti fragility, yeah, that people's first thought might be,
oh, that requires a whole bunch of front thinking about, you know,
predicting the future, when in fact it's exactly the opposite, isn't it It
is, yes, it's we're letting go of this false belief that we can
predict the future. And I think that's one of the things that enterprise architecture
and waterfall approaches try to sweep under the rug for generations that yeah, we
can predict the future, we can build a scenario analysis that's realistic, and
in this methodology we give up all of that. You've never been able to
do it, We've never successfully done it. If you're working in a system
where you can predict the future, then you probably don't need an architect.
It's everything. Everything's easy in that kind of system. I've never really worked
with any of those, but you know, it's they may exist. I
haven't seen one well and they're I mean, I've certainly worked in environments where
they're building the same app over and over again, essentially, and so the
template does make sense. But yeah, not every most people don't work like
that generally. If you need to build the same app over and over again,
that app order exists and you shouldn't need to build it at all.
Yeah, if you're building the same app over and over again, some SaaS
company is going to come along and steal your lunch. And so if you're
not building that SaaS company, then you're going to be in trouble. Eventually,
you're going to get sass. Yeah, but yeah, I mean in
the end, now, recognizing the body of work that's in front of us
when we're building new software, it's because it doesn't exist, which means we
really cannot predict the future. So it makes far more sense for us to
just explore the space over time and come up with solutions as we go.
It feels to me like the way you're describing this now is that the technical
debt we're talking about is assumptions made early that built stuff we never needed.
Yes, the reluctance to change it once we do realize it because of the
commitments we made to it. Yes, that's very, very dangerous. You'll
find the one way that architects combat this kind of thing in the past is
that we'll pick a platform, we'll pick a pattern, and we say this
is the way that we're doing this, and we're because we because we're very
wise, and we've looked into the future, and we've decided this is how
you solve this problem, right. And what happens is that as soon as
you put that architecture, rub it up against reality, it starts to break
a little bit. And the way we talk about that breaking is we call
it edge cases, and we say we can't possibly be wrong. So everything
that doesn't work is just an edge case. It's just a tiny little distraction
on the edge of our of our knowledge. And that's where you get technical
debt because you start building and hacking to keep those edge cases and you won't
let go of the architecture. And so these ideas give us a chance to
question those assumptions very very early. And the way that the method is set
up, it gives everyone, developers, everyone and the team a chance to
come in and push the architecture around and say, you know, is it
going to hold in this timeline. And one way that we've dealt with that
in the past is we use probability and we say that's never ever going to
happen, right, But by analyzing this from I'm always afraid of that kind
of certainty. Yeah, exactly. But one of the things that we've discovered
is that if you take a bunch of things that are going to happen in
the system, they don't have they don't have to be accurate because every happening
is part of a class of happenings that will push the entire system to the
CM kind of state. And if you solve for that state, then you
solve for many, many, many things. And what we've discovered is that
there's a simple mathematical leverage. If you look at it from a complexity science
perspective, a software application is never actually complex. It's a complicated, constrained,
very ordered thing. We tell it exactly what to do, and it
only does what we've told it to do, and sometimes what we think we
told it to do is a little bit different, but it's very very ordered.
The real complexity in the software system is in the people and the markets
wandering around changing their minds all the time. Generally don't have you know,
class libraries that change their minds, not yet anyway. And and which is
this kind of there's this simple software system has to live in this massively complex
business environment, and there's such a huge difference between them that there's a mathematical
relationship between the number of possible states in them, which means that you don't
have to capture every little bit of information about the complex state. And so
those waterfall approaches to requirements were always wrong on a scientific level, the wrong
way to approach things. And so you only need a sample of those two
to configure your architecture. And that's why we use the random simulation to make
that sample as broad as possible, rather than narrowed by by language or by
engineering metaphors that we've imported. And Barry, I'm gonna interrupt for one moment
for this very important message, and we're back. It's done at Rocks.
I'm Richard Campbell. That's Carl Franklin. Hey, talking to our friend Barry
O'Reilly, who we see regularly at conferences but don't talk too near enough.
And I want to push back a little on what you were saying in the
first half too, because a lot of that sort of random simulation is like
it still feels like could we just call that requirements gathering, you know,
but maybe we're testing it with a bit of code and so forth. This
idea that we're going to catch all this early, I just don't know that
I buy it, Like, I think you are going to find things later
on, and the question is how tolerant are you to being able to adapt
later on? Yeah, And the methodology builds on that. So one of
the most important concepts is that we don't know what's going to happen in the
future. We are always going to get surprised. And if you look at
a business environ a business environment has tens of thousands and hundreds of thousands of
people in it. It has technology, it has markets, it has politics,
all of these things and they're all moving all at once. There's no
way that you can predict the movement of all of these things. And so
the biologist Kaufman showed that when you have many, many, uncountable number of
things connected to each other, you don't actually have to predict every tiny little
state of every tiny little piece of a system. You just have to predict
the presence of a certain number of things called attractors, and so a large
complex system. You don't learn a large complex system by grabbing every individual and
understanding their needs and their wants and their habits and their peculiarities. You observe
the system as a set of attractors that these things all influence each other and
push each other into And so the way that you need to work as an
architect is not to identify everything, but to identify as many attractors as possible.
The moving parts will exactly and once once you expose your architecture to those
attractors, you start to reach something which Kaufman the biologists called criticality. And
criticality is the property of a system where when you start to see that it
survives things that it hasn't been designed for. And so a very good example
is humanity or a human being. We survive things all the time that we
don't even know are there. Our immune systems survive things that they haven't seen
before because of the way they're configured. And Kaufman has this thing called the
NK model that explains it very well. And you have n is the number
of components that you have in a system, and K is the number of
links, and by optimizing for N and K in a system, you improve
the system's ability to survive things that it hasn't been designed for. In software
engineering, we've known this for a very very long time. We know that
if you allow the number of components in your architecture to explode, and going
to be harder to kill them all, but it's also going to be extremely
hard to manage them all if you've got lots and lots of them. And
as you allow the number of connections between those components to grow, it becomes
incredibly difficult to maintee and because you have to traceure your messages as they move
between thousands of different channels. And Kaufman discovered that this exists in biological systems
as well. He calls it the NK model, and he noticed that as
you tweak certain things the number of components, the number of links, or
the way they interact with each other, by constraining how often or when they
interact with each other, then you can control the number of attractors in a
system. And he notes that criticality is the property that you reach when a
system arrives at the perfect balance of these things which we call the edge of
chaos and complexity theory, because isn't the corollary also true that if we have
too few components, we also ended with a lot of inertia because it becomes
difficult to change them. Yes, and it becomes very easy to kill the
system because there's only a few components, and if you take out one,
the whole falls apart. And so there's something we've always known this right as
software engineers. There's something in between the monolith and the micro service, and
it's never quite the same thing in different projects. But you have to find
that line, that magic balance, and that magic balance can be fined through
this random simulation, through stressing the system until you see that it stands up
on its own. And so instead of architecture being I'm going to design and
build this beautiful cathedral, architecture becomes a little more like Bambie on the ice.
Right, We're gonna slip slide, but eventually we're going to stand up
and everything's going to be okay. This is absolutely fascinating to me. I'm
just drinking in everything that you're saying here. And so you have do you
have paper? You said you had a book or do you have papers other
than your book or is this all in your book? So it's all in
the book, which is on Lean pub I have a link to it in
the show notes. Yeah, awesome. The book is based on a series
of seven academic papers that I wrote over the since we last talked, over
the last six years. And what I find was I got a lot of
feedback from academic people who said this is really good, and from a lot
of complexity science people who said this is really good, and from a lot
of software developers who said, this makes no sense to us whatsoever. They're
removing the cheese man. Yeah, and I've built a whole career around introducing
complexity to the beginning of a project. Why are you ruining this for me
with you exactly? And so what I what I did then? So I'm
working with the conference. The conference is called the Domain Driven Design Europe,
and I've spoken there a few times on these ideas, and they called me
last year and said, hey, we want to give a little book out
as a present to all our attendees. Soul you could you write us a
little book? So I wrote the book, and I sat down and said
I have to make this something that that's you know, more in the language
that software engineers are going to grasp rather than all these fancy complexity terms from
biology and philosophy and all of these things. And so that book has has
just been released on Lean pub and it's it's much much easier to understand.
And there's a second book in the works coming that's more on the philosophy.
So I ended up, you know, digging, going all the way back
to the Presocratics and saying, well, this is where we went wrong.
About three years ago in ancient Greece, a guy made a decision and we've
all been suffering ever since. And so there's a fair body of work out
there, and there's going to be you know, a PhD thesis published in
the coming months, and there's going to be, you know, eventually a
bigger book. Just I mean not to dive straight into the academics of this,
but part of what makes this methodology work is that software isn't tangible,
That rebuilding the temple is a button press away, yes, you know,
or accepting a pull request away. We did a lot of planning when building
in stone, because getting it wrong takes a long time to clean up and
replace. It's very difficult to go from straw to wood to brick where it's
just not in software. Yeah, the analogy that I use in the book
is the manufacturing of cars. So and I say, you know, that's
a very planned process, and it has to be it has to all work,
everything has to fit together. And that's a kind of engineering. And
whenever software engineers back in the fifties and sixties, we're looking for credibility,
they wanted to say we're engineers too. So we borrowed a lot of the
engineering metaphor from industries like car manufacturing. And what I say in the book
is, imagine what it would look like to be the designer of a car
if gravity wasn't constant, if aluminium constantly changed its properties, if rubber melted
and reformed randomly. Because that's what our stakeholders do, because we're building on
their ideas, their opinions, and so using the old school kind of engineering
way of thinking about the world doesn't work for us. Software is so connected
and so malleable, so connected to our ideas, and our ideas are fluid
in a way that physical substances just aren't there. We don't have the equivalent
of the laws of physics to constrain us. No't we can do anything.
Anything can happen in a software project. And as soon as anything does happen,
the rigid, sort of stiff software, the mechanical part of what we
do falls apart, very very quickly. And that's what I describe as as
the fundamental issue in software engineering, that things are time changing, uncertainty are
driving what we do. And we've been spending all our time trying to nail
it down and make it certain right when we design, and most of the
time we've been lying. And one of the things I reference in the book
which most people haven't discovered this, and people have been emailing me saying this
is absolutely hilarious. But David Parnas wrote a paper in nineteen eighty six called
a Rational Design Process Hi and Why to feake It? And so this is
one of our is what this is one of our finding fathers of software engineering,
and he says it's impuls in this paper, it's impossible to have a
rational, structured way to design software. It just never happens the same way
twice. We're making everything up as we go along. It's pure mayhem,
and he turns round in the paper and says, but there's a lot of
value in pretending to management that we work in a rational and structured way.
And so basically this article is a call to all software engineers to just lie.
Why just make it look like it's engineering. And that's absolutely fascinating.
Well, because it makes it makes the mortals comfortable, it does. Yeah
right, I mean that's really what it comes down to. And so he
knew this forty years ago, and so we've been lying to management ever since
about how how unpredictable and unstealable our methods and our processes are. And you'll
find that in a lot of the approaches that we use, they promise certainty,
they promise stability, they promise a repeatable process. If you look at
things like TOGAF and things like see If that we have nowadays, it's all
about repeatability and everything being concrete and measured and well understood. And our world
as architects simply isn't like that. Yeah, wow, Mind Blown Partners is
still alive. He's actually in Victoria, Like I'm almost looking across at where
he lives. Yeah, he's in his eighties. Like, yeah, man,
I had some email contact with him a few years ago, and yeah,
we talked a little bit. So I pointed out that maybe, you
know, maybe the problem that we've had is not that things change, but
that we don't know what's going to change, which, well know, I've
been wondering the past few years that the inertia of our systems have been pretty
stable, and so it's felt more and more engineering recently. But at the
same time, you know, I spent a lot of time looking outward.
We're right for disruption. Yeah, you know, the smartphone's going to change,
likely to augmented reality. These language these non deterministic language models represent a
whole new X paradigm, Like there's a bunch of funny things that could happen
here, and boy, oh boy, all your plans can be blown up
pretty good. Yeah. I think one of the things that's happening is we're
getting better and better at what we do at the at the software, the
rigid ordered part of what we do. We're getting better at delivering it.
We know that we know how to test it, we know how to deploy
much more often, We're getting better at so many of these things. CLOUDE
is this amazing collection of architectural patterns that makes everyone's life easier. But at
the same time, if you look out into the real world, people who
are doing business and executives and companies and governments, they're finding the world that
we live in harder and harder to manage because they don't know what's coming down
the pipeline or if anything's coming down the pipeline, and that makes their world
difficulty. Management loves predictability. Yeah, at the same time, we've sold
all of the stuff is predictable when reality what is it should be allowing for
extreme flexibility. Being able to pay for your your compute by the minute makes
you more flexible. Being able to automate the pipeline deploy software means you can
iterate more quickly. These are all elements of getting to where what you're describing
Barry, which is you don't have to predict the future. You can keep
adjusting to the present as it comes at you because all of this infrastructure makes
it very easy to do. So. Yeah, it's the question, you
know, when a customer asks you, do you know how to you know?
Can you do this? Not do you know? But can you do
X? And most of the time I answer is yes, even if I've
never done it before, because I trust in my ability to be able to
figure it out and get it done. And so that is the certainty that
I have that I believe in myself. I believe in my team. Yes,
and I know that we can do this because we have the ability to
test things out, to look things up and see where the mistakes have been
made and figure it out. Yes. And this is This is a big
conversation I have with architects when I'm teaching them, And a lot of times
teaching architects is more like therapy than actually teaching them anything. Sure well,
one of the things I try to get them to understand is to say,
why were you successful in your last project? And a lot of architects will
lean back and and say because I use this framework or this pattern or this
tool. I was like, no, it's you. You are the success
in your projects. The way that you think, the flexibility of thought that's
required to build good architectures is what makes it succeed, and that that exists
only in you and the people around you. It doesn't exist in a free
A framework boils away that human essence that makes it possible and that's why one
of the one of the best forms of feedback that I get is when I
teach these classes or I teach these workshops, and I have some senior architects
who's built mad stuff that I respect, who says, this is how we
actually think, this is what we do. That's how I know that I've
hit the kneel on the head because what I started out, what I set
out to do, was to answer the question, how do I turn a
developer into an architect? What do we teach them? Because it's definitely not
too gap, and it's definitely not requirements, and it's definitely not lines and
boxes. And what I've done is I've turned that negativity of senior architects into
a two that we can do so that we're not dismissed anymore, and so
it'll be quiet negative, Nancy. We just want to get this built,
and we can say, well, actually, this is the process. And
it turns out that when you start to work in this random way, it's
so much more fun. And it's one of the coolest things to watch at
the workshops is when people start building systems by stressing them, by making mad
stuff up, by picking words out of a dictionary. Then the fun that
it starts to happen is it's really really cool, and at the end of
the day you get a verifiable architecture out of the work. I also wonder,
I mean, there's a psychological aspect of this. By doing random things,
by not, you know, doing all this careful planning, you're not
upset when your software breaks, like you break it all the time. Yeah,
so it's I mean, you take away that stigma and so, you
know, and you externalize it. It's like, oh, we did this
thing, we got this result. This is not personal. This is not
you've failed. This is we were experimenting. We battered the software, the
software got battered. Good on us. Now let's try something else exactly.
This reminds me of a situation. I was actually lucky enough to be hired
as a session musician on an album that was recorded at Levon Helms Studio and
would stuck right, and so I was hanging around most of the you know,
all I had to do is play one solo, right, and they
finally got to me on the second day. I hang around all day the
first day, second day, about three o'clock. All right, Carl you're
up. And I told them right at the outset, guys, it's going
to take me about ten nine or ten takes to really nail this solo.
And they looked at me like I was crazy, Like what you can't just
like you know, and I said, no, I really have to play
it a few times. I have to feel it to get some ideas,
and you know, by the by the tenth take, I swear to God
it's going to be magic. Turns out I nailed it. It was like
ninth take boom, that was the one everybody said yes. And it's because
I understood myself and I understood what my you know, abilities are and how
I can get to it. And so at the end of the day,
I wasn't sure if I should have divulged that or I should have just done
it. You know that if I hadn't said that, maybe they would have
been angry with the fact, like, like, you're wasting our time,
you know, you should just come in nail it. I don't know,
but it makes me. It just goes back to that whole thing with having
a belief in your abilities and your ability to figure stuff out. Right.
It don't look at it as an unknown look at it is what usually happens
when I don't know something? And how long does it usually take me to
figure it out? Yeah? And I think we have an attitude in our
industry which I call sigalism. And as you know, when you watch a
Steven Siegal movie and he goes into you know, he goes into a bar
full of bad guys and there's like fifty of them in there, and they
all rush at him and he beats everyone up and throws them out through windows
and things. He comes out of He comes back out of the bar without
a scratch on him. He has a tiny little bit of dust on his
lapel which he brushes off and then he goes he drives off again. And
you're thinking, in terms of plot, why would you go in there?
That makes no sense. But in in tech we tend to think that,
you know, we need to know everything. I need to go in and
nail this on the first take. I need to be seen at a meeting.
I need to be able to answer all of these questions about how this
particular as your component works, and you know, I have to know this
stuff. And that leads to a sort of entrenched behavior when we're gathering requirements
both in us who won't admit that we don't know something, and it bleeds
onto our steakholders, and they're caught like a rabbit in the headlights. They
have to say something or else look like they don't know what they're doing.
And what they say then, in those circumstances is very often they're just making
it up to get us to go away. Whereas this kind of this kind
of analysis allows us to take what they say and stress it and push it
around and find things outside of it that we haven't understood. And it takes
all that for us on the psychological side of things. It takes a lot
of pressure off steakholders as well, once they realize that, you know,
so, the question I asked to a steakholder isn't what are your requirements?
Anymore? It's what keeps you up at night? What are you free it
off? What could go wrong here? And they'll happily talk about that because
they're under no pressure to be present size. And the worst thing we can
do to our stakeholders is forced them to be precise in their language or precise
in their description of an unknowable future. We've heard and We've heard this before
with this, and I've certainly had this experience that you don't interrogate like you're
a lawyer, like you're preparing for the lawsuit of this failed project. Let
me get statements from you that hold me I'm no longer liable for this thing
failing. Yeah. I usually say that we have a fantastic if you're a
consultant or you know, an architectal large company, we have a fantastic business
model, and that is tell me what you want, and I know it's
wrong, and I'm going to build it until we find out that it's wrong,
at which point I'm going to blame you because you didn't tell me,
and you have to pay me a giein to do the same thing. And
it's this perfect business model where we have no responsibility for what we build,
and this is an attempt to break out of that and change that or change
that mold. I'm also impressed against the idea that there's there's nothing in the
Edge Manifesto against any of this either. Arguably, this is what real agile
was about, was lots of communication, minimizing what you were writing, responding
to change. It's just that it got well folks pushed back on that sort
of sort of agile button mindset, right, that it keep water falling anyway.
Yeah, I think that going back to without getting too philosophical, but
there was a split in our think in three thousand years ago in Greece,
and we chose the path of order and structure and abstraction, and we stayed
on that path for thousands of years. And it was good that we did
because that gave us computers, but it also has some weaknesses that causes us
to think and in a particular way. When we look at social systems,
we try to treat them like manageable ordered machines. And the agile movement was
actually a huge philosophical event from that perspective because they said, hey, we
don't know what's going on. We can't actually write all this time. That's
a huge philosophical step. It's a huge challenge to our industry well, and
arguably said can't know right, like it is not impossible? Why should Why
do you want us to keep lying? Right? It's sort of a pushback
on Parnass to go, hey, you know how we've been vaking it all
this time? Can we just admit we've been vaging it exactly, and that's
why the agile movement then got suppressed. Actually, and that huge message has
been pushed into all kinds of frameworks and things that go back to the goal
of giving us certainty and structure and order. Yeah, which is I think
against the spirit of that original manifesto. I'm not sure how many people behind
the manifesto actually realized that they were they were taking a swing at something really,
really big when they said that really big that actually swung back quite successfully.
I remember the we talked about this in one of the Fusion Geek outs.
The Ider project, the giantic project was the whole thing was get the
first billion dollars spen because then they can't cancel it. So we can't tell
is what we lie until they're too committed. Then we just work on what
needs to be done. Yeah. Yeah, And a number of mega projects
that are like that, where it's like, if you actually knew the full
scope of this, you may not do it, and we really want to
do it and it will be beneficial, but we can't tell you the truth
because then you won't do it. Yeah, that's a great note there,
Berry. This is so cool. I mean You've really made my day and
I'm loving this well, and congratulations on what is clearly a very cool book
which you can get for free, but do yourself a favor at least pay
the minimum price for you absolutely lean pub you have a choice. Yeah,
and it looks like a fun read. Yeah. The artwork has been done
by my eleven year old who's incredibly proud of it. Right, I think
that's one of the coolest things. And it's our project. But what's their
name, let's call them out. Alexander all right, who did the artwork
and is now demanding royalties and I like it even more. Yeah, Well,
this is great and you've changed the world. So thank you for spending
the time with us today. And well, I'm sure we're going to talk
to you again. Cheers, guys, see you soon at a conference somewhere,
Okay, cheers, and we'll talk to you next time on dot net
rocks. Dot net Rocks is brought to you by Franklin's Net and produced by
Pop Studios, a full service audio, video and post production facility located physically
in New London, Connecticut, and of course in the cloud online at pwop
dot com. Visit our website at d O T N E t R O
c k S dot com for RSS feeds, downloads, mobile apps, comments,
and access to the full archives going back to show number one, recorded
in September two thousand and two. And make sure you check out our sponsors.
They keep us in business. Now go write some code. See you
next time. You got Jamtavans and