Hello, and welcome to This Anthro Life. I'm your host, Adam Gamwell.
You know when you're watching the news or a YouTube video or a documentary
and my talking head says, they're scientific consensus that climate change is man made,
or there is a preponderance of evidence in the literature that meditation is linked
to increased feelings of well being. Have you ever wondered, well? Cool?
But I'd like to see that consensus or maybe be able to follow the
trail of scientific claims made around mindfulness to be better informed. Well, I've
got good news for you. Getting to the consensus behind the claims is what
today's episode is all about. One of the big goals of This Anthro Life
is to amplify the voices, tools and technologies that bring our creative potential to
life. And I'm really excited to be joined on the podcast today by Eric
Olsen. Now. Eric is the CEO of a startup called Consensus. Consensus
is a search engine that uses AI or artificial intelligence to instantly extract aggregate in
distill findings directly from scientific research. Sounds pretty cool, right. It's built
around simple searches like our COVID nineteen vaccinations, safe or the benefits of mindfulness,
or linking together ideas such as poverty reduction and direct cash payments. It
then searches across two hundred million papers, adding more every day, and returns
results that highlight the scientific claims made in each of these different articles. This
is amazing for a few reasons. One, it doesn't tell you what to
think. It simply shows you, at a quick glance, what are the
majority of scientific claims saying about your query. And Two, we're seeing the
emergence of a new wave of natural language processing startups in tech that's aimed at
democratizing scientific knowledge and access. What a time to be alive. Right now,
we're going to get deeper into the why and how of consensus in our
conversation today, So to kick things off, I just want to lay out
two big challenges that tech and services like consensus face for us as listeners to
think about. Whether we're social scientists in academia researchers, or designers and industry,
or good old fashioned public citizen concerned with literacy and public debate. With
the rise of research and information technology, it's surprising when you stop to think
about it, that we haven't also found ways to build consensus around expertise,
and there are a few challenges here at play. One is that Pew reviewed
scientific literature in journals like Nature or Sell if you're in the biological sciences or
American Ethnologist if you're an anthropologist, are behind memberships and paywalls. So if
you're a student at a university, you may never notice this since your department
or university often pays for that subscription. But depending on where you go to
school, you may have noticed that you only have access to some journals but
not others. So this is problem one scientific literature is locked behind a paywall
that most individuals and even some organizations can't afford. Then the second is an
issue around accessibility. If you reviewed literature is often written in heavy jargon,
that is, specialist language that makes it difficult to end time consuming for non
expert audiences or readers to digest. Now, this is more of a stylistic
challenge, whereby academic writers follow along with and reinforce communication styles that work for
the academic system but not for public access. Now sidebar Vita's debated, of
course how well this kind of jargon works for academic systems too, and This
is actually one of the reasons that I started this anthro life back in twenty
nineteen, well nine years ago at the time of recording this episode in late
twenty twenty two. Okay, so these two issues contribute in a big way
to why we haven't seemed to be able to find ways to build consensus around
expertise despite having mountains upon mountains of scientific data and researching our collective human database.
Now consensus is working to change this conversation in really fascinating and impactful ways,
and as well discussed in today's episode, we also need to ask ourselves
and ask of society what we want in need out of access to scientific knowledge.
So we'll be right back after a quick message from this episode sponsor,
and can't wait to dive in with Ericles, you know, to kind of
kick us off. I'd love to get a bit of a sense we kind
of think about your superhero origin story, you know, the bit of your
tale of like how you kind of came into the arena that you are now
around the idea of using natural language processing and aiming to disrupt the search engine
industry, which I love this idea, But tell me, how how did
you get into this space in the first place. Yeah, thanks for having
me, adam a great question. So the real origin of it is,
you know, I come from a family of aconomics and scientists, but I
am not one myself, and it's kind of created this this outsider complex within
myself that made me really really interested in science, but made me an incredible
amateur consuming it. So I found real value in consuming content that had scientists
who are breaking down what the research said about a subject in a way that
a non academic like myself could understand. So it was from getting that value
that, like it was actually a six seven years ago, came up with
the idea of what if there was a way to automate this process and what
if there was a way to get answers to these types of questions that I
have on demand from research from peer viewed sources. And then fast forward five
or six years and I was actually my co founder of Christian Salem who I
pitched the idea to all those years ago, that came back in the middle
of COVID and was like, remember that idea you had, I think the
world really needs this right now, And that's kind of how we got started
on our journey, and then the real light bulb moment for us was after
that we started to dig into what was the state of the technology, could
technology actually solve this problem? And then learning that natural language processing and the
advent of these large language models had really just occurred, and it was really
this kind of perfect why now synergy of societal demand and technological feasibility interesting and
it seemed like on one level two some of the best science and scientific discoveries
are some kind of synchronicity, right where it's like the tech came together with
the moment of the time and then and what people were looking for. It
seems like it's a good triangulate that you know, Yeah, it's lucky in
a lot of ways, right, Like things are more random than than we
wish them to be. And both of those things had me and my co
found or had nothing to do with us, right, there were completely things
out of our control that kind of coalesced to make this possible, you know.
I mean that's it's also like this idea that ethnographers talk about kind of
from human sciences, anthropology, sociology, that some of the best power and
innovation work comes from when we can see those connections of other people might miss,
and so I'm interested in thinking about this idea and in terms of these
pieces that came together, I mean, that's that's actually I think one of
the powerful elements here too is that as we have this kind of perfect storm,
as it were, for this need, I mean, one of the
pieces that stands out you're talking about, you know, six seven years ago
up to today. One of the big topics that comes to mind is things
like the rise of misinformation online as a challenge point in terms of you know,
fake news is the meme or hashtag of the day, right, but
just this idea, I mean, how how did things like this filter into
that that you're kind of thinking process? And I think that this is I
think one of the big challenges that a lot of people face today is I
don't know what information to trust online? And you know, how can technology,
how can tools help us do that? Especially when sometimes those seemingly same
tools are the ones that like make it so we can't trust it in the
first place, right, like algorithms and advertisement. So how do we think
about this kind of model, How can we set this stage of what is
this problem space that we're coming into that we want to solve for. Yeah,
no, no, really, well said across the board. So I
think it might be helpful to like first just like level set on exactly what
it is we're doing, and then I can kind of use that to segue
into answering your question a bit. So, Yeah, we are Consensus,
and we're a new search engine that allows you to type in plain English questions
and be returned relevant findings from pure viewed literature. So basically the ideas instead
of going to Google when you have the question of does magnesium actually help me
sleep? You can type in that exact question. You don't have to do
any special boolean searching or anything. You can actually just type in that question
and our NLP will look through research papers and try to find claims being made
about that question and deliver them to you in this nice aggregated, easy to
consume list. And yeah, to go back to your question. Yeah,
this is another part of our kind of why now story. Obviously, the
societal demand and the clear currents of misinformation and that people were frustrated with their
inability to get good information, but was really you know, a key part
of this is kind of like identifying why that is the case, and you
said it really well yourself that it's like, you know, we want technology
to solve all these problems, but as it turned out that technology was what
was creating a lot of these problems. But there's one real key component of
it too that is driving That is that you know, most of these applications
that we use and websites we visit, and specifically, you know, if
we're talking about Google, the way that we search for consumer information, all
this information has passed through this advertising filter. And that means, you know,
explicitly sometimes just being shown ads. If you typed in downs magnesium helped
me sleep, you'll get a bunch of ads for magnesium supplements. But the
undercurrent of it is that because their incentives are to sell more advertising space,
what they optimize for is continued engagement. So you know, the Google or
Facebook or any of these places misinforming you isn't some giant back channel conspiracy theory
of people in rooms and suits B and mic we want to try to make
Adam believe something that isn't true. It's really just these organizations following their incentives
and optimizing for what's going to making the most money, and that's continued engagement,
because that means continuing selling ads to people's eyeballs. So in turn,
that creates systems that are not designed to give us good information. They're designed
to give us engaging information. And the problem is is that when algorithms try
to learn what's engaging, a lot of times what they find out that what's
engaging is content that is flashy and fiery and controversial. So in turn,
you get shown lots and lots of that that has nothing to do with how
factual the information is. So our real pushback on that is that we think
that the future of search has to be in this premium verticalized subscription model,
because that means that, you know, if we're not selling advertisements, all
of our incentives will just be to deliver you the best product and deliver you
the best information possible. And the hope is is that with NLP taking off
so much, that products can get so good at these services that people will
be willing to pay a small subscription fee for them. And that's when you
can actually see this inflection point of some pushback on these free advertising based tools.
I mean, that's super interesting and it has a bit of that If
I'm a David and Goliath theme to it, right where certainly, how does
that feel? I mean, I think about an exciting bit of challenge point,
you know, how do you kind of a process idea of Like I
think it's a what you're saying is super resonant that there is this ongoing challenge
point that if the way we're presented data is not based on what's most factually
correct or that even necessarily answers my question, But it's based on an advertising
model that has made a multi billion dollar global search industry, how do we
even disrupt them? You know? And so I think it's a really interesting
question. I mean, maybe you have, like to're metaphere, you have
the small stone, maybe have NLP to help us. I mean, that's
exactly what my answer is going to be. Like, I'm not here to
say that we're you know, on the precipice of disrupting Google and making them,
like, you know, really taking away all this all the money that
they're making. But the way to make a giant change is, you know,
to kind of follow that David and Glide example somehow and find your wedge,
find your stone. What is the thing that you can do really well
and start small there. And for us, we think a perfect place to
start is with scientific literature because it's, you know, the information that's the
most valuable source of data in the world, and there's insights to question all
sorts of questions that people have, and they're insights that are really hard to
get by using the currently available tools. So we think it's a great place
to start to maybe you know, disrupt the tiniest bit and then go from
there. Right, you can't start somewhere if you just try to show up
and say, hey, we're going to knock down Google, you have no
shot to do it. You start somewhere, you get really good at that,
you build an awesome product in that space, and then we'll see where
it goes. That makes a good sense, and I think also also wise
right, it's like we do what we can, but like picking that right
movement. So I think it's it's really compelling that scientific literature kind of functions
as that full comment as the piece we're using to then be able to rethink
how search works, you know, and like so it's a very specific example,
and so to kind of set the stage here in terms of the problem
here, I think there's something interesting, right that. So I've gone through
grad school a lot of my colleagues have, and you know, one of
the big pain points, like you don't notice it when you're in school,
is that you know all the journals that you're reading for research are behind paywalls
because usually your school pays for those. But it's interesting, like, as
I spent more time talking with other colleagues across the years in different disciplines or
that went to different schools I had less funding, they wouldn't have access to
some of the same journals I found. And that was interesting to note.
It's this privilege of information and that feels worse when you think about it as
scientific information rights, as peer view literature that we don't have access to.
So I want to think about that with you, like this the impact of
like inaccessible or making accessible scientific information, and I mean, I you know,
we'll just straight up say, I think the current model is broken,
right, like behind a paywall where a lot of research sits. I mean,
this is also interesting in conversation with the fact that at the time of
US recording, the Biden administration just passed this law that at the end of
twenty twenty five into early twenty twenty six, any federally funded scientific research has
to be immediately available upon publication, which is really interesting, and like saying
that feels kind of crazy because it's twenty twenty two and we're still a few
years away from that even being a thing, right, So that's it's crazy
realized. Up until this point, scientific research is still behind a paywall for
the most part, especially the most prestigious things like Nature or sell for you
know, science or biology journals. Anyway, So I'm a rambling now,
But tell me about this process and like how we can think about it,
about this idea in terms of why we need accessible information and like and how
do we think about this the current paradigm today. Yeah, you know,
and when we've talked about like what our problem statement is the broad and way
that we get you know, people's eyeballs to light up and what like the
broad vision of it is this that which is broken. It's really hard to
find good information. Like that's the way to paint it with this this giant
brush. But what we call our sub problem statement has been that the most
valuable and insight filled source of data on the planet is sitting behind paywalls.
But not just sitting down paywalls. Because of that, it's only consumed by
the same people who create it. And there's no way for somebody like myself
to engage with literature unless I'm a part of a university and I'm doing research
myself. And that's a shame on a million different ways, one of them
being like the very obvious, just like economically, the way the system is
just broken because it's many of these things are funded by public dollars, yet
they're not accessible to the public, and that just is incredibly backwards. And
but the good news is the trend is in everyone's favor. Like you said
that, the Biden administration just passed this legislature, and even before that,
the trend has been moving toward making things more open science or making things more
open access. And our data partner they're called Semantic Scholar, and they're run
out of the Allen Institute for Art Official Intelligence in Seattle. It's actually Paul
Allen, Microsoft founders AI Research Arm and they're basically a Google Scholar competitor.
They're a research data aggregator and they try to focus a lot on open access
papers. So a lot of our corpus actually is entirely open access, not
all of it is. We can still do our analysis over the papers,
but if you're to try to go to the full text from some of our
search results, you will run to the paywall unfortunately sometimes, but a good
chunk of our papers are open access, and like you said, that trend
is going to continue to be working in our favor. Very cool thinking about
that. Actually, the idea in terms of I was curious to think about
this and then maybe through Semitic scholar, but how do you approach partnerships in
terms of like working with in terms of how does that consensus plug into partnerships
in this regard in terms of aggregators versus search engine versus kind of do the
NLP the text act that helps put these pieces together? And with that,
I'm curious, you know, so, as an anthropologist, do we get
anthropology journals as part of this as well? Are there? I know they're
not as fancy as oftentimes by your chemistry, journals don't get as much pressed,
but I'm just curious in terms of that, like how do you select
for the kinds of articles journals that are included in the kind of the search
process. Yeah, so to kind of start with, like the partnership's question,
we're incredibly fortunate that we're operating in a space with lots of research organizations
that are nonprofits and that are can be fairly open to partnerships like this.
There is like a big caveat though, that you many times you need to
basically prove that we have mission alignment and stay true to those ideals of promoting
open science and promoting access to scientific literature. That is basically how we got
the partnership with Semantic Scholar was. You know, they're willing to do partnerships
with other research organizations basically without any vetting, but we had to go through
this vetting process as a commercial entity to say, hey, here's how we're
going to be using it, here's how you benefit, and here's how we
push forward your mission. We really tried to work hard to abide by a
lot of those values so we can be attractive to have partner. I mean
one, because we also want to abide by those values, but also it
has the other benefit of being able to partner with a lot of amazing organizations
that have incredible mission statements of promoting access to scientific literature as far as anthropology
journals, I don't know that answer off the top of my head. We
have pretty dawn good coverage. We'vebout two hundred million papers in our corpus,
so I would be shocked if there are not some anthropology journals is a part
of it. Fortunately, I don't have that answer off the top of my
head. And as far as like selecting, it's really whatever we can get
our hands on it. We're not trying to select by domain. The way
that we've trained our algorithms is to try to be domain agnostic of extracting findings
from papers. And I think one of the great surprises of our early product
is how well it does with non medical healthcare related questions I publish. If
you sign up for our product and become a free user, I send out
a newsletter every week that's like, interesting thing I learned this week query of
the week basically, so it's like it's a question that I asked and what
the results were, And this week's was does the death penalty reduce crime?
And our product did an awesome job of servicing research findings about that. So
using that as an example that the domains can be a bit you know less
hard science than you typically think of when you hear a peer of view of
literature, and our product can still do do a pretty decent job of finding
relevant conclusions about those questions. That's super interesting. I think something that stands
out to me here too is a that's really awesome to hear that the algorithm
that you'll put together is able like kind of thinking like a thinking domain agnostically,
I think is really important. It's making me think of, like,
you know, my dreams of being a zocast and journal or that I can
like connect ideas together with double brackets and like be whatever subject agnostic and just
connect ideas as they as a link together and also mean more like how the
human brain works too, So I think that's something else that I think it
gets me. You're really excited about the possibilities that you know, kind of
what the emerging world of natural language processing can do is that we can train
it and train ways that will kind of use the same pathways that our brains
might write, that we don't get stuck in silos of like I only want
to get journals that are from biology or just from anthropology or whatever it is.
But actually the death penalty question is really interesting in terms of probably the
diversity of results that you got back. I'm also thinking of another example.
I saw the blog you recently did on the use of psilocybin medication, like
based on Michael Pollan's How to Change Your Mind series, and like that was
a really cool I think example too in terms of a showing the lack of
a better term, the pop culture application of asking of scientific literature. I
mean, that's really what Michael Pollan's work is all about, right and death
thot. He's a great question too in that regard because also the other thing
too is even that space, how do we even know how much is out
there? Right? And so thinking of the power of that, we can
use big data in ways that the human brain doesn't quite do so well.
There are just like literally searching Alexis nexus right or bioarchive, you're only going
to see so much in terms of returned results that you can only read.
But again, this idea in terms of how can we kind of cross those
boundaries, I think it's really exciting. So is that what you've found too?
That is it surprising how good natural language processing has become You've you've kind
have been steeped in it for a while. But you know, how how
are we seeing it evolveal So, you know, even through consensus, right
as you think about partnerships and what you're searching for, how have we seen
that change? Yeah, even we work in the world of NLP, it
continues to pleasantly surprise me every day. I would earnestly say, what you
said is really interesting of like how it kind of operates in some ways the
way that the human brain works. And the real advent of these the new
technologies. They're called large language models, and the unique feature about them is
that they come pre trained and they come with this underlying knowledge of how human
language works. So the big one that you've likely seen the news and listeners
have probably seen on Twitter or something is GPT three open aies model. It
is the part of the underlying technology of DALI, which is that you type
in a prompt and you get back like an art image. People have been
loving that on Twitter, and that is trained on basically like the entire Internet
of articles, of text articles, and the way they teach it to understand
language is they showed examples of articles with certain words blanked out, and they
try to make it predict what word goes in those spaces, and then you
show that literally billions of examples, and over time it can learn how to
accurately predict what word should go here, so it actually understands like the whole
context of what's going on around it. And yeah, it's shown the entire
text before it makes that prediction, so it's trying to understand text holistically in
the way language works. And then what happens is when you try to do
one of these specific tasks, like we're doing that consensus, you do a
process what's called fine tuning, and you give it more custom training data specifically
for the task that you're trying to do, and then you teach it how
to do that task. But it comes with that underlying knowledge first. And
if you think about the way a human would learn how to do a task,
it's very similar to that. Right, if you're being taught something,
you don't come in as this just blank slate. You have all of these
patterns that you've learned about about other tasks you've completed in your life. When
you come to a task, will learn how to do it. So it
really is in some ways I'm mimicking the way a human would learn how to
do something, and in the context of how we do it, Basically,
yeah, there's a pre train model, and then we give it examples of
We hire scientists to go through research papers and mark up our authors making their
claims. So it basically it's just a bunch of ones and zeros next to
sentences where it says zero, this is background information zero, this is methods
zero, this is more background information one when they say these results suggest that,
so on and so forth, and then when you give it enough examples
of that, you can feed that to a model to learn how to pick
out those sentences. Very cool. Even this idea, I think is really
exciting in importance of context, right, and part of it is like,
hey, how do we train models? But then also recognizing that I love
the way you said that too, where it's that we're never just born as
blank slates, right, we always have a set of pattern recognition and context
that like shape who we are. And like the fact that we are able
to build technology that follows that too is exciting. You know. Obviously it
means we got we got to watch it and it makes it super flexible,
right, Like it makes it so you can teach you to do a whole
host of tasks. Yeah, I think that's right on too, And I
think that's that's in some of the most important power. It's like, at
the same time as we're through neuroscience through psychology really coming to understand the plasticity
of the human mind, right and the fact that we can't even reshape neural
pathways for training through different kinds of lifestyle, even that we can see this
kind of happen also on a technological software level is really fascinating that they're kind
of happening simultaneously, which is interesting. And I though that's because we just
learned to see one and that we can see the other one now, or
kind of how that happened. Maybe a chicken and egg kind of problem there.
You know, it's kind of funny to seek some co evolution there,
I suppose. So I'd love to kind of think with this idea. So
as we're training our software and also working with scientists. It's a very cool
point too that we've got humans as part of that. I mean, it's
one of those things that I've conversation I had with Byron Reese on the podcast
a few years ago and more recently he's the CEU of giggo Oman like did
a podcast on Voices and AI and it was really really into the role of
AI and changing how technology works and like obviously going into things like NLP and
you know, is this interesting idea that a lot of times that there's a
common kind of you know, pop cultural notion that AI is something that will
become generalizable, that it will become essentially conscious at some point, right,
And like this you know, filters into our you know, bigger fear narratives
of like the terminator and the matrix and things like that. But realistically,
you know, most models say a I can't do that, and it won't
and even the way that we're talking about this and that AI often at this
point needs to work in partnership with people still, right, Like it's learning
how to interpret context with humans because we're the ones that do without even thinking
about it. Right. I think this is like an important piece for folks
to like pause and think on. Is that AI is super powerful and it
gives our brains a super power to connect ideas. But really it's also to
your point, is based on a lot of how we do interpretation as people
and learns from that. So it is interesting too that it's that as much
as we're students of the way we can connectnowledge AI as a student of us.
Also, we're going to take a quick break. Just wanted to let
you know that we're running ads to support the show down. We'll be right
back. Yeah. I was on a podcast the other weeks. We were
talking about NLP and he was talking about it as like he was reading an
article that there are gonna have to work to like fight plagiarism in schools because
students could use AI generators to write papers, which totally can happen, right,
and I can do that now, But I was saying my response to
them was, if you're a student that's trying to do that, you know
there is like a prerequisite step in some ways that you have to get a
bunch of training data of a papers show up model to then learn how to
do an a paper. So the work that it would be required to you
know, find tune a model to do exactly what you needed it to do
for your class, you know, is way more of the way more work
than just writing the paper yourself. Now, with that said, like a
generalizable generation model could write you like a paper that could pass as a human
paper, but it probably isn't very good for your class. And in order
to really get it specifically at the task in the way that you would need
it written for your class, you would probably need examples of that. So
a prerequisite for having a model spit you out a papers is to write the
a papers yourself. That's a good point, right, It's like you can
plagiarize, but right, yeah, you can hire someone else to do exactly
right. But that's I think it's really interesting, and I think that's right
on where there's can be a fear of adopting technology, especially one like that
feels like it gets close to home, right, like how am I writing?
You know? But it's it's based kind of in the similar concern that
we kind of talked about up top, where there's there's a lot of mistrust
online on one level, like I don't know if I can trust the information.
And so we've also seen, you know, in the past ten years,
the rise of plagiarism checkers, like in word, Microsoft Word has one
you know, Google will look forward to. And so I've even seen AI
writers that then also plagiarized check itself, which is interesting because it doesn't know
if it's plagiarizing until it looks and says, okay, I wrote the thing.
Oops, yep, I borrowed that from Wikipedia, which I think is
interesting as well. So like it doesn't know it's plagiarizing until then it writes,
then it checks, you know oftentimes or the ones that I've seen in
anyway, But I think that's that's a really interesting question too in terms of
there's the the work that we have to do in terms of providing more access
to scientific literature and making it more accessible and easier easier for folks to get,
especially if for lay people. Right when we already looked into this question
of the paywall problem, But then how do we make this usable for your
average lay your lay person too who's never going to get a master's at PC,
he doesn't need one, doesn't moment, but they're interested in like yet
is our COVID nineteen vaccines effective? Right? What are the benefits of mindfulness?
There's some examples on your website. I thought they were the good thought
starters. So how do we think about that, Like, how do we
help build this in a way that's useful for everybody. Yeah, it's a
great question, and I think that it's it's finding the balance of not oversimplifying
and perverting meaning while trying to use these tools to make things consumable and digestible
and easy to use. I think a real big emphasis on that last one
of easy to use, because that says nothing of the digestibility some too,
always of the text that you're showing. So I'll kind of tie this all
together that what we're doing is we're not generating any text, because we think
that is a fairly dangerous road to go down of trying to summarize what a
sign this is saying without any checks on what does the source text actually say,
without being able to show that to the user. So we want to
do our best to always just extract the information and show them a real quote
from the literature. So I'm going with all this is that that is how
you use the tool to make it easy. You try to find the sentence
where they're answering the question and then just extract it. And you might be
sacrificing a little bit of digestibility because you know, scientists still love to write
in jargon, and that sentence maybe jargon filled but we kind of think that,
you know, this is by no means perfect, and we're going to
continue to iterate on it, but we found that to be this kind of
nice balance of using these tools to make things easier by finding the answers,
but not going overboard and not oversaturating and oversimplifying it by doing all this like
super advanced generation, where we could be totally perverting the underlying meaning of what
the scientist is trying to say. So I think in total, my answer
is like trying to find that balance of where can you use these tools to
circumnavigate things and make things easier while not go overboard and oversaturated and oversimplifying.
Yeah, that's fundamentally important too. And even the software itself, I actually
really appreciated this. It has a beta tag on the website and it says,
do your due diligence from this work. You know, it's like,
don't just take this as gospel, but but you know, still be a
scientist. Still be scientific, and you're thinking and say, look at this,
and then continue to look around as part of your process. I think
that's actually really fascinating too, because I think you're one hundred percent right where
it's like in the in the world of view X and usability, it's like,
we want to make things easy to use for our users, want them
to be frictionlessen to be a good experience. But especially in this case,
like when we're looking at also scientific data, I think it's really really fascinating
and important that the point that you've noted that it's we have to make sure
that we're not either oversimplifying the complexity of an answer, but then also that
means we're doing interpretation of it right, and that then may change what we
are then changing the meaning of if we change the wording to make it sound
more or less jargoning, we realize we're adding a second layer there in terms
of interpreting what we're seeing. And that's that's an interesting and like big conundrum
that you know, whole different like can of worms as it were, but
interesting to think about. Yeah, we're actually training some de jargonizing models.
But we were just having this conversation earlier today. By how we want to
you know, we're still a ways away from having that in the product,
but our idea for how we'd want that in the product is a little button
that lets you toggle that on. So you're you're basically saying, I am
acknowledging that now I want to see the slightly more simplified version of these claims.
And you know, you're very acutely aware that you should be doing more
due diligence when you're interpreting one of those results with a little like toggle bar
at the top right of the screen. And yeah, I appreciate you calling
out the beta dag because yeah, we think that's that's important. We're still
early, We by no means do everything perfectly, and even if we did
extract all these answers perfectly, like, it's still is important. If you're
going to be, you know, making an action item in your life because
of one of these answers, you should probably pop up in the paper and
read all about it. Like it's it's great to be able to get a
landscape of evidence really quickly. That's why we built the product. But context
is always incredibly important. You should do your due diligence when you're when you're
going through these find And then yeah, to your point about friction less,
Yeah, that is we've tried to make the product really easy to use,
because that's a way to make it approachable without having any risk of anything to
do with perverting the signs, right, like justin the log inflow and the
way you can search and the try searching buttons and all of those things,
like, right, those are things that are completely outside of the information.
We're showing their usability in a much more like aesthetic way. So we're trying
to do everything we can to be really good at those things, because those
are ways we can make this more approachable for anybody without having to worry if
we're dumbing down the signs too much. That's a great point and I think
super valuable too, because you know, one thing I have to hear too
is that you know, working in consumer research, whether it's working and doing
kind of design research with different clients, that there can be this I don't
fear is too strong of a word, but just you know, if folks
are not used to scientific literature research, like it's a scary space, right
because it is. It's jargony, it's behind this weird pay well, it's
complex, you know, how do I understand it? And so I think
there's this, you know, to your point, like a real value in
making the experience, the digital experience of coming even to the search itself welcoming
Pogle scholar hasn't changed in twenty years. It's the same intervas for the best
decade. That's so true. That's so true. And it's hell to try
to start something on there. Right, you can't get like internal parts of
papers. You can get titles, but you know, there's nothing in terms
of really finding a way in so to me, also this really fills a
big gap like that, right, in terms of running that question of access.
But then also the rise also in you know, natural language itself.
I mean, this is something that I as a consumer side, came into
being interested in this, Like with calendars, right, meet with Eric on
Tuesday at two pm, you know, like I could type that in then
it would at an event to my calendar. I thought was super cool,
you know, I think then being able to do this and asking scientific questions
is incredible, right, what a really interesting way of having to then think,
okay, you know, how do I formulate my hypothesis in a three
part question you know for a paper, way harder than just saying, yeah,
what are the benefits of mindfulness? And then kind of returning results in
that space I think is really interesting and important because that also plays it,
I think, an important role in accessibility. Now we're folks like they have
a point of access, they may have not hit otherwise. Yeah, we
think there's three main differentiators between us and a search engine like Google scholar,
and we think it's like the three things that are the worst part of that
experience of using Google scholars. So one is exactly like you said, of
the actual trying to get information and related to your question, Google Schullar does
not allow you to type in a plain English question. If you type in
your research question, you're not going to likely get back very relevant results.
You have to do this like you know, kind of insider exclusive boolean searching
to get it to come up with what you want. And then there's that
next part of all they do is just give you links, which you know,
if you're doing a giant lit review and you're a PhD and you know
you have to go dedicate you know, forty eight hours of or fifty hours
of your next week to do that, fine, like whatever, But you
know, your everyday consumer who just wants to ask a question and get it
from good sources. You're not about to go through all those papers and spend
considerable time. So we wanted to make a way that makes it easier to
get information from those papers where we're not just delivering new links, we're actually
surfacing the insights to your question. Now, if you're then objective is to
go deeper and this is a better way to surface papers to look into.
Great. But if it's also just a way to quickly get an understanding of
what the research says about a subject or a question, also great, Right,
And then the last one is a little more subjective. But just their
interface sucks, and they're going to always continue to try to build it to
be a real product experience and not just some clunky academic search engine. It
matters though, I mean, like Apple really kicked up a consumer design language
as needing to be important, you know, with iPhone, and so I
mean through today, it's it's like, it's interesting to see that it's been
such a broad respect now for things like color palette and typography and as matter,
right, and like, where is the affordances that I can easily see
I can click on on a website make a difference, you know, And
like again, having these come together around scientific research, I think is like
that itself is also somewhat revolutionary. I guess I'm trying to get at like
that we don't often have accessibility front and center as a point of value as
a mission orientation for scientific research, which I think is super important in that
regard. I think that thing stands out that I'd love to hear about is
you know, we talked a bit about semantic scholar, but then you've also
talked about I've seen your website like other partnerships like with sie Score, and
this is important because it's how do we build trust in what we're finding?
And so this partnership tell me a bit about this too, or if there's
other ones that like this that are kind of helped put together the bigger web
of how do we build the tools and the trust. No, it's a
great question. Yeah, we're extremely honored and privileged to be able to partner
with size Score. So they are they're actually another private company, but similar
We have a similar mission statement and are after the same things. And what
they've actually done is it's the largest analysis of scientific methods ever conducted. And
they did an analysis I believe it was one point five million papers and looked
at the actual methods that these studies undertook, what percentage of them were randomized,
blinded, had ample statistical power, had diverse population sets, and they
graded the papers and then use that to grade the journals that they came from.
So the whole thing is just a giant pushback on traditional bibliographic ranking systems.
The one everyone knows about its impact factor, right, which just is
a measure of influence and incitation counts of journals, And this is basically a
pushback on that. And they actually published their own paper that showed that it
was completely uncorrelated with impact factor, which is a pretty big red flag for
those who are using an impact factor. And it makes sense, right because
it's actually looking at what the researchers within these journals we're doing. It still
is a proxymetric, like just because something comes from a journal that typically has
really high standards doesn't mean that a given paper doesn't. And that is a
slight limitation of those tags that we have in the product that it getting a
good score just means it's from a good journal. It doesn't inherently mean that
that paper is better. It makes it much more likely than it is,
but doesn't inherently mean it, but it is a real step forward in terms
of these proxy metrics, and then the future is being able to take some
proxy metrics also looking at the study itself and looking like, what were the
actual methods of this paper, kind of rolling that up into one and saying,
you know, what is the rigor criteria of this score? What is
the reproducibility of those findings? As something we are actively working on and really
hope we can get into the product. That sounds amazing and super interesting and
important, especially because I'm trying to think that my partner is a microbiologist and
so that's how I learned about bioarchive, you know. But this is an
important point in terms of like the reproduceability and that as like a product roadmap
feature. I think it's super interesting too in terms of how could we like
ask these broader questions of methodology, which is something else again not really done.
So I appreciate how up top we're talking about this idea that we're working
with scientists with some zeros and some ones to kind of mark where we're talking
about claims. And then this is like kind of another out of the second
But there's another step in terms of bringing in the questions of methodology, how
those are discussed, what do they look like, and how do we rank
that, and then also what might that mean for things like reproduceability of an
experiment for interesting and can we automate it? Yeah, can we automate that,
like sign me up? That sounds that sounds really cool, and maybe
we can use that to then do a little debunking of some of the behavioral
science that like exactly. We've actually met with the group, the Center for
Open Science, and they've done some awesome work in this world. And the
head of the Center for Open Science is Brian Nosek, and he is actually
most famous for one of the things. He's an amazing, amazing figure in
this space, but one of the things he's most famous for is debunking the
psychology study about power poses and how if you stand with your arms and legs
really wide, you're more likely to succeed in an interview. If you stand
like that beforehand and then go in, you're more likely to speak well or
speak confidently, do well in interview with public speaking. And he showed that
that was not reproducible, and then that was a kind of bogus finding he's
been a huge pioneer and in debunking some of these behavioral psychology studies, and
we've talked with him about these reproducibility metrics and how we could actually incorporate it.
And another example. We're not as formal of a partnership where we're using
the data like we are with side score in the product, but it's a
team that we meet with nearly monthly and continue to talk about these things and
our privilege to be associated with them. Very cool. This is just like
a giant nerd fest of awesome that I know. All the listeners right now
we're being like, man, how do I don't get into this? This
is great to be able to plug into these questions. We are totally free
for now, and it is totally free to sign up and create an account.
So hide over a Consensus Dot app and create an account and ask some
research questions and yeah, giant nerd fest and awesome. We might have to
put that as like a tagline in the products some one. You're there here,
folks, Giant nerd fest of awesome. Same, No, totally,
I think, And that's great. Actually I just signed up myself too,
so I'm excited to be digging around and see what we can we can find
here. So I think kind of as a what's next question, I mean,
you're kind of pointing a little bit or hinting a bit about what might
be down the product roadmap, but you know this bigger question too of like
we're around the precipice of this wave of NLP based technologies and startups, and
so where are we going? Like this is a really interesting space, right,
I think we're in the next frontier in terms of what tech can help
us do around NLP. And so what are you most excited for in this
space, Like whether with consensus or just what else you're seeing that's kind of
shaping the industry or what we could do in the next five ten years.
You know, I'll give you an answer that applies a bit to consensus as
well as like as a more broad answer of like what is to common apologies
that this isn't totally revolutionary and is actually what we've somewhat talked about before.
But what I'm probably most excited for is how these will be used to build
really really amazing specialized search tools because of the ability that you know, I
think they're going to start to pop up in certain verticals. You know,
I'm biased because we're popping up in the science vertical, but there'll be other
ones and other verticals that will build use these tools to build amazing information retrieval
information synthesis engines where you can type in questions or do all these different types
of searches and instantly be returned interesting results. And I am just so excited
to see the innovation with products like that, because I think that is our
opportunity to really change the landscape of how we take in information. If these
products can actually become good enough that they can operate on subscription models, they
have a chance to change the way that the world consumes information. And as
far as yeah, like what's to come with consensus, You know, I've
talked about some of this that reproduce a vability score is something we're actively working
on. And another thing we're really excited about is trying to do like I
kind of mentioned that like toggle button of potentially de jargonizing, is trying to
figure out the right ways to introduce some synthesis into art results. So right
now we're pulling out the quotations, and we want that feature to always be
available to people because it's really valuable, it's nice and raw. We're not
perverting any meanings, but we do want to experiment with ways to synthesize some
of these results and give people a need maybe even quicker snapshot of the landscape
of evidence just on one summary screen. A lot of questions to answer there,
and there's gonna be a whole other things to consider when we do that,
of making sure we're not perverting meanings and representing things that shouldn't be represented.
But there's definitely going to be a place for that in these NLP products
are only getting better at large scale synthesis across a bunch of different sets of
results. Yeah, that makes me super excited too. It makes me think
too, because there's been a you know, like explosion of read this book
in five minutes summary book services, you know, and oftentimes those are then
written by people right there, they're like human made synthesis synthesizes, And that's
where it all started, right is like we saw, like whatever products you've
seen that are like manually curated products around like tech summarization, those are going
to become NLP companies in the future. So like Blankest is a perfect example,
right, it's the book summaries. Those are manually curated right now,
no world where in some order of time that isn't going to be just done
by machines, And like, those are the types of products that we're going
to see pop up. A really interesting thing that I've thought it all before
is like the place that we're seeing a lot of these models take off early
right now, and these applications are in generating like subjective content in some ways.
So like one of the most prominent applications we're seeing is using these these
language models to generate marketing copy. And then the other really famous one,
obviously is this art generation. And I was saying, it's kind of funny
that, like, if you think back to a decade ago when we were
all hypothesizing about what AI was going to do, we said, everyone always
thought the place that they'll come last are for roles that are truly subjective and
like art based, right, Like everyone's like, oh, you know,
if you're a painter, like there's no AI is going to come for your
job. And it turned out that that head prediction is like completely wrong.
And I think it's wrong for two reasons. One because when you're generating something
like an art image or even like a marketing slogan, you have a lot
of leeway to kind of like do it in this like party trick way,
and there aren't real consequences for what you're delivering back, right, Like when
you type of prompt into Dala and you say, what generate me this random
image, It doesn't really matter what it spits back to you. The fact
that it's spitting back as something relevant to your prompt, like that creates the
moment that they're seeking to creative. Oh very cool, right, But it
isn't like there aren't these stakes of did they answer this question perfectly and find
the relevant information inside a scientific document? And the same is kind of true
for marketing copy, right, Like, there isn't like a perfect way to
generate a marketing slogan. You just kind of throw things out the wall and
see what sticks. So guess what a machine is really good at doing throwing
things in a wall and seeing what sticks. Right. And then I was
talking about this in my co founder and he brought up another really interesting reason
why this is happening is because the Internet is filled with just subjective content and
it creates these giant training corpuses to build these models. So like the Internet
is literally just filled with marketing copy right, like where you have titles to
articles, so you could say, all right, here you have all this
training data already built for you, where it's here's the text of the article,
here's the header. Okay, build me a model that generates headers for
articles, right, Like, anybody with NLP skills could build you that model
and not that long of a time with the giant trainings that data to do
it on. And the same goes for these images, right Google image.
All the images across the Internet have these captions and titles of what they are,
Like if you'd Google search something it has whatever, your images will have
a little like description of it. Well, there's your training data. You
have all these pictures and all of the prompts. Now you can train a
model that can take in a prompt and spit out an image. So anyway,
I think it's really funny that everyone thought that the more subjective thing,
like it's going to be the hardened like one or zero tasks that AI does
first, because like those all things were automated and all the subjective things that
make us so uniquely human they'll never come for that, and then that was
completely wrong. Watch them, watch them come, you know, it was
funny. It was actually only recently I feel I feel like I got duped
that I realized or found out that, you know, when you're doing those
like captift phrases, like you know, either like the pictures of a boat,
we're just training an AI that's always are all street pictures because it's training
in self driving cars. That's why they try to make you pick out what
the stop sign is, or what the motorcycles are, or what the stoplight
is. It's training data for self driving cars. Does that count as publicly
funded research? You know, it's probably an argument for that. How do
I gotta cut of that? You know? Eric, thanks so much for
joining me on the POT today. That's been It's been really great. I'm
super excited to have listeners check out consensusal but link and guessing folks to kind
of test it out and see and get their thoughts of what it's like,
because I'm really curious to see the human science community dive in and see what
we can find in there. Again, taking some thoughts with the benefits of
mindfulness that the question around death penalty thing is really interesting as well, so
an inspiration for for folks to kind of dive on in. But thanks much
taking the time of talking with us today, and yeah, we wish you
the best luck and hope we can keep in contact and you know, dive
on in. His consensus keeps on building. Absolutely, this has been a
blast. Thank you so much, Adam awesome. Thanks thanks again to Eric
Olsen from Consensus for joining me on the podcast today. I am really excited
at the possibilities of making it easier, faster, and more digestible to see
what the latest sientific consensus is around all sorts of ideas. And in case
you forgot, Consensus is currently freewill in beta, so definitely check it out,
give it a spin and let me know what you fine. Really curious
that the new kinds of connections and possibilities that we could find, so I'd
love to see what you're looking up and the different ideas that you're bringing together.
Now. In fact that this Anthro Life and Consensus did a little collaboration
that I'd love to share with you. You can check it out on the
latest post from the TL newsletter on sub stack. The link to Subscribe is
in the show notes below, and as I've been mentioning for a little while
now, it's been a goal of mine to do more writing and expand the
TL mind share across mediums. So I hope you'll check it out and subscribe,
and if you get something out of this Anthro Life, please take a
moment to give the show a review on your podcast listening of choices, but
that's an option, and share it with a friend who you know we'll love
it too. It's funny, despite podcasting being a thing for years now,
if the culmus around twenty years, the best way to discover new shows is
still through human connections and sharing with your friends. So I'd be honored if
you find the show worthwhile to have you share it with a friend, and
hey, let me know about it. Should me a message at this anthor
life at Gmail or getting contact through the TL website, and maybe we'll have
some new podcast love stories to share. An air never a bad thing,
right and is always to thank you so much for joining and being a part
of this conversation. The invitation remains open to pitch and submit ideas for episodes
or blog posts, and that includes things I can make that you can make
or that we can make together. So if you have an idea, let's
get in touch. Much love my friends. Check out consensus dot app,
give it a spin, and I can't wait to see you next time.
On the next episode of This Anthrow Life, I'm Adam Gamwell, we'll see
you soon.