#204: Accelerated Learning with Dr. Lia DiBello

Brian Marren: 0:03

Okay, well, welcome to the show, Dr. Leah DiBello. We really appreciate you coming on here and talking to us to talking to us today.

Dr. Lia DiBello: 0:13

I'm sure it'll be a lot of fun.

Brian Marren: 0:14

I think so. And I do want to warn the listeners and you Greg is a little under the weather today. So he's not his usual self. So maybe he'll only be at like a normal person's speed.

Greg Williams: 0:26

What is my usual Yeah, I can tell you this. Our viewers and our listeners really don't know the real Greg. So

Brian Marren: 0:34

there's no hiding after this many episodes. know maybe exactly who you are.

Greg Williams: 0:41

So, Brian, no matter how under the weather I am, I'm so excited about this episode. Because one of my favorite humans on the face of the planet is on and we get to talk to her.

Brian Marren: 0:49

Yeah, well, so that is kind of a good place to start, Gregory because Lia, we we reached out to you after we were introduced by a mutual friend, Matt Fennell, we have we've talked about him on the show because of the work up at the infantry, immersive training camp Pendleton. But some of the the main reasons why we wanted to have you on today is one, you're a cognitive scientist, you're actually the Chief Science Officer for ACSI Labs, which we'll talk about, but you're caught you're in, you do a lot with cognition and decision making. You're a practitioner. So you work with companies and organizations not so much in a lab, or sort of academic setting, you're really out there in the real world. You know, plying your trade and doing what you do best. And you're in simulation, and VR, and you work with subject matter experts. And so a lot of our listeners, kind of like I was briefly mentioning, are a lot of our subject matter experts in their own field, whether that's some of them are law enforcement, some of our highly technically skilled people working for major organizations. Some are, we have a kind of a wide stretch of folks that listen to our show, but all of them are very interested in the thing that they do. And that's kind of why they like us, because we talk a lot about cognition and sense making and problem solving, decision making. And everything that we do. You know, this is obviously more informal format than what we do at work. But you know, those, that's kind of the real reasons I wanted to have you on today. And because your your whole backstory is amazing, and I'll have links in the details for him to check out even even more about about one of the things that your your expertise is in is accelerated learning. So I figured, could we start there and define, you know, what you mean by what is accelerated learning?

Dr. Lia DiBello: 2:37

Well, it does depend on who you ask even the co authors of the book that I wrote on accelerated learning. We don't all agree on the best definition. So they're all in that book. But my definition of accelerated learning is basically accelerated expertise. So let's, let's define an expert. An expert for me, is a a person who has a first principles, intuitive grasp of the organizing forces of his domain. So a physicist, for example, sees physics very differently than the rest of us, and sees it intuitively. And I would say that that's true of all of us, we all have some form of intuitive expertise. And when I was a professor, I used to illustrate this by writing something on the board, like, you know, some phrase and say to my class, okay, now look at that, don't read it. And they couldn't do that, because we're all intuitive experts at decoding the alphabet into some sort of meaning. And we can't not do it now. And so when you get to the point where you understand a domain, where it's a priori, you can't not do it, right. That to me, is what we call an intuitive expert. Now Hubert Dreyfus has a nice little taxonomy. He has five stages of the development of expertise, a kind of genetic epistemology model. And for him b, stage five, right, whereas before that you're kind of, but the other stages are interesting, too, because they help us understand when you're not an expert, what you're like, right, and you want one thing,

Greg Williams: 4:26

Brian, that I absolutely love about this, we I'm the comic relief, I guess on the show, but my job is to street things up when things are big and scary. And the concept is too broad for most people to understand because it's out of their field. I like to bring it down into something that they do every day. Well, you just did that and your definition because when you're talking about a physicist as an expert in the realm, you're also talking about an expert model that comes from a bass fisherman that takes people out on an afternoon fishing, and understands that the color of the water or the color of the lure, or structure has something to do with when fish bite or when they don't, and he also understands that fish don't know when it's raining, right, they understand certain things that that come together. And when they coalesce, that creates a body of knowledge that's important in that person's day to day. And so what you're doing is not, you know, high brow for only Chief Science officers, what you're saying is every single field no matter what it is, if we broke it down into critical skills, there would be people within those skills that we would deem expert. And we can follow an expert model and mimic some of those good behaviors. Is that an accurate way of expressing that?

Dr. Lia DiBello: 5:43

Well, that's a that's a, the last thing you said is a bit of a point of debate. Some of my colleagues think that you can become an expert by mimicking experts, I believe there is a developmental trajectory, and you may not be able to skip steps. We know that when kids are learning to talk, even though they never hear anybody say, we go to the store, they say that, because they're starting to understand that put an ad on the end of a verb makes it past, right. So there is there are these interim stages in the development of expertise. And there is some decolletage at at a stage transition. And actually, my dissertation was on that. But what's really interesting is, once you're an expert, I'll give you an example. I was developing expertise and supply chain management technologies among blue collar workers at New York City transit. And I thought, well, this will be interesting, right? They were very nice to me, they helped me with my study, because they figured I'd never get a job. I didn't have any qualifications. And they weren't, they thought I was like a community college student. And they were trying to help me. So and I was Italian. And so were they so we had a lot of cappuccinos together on the shop floor. The thing that, that I noticed was, I would use card sorts to see how their operating mental model was working. When it came to the work that they did was they, they remanufactured air brakes. And you know, they had a very specific kind of cognitive bias and how they looked at, at manufacturing, that was very counter to the way Supply Chain Management, advanced technologies work. Roughly speaking, they start with the end and go backwards. Most people put everything they're going to need for a recipe on the table, and do it bottom up. And they were no different. Once we had done some work with them, and some gaming to transform the way they looked at their work. When I did the card sorts with them again to say, you know, show me basically what you need to get this started. They couldn't replicate their novice behavior. They said, oh, and I did it. I said, Well, how about this? They said, No, no, nobody would ever do it that way. That's a stupid way to do it. You know, you're gathering all this stuff before you need it. And then I showed him a videotape of that they did it that way. And they're like, Wow, who is that? Yeah, you know, and but I learned two things from that once when you are an intuitive expert, or you've reached another stage in your operating mental model, you can't go back. And also conscious awareness has nothing to do with it. Right? Yeah. And that's when I got really interested. I said, maybe what I'm doing with gaming is not teaching people anything. What I'm doing is I'm reorganizing an operating mental model. And then I started looking at what's going on there. Right? And it seemed as though and this is to answer your question, Brian, what's accelerated learning? What's really accelerated learning, in my opinion, is the implicit reorganization of mental models through iterative trial and error cycles. And I started to notice that, and I guess I should have known this because I worked with Catherine Nelson, who studied how the concept of time is created by language that the brain has no sense of time. You know, this whole 10,000 Hour Rule

Brian Marren: 9:32

Is junk.

Dr. Lia DiBello: 9:33

Yeah. Yeah, it is quite a logical time is just an accident of how education works. But actually, if you have enough cycles of trial and error, especially under certain kinds of time, pressure, the brain will adapt, and a story and we've with some of our gaming environments, we've really pushed this to the point where we got two to three areas of expertise and six hours, that was the fastest we've ever gotten. But we had, are now answering it all. Sorry.My phone is ringing.

Brian Marren: 10:13

Not not a problem

Greg Williams: 10:15

It's one of your fans.

Brian Marren: 10:16

Yeah, that's that's Malcolm Gladwell calling you right now and saying, Wait,

Dr. Lia DiBello: 10:21

I think he wants advice on for hair.You know,he could use it. Anyway, don't edit that out. The point is that, what we're what we've what our whole mission has been, since our that insight is, we know that the adaptive unconscious is there. And we know that it learns much faster and much more holographically than things in school. And we shouldn't be surprised because human beings have been learning for 170,000 years. And school has been around a couple 100 years for the average person, right? So it hasn't caught up. And then if you look at evidence, like people who are blind sighted, you know, who can see even though they don't have an occipital cortex, a visual cortex, yeah, we know that if you have a goal and you need to adapt, your brain will learn. But you may not experience consciously, what what it's doing.So then the challenge is, how do we reorganize the adaptive unconscious, for the most powerful form of expertise, because it does whatever it wants, it's not like you say, Okay, now be aware, for a few minutes. And, you know, try to control all these incredibly powerful and rapid processes in the direction I want you to go, doesn't happen. So what we do, because the adaptive unconscious is very idiosyncratic, it's got ADD, it looks at everything. And it's kind of a free association chaining machine, we create virtual environments. Now, a virtual environment is like a really good movie, it actually isn't capturing everything that's going on, it's actually very specifically focused on what the designer wants you to be overwhelmed by everything else is kind of back in the background. But your brain doesn't experience it that way. When you're in a really well designed virtual world, you experience it as the whole world.

Brian Marren: 12:42

So you're you just hit on, we could just do the whole couple of hours on everything you just brought up one because you also validated our approach our methodology to how we do training as well, and it with everything you said. So before we get into the virtual world part, because I want to get there and I, you know, I simulation training, I, when I call something simulation training, I use that the most general sense of simulation that could be to people, you know, not not on a computers in person simulating some event, right. So that's what I mean by simulation stuff. But you brought into a lot of everything that we address in terms of the unconscious and how it is and, and when we specifically when we're getting into human behavior, because you brought up different cognition biases, and how the brain actually learns, you know, we come at it from such a different approach, because we know that those fundamentals are true. And while we use a lot of images, and not a lot of words, and you know, let you use, like we teach a lexicon, but it's also kind of influenced by you. So it's your own lexicon, not so much to remember exactly what this term is, and what it means. It's what does it mean to you? And that sort of that sort of unconscious free play, you know, when we're talking specifically about human behavior, one of the things we tell people because you got into this sort of the Dreyfus model of subject matter expertise. And there's another one that talks about, you know, Kant being unconscious competence, and unconscious, competent, and what that all means, and we actually sort of flip it upside down a little bit. And what we tried to do for a little bit as we say, alright, in when it comes to human behavior, you're actually almost, you're an unconscious competent you, you have the skill set, I'm going to teach you, you're not aware of it yet. Because maybe you didn't get that language, maybe you didn't know how to articulate something. So we provide sort of a scientific lexicon that's legal, moral, ethical, that you can use to articulate so you almost become that conscious, competent. But then at the same time, we have to explain to people like, look, conscious awareness is don't worry about that part. You actually want it to be completely you want to get to the level where it is unconscious. So you brought up the learning process of that specific folks. It was the New York Transit System, I think, and like, you know, we love seeing that because we have those same experiences where someone's like, oh, no, yeah, I knew it was this and we're like, Yeah, But five minutes ago, you said it was at like, No, I would never say that. And you're like, Okay, well, I know learning has occurred. So that's good. And they've taken ownership of it. So they're like, No, I got to the answer. I was like, Well, yes, because use the framework. So I'm happy with that. But that sort of, I look at that, almost as we kind of explained it, like, top down versus bottom up processing and how people look at things, right. And I tell a story to buddies growing up with, and they're the most mechanically inclined people I've ever met, right, they fix everything, they own apartment buildings, their whole family did, they worked on everything, right. And one of them could look at literally like a pile of wood, and tell you exactly the size, you know, house and how many bedrooms you could do, you could build with that. Or the other one would look at like a completed house and go, alright, if you want to build that, here's everything you need, right? It's interesting how people, you know, we you can use both of those processes, it just depends on where you're at. And I think when you're getting into, you know, maybe you can explain it better when you get into that area of subject matter expertise, you sort of created whichever process works best for you, and then seen it over and over again, is that like kind of what you're getting at?

Dr. Lia DiBello: 16:06

I think what you're, what you're talking about is, we see a lot to where people are already experts, they're just experts in a way that's not working anymore, right. And they have all the content, because they've been doing this for decades. But it's reorganized into a sort of autopilot model, that's not going to work for the for the changes in the situation. So what we're doing there is, instead of going one to five and a Dreyfus model, we're going five to five, and you're reorganizing the content, but the process is very similar. It's just that the person who's already a subject matter expert has a lot more to work with. And in some ways, they're much easier, because they do have these incredibly rigid biases that are really getting in their way. But once you get them into that a trial and error activity cycle, what the brain is really doing is unlearning that and

Greg Williams: 17:08

And that's one of the things Brian, I want to hit reorganizing for something more adaptive, but the same content. And it's very interesting to see it happen. Whereas novices going through the same exercise, they're really starting with very little in the closet there, so they don't come up with as good of an outfit. And we you know, that's one of our sort of points that we make a lot is, is that don't give up on the old guides, they actually have quite a bit of content and can be sort of converted to be intuitive expertise, if they get the right methods of doing it. And it's gaming and, and rehearsal, and simulations. on. One of the reasons that I fell in love with your brain a long time ago, is that we're meaning Yulia, we, Brian and I are in the business of accelerated expertise in extremis, the situations being very kinetic war, or cops on the street involved in a very dangerous situation, where lives hang in the balance, they come very quickly. And there isn't a great degree of learning curve. And so what we have to create is context dependent, strategic or tactical, or operational rehearsals, and we have to reorganize, cognitively, just a certain amount of detail to create a response. So our decision point has to come much earlier in the process, and the decision can be okay, not great. And that's another thing that you do in your work, being able to identify those decision points early enough, that will have a profound outcome, temporarily. Right. And I love that because that's what we do. So we deal with, you know, body bombers and IDs and vehicle borne IEDs and snipers and those types of things. Well, just because you're dealing with the mining industry, and you're worried about the tread wear pattern on specific vehicles over time. It's the same issue. That's the beauty of it is that as long as you can create a context dependent world within which to rehearse realistically, I think that's the magic in Am I close on that? Or am I way off? You're right. In fact, a lot of our stuff is very relevant to law enforcement, because, you know, and I would say, the one of the things I've discovered working with a lot of domains is how much overlap there is. And also how like mining is very similar to pharmaceutical drug development because you're both in both situations. You're intuiting the value of of a raw material and transforming it into something valuable and mining is very similar to law enforcement because it's dangerous. Yeah. And one wrong move everybody is toast, you know, you can, you can, you can neglect tire tread on a on a multitone Hauler. And if the thing catches fire, the person driving the truck is history and historic. It's not like losing your tire on your Prius. So I think but but at the same time, when you become comfortable driving out multitone haul truck, you start to feel like it is your Prius. And you may Yeah, so helping people in simulations really show that the consequences are much more dire. And we can kill people, and especially virtual world simulations. And it's pretty traumatic for them, even though they're not really dead. I want the consequences piece as well. So the beauty of repeated rehearsals within a realm where you can make mistakes, and you can try to test hypotheses and you can skin your knee and go back over and over and over. And then consider the outcomes. We can't do that on the street, Although some people are, right?

Dr. Lia DiBello: 21:13

Yeah, they're doing it anyway.

Greg Williams: 21:16

There's discovery learning going on out there. And that's called case law.before the Supreme Court, no, no, you get what I mean,

Brian Marren: 21:24

that's, that was me, I would say, as a kid, I was the one who like grabbed the handle on the stove, burned their hand and then went with the other hand and did it to make sure that's gonna be like that every time. But you're, you're already you're so let's get into exactly what you do then with with ACS labs, and his virtual worlds and how you got into that, cuz you guys have and I'll have the links up in Episode details for folks to check out, just your your future view platform, and you've got all these other things that you work on, you know, because you can create a virtual world in simulate some of this right? So what how do you take what you know, is your expertise and what works and put it into a gaming engine? I mean, what is that? Like? How does that work?

Dr. Lia DiBello: 22:03

Well, first of all, I again, I have to be very clear about this. We, we have tried putting our stuff in gaming engines, like all of open sim, etcetera. And we got frustrated, because even though there was a lot of value in that, if we're cognitive scientists, we need something that does cognitive science, right? And there's no gaming engine for for gaming purposes that does that. So we said, Okay, we're gonna have to make it, and how our environment is different, even though it looks like a lot of multiplayer gaming environments, is it's a lot bigger, you can have a whole city in there if you want, if that's your, your concern, full object. In other words, if you need to train law enforcement people to be vigilant about the patterns of activity and a whole city, we can put the whole city in there. The other thing that we do that's I think important, is we track every decision, every micro decision by everybody. Where did they walk? When did they do it? How close did they get to an IED? Or to a bad guy? Did they notice any subtle indicators that the bad guy is doing bad things in that building? That you know that the simulation is what we call agent based, so it has a self knowledge that records your response to it, like he just walked by us into within two feet of what expert law enforcement person would see as an indicator of illegal activity, any missed it? That's data. Okay. So we make these smart environments, and we have people, we get McCall, I'm designing one right now. And not only do you have to find the bad guy, but you have to decide what to do about it, and evaluate all the risks and benefits. And the bad guy and all of his or her environmental constraints know, what you should be looking for, and are reporting on you. So it's like a smart world. And then all of that is stored in a very detailed log. And we can use that to drive detailed metrics. So that we can give you a heads up display, how many misses you had in that the last five minutes? And how many hits and you're going, you know, I didn't see anything well, it was there to see and you missed it. So you get that, feedback.

Brian Marren: 24:42

because it's not just what you do. It's what you

Dr. Lia DiBello: 24:42

And for businesses like mining, we also failed to do. It can sometimes be just as important like you I mean even to tie back to the to the tire tread example you get calculate the financial cost, right? precise, very detailed failing to inspect that tire could lead to a catastrophic

Greg Williams: 25:33

But those tires are cost prohibitive, there in lies another form of bias or fiduciary bias, because replacing those tires at the optimum level of replacement and going sending them back for retread or whatever. That's, meltdown of that vehicle then therefore that entire operation that's a big cost. And that cuts into my bottom line. And that means that if I go another day, or another days, or another week than I make more money, and who's really going to notice those little, you know, outside that well, when you go to the gaming engine, and you can see the consequences of your They're for that company. I mean, that's so significant. actions. And you can see that those mistakes, not only killed, but it kills productivity. And it costs more in the long run. That's where we are in law enforcement. Now law enforcement thinks that they can fix the dike by sticking, sticking a finger in it, and we've got plenty of fingers, we're doing Greg always sums it up is, you know, especially with law level, like, you just degraded the value of your asset, besides fine. And the idea is okay, but what about tomorrow? What about next week, and it's not, it's the cumulative effect is so damaging. And back early on, in my experimental phases, I looked at Hickson zip, because I was working on predictability, enforcement, you know, every tactical decision you make what's likely going to occur. So if a person was going to run from a copper, I could predict which backyards they would choose in which they wouldn't. For example, if there was a shed in the backyard, you're more likely to choose that if it had wood fence, instead of a cyclone fence, you're more likely to creates an operational certainty and a strategic unknown. So you choose that if there was a trailer parked in it, and no dog, you could use that to boost yourself over the fence and hide in the shed. So I was creating these mental models for campers with no computers, and, you know, no, no internet type of thing. And what was happening is they started becoming a best have to balance all of those things out. And I think a lot of practices, but you can prove it, you actually prove it in the world in the game, because I'm immersed in it. And I see the effects over time of all my decision. That's magic. Tell me about that. Well, and you make the mistakes. So in other words, we tell setting yourself on fire, you know. people do miss it, I just want to hit on that point, because people, okay, we're not going to tell you anything except what your goal is. And you got to figure it out. And you've got, you know, 10 minutes or less. And what they do is they do what they would normally do, they make all the mistakes they would it's incredibly measuring what you did do what we did. normally do, and they suffer the consequences. Or they miss all the cues, and they get to see that. And we don't tell them what which ones they miss, we just tell them, go back and do it again. It is a safe way to iteratively. You know, rehearse over and over again, until you figure it out. And our event generator, there's a couple of things about our platform, our event generator also changes the context that like reshuffles, the deck, right, maybe there won't be a shed in the backyard next time, so that you don't just memorize a solution. But you actually start to pay attention to how these things were organized and forward, simulate in your mind, what could happen. And then I think that the we're not sure why. But in businesses, when we show the financial cost of these things all the way up to stock price and market cap and also more locally and what it caused, that really affects people. Yeah, and they really start to understand how they're located. In the organization, it's almost like I used to explain this to students is the DNA of your whole body is in every cell. And that tells your cell what its job is you have to have that when you're working with an organization, you have to have not just the local consequences, but also how you affect the value of your or effectiveness of your whole organization. But you know, cynically speaking, the thing I worry about is there are people making a living on the chaos. Fixed course. You just nailed it, Leah. Yeah, with the DNA to the lowest common Donna nominator let's Greg is going to St it up now with all this cold medicine on board. One of the things like we go when we conduct a vulnerability assessment of physical structure, let's say and one of the things that they show us their book and they show us your cameras and they show us the protocols. They even rehearse them for us. And then we say okay, we're gonna take a break, have a cup of coffee because we got to catch up on our notes and they prop open the fire door, disable the alarm so they can step outside and smoke, right. That's the DNA of the whole animal being being influenced by the is one sell. And so we try to teach people that if you don't, a janitor sees more than an hour than most of your executives are going to see in a month. So if you don't train and heal holistically the whole event, you said something that's very iterative, but not intuitive. When you talk about a gaming engine that you build and and how yours works compared to a first person shooter, for example, first person shooter, don't worry, I have unlimited lives. And if I really, really, really get killed, there's a place around the corner that I can trade in and get more lives. And don't worry, if I make a mistake, I can keep trying the thing over and over. You know what they don't do, they don't do the consequences. Life has consequences. So if you if you have to map out how I'm going to get better. What's the difference? I can go through the model and never learn anything? Well, no, because the model is suddenly changing your experience. So if you don't follow good habits, you're never going to succeed. And you're not showing them something unrealistic. When you're building the model. You could build a model of Mars. And this is a question, you could build a model on Mars or some future that you've never been to, but you don't you build their business, so they can walk around in the closest thing to reality and pick things up and test them and sample hypotheses. That's amazing to me talk talk more about why that was a key because, you know, we all know, arousal goes up, performance goes down. So there's not a lot of flashbangs and funny, wonderful and shooting and all that other stuff. In your game. Your game is much more like life. Right?

Dr. Lia DiBello: 31:36

Right. Right. But it's if it's if it's your life, it's interesting to you. Yeah, exactly. Yeah, exactly. Right. One of my first experiences with that was actually to when we were making physical models, we worked with a uranium refinement company that makes nuclear fuel rods, and they were never profitable, they are now because of us. You know, you're welcome. And one of the issues was to make a model of their uranium refinement cascade, which is 72 acres, under roof. And we had to make a model of it that worked. And we had to have the little turbans and the cooler sublime errs fail after a certain number of hours of operation. So that's why virtual worlds are nice, because the smart objects were very hard to program. Anyway, when we showed it, these people did not want to be there. We had union and management together, and they thought the whole thing was going to be stupid. We brought them into the room, we took the big sheet off the big model, which was 22 feet long and running. We knew in union, they they couldn't stay seated, they kept. They're like, wow. Oh, they got their hands all over it. We're like sit down, we got to give you your onboarding kids. Yeah, we're not starting yet. Oh, but they did not want to it was irresistible to that. Because it was theirs. And amazing. Yeah. And it's the same thing with mining, you know, I show people our mining simulations, and they go, Wow, this is really hard to figure out, it's so big. But you put a mining person in there and they go, Wow. And they don't want to leave because it's intuitively like a mother tongue to them.

Greg Williams: 33:24

Exactly. There's a concept in a military called left seat right seat. And if I'm going to occupy an area of operation that Brian's unit is in, there's a seamless way of Brian and I going together through it, hey, this is the mosque, these are the cube key players here. This is a community well, then, you know, the cattle over there are from the next jurisdiction, you don't have to worry about that, because they're covered. And you trade this knowledge. And what we found is high functioning units, do a left seat, right seat, that that change of command very well. Low Functioning units, bring all of their biases with them. And so it's just like adversely influencing AI, you know, bad AI. Because if you feed the information poorly, you're gonna get a poor outcome, we could tell the units very early on where we're going to have trouble with, because they followed certain bad patterns, right? But then when you brought it up to him, they were like, No, now this, it's, you're missing the point. The point is, so we had to create a more structured environment, where they went through a series of checks and balances. Now, how do you do that? Because I know you do that. But how do you do that inside the game? What are some of the consequences? And how do you recreate them?

Dr. Lia DiBello: 34:34

Well, the most important thing in our simulations is the goal or the mission, right? Because that has the most powerful effect on the brain. The brain does not like to be wrong. So if you're not accomplishing the mission, and you get minute feedback, that you're getting further away, what you're doing is jeopardizing the success of the mission. Even where you walk, right The brain doesn't like that. And it's good to start to change it, change it up a bit, try new things to get a better get or get better micro feedback. Right? Now, in mining, we use just traffic lights, every time you do something, you get a red, green, or yellow, even a conversation with a subordinate is scored. Whether you walk by a hazard, you don't notice it scored. So you're constantly getting these traffic lights at you. And what we find is that after about 45 minutes to an hour, you're getting mostly green, because you don't like your brain doesn't like getting those threats, and you start being more vigilant and aware. So what we try to do is, we don't have a kind of linear sequence, we have a goal, and then how you get there is kind of up to you. And that allows you to make all the mistakes that you would make when you're on your own and get them reorganized to be more appropriate. But it also accommodates multiple entry points into the problem, right? It's almost a universal solution. And some people are more novice and some people are expert in the wrong way. It doesn't matter. It's like you're you're accomplishing the goal, you're not accomplishing cool, you're accomplishing the goal, you're not whatever you're doing, because you're a novice, or because you're an expert in the wrong way. You're right, getting Rhett's. So you got to figure it out. And

Greg Williams: 36:37

I like how you describe it as an expert in the wrong way. And you kind of hit on that at the beginning, too. And that's a lot of, we've seen that a lot in our work too, especially when you have a lot of someone who's been doing something for a really long time. And you know, the whole thing is because I used to make the joke like, you know, Brian, what you do for when someone asked me, Brian, what you do for a living, I go I teach old dogs new tricks. And they're like, you know, what do you mean, I was like, Look, you're you're an expert, I'm never gonna tell you how to do your job. Let's just rearrange it, we had, I've have seen it, probably you have to similarly with some of your clients where, you know, we had it was a police officer. And after our first day of training, or something came up, he's like, I'm now realizing how many situations I've been in where I, I was the contributing factor that caused that situation to happen the way I handled it and came in, just and they almost feel bad. And I feel like don't feel bad, you were trained, someone showed you how to do it that way. That's why you did it that way. Now, knowing what knowing what we know now, like you can change that. And you already have the knowledge skills, attitudes, attitudes, abilities to do that sometimes. But I'm curious to you, because this is sort of, I'll get to my question in a second, because you brought up a little bit about, you know, measurement and assessment is very difficult. I think people don't always know how to measure and assess whatever problem they're trying to solve. You can look at what happened, you know, after George Floyd was killed, that's that's cost, over $6 billion people have said, so we can do that with a monetary amount. But like, what, what's the cost to that neighborhood to our country to the trust into that you can't that as hard to measure. And and that came down to one person doing something that they shouldn't have been doing and not paying attention to situation. It's like, when you look at at these effects, we in the moment, we can't measure it. And even when I'm an expert in my domain, it's hard for me to measure it, because maybe we have something that our company wants to see what's the bottom line, but maybe, maybe that's the bottom line this quarter, and they're not thinking about over the next three years. And so that's going to change how I look at it. And so what I'm getting to is the cognition biases we come in with as a Smee, as a subject matter expert. Is there any, you know, similarities that you see across the board and people that come in that way? Like, what are the typical hangups? Or what is that bias that that person needs to have that light bulb moment? And how do you get them to they're like, I would love for someone to be listening to this conversation and go, Oh, wait a minute. Maybe you're and that's hard to do on a podcast, you have to do that during training. But but you what are those common things that you see with those cognition biases that get in our way? And how do you guys address that in your your training?

Dr. Lia DiBello: 39:19

Well, I think that there could be ways of measuring the impact of bad behavior, on the value of communities by by indirect things like people don't buy the house. It's in their name. Right. And you're right. And one of the things that that we've been able to do, and some of my colleagues are much better at this than I am is they look at innovation. Companies that have toxic leadership are not innovative. And they have high turnover. And there are pockets of what we call the O ring problem where bad news doesn't get passed up and bad things happen as a result. out, that tends to be a symptom of toxic leadership, for example, and that it's it's actually for us kind of easy to measure. We measure it in all these these financial metrics that already are kind of used by people who, who study publicly traded companies. Now, for something like a neighborhood, I don't think it would be that hard. I think it's the only reason, you know, business seems very mysterious, and very volatile and unpredictable. But people who need to predict, do it very well, I think that the same kind of attention could could be paid. And I think real estate prices, purchasing. Businesses that Do you know, businesses do a lot of research on traffic patterns before they decide to open a store on a street. They don't want to be there, you know, those are and then the tax revenue from the purchases to the city is not there. And those things are very easy to, to quantify and to put in a game.

Greg Williams: 41:08

Yeah, that makes so much sense. And let's look at a very, I like least objectionable outcomes, and I'm always looking at the street. So I'm thinking of like a teeter totter, or a scales of justice kind of thing. And when you look at certain games that Brian and I have been privy to, and it's fair to say, I think that Brian and I, and we are working together on a project that hopefully will will, you know, take off and everybody will finally see it for what it is. Because there's certain things that you can handle notionally, like the construct of the building the smells down on the street, unless they're very specific to meth lab or explosives or something like that. Those are certain things that can be great. We don't need to see them. They're not as impactful to our cognitive brains. But there's other things and specifically in mistake, genealogy, that are electrochemical essentials. Because whether it's good or it's bad, you get that electric, electrochemical neurotransmitter that's pumping in your brain, and you want to repeat that behavior. That's how casino can get us to almost win in combat. While mistakes are a lot like that. Right? So when when I see it in Brian, you brought up George Floyd, so I'm gonna go there. And I never gone to George Floyd on tape. But I'll say this, after George Floyd administration's looked at it as a situation judgment test. And they were coming in with If then, and you know what the bottom line on George Floyd was, we had to obviate the poorest decision that was made, what in the greater good of anyone was it incarcerating or arresting at that time, George Floyd, because it was a low level crime scenes, felony counterfeiting, he had pills, he had drugs, he had all these other things. But was he hurt and he wasn't hurting society, that kind of behavior went on in that society, every single day, somebody chose on this day, at this time, this is going to be the person that's going to jail. And they never once saw the spirals, they didn't see the ripples that were coming out from that major decision, and what I loved about what you're doing, as Chief Science Officer, let's remember that I love what you're doing is you're smacking us in the face with the obvious. And by doing it over and over a little subtly at the beginning, and maybe a little more powerful. We learn. And that's so simple. But it's amazing, isn't it? I mean, it's that simple, isn't.

Dr. Lia DiBello: 43:28

It is that simple. But when you think about how our species has survived over 100,000 years with no evolution, right, and we were living in a cave with the same brain that we're now using an iPhone, we are able to learn when we need need to adapt. And we don't need school for that we just need trial and error. And I also think that, that the reason the simulations work is because when the brain is wrong, at the most primitive level, we see it as a threat to our survival. We are adaptive survival machines, we're prey in the big, you know, and yet, we've figured out what we have to do to adapt equals survive from in our primitive brain. So that's why the games are hard to design, but also very powerful. We have to make sure we don't shut down learning, like activate the fear response, right. But at the same time, we want the brain to think that survival is a little bit threatened here. So I think that's why games are fun. No, I think that's why I think that's why Tetris is fun. There's others. There's a level at which if they all crash at the bottom, you think you're gonna die. And but you know, you're not so that tension is very important to activating the most powerful forms of learning. And that's, that's a good that's a well done Design simulation.

Brian Marren: 45:01

And no and you, you you, that was a great explanation because what you said too is, you know, you're able to learn when we need to adapt. And that need has to be there, right there we have whether that's, you know, we made up the need or there is an actual need to be there, right? It, we need to have that kind of electrochemical response from our brain, it has to buy into that. And we have to tie into that survival. But you also sort of put a limit on you said, you know, enough, but we can't create fear, right, we can't create this, this, oh, my gosh, you know, everything's gonna, you know, I'm gonna, if I do this wrong, you know, everyone's gonna die. Because I've seen the good and the bad and ugly in training. And that was even with in the infantry, immersive trainer up in on Camp Pendleton, where I'd see people setting up these scenarios. And at the end of the scenario, it's like, they get overwhelmed, and they all die. And I'm like, What are you doing like that you're, you're training that marine to think when this occurs, there's no way out and you're all going to die. Like, you have to have a way there has to be a way out, there has to be a win, I have to be able to at least see taste, touch smell some part of a victory here to get kind of my brain and I'm talking about in a training environment to be great, because I see a lot of, of what I would say, you know, well intentioned ways of utilizing technology that are just completely the wrong way to do it. And so it's virtual environments. But it turns into almost a first person shooter game, or, let's let's make you do a bunch of let's run some laps around the building, do some push ups, and then put the VR goggles on and go through. It's like, wait a minute, wait, wait, what, what are we trying to simulate here? And so that's kind of things like, what are those limitations? How do I set that up? If I'm listening to this going, like, hey, we do some virtual training, or I understand simulation a little bit like what, what things need to be there and what shouldn't be there? What should I focus on? What should I not focus on?

Dr. Lia DiBello: 47:00

Well, you know, we make our world VR accessible with the goggles, but everybody takes their goggles off. And I think it's because when you're trying to learn how your behavior affects the big picture, you do much better seeing yourself in the scene, and you'd like you do much better seeing the whole scene. And people intuitively I think what's happening is they're their pre conscious brain says, you know, how am I going to win, I'm not going to win and with within a first person view where I can only see four feet in front of me, and I have to, you know, I need I need almost a drone view of my life right now. And we make that available too. So but you know, VR and AR have their place. So I guess what I think about all the time is back in the 90s the internet was one thing. And those of us in academe had email, we had internet, my friends and I cracked, you know, hacked into the Kremlin. It was really fun. But, you know, the idea was that the general public can't have the internet. It's too dangerous. And I thought the internet is not one thing. It's whatever you do with it. And VR, or the metaverse is the same. Yeah, some of my colleagues are looking at what Matt is doing with their worlds, for example, and we go, wow, that is like so 2005 We met we made those mistakes, you know, and we're not, you know, we're intuitive experts, we're not even sure. What's so stupid about it. Exactly. I don't know if we could explain it to Zuckerberg. But the fact is that nobody's going to feel like they're in that world. Nobody's you know, that's like, you know, watching cartoon it will

Brian Marren: 48:54

Yeah, that's such a such a perfect sample of what I mean. And it's like, you're you're trying to recreate a real, you know, existence or real scenario or real experience, I should say, in this virtual world, why you don't you don't want to that's not your that your brain doesn't need that it doesn't want that it doesn't care about that. It's the certain elements that allow it to think and you just brought up one, you know, needing to see yourself in the scene in a you know, correcting the wrong but you're really talking about perspective, like if I have literally an overhead perspective, and I always tell people like because, you know, you hear the sayings, hey, you got to walk a mile in that person's shoes. You got to see it through their eyes. It's like, okay, psychologically, do you know how hard that is for human beings, your world through another person that sits next to impossible, which you can gain a better perspective, both literally and metaphorically, by sometimes like that, like, what's an overhead view? I see. It's funny when I see law enforcement agents I'm working with and they got this they get really nice drones and all this stuff and like they're using it to film their SWAT rate. I'm like, Hey, man, like we're this way. Let's let's let's open up this the capable when he's on the second a little bit, you know, really cool camera, but what are we using it for? How do we gain some advantage over the situation just to get more informed awareness here, right? I want to make a better decision. And so that perspective is huge. And that's also what I love about the whole idea of what you guys have built is like, Okay, I was not surprised when you said, well, people just end up taking off

Greg Williams: 50:17

Okay, physiologically, we have a the goggles when they stare at the screen, and they want to be there like that does not surprise me at all. functional field of view. And and we see in certain degrees, okay, if we're boys or girls six to 11 degrees of field of view, and we have a peripheral vision to make us orient towards problems. So AR not bashing augmented reality, or VR or any of the goggles. What they're trying to do is they're trying to say, you have to walk in these shoes at this time and look around your environment, which is great. But what happens is they lost sight of that's not physiologically how humans are. That's not chemically how we learn. And you know, Leah, when we were working back in the early days, and Iraq was kinetic. And people in America didn't know a lot about roadside bombs, what they built and they brought me to evaluate was an IED lane. And I said, Okay, it's an ID lane, you got to change the name. And they go away. And I said, Well, you're predisposing everybody for what they're going to find in your, why don't you call it a Domino's Pizza lane. And then when somebody comes in there, if they find the pizza, they get a free pizza, but every once in a while you blow somebody the fuck up. Now when the engine, right, and the idea was, at first I was I was advised that my methods weren't going to be it. Six months, nine months later, we had combat Hunter and it becomes a law of the land. The idea is that people sometimes think that they're doing the best thing to make VR and augmented reality better. But what they don't understand is they're needlessly and laboriously, laboriously waiting down cognitive load is being added where it doesn't need to be. Would you agree with that? Yeah. In fact, I really liked what David Eagleman said the neuroscientist that what we're always doing is creating a simulation in our own heads. And we have these two little holes in our head that gave us all the visual stuff. And we have some tactile stuff, some of us have, you know better than others. depending on you know, how well we move, and we still managed to construct a whole damn world with that very

Unknown: 52:30

well, we're not paying attention to everything we're paying attention to, you know, that that experiment with a gorilla runs across while you're trying to, I always miss the gorilla.

Greg Williams: 52:44

Basketball, right?

Dr. Lia DiBello: 52:46

I'm like, and I say, Yeah, I'm focused, like most people who are trying to solve a problem. And if you design your world, so that they're easy for people to reconstruct the experience in their own heads. And we've, we've put EGS on people to make sure that we're activating the, the motor cortex and stuff, we want people to feel like they're there. And not that they're watching somebody else's vision of what they would like the world to be like. And I think that, you know, for me, I think virtual worlds are the best for with a big screen are the best for changing how we make decisions in context with long term effects. But I also understand that, you know, even I didn't get that 20 years ago. And I think that, you know, we maybe it's like, I had my own trial and error cycles. And now we come up with something very powerful. And you also need to track the world has to tell you what, you know, your impact. That's why we use smart worlds. That's why we programmed our worlds to be smart and agent based.

Brian Marren: 54:00

And because that's how humans learn. I mean, I always looked at kids, how does a kid learn, like they fall down, they get back up, they try something a few times, they go back and you know, I mean, like you you just that that's what you're trying to, we always try to create in a training scenario. And I know when I have it, I love it. You know, when like you just said, you know, you'll actually hook people up to measurement. And we can just see it. Sometimes we're we're doing a observation exercise on two people meeting across the street somewhere. And you've got people on the radio and their voices up, and they're going, Oh, my God, he's coming over here. And they're trying to take notes and they're yelling and like that physiological arousal is happening. And we're like, Man, this is that to your brain. That might well be the real thing. It cognitively

Greg Williams: 54:40

Your brain doesn't care. It is it is it's constantly close enough. And you know, what we hear we hear from other trainers we hear well, we have a crawl, walk, run approach with a failure, your brain processes that environment. So stop doing it that way. And they're like, Well, what are you talking about? Well, you know, we've done this for years and you guys are coming in here. Oh my god. So we live up to the disrupt the name, but it's nowhere on our brand. But sometimes you have to disrupt it. Sometimes you have to kick the gosh damn money lenders out of the temple for somebody to take another look at what's going on. Right? Yeah. And what we love about your stuff is your stuff is all subtle, but it's profound, it's got a profound impact at the end, our training is specifically our in person training is much more in your face. Meaning that you've got to say, Listen, you have to disabuse some people of the idea that they can keep doing it the same way. And it's going to somehow change over time. It's like if it's a failure today, and it was a failure last week, and most of the things associated with their failures, you're probably going to fail. And it's because they're not training the right part of their brain, we see people again, will start where we ended with the 10,000 repetitions, and they say, you know, you can practice it. Well, you know what that reload drills never made you a better decision maker under stress. And so we stick to what we know, we're so laser focused about our lane, that we know where the lane markers are, how wide the lane is, and what the speed limit is. And a lot of trainers need to get back in. There's How do you cuz you're expensive. And you're expensive, because this stuff, because it works too. But I want to ask a question in here. I've had so many people that will come up and throw something in my face. Like they'll go well, in Canons book. And I'm like, Have you ever really read anything can't even wrote?

Dr. Lia DiBello: 56:25

Or talked to him

Greg Williams: 56:27

I know what I'm talking, yeah, Klein for that matter we've got to go out there on the street and actually use this shit. So so don't just throw me a bunch of platitudes, you know, because I know that they work. And the thing that that drives me to that question is you said something that I absolutely love you, I'll paraphrase said that scientists learn to, and that when we find a better way, we adapt over time. And that's amazing, because you're not who you were 25 years ago, and I'm certainly not that person. How does that strike people that have known you that whole time? I mean, do they come to you and go, Hey, you're, you're going back on your earlier work? Or you're changing your mind? How does

Dr. Lia DiBello: 57:07

Um, you know, actually, they don't, which is that work? kind of disturbing. You know, they, they, what they noticed about me is that I was always a moving target. Like, I was always asking myself, What am I doing wrong? You know, we know that, but just having a human brain that we're missing a lot. So we're always asking ourselves, what are we missing? And because I was always involved in that kind of self interrogation? I probably haven't changed in that sense. But what we do changes all the time. You know, we do we do expert worlds now, where we have a very specific capability that we're designing for, like, servant leadership. Right? So I mean, I would have even five years ago, not been sure that we could have done that. But I'm like, you know, what, how was it? Let's try it. Theoretically, it should work. And I think that, just being able to say, we don't know. I mean, the James Webb Telescope should be telling us, we don't know anything. I mean, we're, you know, a couple of years ago, we thought the world was flat compared to what we know now. So just always take that attitude. And I think it will be a good thing. But I think also, to that point. It, we have to have the same social bravery, we're going to make mistakes, we're going to be wrong. We're going to be ridiculous looking and 20 years from now. So what we got, you know, you gotta get there.

Greg Williams: 58:48

And I will get that takeaway. Hold on, hold on right there. That's a beautiful key takeaway, Brian. And I hate to interrupt you. But look, what Leah, what you're talking about is in police work, police are going to make mistakes, they are going to stumble, they do a lot of work out there. And what we have to do is we have to say we're giving them this really, really, really hard job. And we're giving them all these complex situations, sooner or later, that's going to fail somebody. So right now, what we're trying to do is we're trying to journo our way out of it by making them look like boobs, or we're trying to prosecute our way out, but pointing to them and going You're the problem. You're the one that made this mistake, rather than trying to fix it. And if we could take a more scientific approach, what do we do in science, okay? When we see that, that tech doesn't work, we go back to the hypothesis and we do hypothesis testing, till we find a new model that may work and then we test that one. If they would just listen to this broadcast. You said something else, Brian, just give me a minute. You said something else. You said that in the virtual world. People take off their goggles, we've seen the same we've seen it over and over. And if you can recreate it on your laptop computer, and it's just as powerful just as impactful. Then why spend all that money, comma, okay, I'm not gonna go there because we'll get crapped on all over the place, but He also said that immersive environments work. And we know that too. So it doesn't need to be all bells and whistles all the time. It doesn't have to be. So, you know, mind numbingly bright and flashy. What it has to be is it has to be real. But the real than it has to be as cognitively real. Is that a fair assessment?

Dr. Lia DiBello: 1:00:21

Yeah, I think so. And I think cognitive reality is relative. You know, if we go back to our uranium refinement cascade, I mean to anybody else, especially TSA, when we were trying to fly with it thought it was bizarre, and maybe a weapon, you know, but the people who work with those that equipment their whole lives, they thought it was gorgeous. And they thought it was wonderful. And I think that, that it's all about what's meaningful to you. And I think the mistake people are making with the metaverse and now I'm gonna get on my soapbox. Well, you do. Yeah. Is, is they're you know, they're they're gaming companies that are trying to be cognitive scientists, more cognitive scientists who are living with the reality that we have to use a game because it's easier to make a smart world in a virtual environment than it is to build a physical smart world. And it's more scalable and reusable. So it's a, it's a means to another end. But you know, when you start making stuff too flashy, first of all, it's creepy. Second of all, you're given people the message that you don't respect them, that you need to entertain them and entice them. And if you make it more like their real life, that's a much more respectful approach. You know, we made a mining world and we were showing it to some mining engineers, and they're like, yeah, it's too clean. You know, let it be dirtier. You know, it's, it's too cinematic. So we made it dirtier, you know, we made it sort of more gross. And they said, Hey, this, this is what we're talking about. That's the spirit. You know.

Greg Williams: 1:02:07

We made Hoberman, and we said, we're going to cut out all the flash and the whistles and everything. We're just going to use students as the actors. And because the actors have to record some pre recorded responses, because it's interactive. There's a time that they'll say, Yes, pause three to No. Yes. Pause it. So the feedback we get is, yeah, but compared to some of the other stuff that's out there, your actors blow, and it's like, okay, if you're focused on the actors, you weren't focused on the message that solve the problem, then they go back and they go, yeah, it was a wicked hard problem, but we were able to solve it. That's the key. The key is can Yeah, you know, it, like like, you could dress answer me. If I'm wrong, please tell me, you could dress your environment and mining and make it look like the Wizard of Oz, you're gonna have the flying monkeys, you could have all that would be gold on the street. And your brain will accept all of that, and still want to solve the Gosh, damn problem. They miss. And there's carnival atmosphere that we're getting. And those folks are at the shows Brian and I are in at the shows where we're teaching in the trenches, right? And you're going to be the show, make sure you tell us about that. But But Brian, and I don't get to go to the show. And at the show, they have wine tasting, and, you know, crew details and all that other stuff. And they talk the big thoughts, but you know what they're not doing, they're not moving the dial. And we're saying sometimes you got to take a giant evolutionary step backwards to make your training more cognitive. And they don't, they don't get it because they're saying, Okay, well, ours has a pneumatic device that makes the gun shake, and we have the limb immobilizer. And we have all of these things. And you know, that's wonderful. But I bet 10 years from now, iPhones still going to be around your shits not? Yeah.

Dr. Lia DiBello: 1:03:45

Right. Well, I mean, D Andrews, who used to have human performances, labs, and is a coPI. I with me on a project said that when they built the flight simulators, and they had all the turbulence and everything, experts don't pay attention to turbulence. They only pay attention to dials. And when I Yeah, and when I was in a flight simulator for a helicopter, I knew exactly what he meant. You are not focused on any of the other stuff. You're only focused on what the dials are telling you. Are you going to crash or not? And I think I think that that, particularly subject matter experts have, you know, they pick and choose what they focus on. They don't look at everything. And you've got to have that symbolic density there. That's what we call it.

Greg Williams: 1:04:35

Exactly, exactly. And so, too much cognitive load bad experts, the better they train themselves and experience, experiential, tacit, whatever knowledge they get, they can actually offload some of those things. And Brian and I have always looked at a computer to be just that I use my computer to help me free up this space so I can think about the hard thoughts that you know are the problems that need to be solved right now that are gonna impact our destiny? And it's, I gotta tell you, Brian, favorite episode, because when you're talking to somebody that's just smart. And you're Wile E. Coyote, super genius. So it's always fun having you on. And you challenge us. And that's good, too. Because there's a lot of people in the industry that are spending a lot of money on VR, and billions and trillions of dollars on VR. And AR could be better, better spent elsewhere. And and improved training at the same time.

Dr. Lia DiBello: 1:05:29

Yeah, no, I agree with you. And I think, you know, one of the reasons in my early career when I, my mentor was Silvia Scrivener, she was a pioneer in workplace culture, she wouldn't hire anybody that had not worked. She didn't want any peer students in her PhD program. And when we started doing research on workplaces, we had to go into workplaces and be almost professional apprentices, we had to learn what those people are doing, we had to be on the shop floor, coming from a business family, it was handy, because I knew how to do that. But I decided, you know, I'm never going to do research in a lab again, it's always gonna be with real people. Because, you know, you get 30 undergraduates, and you think you've learned something about how law enforcement is done from that. It's, you're just fooling, you're just trying to get tenure. on the backs of a lie, you gotta go at the street. And you've got to learn how to be there as a researcher without disrupting too much of what's going on. And a lot of our colleagues can't do that. I brought some when I was a professor, I would bring people, colleagues to like factories and stuff to show them, they always got themselves thrown out. I said, you got to realize nobody wants to see you walking around with a clipboard, looking down at them from the your glasses, you got to say, Look, I know nothing. Teach me imagine that I got to do your job in a month. What do I need to know? You know? Yeah, and and totally understand that I'm never going to get there. But I'd like to know a little bit of what it's like to be you. But at the same time, you know, one of the things that we learned with New York City Transit, for example, we ended up training 3000 people on a complex technology. And we increase the MTBF. The mean distance between failure of this old equipment, and the first pass yield on repairs was was way up. And we saved the property, hundreds of millions of dollars. And then I realized, we don't know how they're doing it. And we started analyzing their data entry patterns. And we realized there was quite a bit of homogeny, in how people saw the same problem. But it was way beyond us in terms of what they were seeing. And I said to myself, we don't need to understand this, we just need to know that they do. Yeah. And the only statistics we did was a shift a test to look for homogeny. And we knew that we know that experts are more alike than like each other than novices are, right? So I said, Let's just look for that in the data. And of course, the financial impact was huge. But you got to have that humility. When you're doing this research. You're never going to be that level five. Whatever. Let me just tell these people, right there. Yeah. Perfect that night.

Greg Williams: 1:08:45

And Brian. Well, while she's doing that, I'll tell you right now that Maslow is waiting his fist at us from the grave. Because that's exactly what we're talking about is that, you know, when you do in the lab, it's a great theorem, but that has nothing to do with real life.

Brian Marren: 1:08:58

You know, you have to get up knew and she brought up a good point there, too. And Leo Sowell will be respectful of your time here. I know you got we've all got other stuff, too. And we, I'm pushing off our dreaded call that we have after this, which is going to edit that out. Yeah, that's right. But, but know that the you brought up a good point there too, about sort of that being a good student, right. And we've had other folks that invited us, hey, we want you to come to our course. Well, it's not gonna be how you guys do it. And you're gonna be a little bored. Like, don't say that, like, I will come in there like I want to be. I'm going to have an empty notepad and I'm going to be staring at you and I want to what what is this? You know, let me take it all in like and that and I try to do that almost with everyone I meet too, because you find out if they're you because you'll find people like, we'll go to an organization like hey, this this girl over here, like she's really good at her job, like look at how much he does, and people don't even notice that stuff because of personality or whatever they gets overlooked and just just simply lead People tell you what's important to them. And then you brought up to a little bit ago, which I think is important, you know about sort of updating your hypothesis and looking at things differently and how such changes over time. Like, my whole thing is that I hope, I hope that someone 25 years from now listens to this podcast and goes, these people were morons, I can't believe we used to think this way. You know what I mean? Like, that's the goal, like, you look back at certain stuff and go phonology How did anyone ever think that the bumps on your head told you

Greg Williams: 1:10:29

I was the assistant for now, I put the bombs on a lot of people.

Brian Marren: 1:10:34

So it's, it's good and it but but again, a lot of people have a hard time letting go like, this is how it's worked for me. And this is how it's done. It's like you but it's a different world, it's a different context, you have to update that you still have all of this really, really, really great tacit knowledge from which you can draw on use that but conceptualize it in maybe this new context or with this new situation?

Dr. Lia DiBello: 1:10:55

Well, what I'd like to see in 25 years is that accelerated learning through iterative trial and error with gaming environments is like a no brainer. It's like, nobody is going to listen to a lecture, or do a PowerPoint, death by PowerPoint again, and everybody knows how they really learn. And it's just taken for granted. And, you know, even in my career, it wasn't taken for granted that adults could learn, you know, in my early career was assumed you get baked by the age of 21. And you're done. We now know that people change careers, or at least suffer big changes in their current career at least 12 times before they retire between 21 and retirement, we got to you know, that's got to be a given that that's not hard, it's not threatening, it's going to be fun to be different. And people embrace the journey, instead of trying to take a course. But we have competition, we have people making a lot of money, telling people that you got to do this for four years, you don't takes it takes maybe 20 hours.

Brian Marren: 1:12:07

Yeah, that's great. That's the way we love it, that because that's our approach to like, one, we're never gonna waste your time to you're going to be doing something the entire time. I'm here, like, I'm exhausted after teaching after a few days, because we have to constantly engage that students brain, no matter what, everywhere they turn, there's, there's some stimulus that that relates to the topic at hand. And we always tell them, Look, you're not gonna remember everything right now, in a week, you're gonna come back and go, that's exactly what they were talking about, you're going to see something and go, this is one of those things I need to act now versus waiting for this thing. And that's, that's that whole concept. But we try to do that accelerated learning, you know, in person, as much as virtual as we can with some of the folks that we work with. But you know, you know, how the tech folks are, it's like, well, it needs you know, to be better refresh rate on the screen, and the graphics need to be better, and I need more. It's like, No, you really don't, you just you just don't, you don't need any of that stuff. I just need to engage you cognitively. And then I'll have eaten the popcorn and you'll be watching. Yeah, my, my colleague calls it, pick and flesh it out of the pepper. It just keeps you busy. It's not adding any value right now. That's

Greg Williams: 1:13:25

Brian, I just got to tell you my favorite episode when I get to interact with somebody as smart as Leah Dibella. Leah, it's a lot of fun. I'm not saying that some of our guests are rocks on the floor. Some of them are some of them are on the show for very different.My sweet

Brian Marren: 1:13:45

things they've done wrong.

Greg Williams: 1:13:48

Now we want a fun time.

Brian Marren: 1:13:50

Yeah, we appreciate you hopping on here and talking about the stuff I could talk for hours. It's really fascinating. I love your approach and just your experience and how you do it. I took a couple pages of notes. And also I did it on is, like I said the beginning as a validation of a lot of what we do like we're like tough people see, like, I know, we put on a show and we're goofy people but we also really know what we're talking about here.

Dr. Lia DiBello: 1:14:15

So I was looking forward to this. It was really a lot of fun. We appreciate that. And thanks everyone. for tuning in. I'll put all of Leah's contact information for the ACSI she's remember, you know, the director, excuse me, Chief Scientist,

Greg Williams: 1:14:31

Chief Science Officer

Unknown: 1:14:34

will have the links up so you can check out their future view platform and everything she's got going on and link up with her on LinkedIn. And I do appreciate you coming on and everyone out there. Don't forget that training changes behavior.