Data Chefs Can Come From Anywhere, w/ Gartner's Lori Rodriguez

Lori Rodriguez has a unique perspective on business, she's been collecting data about and for C-suite technology leaders as Vice President-Executive Programs at Gartner.  In this episode, she shares her knowledge about technology and business as an expert in both fields.  Lori is an advocate for women in business and STEM fields and has authored the book due out soon-We Want You To Stay: The Hidden Lives of Twenty Women In Stem.  You can preview a chapter of the book and learn more HERE. Episode Timeline: 3:45 - Lori knows more about CIO's than just about anyone on the planet 8:20 - Digitalization and tech changes EVERYTHING and that change takes time to develop 21:20 - Role and responsibility swings between IT and Business 29:40 - The bullying of IT by the villains and the importance of process discipline 37:55 - Is the real villain the software industry? 44:20 - The first MS software that didn't suck, and the idea that software development is prejudiced or gender-biased 48:40 - Some startling statistics about women and minorities and why women aren't staying in STEM fields 1:02:25 - The P3 Power BI class gender statistics and the P3 hiring process yield some insight into the numbers of women in the data field 1:16:20 - Lori has some great advice for fathers with daughters about to enter college 1:25:10 - The interesting history of Lori's book, We Want You To Stay: The Hidden Lives of Twenty Women In Stem 1:31:55 - Rob makes the Ratatouille reference of the decade , The Data Cook can come from anywhere 1:37:10- The clearest explanation as to why P3 is the quickest to results BI consultancy-5 business days for a first tangible project iteration is common for P3's team 1:43:10 - Another crystal-clear explanation as to why Power BI is the best BI tool-with a nice bonus sports analogy! 1:49:25 - Lori shares her great perspective about the human side of Gartner's processes 1:52:35 - Power BI's dominance in Interoperability defined and discussed-The BI in Power BI should stand for Business Improvement: That's Rob and P3's philosophy! 1:59:20 - Rob's theory on something big that Microsoft is planning Episode Transcript: Rob Collie (00:00:00): Hello, everyone. This week's guest is Lori Rodriguez, a Vice President at Gartner, a company you may have heard of, and also author of the upcoming book, We Want You to Stay: The Hidden Lives of Twenty Women in STEM. Going in, I knew this was going to be a compelling conversation, but I honestly had no idea. This is by far the longest podcast we've recorded so far. And we weren't rambling, at least I don't think we were. It was compelling to me every single step of the way. Otherwise, we would've called it off. We would've bailed. We cover a lot of ground. Of course, we talk about her work at Gartner. We talk about her book. We talk about the unique challenges facing women in STEM fields. But the thing for me that I think led this to be such a long conversation was just how much valuable acquired wisdom was on display. Rob Collie (00:00:57): Lori has just a tremendous amount of experience and right at that critical junction between technology and business, which we're always talking about here on the show. And it's not just that she's worked at that intersection for so long, it's also the way that she's been observing it. This is someone that's constantly, it's just obvious that she's always been synthesizing and revising her models and her understanding of the world around her, the world that she's navigating through. And I like to think of myself as being similar. So we just had a great time. At one point, Tom had to drop and we continued for like another 90 minutes after that. That's how much fun we were having. Make sure you catch near the end, the reference to the movie, Ratatouille. I've been saving this Ratatouille reference for about 10 years for just the right moment. And it turns out I found it. I found exactly the time to use it. So, that was really gratifying. It's a long one, but I think it's worth it. You'll be the judge, of course. So, let's get into it. Announcer (00:01:59): Ladies and gentlemen, may I have your attention, please. Announcer (00:02:03): This is the Raw Data by P3 Podcast with your host Rob Collie and your co-host Thomas LaRock. Find out what the experts at P3 can do for your business. Go to powerpivotpro.com. Raw Data by P3 is data with a human element. Rob Collie (00:02:22): Welcome to the show, Lori Rodriguez. How are you today? Lori Rodriguez (00:02:26): Hey, Rob. I'm doing great. How are you? Rob Collie (00:02:28): So good. So good to have you on. I'm really glad we get to do this. Lori, you work at a company that people have probably heard of. Can you tell us what your job title is and stuff like that? Lori Rodriguez (00:02:38): Sure. So I'm Vice President Executive Programs at Gartner. What does that mean? I'm on the business side of Gartner. I'm not an analyst. Opinions are my own today. I'm also not an IT, I'm not a consultant, and I don't run any of our conferences. So I work on the business in the business, if that makes any sense. Rob Collie (00:02:58): Yeah, it does. But of course, no one would ever think that such a thing exists. Like from the outside, of course, Gartner's just a bunch of people that are experts in industries that write articles and review software and stay on top of trends. And that's it. That's all it is, right? And, of course, there's more behind the scenes, isn't there? Lori Rodriguez (00:03:16): Yes, there is. So specifically what I do is I lead our innovative initiatives and I build products and services just like you all do out there. And for my audience, it's CIOs, at least it used to be. And now it's all of those other senior, most tech leaders with all the little alphabet letters behind them, chief technology officers, chief data officers, chief data protection officers, and anything else they throw in there that has to do with the tech side of any business today. Rob Collie (00:03:45): Yeah. There's certainly an innovation in titles that has happened over the last few years. Like, people realize that the word chief, it could be thrown in front of many things. It didn't just have to be like the usual, right, we could put chief in front of anything. Lori Rodriguez (00:03:57): Absolutely. Chief success officer. Rob Collie (00:04:00): Oh, yeah. Lori Rodriguez (00:04:00): Chief customer officer. Rob Collie (00:04:02): Yeah. Lori Rodriguez (00:04:02): Chief customer experience officer. You can keep going. Rob Collie (00:04:05): Chief success officer, that seems to cover a lot of ground. I think that might be the end all, be all. Lori Rodriguez (00:04:10): Kind of think it isn't at the CEO, I don't know, but... Rob Collie (00:04:13): I don't know either, but success seems like it's important to everyone. Okay. So would you say the people with those sorts of titles are kind of like your customers? Lori Rodriguez (00:04:22): Yep. Those are my customers, not from an IT perspective view of customers, but external end user customers. So my job is to go understand what their needs are. I've interviewed over a thousand CIOs. I've actually been Jane Goodall, took my notepad and followed them around all day long, writing down who they're talking to, what devices they're using, things like that. I've done some of that work, day in the life work. Then I understand, well, okay, what is it they're trying to do? How are they getting that done today? And then where does Gartner fit into that model? Or not just Gartner, but anything that has to do with information sources, what are they using today? And where are the opportunities for Gartner to improve their products and services? Or if we have a gap in those, what else can we do to fulfill those needs so that our customers are happy, successful, and keep coming back and buying our products year after year. Rob Collie (00:05:17): So you're going to write a book called CIOs in the Mist. Lori Rodriguez (00:05:21): Yeah, could be, could be. They are strange animals. Rob Collie (00:05:24): Yeah. Yeah. And over a thousand of them, that's fascinating. Lori Rodriguez (00:05:27): Yeah. Rob Collie (00:05:28): How many people on the planet have had a similar experience to interview over a thousand CIOs? I mean, I bet it's single digits, single-digit human population that has had that kind of experience. And that's fascinating. One of the things that I've really appreciated in the last 11 years since leaving Microsoft, the nature of our business is so up tempo, so fast. Our whole ethos is to burn through projects as fast as we can on behalf of our clients, no padding, no overhead, nothing. Right? And one of the side effects of that is that we do get such a broad sampling of the world. Like if you're doing projects quickly, in order to stay in business, we have to do a lot of projects. And so we're just seeing, we're like drinking from the river of everything. Microsoft, they would send me on a field trip every six months maybe, to visit a customer. And I'd come back all full of customer energy like, I know what we should do now. Like that one one day spent with one customer- Lori Rodriguez (00:06:30): Data point of one. Rob Collie (00:06:30): Yeah. And so you end up with some very lopsided opinions about the world. But when I hear someone who has that breadth of sample, especially given our experience, I take note. Lori Rodriguez (00:06:40): Cool. Rob Collie (00:06:41): Like you really know what's going on in a way that most people don't get an opportunity to. That must be really fascinating. Lori Rodriguez (00:06:46): Yeah. It's why I love Gartner, why I love what I do. And it's a very unique perspective because I intentionally mentioned where I sit within Gartner because my audience are CIOs, they run IT departments, but I build products and I need my IT department to help me build that product. So it's very interesting when... And it's getting better. Thank God. But for so long CIOs tended to be far too functional and not businesslike. So I would always hear from them as the market research person, oh, we need to get a seat at the table. The big bad business is beating us up, blah, blah, blah, blah. I'm the business. And I'm listening to them talking in tech terms going like, no kidding, listen to you. I'm the business. Lori Rodriguez (00:07:34): So on the one hand, internally I'm going, yeah, you really need to understand business value of IT and speak in business terms, because you're not speaking to me in those terms. And if you use those terms with your business, I can see where you're having those issues. And on my side of it, I'm the business working with IT. Right? So I understand those problems from the business side as well. So it's a very unique perspective. I thought coming into things that I'm building these products for people and I'm using those products as well in IT. And I could see this huge disconnect. So I would go back to our research organization, I'd say, look, we can keep telling CIOs to talk in business terms and to move from being a functional CIO to more of a business strategist, right? You still have to maintain and manage your IT department, but if you want that seat at the table, you need to take it. And to take it and to lead there, you need to speak as the business in the business to lead the business. Lori Rodriguez (00:08:29): And we would always say that on our research. We said, we need to do more because we're not moving the needle. I keep hearing this year after year, after year. And there's been a significant shift in the last say three years in the conversations that I have with CIOs and senior IT leaders, which has been very positive in that business direction. And they had to, they had to because of digital business. You have to have your IT organization to be at a certain level of maturity before you can take on digitalization, for example. So that was happening and then COVID. And now we've seen all sorts of stuff, all these things that couldn't get done before, telemedicine, remote work, all that just got thrown on the table very quickly. Some people were prepared, others weren't and we've accelerated where we are in the maturity of IT organizations considerably. And I don't know the numbers, but I would imagine that a lot of CIOs maybe lost their job or will be losing their job because they were not up to speed. They weren't prepared. Rob Collie (00:09:28): On a previous show, I mentioned that the famous physicist Max Planck said that the whole notion that science moves forward through a meritocracy of ideas and the best ideas flow to the top, he basically said it was all bullshit. He said- Lori Rodriguez (00:09:43): Beta versus VHS. I mean, come on. Rob Collie (00:09:45): Right. He said, no, here's what happens. The better ideas come around and they're rejected. They're locked out. But then the old guard dies off. Then there is sort of a little bit of a meritocracy of idea, but it doesn't really have a chance to gain a foothold because basically what he was saying is, is that entrenched professionals don't change, I think is what he was saying. Keeping in mind, of course, you are speaking as yourself. You are not speaking on behalf of Gartner. I'm just dying to ask you, the progress that we've seen in the past few years, do you think it's more along the lines of Planck? Like it reflects a changeover, there's different people in those roles? Or do you think it's people actually kind of finally getting it? Lori Rodriguez (00:10:26): I think it's a combination of both. So fall of 2019 maybe, it was in 2019, early spring or fall, I did a set of interviews and I forget the question I was asking, but I kept hearing over and over again, CEO change, CEO change. So, chief executive officer had changed in the organization. And I was like, wow, random sample. So I wanted to go check that. So I did some research and it was true. I'm going to mess the numbers up, but it was something like in the course of two years, give or take 10%, 50% of... No, maybe 40% of the CEOs were turning over- Rob Collie (00:11:08): Wow. Lori Rodriguez (00:11:08): ... in the course of two years. Huge. And the board was bringing in CEOs who had more technology savviness about them, whether it was a tech background or they just were more savvy. Right? Changing of the guard, like you said, right? The old guard was changing. The board understood that to survive, they need to become, I hate to say the phrase, a technology company, but they had to leverage technology in a way they hadn't thought of before. Lori Rodriguez (00:11:34): And so to that point, there was this recognition we have to change. And when you change the CEO and they're mandated to change and use technology, it puts a lot of pressure on CIOs. So the downstream effect is, geez, CIO, if you're still over here and haven't matured your IT organization, you're probably in some big trouble in the next couple years, and then COVID hit. So, that just accelerated everything. I haven't looked at it, so I don't know whether companies kept the CIO that they had, because they had to because things were changing so quickly or if they were like, dump them and let's go with somebody else quickly. I don't know what that fallout was, but you'll see that change from those CEO shifts hitting the IT organization and then really making that cultural shift. To answer your question, where does that fall in terms of which side of the coin was that? Rob Collie (00:12:23): Yeah. I agree with you. I think it's both, right? Good ideas are one thing. Things that you can nod your head to and say yes, what you're telling me sounds correct, that's one thing, but it's different when it has that visceral power of reality behind it. Like you're watching your friends and peers maybe lose their jobs, lose their positions because they weren't flexible enough, that'll wake you up in the morning. So yeah, the world doesn't really move until it has to. Yeah. But I guess in the world of physics there weren't CEOs being changed over that told the physicist what to think. So Planck's rule, maybe it only halfway holds for IT. Lori Rodriguez (00:12:59): Yeah. But the thing is, change is slower, fast, depending on where your horizon is. So if you were looking... If your arc was pretty far out there, you saw this coming. You heard me talking about, I've been talking about this for years, digitalization is going to change everything industry by industry. You could see that coming. You had decades. And I'll talk about my journey. I come from marketing where everything was being bridge board and markers. We did everything by hand. And then they put a Mac IIcx on my desk in 1989 and everything changed. And in 12 to 18 months, it was really, really fast. These industries that were a hundred plus years old gone. Gone. Lori Rodriguez (00:13:48): There's a really complicated apprenticeship role called a stripper, of all things. And you had to practice for decades to do that. But once you obtained that level of expertise, it was a high-paying job and you were kind of set, right? That was gone. The strippers weren't needed anymore once you had access to Adobe, it was called Aldus at the time and other software that was coming out at that time, everything. And I was like, this is amazing. Fell in love with technology. And I could see then, this was just marketing. Who cares? It's bullshit, right? Like, you're marketing stuff. But you could see with this technology, what it could do, if you looked hard enough. You could see what it would do for government or get some sense of it, healthcare, education, buying a car, voting. You could look at any industry and see how digitalization was going to change that industry. Lori Rodriguez (00:14:41): And that's kind of where I just fell in love with technology and where I kind of eventually found that Gartner was doing that. And I wanted to be part of that. I wanted to be part of accelerating that digital change so that we could take advantage of what I saw when I was working in a creative studio in marketing. Rob Collie (00:14:59): Yeah. I love that journey. We talked about this a little bit backstage. To me, the future of most valuable things is at that junction between IT and business, between the subject matter expertise and the technology expertise. And the closer you can get the two together, the more effective you're going to be at the distance between them. And so the people who can speak both, the ambassadors, I sort of think of the ambassadors in some sense is sort of like, that's my tribe. That's who I belong to. I just love looking around and saying, ah, my tribe, our day has come. We are really important these days. I was fascinated by your marketing background in the beginning and the path that you've taken. In Mad Men, there's an episode where they bring the giant computer into the office, like in the early seventies or late sixties. And they're like, this is going to change everything. And, you know, watching it, it doesn't for a long time. Lori Rodriguez (00:15:50): A long time, a long time, 1989 to maybe like five years ago where people's... And we're still not there yet in a lot of industries, but the tipping point is happening. There's no going back. And how often in a lifetime do you get to go check the box on your mission? I mean, that day, that that box appeared on my desk became the day that my mission became clear to me. And here we are. That's so cool and to have a front row seat and even be able to make what small measure of progress I could do, that's been fantastic. Rob Collie (00:16:26): You know what's really funny, I've been going around for years telling people... And it's the truth. It's not a lie. I've gone back and looked. Most of the personal computer advertising that was done in the eighties, if you go back and look at the ads, you see that it was actually spreadsheets. The pictures are of spreadsheets and of charts. And so like the PC, everyone knows this, if you watch the documentaries, the killer app for the PC was the Spreadsheet. But there was also another killer app now, wasn't there, which was Publishing, the production of creative materials. And this is where the strippers lost their jobs, right? Lori Rodriguez (00:16:59): That's where the strippers lost their jobs. Rob Collie (00:16:59): That's right. Pagemaker put the strippers out of work. Lori Rodriguez (00:17:05): So Apple at the time was about to go out of business around that time period. I know that because when Steve Jobs came back, I went and took my entire 20-year-old retirement savings and bought Apple stock, which I still have today. Rob Collie (00:17:20): Well done. Lori Rodriguez (00:17:20): So it was good, good move. Effective rate of 33 cents a share. Thank you very much. And I knew, because I knew fonts and colors, it would take several years before any non-Apple product could replace the font system and the color management system that Apple had and publishers had invested millions of dollars of equipment that you couldn't turn over on a dime. So that was going to take long enough for Steve Jobs to come back and get back in the saddle of this organization and change it around. That was a bet I placed. It was a pretty good bet. Rob Collie (00:17:56): Did you also buy Bitcoin? Lori Rodriguez (00:17:58): Yes, I did. Rob Collie (00:17:59): Oh, man. So next stop, you're going to run a hedge fund. Lori Rodriguez (00:18:05): Just a one or two trick pony, so that's all I got on that bit. Rob Collie (00:18:07): Those are good tricks though. Lori Rodriguez (00:18:08): Yes. Rob Collie (00:18:09): If you're only going to have one or two, those are pretty good ones to have. Lori Rodriguez (00:18:12): Yep. But it's funny you mention Lotus 1-2-3. So, spreadsheets. Prior to that, it wasn't the first time I touched a computer. I was working as a secretary, they called them secretaries at the time, in a marketing company and I was reporting to the CFO and the CMO. And the PC came out, IBM PC DOS, black screen, green dots on it. And I went out and learned it. I brought it back and our CFO, all he did the whole week was about five spreadsheets. There's maybe 10 numbers you plug in, and he spent the entire week filling out by hand these spreadsheets. Lori Rodriguez (00:18:47): So I went and learned Lotus 1-2-3. It took me about a week to take those spreadsheets and do all the formulas. And then I plugged in those... The CMO got the numbers. So I would call, get the numbers. I'd plug them into the spreadsheet, hit go and printed out... And the guy was just jumping up and down. He was so excited. Wow, this is amazing. That was week one. Week two, I printed them out and he was like, okay, okay. Week three, the computer was gone. He took the computer out and we were no longer using the PC. Rob Collie (00:19:19): And somewhere Max Planck nods. Lori Rodriguez (00:19:22): Because he was this old dude. It's all he did all week. And he was made completely redundant by this little box that sat in the corner of his office and he removed it from the office. Rob Collie (00:19:34): You want to tell the alternate story, which is, at that moment, the CFO goes, oh, I am now free to do... I use a lot of sports metaphors, even though I wasn't really part of a lot of organized sports growing up. Doing the spreadsheet every week is sort of like a defensive thing, right? It's just sort of to keep the lights on. The opportunity to improve things, to go on the offensive, to advance a new initiative or a new line of thinking, like this guy now had so much time that he could have used effectively. But when moments like that happen, you find out that unfortunately, most people, not everyone, but most people do have a bit of a defensive mindset. Rob Collie (00:20:14): So yeah, this notion of offense and defense. And I wonder if the thing you were talking about earlier, like the CEO changeover and sort of the changing mindset amongst the CIOs, that's one of the things, when we do talk to CIOs or IT directors, we don't really come out and say it this way, but that's essentially what we're trying to say to people oftentimes is like, you can be part of the wins and not just the people who are noticed when something goes wrong because a lot of times trying to get people to do something different, their first response is this is going to land on me, isn't it? When things go wrong and we're like, well, that's how it is today. Imagine being involved in one of the wins, like something goes right and changes everything for everyone. It takes a little warming up to that idea. But it sounds like, just from what you've been saying, that even on a broader scale than what we see, things are kind of headed in a positive direction there. Lori Rodriguez (00:21:10): Yeah, absolutely. Completely agree. They had to. You can't be a digital organization in a defensive posture. And so a proliferation of those roles with all those different titles was a response to the fact, there wasn't anybody in the organization who felt responsible for bringing technology to those wins, right? Or enabling those wins through technology. A few years ago, it was interesting. I was like, where's that going to play? I actually mapped it out. I created this scenario of roles and tasks and then threw titles on the top of it, just to see, and then just played out for fun, because I'm stupid geeky that way, where could I anticipate these responsibilities heading because nobody was responsible for them? Lori Rodriguez (00:21:56): Well, let's take customer experience for example, who owns that? It's not the IT department, right? They're not good at that. They're math people. They're not psychologists. So then you think, well, who's responsible for any of that today. Well, the closest thing you come up with is the CMO in marketing, but they don't have the technology background. So that was a short-term play where there was a potential that things would go over there and maybe they still are in some places, but they weren't the right people either. So now it's sort of swinging back to IT, but I don't think that's landed really, who owns that yet? When I ask where does customer experience sit in an organization, it's reporting structure, it's not in a satisfying place right now. So we'll see where that goes. Lori Rodriguez (00:22:38): But that's a landscape that the CIO could take on, right? Because as an IT department, you are in a very unique position to have eyes on every part of your business. There's not many roles that have that and customer experience is that. It's the DNA of the organization or it should be. And so it does kind of pair nicely in the IT organization. If the IT organization is completely focused on governance, their favorite word, and effectiveness, you can't take on customer experience because governance and effectiveness is down here in a pace-layering model that changes slowly. And what you measure is very different in effectiveness than what you're measuring in customer experience and innovation. Lori Rodriguez (00:23:25): So you talked about speed. You got to have a lot of speed up there. So IT has to be able to wear those two different hats. Like one is fast and failing a lot, right? Their effectiveness is speed, agility and failing fast forward. When you're managing systems where you can't have failure, particularly if you're NASA, for example, or any business or anywhere, there might be a potential breach in that data that's going to cause harm to your organization or to your customer set, you can't fail. Like, that's a completely different model. So IT's got to reconcile how they would do both, but if they can't figure it out, that's a nice pairing. Or you end up with a chief technology officer or chief customer experience officer, whatever that other CXO is and you have to work very closely together because all of that innovation stuff has to be integrated fully into your systems or you end up with very siloed experiences and your customers are like, this is awful. This is an awful experience. Rob Collie (00:24:29): You mentioned something I thought was just absolutely spot on, which is the CIO does have sort of guardrail to guardrail exposure to everything that's going on, whereas on the business side, that usually isn't true. It's a bit more departmental. And I've got a short story to tell you that I think amplifies your point. This is now coming up on seven and a half years ago. This is a really cool story, especially when you put it in the seven and a half year ago context. So one of our clients, which I won't name, big company, 16 plus billion dollars a year in revenue, not exactly a small shop, they knew that the services they provide their customers are essentially commoditized. Like the types of equipment that they install for their customers, it's the same equipment that their competitors install. And so they knew deep down in their bones that the only thing that differentiates them, win or lose, from their competition is the quality of customer service that they provide. Rob Collie (00:25:29): But that is a long journey. It starts from a data perspective. It starts in the CRM. Someone inquired for a quote, did we even answer? Did we even get back to them? And it goes all the way through things like third-party surveys, like J.D. Power and things like that. The CEO of this corporation knew in 2013, knew all of this and knew that the number one thing that they needed to do strategically at their company was for now, first and foremost, just get a scorecard that told them how well they were doing. Improvements aside, how well are we doing everywhere? But they didn't have one. They actually had nothing measuring the quality of customer service because what they had was thousands and thousands and thousands of reports each coming from its own single-siloed system. And there were nine different systems that had something to say. Lori Rodriguez (00:26:27): And can you imagine the data architecture. I mean, even organizational size, what's the value of organizational size? Is it by number of employees, number of revenue bands? And are the revenue bands across all of those reports equal? You can't even have the data talk to each other. Even if they're just sitting in different components, the data doesn't even talk to each other. Rob Collie (00:26:46): That's right. As usual, the C-suite turns to one of their fixers. One of the things I really, really, really wish in this world, if the fixers had a consistent job title, because the fixers love us and we love the fixers, but they never have the same title. I've never run into two with the same title. They just happen to be like that lieutenant that someone turns to and goes, go kill. And this radical, absolute radical... In 2013, first of all, he went to IT and said, look, the CEO says, we need to do this. It's the most important thing the business could do over the next five years? IT said, yes, we understand. That makes sense to us. We'll get started in a year and a half. We're sunk amongst other things. There were some M&A or some spinoff types of things going on at the time. And they were really up to their eyeballs, but they're always up to their eyeballs, right? That's just the story. Rob Collie (00:27:33): He tried us. Again, that's what I mean by what a radical. Like the technology really hasn't changed. The stuff that we do at our company with Power BI, it's fundamentally the same as it was seven and a half years ago in terms of the real important things under the hood. But it was not a responsible choice back then, right? It wasn't established enough. Like they didn't have the reputation yet to be a safe career move, to try it. But we just knocked it out of the park. I mean, you talk about the data, doesn't talk to each other. It never had to really. It only came together in this data model that we built. Power BI didn't exist. It was still Power Pivot in Excel. And this thing, we were done essentially end to end in three months. Lori Rodriguez (00:28:17): How did you reconcile data values that didn't match up? Did you create your insulation engines? Rob Collie (00:28:24): You mean like IDs and keys? Lori Rodriguez (00:28:26): No. The example I gave you, like what a revenue band is for an organizational size, right? Is it one to $4 billion? One report would say, is revenue band A, let's say. Revenue band A is 4.1 onto 10. That's one report, right, has it modeled that way. Another report has it modeled zero to 1.5 billion is A. So they're completely, they don't match. You can't say this revenue band matches this revenue band, even if it's in different reports, because the values are different. Rob Collie (00:28:58): Well, we got to a lower level of source data. So it was down to like individual- Rob Collie (00:29:02): Of source data. So it was down to individual contracts, agreements with individual customers. So for example, one of the things, one of the factors that we measured was attrition. It was essentially a subscription business or like the monthly revenue in force. And we can see when contracts got canceled, you could assign attrition and it was dollar-weighted attrition. And the whole point of this was to make it drillable so that you could see that it was the same report at the C-suite level for the entire company, as it was for someone managing an individual office in an individual city. You just drill down and you'd see their versions of these same metrics. Lori Rodriguez (00:29:39): Wouldn't that just be what you would think, right? Rob Collie (00:29:42): Yeah totally. Lori Rodriguez (00:29:42): You have a view across the organization and everyone up and down and across, sees the data in a way that makes sense for them at their level permissioning and all that other stuff. But fundamentally, you can compare apples to apples. It seems so easy, right? But anyone who deals with data, and clearly your audience does, it's hard, it's the hardest thing. From the business side, you're going, "why can't you just, I don't understand. Just give me a report." But does this? Rob Collie (00:30:10): Yeah. At P-Three, we run a really, really tiny boutique competitor to Gartner in the form of me spouting opinions. I'm not restrained in any way. We're small enough that I could just speak my mind. For years and years and years I've absorbed so much messaging from the respected figures in industry, talking heads and analysts and things like that. They're always talking about what the next big thing and data is going to be. It's going to be this, it's going to be this, it's going to be this, and I keep looking around going, next big thing in data is doing the basics right for the first time ever. And it's a green field. Lori Rodriguez (00:30:47): Yeah. Clean data. Can we just start with some clean datasets and good data architecture and some governance around that? I'll take that to your point. You, we can do amazing stuff with data, but it's a house of cards if the foundation and the basics aren't done and the business doesn't understand that. Like I said, I wear this weird hat. And as the business side, I'm going like, oh, I don't understand any of that. Just make it happen. Order-taker status I get to have. On the other side, having to produce things and dealing with the folks who have to develop it, it's fricking hard. It's not that easy, business. Lori Rodriguez (00:31:25): It's really hard. And if you guys could get your act together across businesses and define what attrition means, define your KPIs in a way that you can actually have data sets where you could do some of the trending analysis and all the cool stuff you want to get to, but it's garbage in, garbage out. And the business itself is often the reason why because we bully IT and we bully our developers and we bully our programmers to say, this is how I want it. And it has to be done that way or I'll fire you kind of thing. Well, then that's what you get. You get what you ask for and we don't have the basics. We don't have clean data that we can build really cool stuff on top of. Thomas LaRock (00:32:04): Rob, remember I've always mentioned to you how nobody goes to school to be a data janitor. Rob Collie (00:32:10): Yeah. I was coming around to the same exact sort of thing like if you want an example of the behavior, you're talking about Lori, bullying IT, you can even go and look at how the business ends up implicitly treating its Excel gurus. The Excel people, which are in the business, the shadow IT that you talk about. They get beaten up in all the same ways that IT gets beaten up. Excel has for a very long time. Been sort of like, I think it's improving a little bit, but Excel has been a bad word in IT circles for a long time, it's a point of frustration. When I get them together, I'm like you folks, don't talk about Excel, talk about the people who use it and go get to know them and you will be stunned at how much you have in common; the two of you. Lori Rodriguez (00:32:55): True, true, true. Rob Collie (00:32:57): The number of times that people who are good at Excel, given a set of requirements, I need the numbers tomorrow morning. Can you give me the numbers? It's not even the report or whatever, like it's just the numbers. They just make it sound like it's such Childsplay. Lori Rodriguez (00:33:09): It's easy. Rob Collie (00:33:10): Yeah. Lori Rodriguez (00:33:11): Why can't I have these numbers? Well, because you've demanded them one way, your sales department has demanded those numbers another way, neither one of you will budge. You won't talk to each other. And so you create two separate. You have people in each side, whether it's IT, or whether it's the Excel business analysts or whomever. And they're creating reports because somebody higher up said, this is what it has to be. And then those two departments get together to present to the operating committee and their numbers don't line up. And then guess who's spending the weekend trying to reconcile the numbers. It's not the folks who won't talk to each other. You're laughing. So sounds good. Rob Collie (00:33:47): This is it. Over and over and over again. It's really kind of neat when you finally like plugged those two wires into each other, the IT and the Excel people. And the reason we have the opportunity to do this in our business, it's through power BI. It's the new stuff from Microsoft that is aimed at that Excel person. That's actually was my job at Microsoft before I left, was building these tools for those people. When you get them together, I'm going to whisper this, you can kind of sideline the villains in the story and the villains are going to be happy. They're still going to get what they want, but the people who are causing the problems are really not that important in the end once you have the right tools and the right culture in place. And it's just been really gratifying to see it. Lori Rodriguez (00:34:31): It is the right culture. It's not too often. I love the value thing, but it's also, we don't have to be, right? We have to be reasonable people. And we have to understand that every side has to hold up their side of things, right? Like you have to say, well, this is the requirements, we are non-negotiable and these are non-negotiable. If everybody understood, it's important to have non-negotiables and then you negotiate and then that's where guiding principles come in. All the stuff that people don't want to do up front, because they don't have the time, create the frameworks, create the guiding principles, create the data architecture or architecture in general. Oh, that terrible word nobody wants to hear. But if you do that upfront, you're just slotting stuff in and you have a very neutral objective way to negotiate those non-negotiables. And then, things move very quickly. Lori Rodriguez (00:35:24): You can get a lot of stuff done once you've done that foundational, the basic stuff, then you have speed after that. But what we do is we don't have time to do it right. So, we get it done and then we never have time to do it right and every project takes exponentially longer because the foundation isn't there. And after three years, you realize, "well, if you'd spent two weeks, maybe, up front doing this other work, you would've probably got three times as much work done at the end of the three years. Minimal, if you just spend a little time upfront negotiating the terms. Rob Collie (00:35:58): Yeah, I do want to circle back to the villain word because that process, you're talking about, process discipline, and things like that, that's hard. People don't really want to do that. Lori Rodriguez (00:36:05): Culture. Rob Collie (00:36:06): Yeah. Even at my own company, process discipline is not my forte. It's not my strength. Our president Kellen Danielson, he's been responsible, really. He's the backbone of a lot of growth over the past several years. Cause once we got to a certain scale, all the things you just said, I'm probably guilty as charged. Lori Rodriguez (00:36:27): We all are. We all are, but somebody needs to stand up there and do it, find a process, hire a process geek. When you look at your team, find somebody who has some love of process. There are people like, we all look at that and go, "oh God, that was horrible, whoever would want to do that." You're like, "There are people who love that stuff." Make sure somebody on your team volunteers. They're the kind of person who volunteers to take over the process things and then the flip side of that, right? I'm very bipolar that way is you can get into analysis paralysis and get stuck by process. The process is not the end game. The process is the rules that define the game that you're going to play. And they're worth putting down on paper, so you don't have these arguments and you end up, you end up with villains and hurt feelings and all sorts of other stuff. Rob Collie (00:37:14): Yeah, at our company, it's a joke, but I think it's the truth. It's that I've been saying for a while that Kellen and I combined are one complete leader. Lori Rodriguez (00:37:24): That's a good thing. Rob Collie (00:37:26): It's been really good. And I've always known that there were people who were good at processing who enjoyed it, but what I didn't do was I didn't respect it. I didn't respect it. I didn't really think it was valuable. If I look back, I would say, "I thought of it as the equivalent of holding the clipboard." I'm seeing with new eyes in the past several years. I have seen what it can do. I am now that visceral believer and the thing that I used to just nod and say, "mm-hmm (affirmative) of, course." Now, I'm the visceral believer in it. And it's a big difference. I really want to get to talking about your book, but I wanted to throw one more thing at you before we do that, which is you're talking about these villains and see what your reaction to this is. Rob Collie (00:38:02): I think that the single biggest villain and the decades of IT, business, conflict, and friction has actually been the software industry. I think that people like me, for a very long time, have built tools that basically contain all of the necessary and sufficient ingredients to create dysfunction. All of these processes we're talking about and a cultural change and getting on the same page and all of that, a lot of those things we're talking about actually can't happen unless the tools facilitate it. The tools have to at least allow for it and so much of the software that I used to build for it departments, I was a decision maker. I was designing this stuff. How does it behave? What are its capabilities? What are its feature set? I was baking in conflict, without knowing it. And I think there's this early, early glimmer of awareness now. This trend in the software industry is even younger and more immature than the one we were talking about with like CIO is coming around to a business mindset. It's like IT software has to be built for that middle ground between the business and IT. Lori Rodriguez (00:39:18): I'd agree, the only thing I'd change is that's been around forever. Create software that doesn't suck. That's been the mantra of open source and other people. So when you have the consumerization of IT, so people bringing in mobile devices and just like, we'd had it, right? We were living a world outside of work where technology was something we were dependent upon and then we'd go to work and we hated it, right? We hated IT department, we hated the tools we use, we hated all of that. Once we brought, we were like said we've had enough and sort of everybody at work created their own manifesto. Then I think that's where it sort of flipped to what you're saying. So it is sort of young in that aspect is that the internal tools and software we're building, we were the last mile, right? Lori Rodriguez (00:40:04): It doesn't matter what the associates are using. We had to focus everything on the client. I'm like, well, your associates are touching the client. And if their software sucks, it just translates down the line. One of the things that's happened and it's still happening today, this whole notion of customer experience, when I say customer experience, and I know IT folks, that's a bad word, right? So I get that, I just don't know what other label to throw. You have four things really under that, you have your customers, the people you sell to. You have your customer's customers in B2B, right? So who are they selling to? You have your associates, which is the part we're talking about building tools. So there's an experience for your associates, associate experience. Whatever you want to call it? And then there's internet of things. So things are going to be customers where you're talking specifically is associate software. Lori Rodriguez (00:40:55): I think. And what's happened is you remember the old prioritization you'd have 1, 2, 3 must haves, nice to haves. The only thing that ever got built were must haves, and those typically were along the lines of the function. Does it function? I need it to turn the lights on when I say "help I've fallen, I can't get up," and when I say, turn the lights on, it turns the lights on. But if it turned on floodlights and what you really needed was reading light or something, or it turned on your red light that spins and "Woo Woo Woo," that's not the kind of light I needed, but we put down as a requirement to turn the light on. So when you're thinking of needs analysis, there's a lot more than just does it do the thing that we said it was going to do. Lori Rodriguez (00:41:43): There's a whole lot of psychological and emotional and behavioral aspects along the lines of a requirement. So you have the logical ones. Does it, do you know the functional piece? Does it achieve the goal that we set out to achieve? Does it drive the behavior that we're expecting, but then you all will be aesthetics. Is it pleasing? Is it adoptable? Will I use it? If they're not going to use it, it doesn't matter all. If you've hit the functional requirements and we're still not there yet, we're much further along on the client side of things, because it's revenue impacting. They won't buy your product if it sucks, but you're forced to use software that sucks internally, but we shouldn't allow that, it's not going to be effective. You're not going to get the gain or the ROI that you expect. 70% of initiatives fail that touch the client, 70%. Lori Rodriguez (00:42:36): And why do they fail? Because they suck. They're not adoptable. So can you imagine how bad it is for associates? And that's why we feel it because it sucks. And we always think, oh, well, who cares? If it's pretty to use, well, guess what? Your users care. They won't use it. And they'll find ways around your software, which we all know. And so you have shadow IT, you have people who just refuse to use it, they build their own things, whatever it is. Lori Rodriguez (00:43:00): So, you know, what, why don't you just give into it, realize it upfront and understand what it takes. Stop thinking about how do we get this thing out the door to meet this functional requirements. Start thinking about really, that's not the end. The end is a little bit further. The end is when you have them adopting it and you're driving the results that you expected. We think over the line is the launch, over the line is the usage. And once we recognize that and we start measuring the usage piece of it as our criteria for success, as opposed to I successfully launched it, redefined success. And that'll change I hope, how we develop the software in the first place. Rob Collie (00:43:46): A note to the listeners here, you can tell Lori has been around a lot of software because she uses the insider technical term sucks. Cause that's what software does. Lori Rodriguez (00:43:56): I think that term came out the moment they launched the very first software that phrase came out and it's been part of software development ever since. Rob Collie (00:44:08): I remember meeting a customer one time when I was at Microsoft and then looking at us and saying, you know, we like to say that Microsoft software sucks less than the competitors, you know? And I was like, oh, what a compliment? I feel so warm inside. That's what we were aiming for. Lori Rodriguez (00:44:24): Microsoft was one of the main reasons why that phrase came out, software sucks. The first time I tried a Microsoft product that I said, "wow, who made this?" I knew it was Microsoft but I really was like, "did they buy a company? Acquire somebody was OneNote." Rob Collie (00:44:39): Yes. I knew you were going to say OneNote. Lori Rodriguez (00:44:41): That was the first time. Rob Collie (00:44:42): I knew it. Lori Rodriguez (00:44:42): First time. Rob Collie (00:44:43): I almost jumped ahead of you and said, you're going to say OneNote, aren't you? Right? Lori Rodriguez (00:44:47): It was awesome. I was like, this cannot be a Microsoft product. I'm like, they must've acquired this. And then I was like, well, find that development team and then clone them throughout Microsoft. So tell me, why did OneNote come out and why was that a good product that didn't suck or sucked a lot less? Rob Collie (00:45:07): All right, I don't know the whole story, but I'll tell you what I do now. First of all, the group program manager for OneNote and Microsoft, the program managers, you probably know this, but for the rest of everybody, the program managers, that was my job. We're part of the engineering team, but we were essentially a hybrid of design and engineering and also customer requirements and research and things like that. We would write all the specifications of what the software should do. We wouldn't actually implement anything, but if the software sucked, it was our fault. If there was a bug in the software that was the developers fault, the programmers or the testers or whatever, just an actual bug, but any design problem or capabilities or usability or any of that kind of stuff, that was all us. We were responsible for that. We made lots of mistakes and think about it. I was a computer science grad, what the hell business did I have in my early twenties? Lori Rodriguez (00:45:58): And you had no cognitive sciences background, either? Rob Collie (00:46:01): That's right. Well, hold on. Now as a little bit of an outlier and that I'd taken a couple of psychology courses, I was a philosophy, math and computer science triple major. So like you could already tell that I wasn't really all that into it from the technical side. So then I went to work for a monster technical company and that ivory tower like mathematical-mindset. Rob Collie (00:46:21): This is one of the reasons why software does suck, I think, is that in order to build it, you need the chess master type personalities that have been deep into code and deep into data structures and all that other stuff which is really kind of repulsive to most normal people. It's not a crowd that you want to go have a beer with typically. And they don't have a whole lot of real world experience. I like to say that they took me from college to Microsoft in a sealed underground tube so that I wouldn't polluted with any real-world knowledge on the way, from one campus to the next. And then sit you down in front of a desk and say, okay, now design software for the world, make decisions on behalf of multiple billions of adults who know more than you do. It's just so bizarre. Lori Rodriguez (00:47:03): And I'll tie this back to the book. So who were those people white males, nerdy white males, right? So then you got- Rob Collie (00:47:11): Check, check, and check. That's me. Yup. Lori Rodriguez (00:47:13): So you had code that doesn't recognize black skin in a camera, right? And you have software that doesn't recognize that moms handle computers differently than some guy sitting in front of a computer who played video games, their whole life. So you run into a world that is kind of like a left-handed person using right-handed scissors. Rob Collie (00:47:39): Yeah. It's this has gotten better. But I actually think at the time that I was doing this, like the late nineties, early 2000s, I think it's even worse. I think you're talking about an audience of people behind the scenes doing this at Microsoft who fundamentally didn't really understand human beings. There's a lot of refugees from humanity in the tech circles, at least back then- Lori Rodriguez (00:48:01): I love my tech buddies. Rob Collie (00:48:04): Yeah, I do too. But you, but you notice though- Lori Rodriguez (00:48:09): It's a different mindset, it's a different mindset. Rob Collie (00:48:10): It is, some of the most eccentric and difficult people that you'll ever meet are also the techies. You can assemble a group of friends at a social circle out of techies and it is awesome. But if you had to make your friends out of all of them, it might be a little, you might think a little differently. Lori Rodriguez (00:48:28): Who's getting kicked off the island first? Rob Collie (00:48:30): That's right. By the way, we won't go into this. But some of those people that I'm talking about that were difficult to get along with were in positions of enormous power at Microsoft. Let's leave it at that. So let's get to the book. First of all, what's the title of the book? Lori Rodriguez (00:48:48): It's called "We Want You To Stay- The Hidden Lives of 20 Women in STEM." STEM being, most of your audience knows, science, technology, engineering and math. Rob Collie (00:48:57): I guess I kind of always implicitly wondered if the M was medicine, but yeah. Okay. So it's math, the stem fields, is the book out yet? Lori Rodriguez (00:49:06): Nope. I'm hoping to get it out late spring 2021. So we're getting close down to the wire. Rob Collie (00:49:12): We're going to talk about the book a lot before I forget if you're listening to this and you're like, I really want to, I want to read this book someday. You can sign up, right? There's an email list. Lori Rodriguez (00:49:22): Go to stayinstem.com. So S T A Y I N S T E m.com, sign up, and you're early enough at this point that I'm also taking beta readers, so there's a free chapter out there and at the end of the chapter, there's a link to go provide me feedback. Look, I took this on like a product, so I'm building it with the audience. So that's why there's 20 women in stem. The bulk of the book is their autobiographies. And they're helping me decide where we focus and what we talk about and what's important and what's not. Lori Rodriguez (00:49:56): I'm interviewing dozens and dozens of young fathers, so men, women, women of all across all STEM across the world, in addition to the 20 women understanding, what are they up against? What are they running into? And in co-creating the book with this incredible audience of people. And now that the first chapter is in beta form, I want to know, is it boring? What did you find exciting about it? More of this, less of this, would you read another chapter and building it like I would do a product or any problem solving, using the frameworks that I've used to build products. So just applying that to the book. Rob Collie (00:50:33): Bringing that process discipline that we talked about earlier. Lori Rodriguez (00:50:36): Yes, exactly. Rob Collie (00:50:37): Unlike me who just sits down and says, all right, this isn't real until I start writing. Lori Rodriguez (00:50:42): Well, I did, I did a lot of that too. Rob Collie (00:50:45): So Stay In STEM, it's 20 stories, it's primarily 20 stories of women in STEM, sort of like their career stories. Lori Rodriguez (00:50:51): Not just their career. It is their stories, everything. It's the conversations they had with their husbands, it's the death of a child, a parent, the suicide of a partner, all of that, because that's what we bring. We bring all of that to the table. And we're human beings back to your point, whether you're men or women or you're human beings. And part of this is we need to understand that. There's three acts, the first act is why does this even matter, right? Like what is the current state? It's pretty dismal. I'll throw some numbers out, but why does it even matter? We started to touch on that a little bit and I'll give you a couple examples. I'll give you one right now, scooting around, looking for stats. I love data, actually. I suck at math, but actually I was really good at math as a young kid. And I was like, I'm done. I didn't see the need for it. Now. I wish I had spent a lot more time. Cause I love math actually. Rob Collie (00:51:46): Let's make sure we circle back around to that. I want to talk about that. Lori Rodriguez (00:51:48): Yeah, so I love stats and numbers and studies and digging into that. And so I have 800 rows of studies in a spreadsheet, somewhere that I use. So there's a current state. And then why does it matter? There's three reasons why. And the first one is pretty shocking. There's a whole book out there that I ran into that I just love. I'd already seen a couple of these numbers, but there's a whole book somebody did call "Invisible Woman." If you are in data, you have to read it. If you're a woman and you're reading that, grab a bottle of wine or tea or whatever, and you are going to get so mad, but you need to just chill to just calm down. So there's a lot of stats in that book. Like this one consumer reports did a study or was part of a study or that's where I read it. Lori Rodriguez (00:52:32): In a car crash, they take all the other variables away. How many drivers, level of what- Any car crash women are 17%, 17, one seven percent more likely to die in that car crash. And 73% more likely to have serious injury in that car crash. Why? Crash test dummies are set for the average male body that includes size and anatomy. So who made that decision? Who was in the room, who wasn't in the room? It's expensive, crashing a car, doing a car crash test is expensive. So you're going to do one or however many the government tells you, you have to do so to do one for men. And one for women you've doubled your cost. So who chose, guess what they chose? There are more women drivers in this world than there are men, driving is about equal, right? And whether you're driving or you're a passenger, it doesn't matter. Lori Rodriguez (00:53:27): The injuries are higher and there are stats all over like that. And so, and it gets worse for women of color. And I'll just give you a stat, it's related but separate. And when you think about artificial intelligence and you're building AI systems, think about autonomous vehicles and the choices an autonomous vehicle has to make. This camera does a real crap job of recognizing dark skin. Particularly if you have a dark skin and a light skin in the same photo, guess who it optimizes for. That's terrible, but that's Hey, I got crappy pictures. Now you put that same system and technology on an autonomous vehicle. You're going to end up with a lot of dead black and brown people before somebody recognizes it and does a report on it. And the old uproar and then we change that. Why don't we just make those decisions upfront in the best way to do that is to have diversity of visible minorities, as well as diversity of thought in those rooms where decisions are being made at every level, certainly at the top, but all the way down to the coders as well. Lori Rodriguez (00:54:35): Somebody has to go, wait a minute. Are you recognizing the color of skin it's disabilities too? I don't know if people are into game theory, again, nerdy thing. I kind of like. Rob Collie (00:54:45): Yeah, yeah, yeah. Lori Rodriguez (00:54:47): I forget what it's officially called, but there's one about a train is going down the tracks and it's going- Rob Collie (00:54:52): The trolley problem. Lori Rodriguez (00:54:53): The trolley problem, it's going to hit somebody, right? Rob Collie (00:54:55): Yeah. Lori Rodriguez (00:54:55): So you divert it. So let's say there's a bunch of people walking across and you got that trolley problem train. If the car keeps going, it's going to run somebody over. Well, it's going to divert probably to a non-human being. So it's going to crash someplace else. So you avoid the human-being, well, does it recognize somebody in a wheelchair as a human being? Rob Collie (00:55:16): Have you trained it? Lori Rodriguez (00:55:17): Yeah, it is going to get increasingly more important that we have people from diverse perspectives in the room where decisions are being made. So that's the first thing, really important. The second one is role models. Women make up 50% of the overall labor force, used to be less, but we are at parody in the labor force, but we're only 26% in STEM. So we got to do better on those numbers. And in some things like computer science, it's actually going backwards. We used to make up in the 60s, women were hidden figures. We were doing a lot because there was a data entry kind of stuff in, I forget the year, 1984, maybe somewhere around there, we were in the 37% range. We're at the 15% range now. So we're actually going backwards in computer sciences, pretty bad. So when you don't have role models, you don't think you have a career path in that organization. So that's a problem. And there's a lot of problems too around, I won't get into toxic environments because there's that kind of stuff too. Lori Rodriguez (00:56:20): But you just, you need role models. You need to say, oh, there's a path for me here. And you need to have that. So you need women in the decision-making levels, right? So up in the operating committee, on the boards and they need to be visible. You need more of them so people enter those fields in the first place so it's cyclical. It gets even worse, 26% of women in STEM, 53% of the 26% leave. So after 10 years, you're down to like 13% of the people. Rob Collie (00:56:46): Eek. Lori Rodriguez (00:56:46): Yeah, we leave. It's sucks to be a woman in STEM. It's just really, really freaking hard. Rob Collie (00:56:52): Okay. Let's zoom in on that really, for a moment. So the 20 something percent number, if you stride, it sort of like by cohort and you go like a number of years into career, that's the total of all human beings working in STEM. If you filter the audience to say with 10 plus years in field, then the numbers skew even further. Wow. Okay. So that I didn't absorb initially. That's amazing. Lori Rodriguez (00:57:17): Yep. More than half women, 10 plus years in leave. Rob Collie (00:57:21): Now you said something earlier that really spoke to me, which is that I like to say that the line between personal and professional is an illusion. It's a fiction invented to serve, I don't know managers, I have no idea. It wasn't invented to serve humanity. That's for sure. And so when I said career stories and you said, no, it's their life stories. I was like, oh, I just got corrected on my own principle, dammit. It's not supposed to happen. So when you said, why is it important, for me, the second reason, the one about if this is something that speaks to you, if STEM is something that is interesting and it seems like your calling, then what a shame to be one way or another discouraged from it. Cause again, coming back to that line between personal and professional- Rob Collie (00:58:03): ... Are discouraged from it. Because again, coming back to that line between personal and professional, we're talking about human happiness here. Rewarding work, valuable work. And everyone should have the equal opportunity to that. Rob Collie (00:58:13): The first thing you were saying about, "If we don't have better representation in software development teams or whatever," right? "Then, we're going to make these, continue to make these sorts of mistakes." I agree with that. Rob Collie (00:58:24): At the same time, I wonder if that's a way of just letting me off the hook. It's implicitly saying, "The white males are never going to change. They're going to keep doing the crash test dummies at a certain height and weight. And they're just never going to wise up." That's the environment I'm coming from. I completely agree that that is a problem. And I had to wake up from that nightmare and become a human again. Even in the latter half of my time with Microsoft, something really changed in me, and I became somewhat of an alien. And the people who were doing OneNote were that kind of people. They had had that kind of transformation. Like Owen Braun? Owen Braun sat next to me in New Employee Orientation in July of 1996. I met him on my first day at Microsoft, his first day at Microsoft. And he went on to run OneNote. Lori Rodriguez (00:59:10): Cool. Rob Collie (00:59:10): And he did an amazing job. Lori Rodriguez (00:59:12): It was a transformational product for Microsoft. Rob Collie (00:59:16): Yep. Lori Rodriguez (00:59:16): It felt like, from the outside. It's interesting to hear the backstory. Rob Collie (00:59:20): Yeah. I agree with you at the time. I was like, "Wow! This is a piece of software that I actually like! I'm happy with this thing. What's going on here?" Rob Collie (00:59:28): And it wasn't just Owen. I mean, there was a whole team there. It was a really interesting cultural outlier, even within the Office team at the time. Oh, it just warms my heart to know that we were both thinking of OneNote when you said that. Rob Collie (00:59:41): Okay. So if 50% plus leave after a certain number of years, that's probably happening beforehand too, right? The initial number, the number that make it to a STEM job, even for a little while, it was probably already whittled down. You go back through school, you go back through everything, right? And there's that same, "Hey, let's call it attrition." There's an attrition process that... It's not like it starts on the first day at work. It started in third grade. Lori Rodriguez (01:00:10): Yeah. There's been a fair amount of people looking into women in STEM, girls in STEM, girls who code. Which is awesome, right? I love it. But there wasn't anybody who was looking at why, really focused on this huge problem of women leaving. And if you think of a bucket with holes in it, it doesn't matter how much water you're putting in. Those holes are big. Lori Rodriguez (01:00:30): And when girls don't see women in STEM fields, whether that's on television, in books, magazines, newspapers, or in the companies that they're applying for, or the programs that colleges that are trying to introduce them or the professors, they're like, "Oh! This is a guy thing. It's not for me" or whatever. "It's going to be hard." Or it's the CIO of NASA that you had pointed out. "There weren't girls in my class." It wasn't just in the class, right? This is another, "Aha!" When I go talk to my friends, they're not in STEM classes. They're not in science and engineering or math classes. I don't have as much to talk to them about. And they're talking about whatever they're taking in humanities or something else. And I become less relevant even in my own circles. Lori Rodriguez (01:01:16): So when she, the CIO of NASA, when she tested high for engineering, she was like, "Uh-uh (negative). No way. I know what that means. I'm going to be alone again." And she became an economics major instead. Turns out, that passion for technology and math and science is hard to get rid of. And her career just kept circling around that. And eventually, she became CIO of NASA, which you can read about in the book and you can download that chapter when you go sign up on that stayinstem.com. Lori Rodriguez (01:01:45): So yeah, it's attrition, all the way through. But again, I look at root cause analysis and I'm looking at it. This is a problem I want to solve. Where can we really be most effective? And I think we have to stop the stem, pun intended, of women leaving the STEM fields. And if we plug those holes, we'll actually accelerate the number of girls coming in. Lori Rodriguez (01:02:10): So to me, this gets down to root cause and the right place to spend some time and money. So my time and my money at this point is... Rob Collie (01:02:22): Well, that shows that you're committed, right? Lori Rodriguez (01:02:24): Oh, yeah. In both meanings of the word. Rob Collie (01:02:27): That's right. Thomas LaRock (01:02:28): I have a question and a comment for right now. And I'm not asking this question in any way to take away from the importance of what I would equate to customer retention. If you have a girl or a woman in STEM, you want to keep them there. I believe that is important. But I am curious, because I'm a data person. How many men leave? Lori Rodriguez (01:02:49): Yeah. Let me see. I have that stat. Lori Rodriguez (01:02:51): So women leave it twice the levels. I forget the number. So if it's 53%, men are leaving at like 20-something. Thomas LaRock (01:02:58): Okay. And the reason I ask is because I was curious of about the overall retention rate for our field. Because I think of things as burnout. And I think of things, like when Rob talks about what Microsoft used to be like. And how some people can just say, "You could end up working in tech but you're working for this horrible Facebook-like company." And you just say, "I can't do this anymore." Lori Rodriguez (01:03:22): It's a very good question, especially from a data perspective. Because you can tell a lot of lies with facts. Thomas LaRock (01:03:27): Yeah. And that's what I was getting at. So knowing that's twice the rate between just those two genders is relevant. Thomas LaRock (01:03:35): So here's my comment. And my comment is actually to Rob. Because Rob? See, Rob used to tell me how he had solved the problem of women in technology. He had solved it. Rob Collie (01:03:46): Oh, I'm being set up here. This is what being set up sounds like. Go ahead. Thomas LaRock (01:03:50): So he used to tell me, though. Every class he would run, half the audience were women. Everywhere he went, his classrooms were, more often than not filled with women, beyond the, what I would say is, I think 30% of IT or 30% of technology workforce is women? Something like that? So anyway, he had a higher rate of that. Thomas LaRock (01:04:14): So I'm just wondering, Rob, are you still seeing a high number? Rob Collie (01:04:19): Yeah. I haven't taught one in a little while. It's been a year since I've taught a class. But I spent 10 years teaching essentially Power BI. Pretty technical. Pretty technical topic. Data modeling, star schemas, and all kinds of stuff. And my classes, oftentimes, it's volunteers, people who signed up for this class, they went out of their way to sign up for it. Other times, it's like a hostage situation where some manager decided, "We're going to teach the team." Rob Collie (01:04:48): But either way, I think, over time, it's been slightly more than 50% female, the students in my classes. So in terms of quantity, we're at parity in these classes. And you go, " Okay. Well, all right. Well, what about quality?" Well, again, every class, in the back of my head, I'm always identifying who I think is the best student over the course of those two or three days. Like, "Who's going to run with it the most?" And again, it's 50/50. Again, maybe a little bit more female. It runs with the population that I see in the audience. It's not like the dudes are just shining by comparison. I mean, I've always been just so happy about this, that it's, whatever the filters are, the attrition factors and things for STEM, when it comes time for these classes, that filter isn't being applied somehow. We're missing that filter. Lori Rodriguez (01:05:41): Did you think you were doing anything different? Were other classes getting the same kind of 50%? Rob Collie (01:05:48): Well, I don't know. You're right. They're possible. And of course, this would be really a really nice, self-serving narrative if there were something about the way that we described the classes or advertised the classes or whatever, ahead of time, that led to this, there might be a sampling bias, right? And that sampling bias is us, the way that we talk about things. That is one of the things that we do is, we talk about technology in a much more approachable way. Almost reflects my journey of being a technologist-turned-human over the course of my career at Microsoft and how much better things got when I became human. So we stuck with that. We bring that with us. But let's ignore that. Because I don't really think that's the case. Because even in the hostage situations, this is the case. Rob Collie (01:06:32): Let's talk about Excel for a moment. In the course of working in business, in any role, any sort of office role in business, you can think of it as a random particle thing. Like you're just bouncing around. You're this molecule, bouncing around. And sooner or later, you're going to collide with Excel. And most molecules, most people, when they collide with the Excel molecule, they bounce off. Fast and hard, like twice as fast, the other way, as they came in. Rob Collie (01:07:04): But some freaks, they stick. And that moment, who sticks to Excel versus bounce off. That thing? This is what I believe. And I don't know what the why here is. But I do believe, with some strong factual basis, that the people who stick to Excel do not skew male, that it is like 55/45 female. Rob Collie (01:07:27): And I like to think; this is where I start speculating as to why. Okay? I consider that first thing to be fact. Everything after this is speculation. The idea that, "I don't want to do the same thing over and over and over again." Like, "I don't want to manually repeat steps. I don't want to do meaningless work." I don't see why that would skew male. Right? Things that solve problems, like solving a mystery? All these things are things that you can get behind. Whereas, in math class, in high school or whatever, right? You have to absorb calculus as something, the reasons for knowing it, you have to take those on faith. Here I am, years later, working... Rob Collie (01:08:10): Basically, I do math for a living. We're primarily a BI company. And everyone was right in high school. We're never going to use this shit. I've never used it. None of it. A tremendous swath of my academic career was a lie that I fell for. But with something like Excel, you can just see the practical benefit. It's right there. At that moment, all the societal filters, whatever; they're just not there. Rob Collie (01:08:34): The one thing that is sad about this; and this is a really, I think, an interesting topic for discussion; is that, despite what I've been telling you, that it's like 55/45 women in these classes, and it's 55/45, best student is female in these classes; so you're checking all the boxes of what you would want to hear, what you'd want to be reality; only like 15% of the people who apply for jobs with us are female. Lori Rodriguez (01:08:58): So I'm going to throw some hypotheses out there. Rob Collie (01:09:00): Oh, I like hypotheses. Lori Rodriguez (01:09:02): Could be completely wrong. But they're based in some fact base, right? So I'm making some dot connections here that may not, should not be connected. Lori Rodriguez (01:09:11): So we talk about confidence, this notion of confidence. I'll give away one of the things in the book that I'm working around. Don't know what the type exactly, the labels yet. So this is new. We've got self-confidence, which is, "I've got self-confidence," right? But with women, there's a confidence and then there's imposter syndrome, right? Like, "I'm in this place and maybe I shouldn't be." So that was a term somebody came up with to take a subcategory of confidence. It didn't seem to fit the model, what everybody typically thought, of confidence. Lori Rodriguez (01:09:44): I think there's something else. Self-confidence and imposter syndrome is something you're feeling about yourself inside. When you're a visible minority, there's a whole bunch of other stuff that's happening that falls outside of imposter syndrome and self-confidence. You can know your shit and you can know that, that role that you're applying for, there is no one on the planet who's better qualified for you. Right? So you've got this level of self-confidence. You don't have imposter syndrome. You're not... Overconfidence, too. But let's just say that's not the case. You really have a good bead on reality. There's a whole bucket of stuff we're not talking about. It's like, I can know that I can do that job. What I don't know is, "Do you believe I can? Do you believe that I can do this job?" Lori Rodriguez (01:10:35): And it's not only that weight of not knowing. You, then, can't pull apart what is reality and unreality in that framework. When you don't get that job or you're not given that promotion or you're not getting that high-visible project, you don't know if it's something you can control, right? Like, is this, "Am I doing a good job?" I'll get around to why this is important with the Excel. "What can I control? And what can't I control?" Lori Rodriguez (01:11:02): Someone came up with this really cool term. I don't know if it's that yet. But I like it. It's called quantum confidence. And she was like, "Because you're confident and you're unconfident at the same time." I'm like, "That's..." Rob Collie (01:11:13): Schrodinger's confidence. Lori Rodriguez (01:11:15): Yeah! There you go! All right! Cool! Lori Rodriguez (01:11:18): So what does that fall into this? Well, women don't apply. And I'll add another dot. Women don't apply for a position till they feel they can do that job at a 100% and men don't. So if you don't feel you can do that job at 100%, you're going to go try to get those skills. Lori Rodriguez (01:11:34): So are they going to these courses? Not the hostage situation. But are they going out to upskill, to gain a level of confidence to apply? And that's the self-confidence to apply. But then, they're not actually gaining that to the degree that they're applying. So they're getting the skill set but there's still being something holding them back from actually applying for positions. Lori Rodriguez (01:11:56): But I think those two factors may have something to do with what you're seeing. Rob Collie (01:12:01): I believe that. Rob Collie (01:12:02): Let me ask you a piece of advice on this topic. So something about us is, I think, unusual, is the screening process that we use, the interview process that we use for hiring. In fact, our PR company has seized on this and we've written now multiple articles for various outlets about our hiring process because it is so different. There's a lot about it that would be interesting to talk about. But we have lots to talk about. So I'm not going to drag us through all the details. Rob Collie (01:12:27): But here's the thing that I think that's relevant here, is that it's incredibly selective. I think we have like a 2% or 3% pass rate on this process. And it's actually, without going into much detail, it's actually very, very, very, intentionally very objective. It's a test. And it simulates the job that you actually have to do. So it takes a tremendous amount of all the human judgment out of it. It's pretty close to blind in a lot of ways. But we never see the person. By the time we talk to them, face-to-face, on camera, or whatever, we've already decided we're hiring them. Lori Rodriguez (01:13:01): Do you black out the names too? Or just the... Rob Collie (01:13:04): No, we don't. We don't. But you either get this thing right or you don't. We really have removed... You never remove 100% of the judgment, right? We're like 98% judgment's been removed. We know whether someone can do the job or not. And a lot of thought went into this process to do what it does. Some of it's secret even. Like if we talk too much about it, it won't work anymore. Rob Collie (01:13:27): So when I'm teaching these classes, right? A lot of times, people come up to me, in the hallway or whatever, like during breaks. And there's, "What about coming to work for you?" The one thing that I hate about our interview is that it discourages 97% to 98% of the applicants. We turn away really good people. Because we need to unquestionably know that you're excellent. The type of work that we do, oftentimes, you're out there on an island, one person, dealing with some very difficult situations. In order to move fast, you can't have a huge team with you at all times. So we have some really stringent requirements we have for the job. Rob Collie (01:14:07): So on one hand, I'm always wanting to tell the woman standing in front of me, asking me. It's like, "Come on! Apply for a job!" Right? But then, at the back of my head, I know, 97% of the time, we tell people they aren't good enough. I hate that. I just hate it. Rob Collie (01:14:25): So if I suddenly went out in the world and actually convinced women to apply at the same rate that men do; let's say I managed to wave that magic wand; I'm actually weaponizing a bad message. Lori Rodriguez (01:14:36): If it's fair and you tell them up front? "Look. I'd love for you to apply. Just letting you know, 97%." People want a chance to prove themselves. Rob Collie (01:14:45): Yeah. Lori Rodriguez (01:14:45): And we're not being given a chance. So that's one. So the one thing is, "Yeah, go ahead!" Tell that person, up front. Just say, "Look. This process is rigorous." Whatever. Rob Collie (01:14:53): I do that. Lori Rodriguez (01:14:54): Or, you could just say, "If you want to, that's fine. I don't discuss this kind of thing. I teach the class," blah, blah, blah. Whatever you want to come up, to either, to pass everybody, or you can say the same thing. Rob Collie (01:15:05): I tell the truth. And I encourage them to apply. But here's the thing. I'm cognizant of whatever this dark matter is, this mystery, this other kind of confidence or whatever, right? If this person I'm talking to is trying to figure it out, they're hovering on the edge of, "Do I belong or not?" If a man is less likely to be feeling that way? Lori Rodriguez (01:15:28): Yeah. Totally. Rob Collie (01:15:30): And the woman is more likely to be feeling that way, I think that our interview might cause more harm to someone who's on that border. But I still do it. I'm like, "Look. This is the numbers. I would absolutely love for you to apply. I want you to succeed." Lori Rodriguez (01:15:43): You said the fragility isn't in the trying and failing. It's the belief that you can, right? Rob Collie (01:15:49): Mm-hmm (affirmative). Lori Rodriguez (01:15:50): The belief that somebody else has in you. You can talk about, "Look. I believe you can do it. You may not be ready right now but you could do it. So apply, see where you are." Rob Collie (01:15:58): Yeah. Lori Rodriguez (01:15:59): That's a very different message than, "you suck at math," you know? Or whatever. Lori Rodriguez (01:16:04): So there needs to be advocates and allies so, when there aren't women in the room, somebody is throwing their name in the ring. And a notion of, "Give people a chance. Tell them you believe in them." Lori Rodriguez (01:16:17): The other thing is, you can go out and aggressively recruit from different places. Thomas LaRock (01:16:22): This is very similar to how, when you're organizing a conference and you find out the conference has no women speakers. And you go to the organizers and they say, "We just can't find anybody." And you're like, "You're not looking. You really not looking." You can't find anybody? That says something, either about you or about your event maybe not being a safe space. So there's so much to unpack with that. Thomas LaRock (01:16:46): But here's my question. My daughter, high school senior, she's been accepted to all the schools she wants to go to. What should I be doing for her at this moment in time? Lori Rodriguez (01:17:00): The dishes. Thomas LaRock (01:17:01): Yeah? Lori Rodriguez (01:17:01): And the laundry. Thomas LaRock (01:17:02): Okay. Lori Rodriguez (01:17:03): And the housework. Thomas LaRock (01:17:04): So I'm going to disagree with a lot of that. Because those are basic skills she'll need in life later on for herself. Lori Rodriguez (01:17:11): No, no, no. You. You as dad. Thomas LaRock (01:17:12): Yeah. Lori Rodriguez (01:17:12): You as dad. Thomas LaRock (01:17:13): Yeah? Lori Rodriguez (01:17:13): Need to be seen. Thomas LaRock (01:17:16): Yeah. Lori Rodriguez (01:17:16): Because this gets to why women stay. Thomas LaRock (01:17:18): Okay. So I do do all those things though. Lori Rodriguez (01:17:20): Awesome. Great dad. At the level... Thomas LaRock (01:17:31): Okay. But she also needs to do those things. Lori Rodriguez (01:17:32): Oh, clearly. Clearly. You need to learn how to do that on your own. Thomas LaRock (01:17:32): Okay. Just so we're clear. Lori Rodriguez (01:17:32): Yeah. No. So we're clear. Thomas LaRock (01:17:32): Just so more clear. She needs to do those things. Lori Rodriguez (01:17:32): She needs to be able to boil water. Thomas LaRock (01:17:33): Yeah. Yes. Thomas LaRock (01:17:34): What else, though? How can I be of the most help for her over this next stretch of her life in order to give her that foundation? And she may choose STEM and then leave for some other reason. But I just want to make sure or that I'm doing what I can control, right? Lori Rodriguez (01:17:55): I love it. It's a great question. Lori Rodriguez (01:17:57): And I was joking. But also, a huge reason why women leave is the disproportionate amount of hours that they have to spend when life gets complicated. Doing the childcare, the elder care, the household management. We'll talk about that separately. That is a big piece. Lori Rodriguez (01:18:15): There's something that you said that actually is key that sounds like you're already there. And that is, the ability for her to be independent and feel like she can do things like cook and travel and whatever. And the belief that their parents have in their children and that their children know. So doing things like, when they're little, but this is something you can still do now. Lori Rodriguez (01:18:36): One of the women, an astronaut in the book, she recalls being a little girl. And her father was a pilot. He had a plane in Farmville, town of 1,100. And she'd go in the hanger. And he'd say, "Hand me the wrench. Can you fix this? Bang on that." So he had her, as 5, 6, 7, as long as she can remember, she was helping her dad build his plane. And he told her she could do everything. And he not only told her, "You could be anything you want." He gave her responsibilities at a young age that, to this day, that carries with her. And looked up at the stars with him at night and said, "Wow! Imagine what it would be like to be up there one day?" Because this was around the time of the Space Race. And said, "Yeah! You could do that. Of course, you could be an astronaut!" Even though there were no women astronauts, and even though you had to be a fighter pilot; so you couldn't even be that because women weren't allowed in combat positions; her father says, "You can do anything." But then, he backed it up by believing that she could. Lori Rodriguez (01:19:38): And we have to be, as parents, you have this dual conflicting feelings. "I have to keep them safe and coddle them." But then, at the other hand, you have to toss them out in the world and go, "You can swim." My dad used to, we'd stand on the side and he'd go, "Good! Jump! Jump! I'll catch you." And then, when you'd jump in the water, and he'd back away. And you'd go under and you're drowning and whatever. And that's how we were taught to learn to swim. So literally, push you in the pool and you're drowning. And he just figured, "You're going to swim. But if you are drowning, I'll catch you." As parents, we have to figure that out. Lori Rodriguez (01:20:10): So as your daughter's going through this, she's going to go through, especially if she does STEM, and engineering in particular, engineering's freaking hard. You're going to fail classes or get grades you've never seen before. You have to say, "Stick it out! You can do it." Don't ever leave because you don't think you can. Leave because you decide you don't like it anymore and not like it because it's too hard. But just, "Work hard and you can do it. Be okay with failing a little bit." And as parents, allow them to do that and be there to catch them if it's too hard. Thomas LaRock (01:20:43): So here's her current field of interest. Forensics accounting. Lori Rodriguez (01:20:49): Cool. Thomas LaRock (01:20:49): Right? I didn't even know this was a thing. Lori Rodriguez (01:20:51): Cool. That's cool. Thomas LaRock (01:20:52): And we're at this school. And we're in the bookstore, which the school still has, for some stupid reason. She pulls out this book. She goes, "Yeah. Here's..." And it just says forensics accounting on it. I'm like... So that's like FBI stuff. That's like real... Thomas LaRock (01:21:07): Anyway. Here's the question. Is accounting a STEM career choice? Lori Rodriguez (01:21:12): It's a STEM field. It's math. So you're doing math. Yeah, absolutely. You're doing the math piece. Lori Rodriguez (01:21:16): There's a woman in the book. Oh, she's just amazing. At 25, she was a CFO. 25 years old. CFO of a celebrity law firm that everybody's, you would know the name of. And then, at 26, one of the clients asked her to be CFO for their organization. And you're like, "26?" If she were a guy, we'd be celebrating, right? "Prodigy, prodigy!" The whispers in the hallways, the whole thing. But it was really, really hostile. This woman is, talk about confidence. I've never met anyone with as much confidence as this woman. She went to Montessori schools. You just figure it out. You just go find the thing you like to do today as a eight year old or whatever it is. You go, "Go do it. Here are things. And we'll kind of help you out." Lori Rodriguez (01:22:02): And so, she started doing internships in high school, on her own. And so, all through college, I was like, "How did you get all those internships? They must've had a really good internship program at your school." She said, "No. I found all those on my own." So every time she wanted to learn something new, she moved on. It was like, "Okay. I learned what I needed from this job. Now, to move on to the next, I want to learn this." Lori Rodriguez (01:22:23): So I added up her experience in internships as if it were a regular pathway on a resume. And she was at like 34, 35 years old. So a 35-year-old CFO is like a normal thing. She had just condensed all that, starting in high school, with the jobs that she'd had. Lori Rodriguez (01:22:42): So forensic accounting, I think, the accounting piece of it, for sure, she can probably find some internships. And don't wait for the school. And give her the confidence that, "It's okay to fail." And that you believe in her. And have her find people to surround her with. One of the things that, a consistent thing that came up, was a board of advisors, career advisors. So find people, have her start collecting people; friends, family, maybe friends of yours; that she can turn to for career advice. Do those sound reasonable? Thomas LaRock (01:23:18): Yeah. Absolutely. Thank you. Rob Collie (01:23:20): Tom, I've got a suggestion as well. Thomas LaRock (01:23:22): Yes, sir. Rob Collie (01:23:23): And this is, of course, me talking, my book. But it's the truth. Power BI is amazing for that stuff. We have found fraud when we weren't looking for it. That thing we talked about earlier, like all the different silos of information in an organization? A lot of times, fraud utilizes that siloization to hide its tracks. They just know that, you're looking at this one report, you're never going to see it. But when you suddenly cross-reference, when you splice across all of those silos, and you have the ability to drill down? I mean, oh my gosh. It is so hard. It's interesting. Data quality issues, people aren't recording the data properly, that something's going wrong in the source data systematically, that's not intentional? That leaps off the page so quickly as well. So it's almost frustrating, how quickly those sorts of things surface themselves. Because now, you've got to go deal with something way upstream, nothing to do with the reporting that you were trying to do. That same sort of characteristic, it applies to basically any sort of anomaly. Anomalies have a very hard time hiding. We've caught people stealing. We were just trying to do some sort of product mix analysis. Rob Collie (01:24:39): So I don't think the industry of forensic accounting has caught onto this either. The traditional training mechanisms and classes and all of that aren't going to feed you this. Because it's just so many steps removed from the way it's been done. So this would be an out-of-band. You could throw the word disruptive on it. Whatever you want. I really think that tools like Power BI are coming for the forensic accounting. And it's one of those things, like we were talking about. You can see it coming. You have the foresight. You can see it. But the depth perception is questionable. You have no idea when it's actually going to happen. Rob Collie (01:25:15): Okay. Back to the book. You started talk a little bit about how you selected people. You didn't just go with your first-degree network, which is kind of what I thought you would do, right? You already knew everybody that you wanted. It was just a question of having to pick the 20. Is it true that you went looking for people that you had not known before? Lori Rodriguez (01:25:32): Absolutely. I'll take a little bit of a step backwards and say how the book came about. Lori Rodriguez (01:25:35): So back in 2012, a mutual friend put some CIOs that she knew together to have a networking session every four to six weeks. So we just got on calls. And we're like, "Okay. You're friends of Janet." Like, "Who..." So we just explained who we were. Because it was a random collection of women CIOs. And I was not a CIO. Well, I used to be. But I was a friend of Janet's. And she brought me in. And we explained our stories. And we were fascinated. Because the myth is, there's this linear path that you go on and you knew that's what you were going to be. So everybody started their story with, "Well, I didn't intend to be in IT or a CIO" or whatever. And then, we went through the paths and we found them fascinating. And being that these were CIOs? Clearly, they have a lot of ambition. They're like, "Okay. So what are we going to do with this information? It's nice that we're getting together but we want to be productive." Lori Rodriguez (01:26:28): So the idea was to start a book. And we're like, "Okay. Well, on what?" "Well, women in STEM. Or women in technology." And we'd joke around like titles. And one of them was, "There's No Line at the Ladies' Room at an IT Conference." So that was like one of the titles. I'm still playing with that one. Lori Rodriguez (01:26:48): But then, we started... Well, the problem we wanted to solve was what we had talked about. There were finally people looking at getting women into STEM but we saw far too many of our colleagues falling out. And at that level, at the C-suite level, every room they- Lori Rodriguez (01:27:03): And at that level, at the C-suite level, every room they ever went into, they were the only woman in the room, the only woman in the room. Being a CIO is a very lonely job, in and of itself. Being a woman or a woman of color, the burden is really tremendous. You're invisible and at the same time, you're highly visible. You can't make mistakes because you're visible, right? You're just this, everyone's looking at you for that. And you don't get a pass on them. You have to do a lot more work. All these other burdens. So people drop out and they don't get to that level. And then, you're the only one. Lori Rodriguez (01:27:34): So we started this book. And then, the book Lean In came out. We're like, "Oh! Okay. Cool. It's all going to be fine." Oddly enough, I was doing a laptop upgrade around nine months ago and I came across my notes. I was like, "Oh, damn! Let me go check those numbers out." We didn't improve over that eight years. In technology, at least. We actually went backwards. The numbers were even worse. So I said, "That's it. I'm doing it." And I just started working out on this book. Lori Rodriguez (01:28:01): So clearly, I went back to one of the women I'd stayed friendly with. Which, at the time, she was the Deputy CIO of the EPA and went on to become the CIO of NASA. And that's where I started. And some of these other people that I knew. And about three or four people in, I was like, "I'm heavily loaded with CIO and tech." And as I'm thinking about and researching the book, I'm like, "This is exactly what everybody else falls into. You go to your status quo and what you know." So I did. I intentionally went out to find, I put in a metric, right? Like, "I'm going to have at least half the women in this book be women of color." So I'm going to be intentional about the numbers. And I'm going to measure it every which way from Sunday so I keep track of who I'm reaching out to and how much. Lori Rodriguez (01:28:43): And then, I just started to look up, fish where the fish are. I started to look at associations and who's writing, who was in articles, et cetera. And also, I was like, I didn't want to lean too heavily on the articles because those people are already getting fame. It's great that the women who had 23andMe and YouTube are there, right? It is amazing. But not that many people have Stanford professors as parents, right? And grew up in Silicon Valley. Lori Rodriguez (01:29:13): And one of the problems with the Lean In book was, that's awesome, Cheryl Sandberg. Thank God you wrote that. But it's unapproachable for many women. And they're like, "That's not me." I have Stanford graduates in there. I have people who dropped out of high school. I have community college. They went to community college. Myself, I dropped out of college. I was like, "Not for me." They were telling me what to think and I wanted to know how to think. And so, I was like, "I'm out. I'm going to go figure that out on my own." Lori Rodriguez (01:29:40): So I wanted the book to be, anybody should go to STEM. Like we all should. If you're interested at all, I wanted it to be approachable. And I wanted to tell stories, at least one of the women in the book could be some way relevant to them, regardless of where they were born or the educational path they took because they were privileged to do so. Or maybe they weren't. Some people couldn't afford it. And all they could do is community college. Maybe they made a mistake along the way. And how do you recover? Or maybe they left the workforce and came back. I wanted all of those stories in there. Lori Rodriguez (01:30:21): And I've done a pretty good job. It's pretty cool. And I also wanted to cross STEM. So again, I was really heavily IT-focused and technology-focused. I'm like, "I have to get some women in gaming in here." So I found this unbelievable story; should be a movie. This CEO of Future Club. Used to be at Riot Games and Lab Zero. Skullgirls, League of Legends, and things like that. It's just, the stories are just incredible. So I got gamers. I've got people on the aspergers spectrum, just all, the whole spectrum. I've got... Today, in the US, I have no idea where you were born. So I have women who were born in Ghana. And so, Ghanaian American. I have a Nigerian Canadian. Lori Rodriguez (01:31:03): And I hate to bucket people. But I also have a colleague. I knew her story was amazing. It really was. South African. Grew up in and still lives in Soweto. So as an eight year old, playing outside in the schoolyard; this is just prior to Nelson Mandela being released. And military vehicles pull up to the school yard and tear-gas eight year olds. She sat on the playground. So, yeah. It's just incredible. So going in that environment and becoming CIO in that environment as a woman is just insane. Lori Rodriguez (01:31:34): So a lot of really cool, as I said, life stories. At the same time, it has this STEM piece to it that, I just wanted to show, anybody has a place here, if you want. So one of the women said, "It's a book about women in STEM that has nothing to do about STEM." And I'm like, "Yeah. Maybe that's not a bad way to look at it." Rob Collie (01:31:52): Stuff like that is always; to me, anyway; a sign that you're on the right track. Rob Collie (01:31:56): I really like that you deliberately made sure that this wasn't like the pedigreed all-stars. Did you see the movie Ratatouille? Lori Rodriguez (01:32:03): I love that movie. People say, "What is your favorite movie?" I'm like, "Ratatouille." And then, there's another movie that's, it's Japanese with American subtitles. Emperor's Tailor or something like that. I'm like, "Those are my two favorite movies." Rob Collie (01:32:15): Really? Lori Rodriguez (01:32:16): They're completely opposite. Love Ratatouille. Rob Collie (01:32:19): So then, you know where I'm going, right? Which is, I've only seen that movie once. But it made an impression on me that I keep coming back to, thinking about it, many years later, which is always a sign that something was a beautiful piece of art, or at least it touched me. Like the chef, the hero chef that's like, that's not even alive anymore. He's like passed away in the story. He's always like, "Anyone can cook." His whole philosophy, that chef, that this young rat bought into, this wasn't some pedigree priesthood. It wasn't off-limits. It was a talent that lies dormant in people across every demographic. Rob Collie (01:32:58): And the movie goes out of its way symbolically. It's a rat. It's not even a person, right? It's something that we all associate with dirty. Like, "You can't have rats in a kitchen." That's how beautifully constructed this story is. The critic, the snobby, snobby critic, who has a thawing and an awakening as a result of this experience and becomes a believer as opposed to the villain. It's just like, "Oh!" It's just this most touching story. And I've written a lot of blog articles over the years. I don't blog as much as I used to. But this, "Anyone can cook." Rob Collie (01:33:33): What I was talking about it earlier, with the data stuff, with the Excel stuff, right? It's kind of like, it's that dynamic. I don't care whether you identified as a math person in high school. I don't care whether that spoke to you. It spoke to me. But it was really, in retrospect, in a really hollow way. It was just a way to have a false identity for me. It was a way that I could tell myself that I was good. I almost like wielded it as a weapon against other people though. I wasn't very nice about it. Lori Rodriguez (01:34:02): You were ego. Rob Collie (01:34:03): Yeah. Lori Rodriguez (01:34:03): You were the critic ego at that point. Rob Collie (01:34:05): That's right. And it's been a long journey for me. So now, I'm the one that's going, "Yeah. I actually prefer you if you weren't into calculus in high school." I don't understand why you would be so sick. It's almost like something's wrong with you if you were into calculus in high school. It takes one to know one, you know? Lori Rodriguez (01:34:24): I hadn't thought about it. But I love that movie. And I tell my kids all the time about that movie. And they know I love that movie. And now that you mentioned it, that's what I was going for, absolutely going for, in this book, and why I was very purposeful and intentional in the stories or the people I reached out to. Lori Rodriguez (01:34:45): And I had a super-high hit rate. I'd send an email or I'd ping people on LinkedIn. And I had like 90% of the people I asked at least talk to me. And I only had one person say no. And it was because they felt they weren't good enough and they gave me another person's name. And I'm like, "I'm going to come back to you and interview you because you are good enough. You're pretty cool or I wouldn't have reached out to you." But I did. Lori Rodriguez (01:35:09): So everyone wanted to tell their story once they understood why. I didn't realize it at the time. But, yeah. That movie, that influence of, "Anyone can cook." Not that anyone can cook. I think ego explains it at the end. A cook can come from anywhere. Rob Collie (01:35:24): Right. I talk about it in our world as the data gene. And I do believe this, based on what I've observed, that it cuts across every demographic at about the same fraction. It's not everyone. It's like one out of 16. Rob Collie (01:35:37): I actually have a number of different research methods that have yielded the same result. Like one out of 16 is the high watermark. It's at most, one out of 16. The data gene can lie dormant for a very long time. You wouldn't know you had it. And then, that collision with Excel. That's usually, that's not the only story, but it's the majority of stories. Because that's the place where you would encounter it. You hit it that first day and you start getting this weird, twitchy, itchy feeling. Like, "Mm!" I agree. Like 15 out of 16 in the data world can't cook or don't want to. Not interested. They're not interested at all. Maybe they could. But interest is a big thing, right? It doesn't speak to them. Rob Collie (01:36:18): But, yeah. I completely agree. The cook can come from anywhere in data. And so, this has been an ongoing fascination of mine. Like, "I want this to be true. I've observed it to be true. And now, I want it to be as true in the numbers that we see. My experience dictates it should be." Rob Collie (01:36:36): And this is why the overwhelming weight towards the male applicants for jobs at our company bothers me. I mean, this is, in a way, we're staffed in a way that is not at all the tradition for an IT services consulting firm, for a BI consulting firm. We are overwhelmingly staffed with people who came from the business, for instance. We're staffed with people who can be the requirements-gatherer, the communicator, the architect, and the developer, all in one. Lori Rodriguez (01:37:05): So you liked, in Renee, the CIO of NASA, you liked in her story? Rob Collie (01:37:09): Yeah. Yes. Lori Rodriguez (01:37:10): That you picked up on that? Rob Collie (01:37:11): Oh, yeah. Lori Rodriguez (01:37:12): She came back to math because of requirements gathering. Rob Collie (01:37:14): Yeah. Actually, I have a fight-starter, if I go to particular conferences. I have a number of things I can say that I believe to be true that will immediately start a fight. Which shouldn't, right? But they do. Which is, I'll tell people that, in a traditional BI project, business intelligence project, greater than 99% of the elapsed time, the cost is in requirements transmission. The whole project. The time where the hands are spent on the keyboard, typing the right code, the stuff that sticks, the reports that actually eventually come out the other end, if you just sat down and typed those out and just typed that code, it'd be over in an eye blink. It's all requirements. Lori Rodriguez (01:37:53): And do you think that's the right number? Do you think that's the right ratio? That it should be mostly requirements and less code? Rob Collie (01:37:59): No. Because the problem is that it's so wasteful, that 99%. And if you measure it in its absolute terms? I don't know how many person months it'll end up being. But sometimes, it might even be like 100 person months. It doesn't have to be that. It could be like six person days. It's the tools. Rob Collie (01:38:18): And this is the thing. I was part of building the old wave of BI tools at Microsoft. And then, I was part of building the new wave. And I also just lucked out in that I'd also had a chance at Microsoft to be someone who had applied the old BI tools. Lori Rodriguez (01:38:32): So when you say requirements, you're talking about the old waterfall document-type requirements in? Rob Collie (01:38:38): That's right. That's right. Lori Rodriguez (01:38:38): Okay, okay. Rob Collie (01:38:39): Yeah. Lori Rodriguez (01:38:39): All right. Got it. Okay. Rob Collie (01:38:40): And the old tools drove the waterfall methodology in BI. Lori Rodriguez (01:38:46): Yeah. "We're going to write a document that's 400 pages long." Rob Collie (01:38:51): Right. Lori Rodriguez (01:38:51): "About this thing that we're eventually going to start building." Rob Collie (01:38:53): Right. Yeah. Lori Rodriguez (01:38:54): "Which means, then, we're so invested in that document, that all the requirements who wrote in there that, when we find out, midway through, as you always do through a project, or at the start, that they're wrong?" Rob Collie (01:39:05): Right. Lori Rodriguez (01:39:05): "It's too bureaucratic to change and we're vested in it too much." Rob Collie (01:39:09): Yeah. Lori Rodriguez (01:39:09): "And we're just going to produce it and launch it, damn it." Rob Collie (01:39:11): That's right. Lori Rodriguez (01:39:11): That kind of process you're talking about? Rob Collie (01:39:13): That's exactly right. Yeah. Rob Collie (01:39:15): And there's so many myths in the requirements document, right? One myth is that the people who are providing the requirements will transmit them properly. The next myth is that the people receiving them will record them properly. The third myth is that the person that they give it to to implement will receive it in a transmission properly. And the fourth myth was that the requirements were correct in the first place. Lori Rodriguez (01:39:36): But there's a lot of plausible deniability built into a requirements document. Rob Collie (01:39:41): Totally. Oh, yeah. Lori Rodriguez (01:39:42): So no one along that path will take any responsibility for having caused the damage that that final product, when it's produced, create. Rob Collie (01:39:50): Mm-hmm (affirmative). That's right. And if you're outsourcing this, if you're hiring a BI firm, like one of our traditional competitors? They love it when this goes wrong. They've been billing the whole way. Now, you're out in the deep water. You can't turn back now. So that change order, the addendum to the contract, is going to get signed. And you're going to keep billing. Rob Collie (01:40:08): I believe; and our results over the last 5 to 10 years bear this out; that any BI project? We can be looking at your first tangible results, not a mock-up. You're not done. But within five business days, you can be seeing your first output. Lori Rodriguez (01:40:26): You've got to get to the output as quickly. Rob Collie (01:40:30): Yeah. Lori Rodriguez (01:40:30): It's just like any other product's development process. You've got to get to your MVP so you can test it and break it. Rob Collie (01:40:34): That's right. Lori Rodriguez (01:40:34): You want to break it early. Rob Collie (01:40:35): Right. Lori Rodriguez (01:40:36): The longer you wait to break it, the more costly it is, because you've invested more time and money in building that thing. And the more vested people are going to be in saying, to keep persevering down this path anyway, the more objections you're going to have to pivoting or changing. So if you can get something out the door very quickly, with a minimal time, minimal effort, and break it? As long as you're going to break it. Because I see a lot of people doing Agile or MVP. It's just condensed waterfall. Like... Rob Collie (01:41:04): Yeah. It's just a... Lori Rodriguez (01:41:04): It's just, we, instead of taking months, we did it in a shorter timeframe, but you still get no chance to iterate, based on the feedback you have. Rob Collie (01:41:11): Correct. Lori Rodriguez (01:41:11): So they're still as vested in that thing that they launched as before. If you're doing that, you're getting it out in five days, with the idea that you're going to iterate based on what you find, you've built in the fact in those five days that you're going to spend more time iterating after. It's not like... Yeah, the expectation, it is done. Rob Collie (01:41:30): It's not done. And we have a saying, which is, again, "Learn the hard way from experience." Human beings do not know what they need until they've seen what they asked for. So even if you manage to achieve perfect requirements transmission, which has happened never. If you got there, the first thing they're going to go is, "Oh! Right! This doesn't actually answer the question that I thought it would. But now, I know what we need to do." And you just peel that onion. Lori Rodriguez (01:41:57): Any parent who's tried to teach their kid how to ride a bike has run into that. You could explain everything. And your kid gets on their bike. You're like, "Ooh! I forgot to tell them about balance" or whatever. Rob Collie (01:42:06): Yeah. Lori Rodriguez (01:42:07): Or anything you're trying to explain that you do from an automatic basis. Lori Rodriguez (01:42:11): So people lie. And you know that. Rob Collie (01:42:13): Oh, yeah. Lori Rodriguez (01:42:14): People lie. And they don't do it on purpose. So when you're asking for requirements, they lie for a number of reasons. They want to give you the answer they think you want. They want to give you the answer that makes them feel, makes them look or seem smart or better or whatever that is. Or they don't know what they need because they think they know what they want. But because of lots of different reasons, it's not really, to your point, when they get it in their hands and they use it or they see it, everything changes. "Ooh, I forgot about this." Or, "Oh! I didn't mention that piece." Lori Rodriguez (01:42:44): So the sooner you can get people to admit their lies and mistakes, the better off your product's going to be in the long run. Is that kind of where you're going? Rob Collie (01:42:52): Yeah. The longer a lie ages, the more intractable it becomes. If you told me one of those accidental lies five minutes ago, you're not going to be as concerned about walking it back. But if that lie has sat on the record for three months? Lori Rodriguez (01:43:08): In paper, in a requirements document. Rob Collie (01:43:10): Yeah. It now becomes synonymous with your reputation. Rob Collie (01:43:14): So the thing that I saw in 2010, with the new wave of tools? Lori Rodriguez (01:43:18): And new wave of tools, explain that. Rob Collie (01:43:20): So for me, it's power BI. Lori Rodriguez (01:43:22): As opposed to Excel? Rob Collie (01:43:23): Yeah. So the first effort at Microsoft with Power BI, we put it into Excel. It was called Power Pivot. Because of the visual canvas that tools like Tableau and others had, Microsoft realized, "Okay. We need to match that level of visual." And they couldn't do that in Excel. So that's why we have the separate Power BI product now. That and a couple of other reasons. But the stuff under the hood that's really the game changer for Power BI relative to the other tools, most of that was already put into Excel in the early 2010s. Rob Collie (01:43:52): And our company, the idea for our company, dates back to that. When I saw that the tool now moved fast enough that you absolutely could handle the requirements process in the form of real-time collaboration with the stakeholder or stakeholders, like from a blank canvas? "Let's not talk about it so much. Let's not try to write documents. Let's not try to do any of that stuff. Let's sit down, load the data from however many different sources." Like that story I told earlier, about the customer experience scorecard. It's made a huge difference for them. That's how we did it. There were nine different data sources. Not all of them were in the data warehouse. The data warehouse is never complete either. We sat down, we loaded it, and we started. Rob Collie (01:44:37): What's the first place to start? Well, let's start with the front of the funnel, like the CRM, like the first measure. What's the first metric? Well, people requested quotes. Let's start with the obvious. Did we ever get back to them? What is our response rate for people who say, "We'd like to maybe do some business with you?" It turns out it's not 100%. Lori Rodriguez (01:44:56): As a business, you know the information's there, right? Like we collect it. Rob Collie (01:44:59): Yeah. Lori Rodriguez (01:45:00): We ask our customers to feed us a lot of information. We do nothing with it. So it's there. It sits there. But as a business, we know it's there. It's frustrating because you can't get at it. Lori Rodriguez (01:45:11): One of the things you haven't brought up, but to me, is so important, is the democratization of data. Because if you can put it into that one place, like you said. And then, you could ring-fence it, with all the permissions and everything else. And then, you could set it free and let people build stuff on top of it. Lori Rodriguez (01:45:28): So if I need to get data, if I need to get information and answers out, or even just go in and explore that data for things that it might tell me that I had no idea that I should know, the fact that I can go do that on my own, or I could get a BA to help me or whatever, and I don't have to wait a year to get a IT budget approved? To get money to have IT do it? You're freeing up the data for use for the business. And that's where I see just the huge potential of this shift that you just talked about, from the old way of thinking to the new way of thinking. Do you see that the same way? Or am I seeing it a little differently? Rob Collie (01:46:06): I do. Because another one of the things that the old waves of software inflicted on us, IT didn't choose it to be this way. The software industry did; is that everything was heavy, heavy, heavy infrastructure. So we have all kinds of pithy little ways of saying things that we've developed over the years. And so, we call it faucets first. That's our methodology. The old way was plumbing forever. "Oh! You need a chart? Mm. Okay. Well, let's not get carried away. We've got to go build a lot of infrastructure before we can talk about charts now, don't we?" Rob Collie (01:46:42): And seriously, like Microsoft's old software, for example, it required the old analysis services. It required that all of its data come from the same database instance. So in order to even get started, you had to first get all of the data moved into one place, which was never going to happen. It started with a known failure. Lori Rodriguez (01:47:06): Yeah. Fatal flaw, right from the get-go. Rob Collie (01:47:09): I visited one customer one time, who said, "You know the most exciting thing to us about Power Pivot?" And I go, "What?" He said, "Well, we have all these data warehouses. The enterprise data warehouse project is on our list. And it's forever on our list. It's become like this running joke that we're going to unify, unite the clans of all these-" Lori Rodriguez (01:47:25): Single view of the customer. It's been on there for 20 years. Rob Collie (01:47:29): Yeah. It's never going to happen. Lori Rodriguez (01:47:32): It's outlived six CIOs and three CDOs. Single view of the customer. Rob Collie (01:47:37): And these people were telling me, "We're really looking forward to the idea of being able to do dimensional modeling, like a BI analytical model, with data from multiple data warehouses." And at that moment, I just, my jaw was on the floor. I'm like, "I can't believe that we did it that way before." Rob Collie (01:47:52): So I sound like a fanboy for a particular vendor and a particular piece of software. And I'm really cognizant of that when I'm talking to someone from Gartner. Even though you're speaking as you, right? Just as a person to person. But I tell people all the time, I didn't revector my career lightly. The fact that I'm really familiar with Microsoft is actually, or would probably, for me, be a reason not to bet on Microsoft. Because I know where the bodies are buried. I know that you become really familiar with the sins of the organization that raised you essentially. I really, truly believe that this thing is something different. And in fact, like now, we're seeing, we're finally seeing the other vendors start to come around to what Microsoft is up to. And they're starting to now go, "Oh. Oh, right." So they're starting to play catch up to Microsoft a little bit. Lori Rodriguez (01:48:43): Yeah. So I mean, you can think of it. Sports analogies. Pick anything. Tennis, American football, the other kind of football the rest of the world plays. When there's a breakthrough, one team dominates, or Serena Williams comes in? Rob Collie (01:48:59): Mm-hmm (affirmative). Lori Rodriguez (01:48:59): It forces all, everybody else, to up their game to compete. So the fact that Microsoft did something more akin to OneNote than what it previously had done, the market's going to tell you whether it's working or not. It opens it up for competitors to beat Microsoft at its own game. So it's helpful when there's been a leapfrog made and then everybody else is going to scramble to fill that and compete. Lori Rodriguez (01:49:24): So I appreciate that. I don't take sides on vendors, just from a, in the business itself, and then as Gartner, I don't represent the analysts. I don't represent anybody. I have to do my work as well. And I look for the organizations, companies, people, that are going to help me get my work done, faster, better, cheaper. And quite frankly, pleasant. Working with people is pretty critical that you can trust them and you have a good time while you're working. Because again, back to that work life, I don't want work to suck either. I want work to be fun and exciting and meaningful. And that has a lot to do with the people you're working with. So appreciate that, appreciate. Rob Collie (01:50:11): Yeah. We call this show Data with the Human Element. It's what we're about. Rob Collie (01:50:13): To do justice to your question, to honor the question that you were asking before, I want to make sure I close that loop. It might not have even been all that intentional. But I think it was halfway intentional. The Power BI sidestepped all that infrastructure. And you talk about how the data doesn't talk to each other. That's true. Your "best of breed," nine different line of business systems that are involved in a particular situation, have no knowledge of each other or interest in one another. In fact, they hate each other. And that data is now able to meet on a common ground inside a Power BI model. And it's effortless. It's not like line of business integration. It's not that kind of middleware. But to line these things up with each other and see across the business, end-to-end, it's breathtakingly simple now. Rob Collie (01:51:04): And so, the requirements process and the infrastructure weight that was required before, I think they are two sides of the same coin, when you have a tremendous infrastructure investment that must be made. And again, another quote I got from a customer years ago is, "Yeah. My team spends six months to put a dot on a chart." Just looking at me, just confessing their sins. That was the cool thing about being at Microsoft. You were like the priest and everyone was going to confess their sins to you. Like in the booth, you know? Lori Rodriguez (01:51:31): So I have a question. So I'm a fan of low-code, no-code, and the democratization. But there are things that you have to keep centralized. Then, there's this middle piece, which is integration, right? It has to be interoperable. And I feel like Power BI isn't there yet. It still has a lot of work to do, in and of itself, for what its current set of requirements are. You've got a great MVP V3 or something, wherever it's at. But it still has some work to do from that perspective. I think the next big leap, from what my limited experience, is the interoperability with the infrastructure itself. So it's got to be able to... Lori Rodriguez (01:52:12): And again, I'm looking from a customer experience perspective where the end result, what I want to get out of. And I'm always thinking of what's next. So I'm all in. I love this thing. And yet, there are things that I need to do to trigger other systems to kick in, right? Rob Collie (01:52:28): Yeah. Yeah. Okay. Lori Rodriguez (01:52:29): That trigger piece. And I'm not seeing that just yet. So is it because I'm inexperienced in this? Or is this the next place that needs to go? Rob Collie (01:52:36): Well first, let's clarify what you mean by interoperability. I think I know. But just for the listeners. Rob Collie (01:52:41): One kind of interoperability is ability to eat data from lots of different places. I think it's great at that part. It's probably best in class for a single software, piece of software solution. It doesn't require you to add three other vendors into the story. Power BI is a world beater in that regard. I think you're talking about like the read right? Lori Rodriguez (01:53:03): Yeah. So for example, let's say that the insights you gained from your CRM system in this piece, right? You put a flag on it that says, "When it hits 83;" making this up; "go trigger this process over here in this other system to go tell the sales guy to go sell." Rob Collie (01:53:22): Yeah. Yeah. Lori Rodriguez (01:53:22): Or whatever it is. That triggering mechanism? Does that... I know you've got Power Apps. But it seems like it's not quite yet that right perspective. Or I don't know what your terminology is. But to go off and trigger the infrastructure to do something else. Rob Collie (01:53:38): Sure. Sure. Rob Collie (01:53:38): So this is awesome. This is like... I don't want to sound patronizing. But it's almost like we're following the same trail of breadcrumbs separately and seeing the same things. Rob Collie (01:53:49): Before I show my cards, let me ask you a question. Are really any BI tools good at what you just described? Lori Rodriguez (01:53:58): No. Not at all. But I don't care. I'm the business. I don't care about the tool. All I care about is, "Here's what I want to happen. This is awesome! You gave me this data." Now, it's like, "Ooh! I've got to do something with this data." Or, "What does it matter that I know it? I need it to do something." And the do something piece isn't there yet. I'm happy. I'm happy that I see it now. Now that I see it, I want it to do something. So I'm always onto the next thing to order the IT department and the Excel people to do. "That's great! Now, your next thing." You're only as good as the next thing. Rob Collie (01:54:31): Okay. So in college, I had a problem, like an ego problem, when I was writing a philosophy paper. And I thought I was the first person on earth to have this idea, you know? And then, I would discover, two weeks later? I don't know. Some philosopher had written exactly my idea. And now, I felt terribly invalidated for some reason. And my professor told me. He says, "Listen. At these moments, you should not take this as invalidation. You should take it as confirmation." Rob Collie (01:54:59): So that's how I'm feeling right now. BI, in a vacuum, means nothing. Nothing at all. Even just the nature of it. Business intelligence. This is a domain, this whole industry, that was so difficult and so unsatisfying and so relatively low ROI, relative to what we could imagine, that it has become almost, like a means to an end has become a goal in and of itself. The I should have always been for improvement. Lori Rodriguez (01:55:31): Yeah. There you go. Exactly. Rob Collie (01:55:33): Until your improved knowledge, your improved vision, it translates into improved action. It's meant nothing. And when you start to judge BI by that standard, suddenly you start to think, "Oh, my God. It's even worse than we thought. It's really, really poor. I've been part of this. I have built reports sometimes that I thought were, "Hmm. That's awesome! That's a hot report." But then, the user of it looks at it and goes, "I don't know what I would do with this. How would this change decisions that I would make or actions I would take?" Rob Collie (01:56:03): How would this change decisions that I would make, or actions I would take? I go, "Mm, yeah. Okay." So, I think that BI software has been given a pass for a very, very, very long time in this. If the dashboard tells you something interesting that you should go act on, why does it just sit back on the couch and go, "Mm, good luck with that?" Lori Rodriguez (01:56:21): That is exactly where I'm going through. You should know what the next step is. Right? You know what the action to take is. How do you then feed that back into the systems to take that action? That's what I mean by the interoperability, or that mild ... the right piece that you were saying. So, it's got this information. How do I then connect it back to the infrastructure, so it can trigger something else to happen along the way? Rob Collie (01:56:47): I've got a lot of thoughts on this. Part of it is reading the tea leaves of Microsoft, as a trained observer of Microsoft. Let's give us all an example that we can use as the testbed. Even if it's not automated, even if the action isn't automated, that's next level. Even if we lower our standards a little bit from that, it's still a failure today. Right? So, the example I've been giving people lately is, let's say you're looking at a dashboard and it's just jumping off the page at you that warehouse six is going to run out of inventory before it's replenished. So, you're going to have an error gap in your supply chain. The dashboard that does that, that tells you that problem exists, today, thinks very highly of itself. It's very smugly satisfied. "Look what I did. I showed you a problem, but good luck." Right? Rob Collie (01:57:37): So, now what have you got to go do? First of all, you've got to formulate a response. You've got to formulate what you can do to address this. But then you have to go and log into some other line of business system. Maybe we can transfer some excess inventory from warehouse four to warehouse six. That's one of the ways that we could do it. Of course, we need to rush an order. Okay. We've got to go to an ordering portal to rush the order of more widgets and deliver them to warehouse six. But there's this huge context shift that has to happen for the user of the dashboard. They have to go and navigate to the right system and drill down to the right context, warehouse six and warehouse four, or whatever. Right? Rob Collie (01:58:19): Imagine, instead, if the dashboard, when it's highlighted for you, it's right there. You see the shortfall, multiple different actions that you can take. You can start to arrange the transfer of inventory, because you can see it right there on the dashboard. Four, warehouse four, has got six months of supply. Warehouse six is in trouble. It's just right there. Why do I have to go? Why can't I just connect the dots there? Lori Rodriguez (01:58:45): Or have it do automatically. To your point, taking automation off the table, at least the information there. Rob Collie (01:58:53): I think that organizations like yourselves, on the analyst side ... This is a prediction I'm making about [Gartner 01:59:00]. Right? Some number of years in the near future, when Gartner is formulating their magic quadrant for BI software, they're going to start using this take action integration capability as one of the axes that they're evaluating in order to ... in terms of completeness of vision, to rank the vendors. Once I came to this realization, all these sorts of things coming together, I had this all crystallized for me, suddenly, I understood what Microsoft has been up to for the past three years. It's like, "Uh oh." Again, this is me opining. So, your mileage may vary on this information. Rob Collie (01:59:40): Microsoft had a great conference called the Data Insight Summit. Loved this thing. Thought it was awesome. But then they renamed it to the Business Application Summit. It became a little less fun. All of us in the data world were a little bit grumpy, because now we had the Dynamics. It was also the Dynamics conference. It was just data before, but now it was Dynamics and data. So, all the Dynamics products, the ERPs and CRMs and all of that, and accounting software and all that kind of stuff. The VP at Microsoft, James Phillips, who had been in charge of Power BI, just Power BI, after a while, when the track record was established and things were going well, suddenly, they gave James Dynamics, in addition to Power BI, but they also gave him all of this middleware stuff. They gave him the Power apps stuff. They gave him the Flow and the Power Automates stuff. Right? Rob Collie (02:00:34): At the time, when they made the change, it just seemed like this random grab bag to me, or yet another Microsoft, pie-in-the-sky, out-of-touch, move. But now, now I'm starting to wonder. In order for the reality that you and I want to see, in order for that to happen, you have to accept that this is never going to be out of the box. It's the most custom, one-off type of equation ever. You can imagine building a dashboard that is, in some sense, one-size-fits-all for an industry. Every oil company that's running on this sort of drilling system or whatever, I could build some dashboards and sell it as a subscription product, for example. But when it came time for them to take action, everyone's got a different ERP. Right? Rob Collie (02:01:20): It is the most custom thing ever, the taking action part of a dashboard. So, it is inherently going to be a development exercise. Now, how low-code, how no-code is that development? Okay. There's a spectrum there. But we need to accept that this is solution building. It's going to be a platform rather than an out-of-the-box answer. Oh, my God, does Microsoft have a platform mentality. This is one of the things that they are really, really, really good at, at least relative to their competitors. Right? As an observer of this industry, now also the Salesforce acquisition of Tableau makes more sense to me, too. Salesforce is also trying to be this middleware operating system for your business type of company. They've long since overflowed their banks of CRM. Rob Collie (02:02:10): If you think of BI, seriously, just really think of the last three or four months, this has been churning in my brain. If you think of BI, effective BI, as a form of middleware, it is read-only middleware. It's the place where all these silos meet, but in a read-only sense. Oh, yeah. Of course, Salesforce needed BI. Of course, they needed the read-only middleware to go with their read-write middleware platform that they're trying to build, that's their core mission these days. Of course, they would reorganize all of this stuff together at Microsoft. I see this battlefield. Knowing the players, I know who's going to win. Microsoft already has the hooks. I'm sure that the hooks aren't good enough yet. The ability to embed a Power app into your Power BI report, they had that a long time ago. That's been in there for a long time. I'm not going to pretend that, "Oh, there you go. There's your answer." It's just that they've been thinking ahead in a very interesting way. Lori Rodriguez (02:03:16): Do you think it's the right direction to put the Power app into the BI, versus the brain into the app? Right? To me, when you're talking about BI, it's the brain. Right? It knows all this stuff. You have the body. Your central nervous system is the middleware that connects the body to the brain. It feels like it's, perhaps, flipped the wrong way. You've got the BI as the thing that you put the body into, as opposed to the intelligence into the body. Rob Collie (02:03:46): Well, if I'm following your metaphor, I just take it for granted, the data model brain that you build behind a Power BI report. I just take it for granted that thing is accessible via API. It's always been. I don't have to use Microsoft's front end at all, if I don't want to, to leverage the smarts that I've invested into that BI brain, the data model, even without Power apps, if I wanted to do the thing that you were talking about, the automated trigger. Lori Rodriguez (02:04:13): Yeah. At the end of the day, in order to get a result, you have to change a behavior somewhere, in somebody. Right? So, if you're designing your experience to change a behavior, you want the intelligence to slot it and feed that experience. It seems to me, you're building the intelligence and then figuring out how to make the experience work around the intelligence you have, as opposed to just saying, "This is a behavior we want to get to. This is a behavior we have today. I'm going to create an experience. It's not with Microsoft products, but the Microsoft products are going to make the muscles move, or do whatever." I'm thinking of it that way. I don't know that there's anything out there like that. Rob Collie (02:04:59): So, are you saying ... Again, I think I might just be struggling to understand where you're headed yet. Are you saying that maybe we take the improved behaviors and the information and inject them into the line of business system, rather than having a jump-off point in the BI? Lori Rodriguez (02:05:15): I don't know yet. I'm not sure yet. I think I'm going ... I'm leapfrogging. This is awesome. You've now got all the data accessible in one place with Power BI. Then the next thing is to be able to take action off of that. You're starting to see that with ... I'm just using the Microsoft terms. I don't know what Salesforce or anybody else has. People are out there building other things like that. Whatever that thing is that some company's building on top of their BI platform, to then start to automate things, take action, that's awesome. That's terrific. Lori Rodriguez (02:05:49): But if I were a disrupter, because Microsoft is building with what they have, taking what they have and trying to retrofit or fit it into this thing, I would just say, "I want this behavior to happen. This is where we are today," and I would build a product for internal associates or whatever, it includes a dashboard in there, that gets me from point A to point B, and slot it in there as the intelligence to make things move, to drive the behavior that I want at the end of the day. That's Nirvana. Right? If I were starting blank slate, I would start there. Then I'd slot in where BI comes in, where automation comes in, into that process. Lori Rodriguez (02:06:32): Again, I'm tool-platform-agnostic. I'm just saying this is how I would build something, if I had that behind me. What Microsoft is saying is, "We had Excel. Now we have Excel Pivot, the next thing." So, you're iterating. Right? You're telling me, "Microsoft is iterating, along with products that they already have," as opposed to Blank Slate or Greenfield, and saying, "This is what it could be in a Nirvana state. Blank Slate, if I were to build this today, I would do X." I'm challenging, because eventually, you're going to get there, but it's going to take you a lot longer to get there in a scientific methodology of experimentation from a starting point, as opposed to saying, "This is where I want to get to. This is where I am today." Like a Google Maps. Right? Just say, "How do we get to that end place we want to go, if we were to just get in our car and design a roadmap from scratch?" Rob Collie (02:07:24): Well, I think ... This is just my take on all of that. I think that your history proves that you're much more likely to be correct in this assessment than incorrect. The ideal, the final form of this stuff, is something we probably haven't seen yet. When you reach such a fundamental realization as all of our BI has forever been given a pass on the part that matters the most, which is the taking action, you know that there's a lot of road left. The thing that makes me comfortable, as a Microsoft partner, which is one way to define our company, I suppose. Right? I mean, I try not to think of ourselves that way. As an organization that is associated with and invested in the Microsoft platform, one of the things that makes me comfortable is that I think that Microsoft, technical distance from this unknown Nirvana, is probably quite a bit shorter than their competitors. Rob Collie (02:08:23): They probably won't lead the discovery of that Nirvana, even though it happens with their own tools. One of the jokes we used to make about Microsoft was, we give you the parts to the Porsche. Have fun. So, companies like mine are going to be involved in the creation of these solutions, using their platform, which is insanely flexible. Again, you're saying that it should be better and closer to that Nirvana out of the box. Lori Rodriguez (02:08:55): Insanely flexible then allows what you just said. If you give the parts, you can create a Porsche. Rob Collie (02:09:01): That's right. Lori Rodriguez (02:09:02): Dodge Caravan. Rob Collie (02:09:03): That's right. Lori Rodriguez (02:09:04): Some car that's never existed. You could strap on a hoverboard and come up with something else. That's the piece. It's got to be insanely flexible to then say, "Well, you know what? We built it so it was the outsides, but what we're seeing is people are using it as the inside, as opposed to the outside." So, the brain and the nervous system, as opposed to the body. Right? Which is the body is the behavior and the intelligence, and the nervous system is driving the muscles, which is the outside, is what you're seeing and experiencing. So, if it's insanely flexible, then that allows organizations to build the Porsche. Rob Collie (02:09:42): Yeah. Lori Rodriguez (02:09:42): Am I understanding that correctly? I think that's probably a good way to get to this blank slate, from what Microsoft and other companies are doing with what they have today. Rob Collie (02:09:52): This is one of the only pieces of Microsoft's DNA that I think has remained relatively consistent from the beginning, at least from the ... I've said this before on this show. Most people think of Windows as a product, as a consumer product. It's the thing with the start menu and all of that. I went to Microsoft and I spent some time with the Windows team. I quickly discovered that isn't how Microsoft thought of Windows at all. The start menu and all of that is just one app, little tiny, tiny, insignificant piece of software that was written on top of Windows. The world thinks of Windows as the shell. That's it. They're like, "Ah, whatever. Those people weren't even all that respected, necessarily, on the Windows team." The difference between the Windows team and the developer division, the Visual Studio division at Microsoft, there wasn't any. They were the same crew. Windows was an API. Windows was a development platform. That's what it was from the beginning, and that's what it always was. That's where the parts from the Porsche thing comes from. You can get a fully assembled moped from Microsoft's competitors. You want a moped from Microsoft? You're going to have to build it. But you can build anything. So, this is the pros and cons, the plus and minus, is that it's so strange, in some ways. The very, very, very first version of Power BI included the ability for you to code your own visual. You could write your own custom chart control from the very first version of Power BI. Right? It's like, "Guys, this is not important. You spent your time on this. We could have had a million other really useful things. But that one? That one needed to happen first?" Lori Rodriguez (02:11:29): You know why? Because people told them that it had to be there. Right? Back to the lies people tell in the requirements. Of course, I want control to be able to do that. Then you get it, it's like, "It's so complicated. I can't do everything." Rob Collie (02:11:44): But only Microsoft would tell that lie. People are going to be falling all over themselves to code custom visuals. Now, it worked out, because it's future-proofed them in a number of ways. That's one of those features, if you don't do it at the beginning, you'll probably never do it. We use custom visuals. There's a pretty vibrant market in custom visuals. Lori Rodriguez (02:12:05): Well, that's where you can layer on from the platform perspective. Right? Rob Collie (02:12:08): That's right. That's right. Lori Rodriguez (02:12:08): If you give people all the parts to build stuff, then you spring up a whole bunch of consultants who will create that moped for you based on ... Rob Collie (02:12:17): That's right. Lori Rodriguez (02:12:18): So, we can create it for you, or we've got ones that are already built and you just buy it. You could have that same model work. Rob Collie (02:12:24): That's right. Lori Rodriguez (02:12:25): Where you allow partners to come in and they do whatever it is that people want. Rob Collie (02:12:29): That's Microsoft's ethos, in a nutshell. I don't think, as a company, that we have done too much work developing custom visuals. Most of our consulting work around Power BI is helping people build data models and reports. We have a partner, one of our partner companies, that has done a financial planning product, incredibly comprehensive financial planning product, integrated with Power BI, that would not have been possible, just a nonstarter from the beginning, if this custom visual framework wasn't in there. Their custom visuals are charts that also allow you to write back to various other sources, because the code is up to the custom ... So, it's not even a Power app. Right? It's just a chart that you can right click on a bar and say, "No, no. Make that six. Show me what the implications of that would be." A lot of the best things are things that Microsoft ... If you rely on Microsoft to anticipate what the world needs, you're never going to get there. They do leave the canvas flexible. Lori Rodriguez (02:13:30): Yeah, but who would anticipate a pandemic? We have to be uncomfortable with uncertainty. One of the women in the book, she spent her entire life ... IBM fellow, National Academy of Sciences, she built her career on uncertainty, did her dissertation on uncertainty. She's like, "Yeah. I'm right there now. I was ahead, looking ahead." Because life's uncertain, which is you have this balance between flexibility and then overwhelming people with too much choice. How do you work through both of those things? Rob Collie (02:14:02): Yeah. If there's 31 flavors of ice cream, if I up that to 600, no one ever eats ice cream. Lori Rodriguez (02:14:08): Yeah, or even 31. Right? It's so cool with neuroscience, as well, that we can look at things like too many choices and figure out how to build products that accommodate how we actually think and what we actually do, versus what we tell developers, and people like me in market research, what they want, which is why Steve Jobs, he said he didn't do market research, and that's not true. But he didn't do the kind of market research that was typical at the time, with focus groups and telling people, "You want blue or yellow?" They're like, "Ah, of course, I want yellow." That's not the case, because it's not reality. Rob Collie (02:14:46): The second half of my career is one in which I respect Steve Jobs immensely. His editorial force is just something else. There was something you said about uncertainty that really also spoke to me. The way that I found my way into my discovery of my own data gene was through fantasy football. In 1996, I was invited to join ... My first year at Microsoft, I was invited to join a fantasy football league. I didn't even know what that was. I wasn't watching football at the time. I wasn't interested, but I did it just to, "Ah, I'm a new guy. I'll meet other people. I'll use this as a social thing." Somewhere along the way, in year three, I read this article somewhere that explained to me what the real game was in fantasy football. I was like, "Aha! Oh, I get it now." The smoke parted. Suddenly, I was up to my eyeballs in Excel. That's what's, in the end, powered my interest in all of this stuff. Rob Collie (02:15:44): So, back to the women in data, I think you'll really appreciate this story. I was in the succession plan on the Excel team. I was the heir apparent. I was next to take over for the group program manager job. I was making it. Then Microsoft decided to launch a fantasy football team, a fantasy football software team, over on the MSN side. I'm like, "I'm out. I'm out. I'm gone." So, really, you think of really poor career choices. One of the things I got to do over there was try to build a consumer-facing, end user-facing stats portal for the NFL. We did it, because it was me. Right? We did it based on Microsoft's pre-Power BI, their traditional BI software, with an Excel web front end over the top. So, it was an analysis services model with OLAP data, blah, blah, blah. Right? Rob Collie (02:16:39): So, that stuff was too hard. I couldn't do that. I had to hire a consulting firm. I got to be a customer of a BI firm. I got to be the customer in a BI project. I was lucky that the consultant that they assigned to our project, from Hitachi Consulting or whatever, didn't know football. He was from another country. He had no idea about American football. So, we had the same dynamic where I had to explain to him the business, the subject matter, over and over and over again, and the requirements process. If I had just lucked into another NFL fan, it would have been easy. I would have missed an important learning, which later on, without that piece of information, without that experience, I would have never launched this company. I would have never known that the world was changing. Rob Collie (02:17:22): The thing with women in data was that we started doing focus groups when we got something that was an MVP up and running. We started doing focus groups. We recruited people that were friendly to Microsoft. They knew Microsoft somehow. They had someone, a family member, who worked there or whatever, they had some sort affinity for Microsoft, but they were also into football. I was so proud. I would sit down in front of these people and say, "Oh, my gosh. Look at this. You can ask any question you want about a sports situation, an NFL situation." What's Tom Brady's conversion rate on third and long in the fourth quarter or whatever? The guys in the room were not interested at all. They didn't care. They were just like, "Dude, okay. Let's get to the end of the tour, so I can get my free piece of software, like you promised." But the women, not all of them, but the ones that were interested ... I did four or five of these focus groups. Every single time, the person who was most interested in what we were doing in the room was a woman. Just so excited to interact with this thing. Lori Rodriguez (02:18:29): That's awesome. Rob Collie (02:18:30): That was my first brush with maybe curiosity, maybe curiosity runs stronger in women than in men, which if it were true, makes me feel bad for men. Right? What's wrong with me? Lori Rodriguez (02:18:44): Is that the digital equivalent of not asking for directions? Rob Collie (02:18:48): I guess. I think it's exactly it. Right? I could see these women thinking to themselves, "I am so tired of listening to the guy at my office go on and on and on about football, like he knows everything. I'm going to go in and just own him next week with this portal." Lori Rodriguez (02:19:04): I love it. My 17 year old daughter got invited to a fantasy football league. My husband and I were huge Giants fans when we were younger, but life gets in the way, and I just haven't watched football, like I used, to in years. So, it's just not on. So, she hasn't been exposed to football, which, wow, she doesn't even know the basics of the game. She gets asked to be in this fantasy football league. So, what does she do? Talk about curiosity. She starts asking everybody she knows, lots of people she knows that are into football, "How do you play this game? What do you do?" She's led the fantasy football league the entire time, from the beginning, by far. Lori Rodriguez (02:19:45): In the last game, another female, same age, cousin in this family, overtook her by 10 points. So, the top two going away were the girls in the group, one of which has never watched, really, other than the Super Bowl, which we do a big thing for Super Bowl, has never really watched a football game. Because she just went to data, and she was able to get lots of different opinions, look at all the stats, qualitative, quantitative, and make some judgements based on what she's hearing, and just objective, very objective, she doesn't care about any team or any player, and she's led the whole way. It was awesome. I'm tracking with you on that story. Rob Collie (02:20:28): So, I can't think of a better place to end. You hear that? That is what data is about. Ladies, just lean into it. I really do think there's something to this. I think, really, it's a weird thing to say, but I think women are better at it. I really do. Not that our guys aren't. We've got a good team, a lot of good people. I want to see that reflected in our demographics on our team going forward. Lori Rodriguez (02:20:53): Yeah. I'll echo that. I heard a quote, and I love it. It's, "All flourishing is mutual." So, when we compete together, when we play together, we all win. We all push each other to make ourselves better. So, all flourishing is mutual. Yeah. We need more women in data. I'm right there with you. Rob Collie (02:21:12): Yeah. Let's make that happen. I'm really looking forward to your book. Thank you so much, Lori. This has been awesome. This will probably be our longest podcast we've ever done to-date. Thank you so much for taking the time. New Speaker (02:21:22): Thanks for listening to the Raw Data By P3 Podcast. Find out what the experts at P3 can do for your business. Go to powerpivotpro. com. Interested in becoming a guest on the show? Email Luke P, L-U-K-E P, at powerpivotpro.com. Have a data day!