Pokerbots, Adware, and Burning Man, w/ Brad Miller & Kai Hankinson

Brad Miller and Kai Hankinson from Agree Media were Rob's first clients at P3 before it was even called P3!  These guys have been close friends and business partners for a long time, and they know a lot about data.  We cover: Their ups and downs in various business ventures How they wrote and used AI software to win at online poker, while living in India Organic farming in Hawaii, also known as WWOOFing Their connection with Snoop Dogg How Agree uses Power BI The Lead Generation Business The differences between Facebook and Google advertising Episode Transcript: Rob Collie (00:00:00): Hello again everyone, one of the gifts of my career is all the interesting places it's taken me during some interesting times and all the interesting people I've met along the way. Well, today's guests, Brad Miller and Kai Hankinson are two of the most interesting people I know. Ivy League MBAs, entrepreneurs, rabid users of Power BI. Along the way, they've written poker bots, lived in India, they've been sued by Facebook, and they've lived off the land in Hawaii. But they're more than just interesting people to me. I'm really grateful to say that they're also my friends. Friends who've been along for the whole ride. They were there before P3 was even a company. And really they were the guinea pigs for the whole idea. When I had a full-time startup CTO job, they'd hire me to consult with them on the side, even though I wasn't taking consulting terribly seriously yet. Rob Collie (00:00:49): So they watched P3 develop from literally zero into the nationwide firm we are today. More than watched actually, they helped and vice versa. I've been along for their journey as well. So what I hope you hear is an authentic conversation between friends. A conversation revolving around data, for sure. But again, that human element looms large, the filter was completely off for this one. We talk a lot about failure, rebounding from it, learning from it. We laugh a lot about how hard it was to get them onto the Power BI train. Even though today, they can't live without it. And I think you'll see there's no shortage of personality in these two. It's a long one, but I don't think there was a single dull moment in the conversation. So let's get after it. Announcer (00:01:33): Ladies and gentlemen, may I have your attention please? Announcer (00:01:37): This is the Raw Data by P3 podcast with your host, Rob Collie. Find out what the experts at P3 can do for your business. Go to powerpivotpro.com. Raw Data by P3 is data with the human element. Rob Collie (00:01:53): All right. Welcome. My California startup entrepreneur friends. Thanks both of you, Brad and Kai for coming on. Really been looking forward to this. You guys have been in business together for how long now? How long has Hankin Miller Enterprises been a thing? Brad Miller (00:02:10): I think we got started what, 2005, Kai? Kai Hankinson (00:02:14): I think we got started in 2004 actually. Brad Miller (00:02:16): 2004, so it's a long marriage. Rob Collie (00:02:20): Yeah, I was going to say, and one of you has already forgotten the anniversary. One of you's already forgotten how long. So it's a realistic marriage, isn't it? Brad Miller (00:02:30): It's a very realistic marriage on a lot of fronts. Rob Collie (00:02:33): Yes it is. So you guys met in business school, right? Kai Hankinson (00:02:37): Yep. Rob Collie (00:02:37): And not just business school, right? Ivy League business school. That's where you guys met. I'm an engineer sort of originally like that sort of what the machine churned me out as, as a 22 year old. From my perspective, you guys were in a completely different world. You were in the essentially like the financial and business world with almost like this made man sort of circle, in a way. Right? But today I think of you as data professionals in every bit as much of the real sense as I am. So Wall Street, private equity, my expectation as a borderline Vulcan nerd is that once you're in that world, you would just stay there. You know, you would just stay in that world. You would stay circulating in that Wall Street and private equity world. You'd have other people doing your data for you. You wouldn't be getting your hands dirty in that. You'd just be too important for it. That's my expectation as an outsider, but that's not where you find yourselves. Rob Collie (00:03:39): And so I just want to start here, why did you leave? Why did you leave that world to go do your own thing? And some of the things you've done, as we're going to get to, are very interesting. What led you to branch out like that? Brad Miller (00:03:52): Kai, you want to take that one? Kai Hankinson (00:03:54): Yeah. Brad and I, we met at the first day at Columbia Business School, and we were assigned this little group of four students together that did a semester long project. Brad and I hit it off as friends, and it wasn't long before we started up a weekly poker game at my house where we would take money off of the rest of our business school friends, who mostly came over to socialize and drink the free beer we bought them. We were taking the poker much more seriously just because that was our thing, and we wanted to win. Rob Collie (00:04:29): I've had friends like you at poker games. You ruin everything. Kai Hankinson (00:04:37): Yeah. So we had a lot of fun playing poker through business school. It was around the time that the Rounders movie came out. No, we went to school 99 to 2001. So we were in Manhattan and we'd occasionally hit up those Rounders style clubs in Manhattan that the movie was kind of written after. Those places used to scare the shit out of us, in a good way. But we would just walk into places, scared to death, just these little business school dorks. Trying to look like we knew what we were doing, and we pushed some chips out there. Rob Collie (00:05:09): And then Teddy KGB. Kai Hankinson (00:05:11): Yeah. Rob Collie (00:05:11): Sitting across the table. Yeah. Kai Hankinson (00:05:14): Yeah. You don't want to mess with Teddy KGB. Anyways, we graduated business school finally and went to for the man. I was on Wall Street for Merrill Lynch and Brad went out to California with Deloitte, but we just kept in contact about the poker because we were really interested in it. We started studying the artificial intelligence projects going on in poker at some of the institutions like University of Toronto, I think was a big one. This was back in the day when the AI had beaten the chess problem, but hadn't yet really made a dent in the poker problem because it was a tougher problem. Chess is a perfect information game. Everyone can see all of the board and poker isn't, and because of that, you get the whole component of bluffing and everything else. Kai Hankinson (00:06:04): And so it's just a harder problem for AI, but we were really interested in their attempts to solve it with AI. We ended up getting this idea that we could break the poker hand up into the parts that were best for AI and the parts that were best for human, and just divide up the workload that way. And together build a best case player. We ended up raising some money. We actually raised the money in 2004, which was the reason for my answer, and we quit our jobs in 2005, which was a reason for Brad's answer. We raised the money and left our gigs, and started up a company dedicated to playing online poker with artificial intelligence assistance. Brad Miller (00:06:51): I will say that it was easier for me to leave my job than it was for Kai to leave his job. Going back to your question, I had always considered myself an entrepreneur and I knew that's what I wanted to do. So my time in management consulting was time I was putting in there. It's also very difficult to get rich charging by the hour. Right? Which is the consulting model. So I knew wasn't really my long term goal. Kai, on the other hand, had landed a very plush gig, trading equity derivatives for Merrill Lynch. It took me a while to convince him to leave. I think, at the time, his wife, at the time, wanted to move back to California, and the New York lifestyle might have been killing him a little bit, and so that kind of helped the cause. Brad Miller (00:07:41): During the whole time I had been playing on the nascent online poker rooms, and there was some really basic software that was available at the time, I forget the name. There were all these upgrades that I wanted to make to the software that I was like, "the guy's missing this, he's missing that, he's missing this." And so I'd keep telling Kai about it, and Kai would agree and be like, "what you're describing is a lot like kind of the software and the UI that I use at Merrill to trade derivatives." And so we started to design what this ultimate software product might look like? Rob Collie (00:08:20): Did you hear how Brad started that? He said, "it's really difficult to get rich charging by the hour." You had to throw that in there, Brad, you had to cut me. As the owner of a consulting firm, don't I know it? You know? Brad Miller (00:08:33): I think the difference is you're the owner, so if you've taken that step and taken the additional risk, right? Rob Collie (00:08:40): That's true. I'm still aspiring to be you guys when I grow up, and you're still aspiring to be someone else when you grow up. It's how the game is played. If I can paraphrase the last couple minutes, we really enjoyed playing poker with our business school friends that weren't as good at it, and taking their money. And then we said, what if we could take that and really scale it? What if we could do that online? That'd be great. Brad Miller (00:09:07): I think that's right. One more thing I'd want to throw in though is that I think both of us live by a kind of philosophy where if you're not willing to publicly admit that you do something, then you probably shouldn't do it. And so it's kind of a motto we live by. I know I definitely felt like I was living two lives at the time. I was living Brad, work Brad, for a consulting firm. And then I was living private Brad, and I just really didn't enjoy that. I was like, I want to be one person, I don't want to have to put on an act, I don't want to have to satisfy the man. Brad Miller (00:09:49): Things like Kai went to this mushroom ceremony a couple months ago and had an amazing experience, and wrote an email to our entire team about it, explaining the ceremony and what he had learned. You can't do that at Deloitte. If you want to go run around naked at Burning Man, you should be able to go run around naked at Burning Man; but Deloitte customers probably wouldn't appreciate it. So, that's what I mean by kind of two separate lives. Rob Collie (00:10:17): There's all these like workplace personality assessment, there's a million companies that do this. The ones that I've done, I've done the same one a couple times at different points in my life. They give you two pictures. They give you one of yourself in the professional environment, and one of yourself in the personal environment. There's almost also this distance score between the two. Rob Collie (00:10:41): The mediator in the first session, when I took did this at Microsoft, told us that those of you with a large distance between these two, the most authentic moment he could offer, he was just going through script for all day. Right? But this one moment you could sort of tell he was telling you something that was real. He was sort of dropping the script and he was saying, "if you have a really large distance there, it takes its toll." He said it that slowly, and then he repeated it, "it takes its toll." I'm sitting there looking at my sheet, and me and one other guy and this room of like a hundred people, like we're these outliers with this most dramatic difference between our personal and our professional. And I'm just like, "Ahh!" You know? Took it again recently, but my distance isn't as great. Both sides of me has changed over the 10 years intervening. They've sort of moved towards each other, and I feel a lot better. Brad Miller (00:11:35): Yeah. Rob Collie (00:11:35): I can certainly appreciate that. Did you get rich with the poker thing? Brad Miller (00:11:39): So scale, we did. We ended up moving the company to India because we wanted to use one of the outsourcing firms there. We were working with one of the outsourcing firms. We ended up putting together a group of a couple of hundred Indians that would use our software that we had designed and developed to play online poker at the various rooms. It was pretty crazy. We were doing things where the software was recording every hand online at these poker rooms. So we had data on every handle that was playing. So that was accessible while you were playing against that opponent. Our software allowed you to play upwards of 10 to 20 hands simultaneously. We then started doing things, because it was all through our UI, we would break the hand down into its various components. For example, we would have an expert that would only see hands that had pocket pairs in early position. Brad Miller (00:12:45): Whenever we got dealt up a pocket pair in early position, we would route that to Indian call center agent number 12. So, on his screen at any point in time, you could have 12 hands, only from the flop forward, only pocket pie pocket pairs in early position. That's all he knew. He was an expert. That's what he did. So we just designed the whole thing kind of similar to how he would design a call center, and how you would route different phone calls. We would route poker hands the same way. Kai Hankinson (00:13:19): Yeah, in order to do that, guys and girls would get thrown into the middle of poker hands. So in order to do that, and also to facilitate the ability to play so many hands at the same time, Brad invented this graphical representation of a poker hand to where, in a quick instant, you could catch up on all the action that had occurred up to that point. It really was one of the coolest things we came up with, I think. It was just so powerful to be able to, even if you were playing in the hand the whole time and you forget that a lot of the little stuff, and with this, you can at any point, look back at every little bit and be like, "oh, that's right, he did call that cold on the flop." Rob Collie (00:14:02): Do you have a screenshot of that laying around anywhere? This is a data visualization technique, right? It'd be a really interesting novel thing. We won't actually do this, but it'd be a neat thing to build the Power BI custom visual that reimplemented what you designed. If we were just laying around with nothing to do, we would absolutely do that. Brad Miller (00:14:22): Yeah. It'd be cool. I'm sure we have it somewhere. It's also just such a neat experience because they don't play poker in India. And so we had to train everyone up from the get go. We also couldn't let them that they were actually playing real poker because they were betting, in some cases, hundreds of dollars per hand, they were only making $10 a day. If they knew that they had hundreds of dollars on the line in any given hand, they would just freeze and wouldn't play optimally. So we basically had to tell them that they were just testing the software and they were trying to train the software up. So it was a crazy experience. Kai Hankinson (00:15:03): That was a wild experience. You would walk around that floor, that call center floor, overlooking the shoulder of all these little kids. They were like 20, 21 to 24 years old, just so innocent and fun. You'd look over their shoulder and you would know what was going on, and I'd be nervous as crap like, "oh please, please don't screw this one up. You got to raise him here, you know, right?" I'm thinking, but I don't want to interrupt him because he is also playing four hands at the same time. What a wild experience. Rob Collie (00:15:40): Now, Luke is usually just the silent producer on these things, but in the same timeframe, Luke was playing a lot of online poker. Luke, now do you see what you were up against? Luke (00:15:51): Well, yeah, I might have played against you, and that's the reason why I sucked so bad. Brad Miller (00:15:58): Now, we were playing primarily at the lower limits in the limit games. So, once you get to the high limit stuff, the software wasn't able to compete. So we were mostly in it for the quantity of the quality. Kai Hankinson (00:16:14): We were playing quarter million hands a day, quarter million hands of poker a day at the end. So, maybe we should say though that eventually we got with the Indian operation to where we were highly poker profitable, but the company wasn't yet profitable. We weren't covering our full operating expenses. We were running out of our runway from the money we had raised. We decided to make a bet that we could automate the rest of the hand, and both cut down our cost structure quite a bit and play even better poker than we were with the human training. Kai Hankinson (00:16:52): We eventually packed up from India. We came back to San Diego, boarded ourselves up in an office, and set out on a six month endeavor before we ran out of money to fully automate the hand. We made it just in time, at the end, we got up to a quarter million hands a day, when it was just servers, no operational nightmares, no humans to maintain and get everybody aligned up, logged in, just servers, just pounding out hands on every table we could get a seat on online. You say we were primarily lower limit, I think the truth is that's only because that's primarily what's played. We had guys at every limit poker seat. It was just that there's very few high limit tables. Rob Collie (00:17:38): Did you ever suspect that you weren't alone in this? That there might be other similar operations going on, and some of your hands you were playing, you were also playing against other bots? Kai Hankinson (00:17:50): You definitely wondered. It's just so unknowable. Rob Collie (00:17:55): Yeah. Kai Hankinson (00:17:55): People would accuse us of being bots regularly. I think that's just an accusation that probably flies around online quite a bit. We had a whole setup where when one of our players was mentioned, it would get routed to a human whose job was to like pipe in on the comment thread and come back over the top, "your mother is a fucking bot." Rob Collie (00:18:20): You had a specialist for that as well. Kai Hankinson (00:18:25): Yeah. Yeah, we did. We didn't like being called bots. Rob Collie (00:18:28): Yeah, those accusations that are true, those are the ones that hurt the most. You know? Kai Hankinson (00:18:34): Yeah, we have a poker reputation to maintain as a big bluffer. Rob Collie (00:18:41): At the same period of time when you guys were doing this, I was working at Microsoft and playing a lot of World of Warcraft because, like you were saying earlier, you get this difference between your personal and professional self. There's a point at which you don't really pour as much into the Microsoft job, right? You start staying up late playing these games. There's bots all over those games. Those games are loaded with bots that are running around farming in-game currency to turn around and sell it for real world currency. You can report the bots, and if someone tries to talk to them and if the bot seems human-like in its response, the bot gets to go on its way. Look at the sophistication of the CAPTCHAs that we have to navigate on the internet these days. We're reaching the point where the CAPTCHAs are harder for us than they are for the bots. We're not too far away from the point where the bots are better at solving the CAPTCHAs than humans. Then what do we do? It's sort of game over. Kai Hankinson (00:19:42): The best defense is if you can pass a quick Turing Test. Rob Collie (00:19:46): Oh yeah, of course, everyone knows about that. The Turing Test, if it turns out that, in those situations, the bots are just routing to a real human to pass the test, what are you going to do? Game over. I asked, "Did you get rich playing with the poker business?" You said, "Well, we scaled." Brad Miller (00:20:08): We did not, we got very poor with the poker business. There were two things that went against us at the poker business. One was in around 2008, they passed some legislation that basically killed the industry for a while. All the poker sites were based in offshore jurisdictions that the US government couldn't touch. So what they did was they regulated the bank transactions between the US banks and the poker sites. So there was no way to get money in and out of the poker sites anymore. Which effectively killed it. Which was a pretty smart way to kill it, if you're going to do anything. The other one was we thought that the poker software would kind of go the way of the market trading software, the stock market trading software, the kind of stuff that that Kai was using at Merrill Lynch. We thought that if you could build better software, great, you deserve to win just like in the market. If you can build better software, you're a hedge fund, you can build better software, great. The poker sites don't like that. We didn't quite realize that at first. The reason they don't like that is because what happens is you take it to the extreme. Brad Miller (00:21:20): Let's say this one guy at the table and he kills everyone else. Basically all the money goes to him. He stands up and he walks away, right? The poker site doesn't make any money. If everyone at the table is of kind of equal capability, then what happens is they just push money around hour after hour. Everyone's just sitting at the table, money's going back and forth. And the poker site takes their percentage out of every hand or out of every hour or whatever it is. So the poker site has a lot of incentive to make sure that everyone is kind of at an equal playing field, an equal level. And so they were very anti the kind of software that we were building. So we had those two strikes against us with were too big to overcome. Kai Hankinson (00:22:10): We were making about $5,000 a day in pure poker profit, and we were on our way to getting rich. We just didn't get the money out. Rob Collie (00:22:21): So you got poor instead of rich playing poker. Which, when you say it that way, makes it sound like it was predestined. I'm sure you both took that really well. Right? How many years did you have sunk in it at that point? Brad Miller (00:22:37): Well, that was about 2008, 2009 around there. So... Rob Collie (00:22:41): Wow. That's longer than I realized you guys were at it for four years. Brad Miller (00:22:46): Yeah, roughly four years. Kai Hankinson (00:22:48): Yeah. Yeah. The hardest part for me was we had raised all our money for that endeavor had come from friends and family. Rob Collie (00:22:56): Yeah. Kai Hankinson (00:22:56): So we were a total loss for our investors. That's one thing that I love about our partnership, Brad and mine, is that we've been through thick and thin together. You know? That's not the only company that we had a bad outcome on. We're on our fifth startup together, and we've had some depressing bankruptcies and some exciting exits, and we've been through it all. I've seen a lot of partnerships break up on the downside. I've seen a lot of partnerships break up on the upside, even. When, all of a sudden when you win and you exit, now there's something to fight over, and partnerships break up. We weathered all that together, we're on our, onto our fifth company together right now. Rob Collie (00:23:36): Yeah. That's amazing. Now, just a random side question. What do you know about organic farms in Hawaii? Kai Hankinson (00:23:46): Well, after losing all my friends and family's money, I got pretty down on myself, and Brad got right back on the horse and went right into private equity and was off to work the next day. I took a year and a half off of work just to kind of regroup and find myself. I ended up during that year and a half over in Hawaii doing something called WWOOFing. Which is working for free on organic farms. That was a great time in my life. I did a lot of meditating, lot of soul searching. Rob Collie (00:24:21): I now know too WWOOFers. Paul, at our company was telling me a similar story one day about, "Well, you know, there's this time where I ended up, you know, over in Hawaii and like, you know, living off the land on this guy's farm that he let us..." And I'm like, "Whoa, whoa, wait a second. I've I have heard this story before." What are the odds? I'm a WWOOFer magnet, apparently. Brad Miller (00:24:46): I decided to buy a one way ticket to Hawaii. Bought one of those kind of load of the ground trikes. I don't know what you call them, but... Kai Hankinson (00:24:55): Recumbent trike. Brad Miller (00:24:56): Recumbent trike. Folded it up, put it in a baggage claim, and then was sitting in baggage claim in Hawaii... Kai Hankinson (00:25:03): Baggage claim. And then, it was sitting in baggage claim in Hawaii, putting it together. And then as soon as he put it together, he just hit the road and stopped where he stopped. Brad Miller (00:25:11): Lived out of a tent. I went months without spending money. It's crazy over there. You can just live off the land, there's avocados and nuts growing everywhere. Rob Collie (00:25:22): I really need to try that someday. You're making it sound so good... All right. So poker, Hawaii, private equity and you did something else. What was the next one? Now we're really starting to... AI and ML definitely qualifies as a data thing. There would've been no real reason for our paths to intersect, especially back around 2010. So what were you guys doing in 2010 when we met? Brad Miller (00:25:57): At that time, we had gotten back together. I had come out of my hiatus and at the time when we first met, we were running a Facebook app called Page Rage. It exploded in popularity, it was back in the Facebook app days. That used to be a huge thing. I don't know that they still are. And Page Rage was this silly little app that let you color up your Facebook profile which was, as they are today, very uniform with everyone else's and all. And make it look a lot more like a MySpace page, which was the big competitor back then. People loved it. Brad Miller (00:26:34): That app exploded and we found ourselves in our second startup together with a lot of big data from all those Page Rage installs. Kai Hankinson (00:26:43): I think how did it go? Robert was like, "We just needed a better BI solution." I don't even know if we knew the acronym BI at the time. We just knew we needed a better tool set to help us understand this data. We were using Excel for everything that was after Excel 97. So that's when it went to like a million rows or wait, no. Am I getting the years wrong? Rob Collie (00:27:05): 2007. Kai Hankinson (00:27:07): Right. Rob Collie (00:27:07): Excel went to a million rows per sheet. Kai Hankinson (00:27:10): That was a game changer for Brad and me. I mean, we used Excel for everything and it was limited to 70,000 rows or something prior to that. But then we quickly outgrew the million rows and that's when we started circling around and I ended up finding you Rob online. And I called you and as it would turn out, you were the program manager on our favorite software tool, that Excel 2007 version, I think. Is that right? Rob Collie (00:27:38): Yeah. You see, this is great. You tell people the truth about yourself, like what you did at Microsoft. And then you just sit back and watch as your legend grows. There's an important word, he says, "You were the program manager." Well, I was a program manager on Excel. There were like 15 of us. Now, I was pretty senior on that team and I had things I was in charge of. I was in charge of the BI features in Excel, that kind of thing. But it does sound a lot better when you say it. "You were the program manager for Excel." And so when you introduced me that way and you've done that now a million times with other people, I just let it go. Rob Collie (00:28:18): I mean, of course, why would I interrupt you while you're... I said a program manager, you heard the program manager and you were dually impressed. I was working as the CTO of my first startup at the time. I had left Microsoft among other things, I had seen the promise of this Power Pivot thing that we've been working on. And so I was at a startup that was leveraging Power Pivot, as its primary software. They told me as part of my employment deal, that I was allowed to do some consulting. Okay, great. But I hadn't done any, I hadn't actually done any. Rob Collie (00:28:55): And so you guys and you know this, but for the benefit of the people listening, you guys were the absolute first clients of P3. And I didn't tell you that at the time. In fact, I even tried to get you to hire someone else instead of me, that's how good of a businessman I am, right? Like, "Nah, you don't want to hire me. There's a guy in San Diego who'll do a great job." And so I just point you his way. I tried to have you not hire me and you hired the other guy, didn't you? It didn't go well. Brad Miller (00:29:26): Hired and fired. Rob Collie (00:29:28): You called me back and said, "No, really, you should take our money, Rob. Really, you should take it." And I said, "Okay, fine." And I was just absolutely stunned at what you guys were doing. It was just the three of us in a conference room for two days. There were other people in the office, around and about, but it was just the three of us the whole time. I remember this moment where it started to dawn on me, what you guys were doing. And the scale of it, when we were looking at the number of active add-ons. The number of Page Rage add-ons that were out in the universe, calling home, letting you know they were still alive. Rob Collie (00:30:09): I don't remember what the number was, but it was like double digit millions. In 2010, it was October of 2010, something like 60 million. Was it 60 million? Was it 100 million? I don't remember, but it was just I had never heard of it. I had never heard of Page Rage. And yet there were like 100 million humans or 100 million devices out there in the world running your Page Rage software. I was just like, "What?" Kai Hankinson (00:30:37): Yeah. It was some mind boggling scale. It never ceased to me, how big the world is. It's hard to wrap your brain around it. Rob Collie (00:30:46): Well, wrapping your head around how large the world is, is one thing. Wrapping your head around, there's a reasonably good chance that if I meet a human being on the street, they're running my software, that's a different thing. You guys were paying what? You paid Snoop Dogg to tweet about how good his Facebook page was looking today, thanks to Page Rage and stuff like that, right? I mean, you guys were doing something next level. It certainly got my attention. Kai Hankinson (00:31:15): We had just got a written, the code tails basically, of Facebook. Because, this was right when Facebook was just exploding. And so we were writing their code tails. I think at our peak, we got up to 250 employees. And it was fast, it happened in a matter of two, three years. The Snoop story is funny. That whole influencer model is a lot more prevalent today than it was back then. But he was one of the OGs back in the day. And you would pay him to tweet something out, but he would make you come up with the tweet. So we had 50 white software engineers in San Diego, coming up with Snoop's tweet, which was pretty hilarious. Rob Collie (00:31:59): I'm sure it sounded very authentic. Brad Miller (00:32:02): We had experience passing that touring test, though. Rob Collie (00:32:05): Yeah. There you go, there you go. Are you Snoop? Check here if yes. So the financial transaction in that engagement, went my direction. Money came my way, you guys paid me. But I definitely benefited more from it than you did. Brad Miller (00:32:20): That's an understatement, Rob. Rob Collie (00:32:21): Yeah, I know. We'll get to that. We're going to drag my name through the mud, don't you worry, we're going to do a thorough job. I have a saying that I have been saying now for 10 plus years. Which is that, human beings do not know what they need until they have seen what they asked for. And the very first time this thought occurred to me was when I was working with you guys. We sat down with what has ultimately become our jumpstart methodology. Which, boil it down in layperson's terms is really just, just get started, is the philosophy. Rob Collie (00:32:56): Which we loaded a bunch of your data and the Power Pivot engine, the DAX engine, you did a great job chewing up tens of millions, hundreds of millions of rows. It was great. It was doing everything that you wanted in that regard. We got to a point where we had the chart, the pivot table, whatever that you guys had wanted that you'd described from the beginning. And then as soon as you saw it, you're like, "Ah, no. That's my not quite right. We need to blah, blah, blah, compensate for this, filter, that." That kind of thing. And we tore through four, five, six, maybe even 10 iterations like that in rapid fashion. Rob Collie (00:33:33): And got you to a place where you were looking at something that you wouldn't have even known to have asked for. And it was clear that you guys were really sharp, so it wasn't a failure of brain power. It was just a basic limitation of human capability. You need to see something that you asked for and then realize, this isn't good enough. There was one other thing that happened at the end of that, which was that you guys realized at that 10th iteration or whatever, which didn't take very long. That, "Oh my gosh, we're not even collecting the data that we need to answer the problem." Rob Collie (00:34:10): You probably don't remember this, but it was a really formative moment for me. Where, you guys realized that you needed to have an extra column of data that wasn't being reported by the add-ons. So you had to go back and modify the add-on code. So that in the future ad, when they called home and sort of reported on their health or whatever, their activity. That you'd have the raw data to do what you needed to do. And it was just crazy to me and how fast that happened in the course of those two days. We went from thinking you needed X to realizing you needed X to the 10th derivative. Rob Collie (00:34:45): And then even that going, "You know what? Not good enough." These ever rising standards, the ability to be greedy and truly chase some thing to the end of where it needs to go and watching it play out. And it just had never, ever, ever worked like that in traditional BI. In traditional BI, you don't even make it to derivative two, you just never get there. You get exhausted, you get worn out, there's just too much time, too much expense. And you certainly never get there with regular Excel either. Because, of all the elbow grease that has to go into every single thing that you do. Rob Collie (00:35:21): And I was just like, I left there even more in and I'd already chosen to bet my career on this stuff, right? But I left San Diego even more in. But of course, it didn't take with you guys, right? A week later, how long was it before you stopped thinking about Power Pivot? Kai Hankinson (00:35:41): We just couldn't do it without you in the room. I mean, it was just so hard for our company to adopt those early stages. And I don't know if it's because the product was a little bit immature, it's evolved quite a bit or we had a shitty teach. Rob Collie (00:36:00): You mean the first guy before I came out there? I agree. You totally did. Kai Hankinson (00:36:04): I don't know. I'm thinking about your second consulting engagement. It didn't take the first time, we're like... But you sold us on the revolution, you sold us on the vision. We wanted what you were selling. We just didn't pick it up on that first trip. So we brought you out for a second trip and it was a two day consulting engagement. And the first day happened to coincide with the day I was throwing a company party at my house. Rob Collie (00:36:32): Oh no. Hold on now, you've got this all wrong. You're talking about the third time that I came out to try to make it stick with you guys. There was already a second time- Kai Hankinson (00:36:43): Okay. Rob Collie (00:36:43): In between. You're talking about failure number three. Kai Hankinson (00:36:47): Okay. Rob Collie (00:36:48): The real problem with you guys and your adoption of this, was that you just had too many resources. You had too little time, which is common to everybody. You had too little time to learn something particularly new, right? If you had to learn Excel from scratch on the spot, it would be hard to take that as well. Takes years to get good at Excel. But the other thing is that you just had developers laying around, left and right. You just had too many technical developers that you could go to them and say, "Hey, I want you to write some code to take this data and turn it into a chart." Rob Collie (00:37:21): And it's like addiction in a way. It always seems like it's the cheapest move, to go ask the developer that you've already got to do something for you. And it is. At that moment, it is the short term cheapest move. But you're stuck in this relative minimum. You're just stuck in this gravity well. Kai Hankinson (00:37:39): Yeah. Rob Collie (00:37:39): Where, if you do take the time to escape it, you end up in a much different place in a much better place. But I just couldn't break you guys of that, we're surrounded by developers crack pipe. I didn't have the right treatment plan for you. So that was the second mission, right? I came out the second time and you had even more people there. And my job was to convince, not just you, but also the new guy. And so we did that, he wasn't really all that convinced. And then you still had the developers. Because, you guys just kept getting back on that horse. Rob Collie (00:38:11): You kept falling off and getting bruised and then you're like, "Oh we should call them out again." And I'm like, "Okay, great. Let's go. I always enjoy visiting you, let's do it." So yeah, the longtime blog readers have even seen a picture of the shame that Kai's about to describe. If you've been around reading the Power Pivot Pro blog from the beginning, I sneaked a picture in of this. So go ahead Kai, embarrass me. Kai Hankinson (00:38:38): I'm sure you had a lot of anxiety. This was your third attempt, you had failed miserably the first two times with your first client. We invited you out to my house with the whole company and I'm just going to leave it at, you drank more than anyone in my company. Rob Collie (00:38:57): Lifetime. Kai Hankinson (00:39:01): You weren't anxious at the end. But I ended up having to drive you home and then pick you up the next morning to come back for day two of your consulting engagement. Rob Collie (00:39:10): In my rental car, by the way, right? You picked me up the next morning in my rental car. Kai Hankinson (00:39:16): When I saw you that next morning, I understood that day two was off. You were just going to be physically present. Rob Collie (00:39:26): And you paid me anyway. Kai Hankinson (00:39:28): It was the right thing to do. Rob Collie (00:39:31): And you invited me both days actually that I was there, to your after or noon KPI meetings. Which again, I learned something you were paying me to learn. It was awesome, I really appreciate that. You guys are the best. You had like 20 charts that you looked at every single day at four o'clock in the afternoon. And it was the same 20 charts, every day and there were like 10 people in the room. Kai Hankinson (00:39:57): About 20, easily. Rob Collie (00:39:58): Okay, yeah. It was a big room. Kai Hankinson (00:40:00): Huge meeting. Rob Collie (00:40:01): And there was no interactivity, you couldn't drill down on a chart. That's why there were 20 charts. It's because you probably had started with five charts. Brad Miller (00:40:08): There were probably 50 times four. There were probably 50 slides with four on slides, probably 200 charts. Rob Collie (00:40:17): Right. You didn't start with 200 charts on the first day of the KPI. There's 10 charts and you're like, "Ah man, these don't answer the questions that we need to follow up." So clearly the answer is more charts. And this is how, by the way, traditional reports explode in traditional BI as well. Like, you got a question? You need a new report. Got a question? You need a new report. Organizations sometimes end up with tens of thousands of reports. Many of which have only been looked at once. Some of them have never been looked at, they were asked for built and never used. Brad Miller (00:40:48): Yeah. Rob Collie (00:40:49): And it's all super slow,.but you guys, I was really, really, really rapidly against this. This idea of you'd have 200 static charts. It was in some sense, offensive to me. Because, I was so big on and I still am... On the concept of interactivity in the ability to dynamically ask questions, as opposed to having to build all the charts to answer all the questions, which is a losing game. You never get there. At the same time though, it was a good meeting. I could not say to you that it was a bad meeting or that it would've been better. Because, it was so weird to me that it was the same I guess 200 charts on Thursday that it had been on Wednesday. Rob Collie (00:41:36): How much could change in your business in one day that you would look at the same charts? That the conversation was completely different both days. Kai Hankinson (00:41:44): Yeah. Rob Collie (00:41:44): It was super valuable. Kai Hankinson (00:41:46): Rich conversation every- Rob Collie (00:41:48): Yeah. Kai Hankinson (00:41:48): Every day at that meeting. I mean, it would just be like either the charts would be stimulating. The changes in the charts would be stimulating the conversation or we'd come to the meeting with business issues that we would then turn to the charts to help us think through. Rob Collie (00:42:04): Yeah. Kai Hankinson (00:42:05): And a quarter of that room was dedicated to supplying that product. That's how it went from five slides to 50 slides. Is like almost every meeting we would put in a request that we'd usually be looking at the next day. Rob Collie (00:42:20): Yeah. Kai Hankinson (00:42:21): Because, we had so much expensive manpower thrown at the problem. Rob Collie (00:42:25): I mean, you were in essence, an IT organization from top to bottom. And so when you have that many resources laying around at your disposal, you tend to take that path. But I did, I learned something from that as well. I came back from that experience, having seen the... I also learned, don't eat chocolate cake, birthday cake at the office and that'd be the only thing you eat all day and then go straight to the Cinco de Mayo party at the CEO's house. That's another lesson, don't get into land war in Asia, et cetera, et cetera. But I came back, I realized that I had been too much into the concept of interactivity. Rob Collie (00:43:06): I refined something about my stance about all of this. Which was that, any report you produce needs to be useful before you touch it. Before you interact with it, it needs to already be starting the conversation. Whereas before, I was being lazy essentially and I was saying, "Look, if the thing is interactive and it can answer any question, it can just sit there like a slug at the beginning of your relationship with it every day. Staring at you saying, 'Yeah, I know I'm telling you something that doesn't matter to you, but you can ask me the things you really care about.'" Rob Collie (00:43:40): And so it was almost like interactivity was required before you could derive anything from a lot of the stuff that I had been producing. That's much closer to where I am today, by default useful. I'm not as obsessed with interactivity, it's table stakes that you'd be able to drill down. But if you have to interact with it before it tells you anything at all, it's a bad design. And so again, you paid me, I drank your booze, I came home wiser and you guys didn't adopt it. Kai Hankinson (00:44:10): Yeah. I mean, you certainly helped convince us that we were on the right path to be using the big data, to answer our questions. And we were sold on the vision that you were high priesting. Rob Collie (00:44:21): You say 'we', but let's be clear. You weren't both as equally in, were you? One of you is a little bit more skeptical than the other, that's true, yes? Brad Miller (00:44:31): I think that's true, but I will give you some credit though. I will say that, you have two main parts to your business, right? And we talk about this a lot when we work on the search advertising for you guys. You have your training side of the business and you have your consulting side of the business. And so, when we kept bringing you back, it wasn't just for your fun personality. It was for the consulting side of the business. And so while we weren't yet sold on being trained on the tool, there's a whole side of your guys' business, where you do provide a lot of value just through consulting. Brad Miller (00:45:11): You guys are experts in your field and so when Kai and I are able to sit in a room with you and talk to you about data and BI for hours at a time, that is super helpful to us, tool or no tool. Rob Collie (00:45:25): I appreciate that, that truly actually really is flattering. I always felt like I was learning, interacting with you guys more than you were. And that's just sort of the nature of imposter syndrome, but you're right. That's another thing that's different; if you all met our company today, things would go differently. Even if you were in exactly the same spot that you were in, in 2010. Things would go differently today because at the time, I had a day job, right? I couldn't devote a whole lot of time to you. So it was just like, "Look, I'll come out and tell you some things and leave you some stuff. Rob Collie (00:45:55): And then you're on your own." That's how the company started. The first three years, I was still full time employed elsewhere. But now this is what we do. This is what we do all the time. And so we would have the time to dedicate a consultant or maybe even multiple consultants to helping you on an ongoing basis. Even with fractions of their time, it wills till be a lot more than what I could have allocated back then. And we would just do stuff for you now. Brad Miller (00:46:21): Yeah. Rob Collie (00:46:21): There'd be more continuity there and I think it would've gone a lot differently. Brad Miller (00:46:25): Yeah. Kai Hankinson (00:46:25): Was it next that you got accidentally added to Brad's bachelor party invitation? Rob Collie (00:46:30): Yeah. He's been really kind about never admitting that mistake. Kai Hankinson (00:46:33): And I'm pretty sure it was on purpose. But you were convinced it was an accident that you got invited, to Burning Man. Rob Collie (00:46:39): Honestly, the reason why I was like a Tinder box ready to explode on that fateful Cinco de Mayo, is just, it wasn't just a professional decision leaving Microsoft. I'd been through some really, really tough divorce and family stuff. And I was away from all of my friends and I wasn't in Seattle anymore, I was in Cleveland. Essentially by myself and hanging out with you guys every time you guys would bring me out there. Hanging out with you guys was the best thing that had happened to me in years, it was just great. I've told you guys this, but I would have dreams in which I was out there working with you guys, living out there, hanging out with you. That was how much, that's how strong the pull was. Rob Collie (00:47:24): Those were tough years. Out there hanging out in San Diego of all places. Leave Cleveland, get off the plane in San Diego and be like, "Oh my God." And then it just felt like living a completely different life. "Oh yeah, of course. I'm going to go to the CEO's house for Cinco de Mayo party. It's on the Pacific ocean?" "Yeah. It's on the Pacific ocean. We're just hanging out in the front yard on the Pacific ocean." "Hand me that margarita." Everything was predestined from that point forward. I mean, I was in need of an outlet and boy, did I find one. When I got the bachelor party invite, I mean, it was just too good to be true in a way, right? Rob Collie (00:48:07): Brad wants me to go a Burning Man with him? And there were so many people on this email. I remember it being, I just assumed that Brad has 1,000 friends, it probably wasn't that many. But I just remember it as being like 300 people on the email or something. And so it was so easy to fat finger that and include the wrong Rob or something. Brad Miller (00:48:32): I definitely did not include the wrong Rob. There were a lot of people on the email because I wasn't sure what the response was going to be when I had my bachelor party at Burning Man, because it was 2015. And so, I just invited everyone from my life, going back to high school that I really liked and really wanted to be there. Even if I hadn't seen the person in 10 years, I wanted to invite them. Because if they wanted to go, I wanted them to be there. So there were a lot on there, but they weren't 300. Rob Collie (00:49:04): Okay. And this is really something I appreciate about you, Brad; is just, you're really good at setting aside everything that doesn't matter. And just focusing on what would be the most human and fun thing? You don't get hung up on, "Oh, is it weird to invite these people, I haven't seen you in a while?" You just set that aside. That didn't get in your way. Brad Miller (00:49:28): Yeah. Rob Collie (00:49:28): And so then I did the responsible thing and not read your emails. As soon as it started to become real, I started to get terrified. Like, "Oh my God, I'm not Burning Man material." And you talked me into it. Kai Hankinson (00:49:41): What a trip. Rob Collie (00:49:43): Yeah. Brad Miller (00:49:44): [inaudible 00:49:44] with 18 of us going and only one had ever been before. He had only been one time with his 80 year old dad back in the day. Rob Collie (00:49:53): That was when we finally dispensed with the idea of, this isn't a professional client relationship. This is a friendship. Kai Hankinson (00:50:03): That saved us... Rob Collie (00:50:03): Well, client relationship this is a friendship Brad Miller (00:50:03): That saved us a lot of money. Rob Collie (00:50:05): It did. A lot of failure, you got out of the Facebook business. Facebook had something to do with that, right? You were riding their coattails. One day they turned around, and said, "Hey, get off my coattails." Brad Miller (00:50:19): Yeah. They did not like our business model. They did not like us covering their ads with our ads, that was coming to an end, but before we ended up selling that company, and we had a nice exit from that one. We had these 60, 100 million add-ons that had downloaded our software. We were looking for ways to pivot, we were looking for ways to expand, and so at some point, Kai said, "Brad, I'm giving you three or four resources. I'm sending you down to the basement for three months. You're going to do essentially a consulting project, and figure out where we're going next. You're not allowed to talk to anyone else in the company, you just of focus on this." We went down, we spent three or four months on it, and we came up with two ideas. Brad Miller (00:51:08): One was a data analytics company, because we just had so much data coming down from these add-ons that like, there was something to do with this data. We ended up exiting that company, and selling it to a company in New York. Then the other spinoff was what's today Agree Media, which is in the lead generation space, having so much data, one of the things we could do with that data is identify possible leads for various industries and verticals. That was the original seed of Agree. After we sold off the main company, the big company, Kai, and I bought out our other partners, and so now we're just owners in Agree, 50-50. Rob Collie (00:51:50): Yeah. That timeframe, when you were in the basement, I remember this because this is when I was out there, when I was contemplating, and in the final stages of contemplating leaving the startup I was at and turning P3 into its own distinct entity full time commit to it, that kind of thing. This was in like 2013. Does that sound about right? I just remember when I was out there pitching you guys on being an investor in my company, which in the end we decided we didn't need any investment, but you guys had such great success pitching your friends and family on investing in your companies. I just said, Hey, I'll return the favor. Agree Media. This is why we're here. What is it? We've taken an hour to get around to the crux of it. Rob Collie (00:52:39): When I talk about you guys in Agree Media, to the people who don't know you, I describe you as experts in data driven advertising. People who know the internet advertising space really better than anyone is how I queued it up. You say, it's lead generation. You didn't say, Hey, here's the idea. Let's get really good at advertising. Let's get really good at digital advertising. Let's get really good at digital marketing. That wasn't the goal. That's not what you came out of the basement with. Right. You came out of the basement with, we should be a lead gen business. Brad Miller (00:53:16): Yeah. Rob Collie (00:53:17): Tell us a little bit about the advertising operation. Brad Miller (00:53:21): Yeah. I mean the lead generation business is interesting because one example where it struck home with me was like, you go to various conferences for whatever industry you're in. Right? With the prior software development company that we were working at, that we owned, we'd go to these conferences, and it would be guys in their early twenties, and hoodies, and the whole Mark Zuckerberg look right. Everyone at the conference, the first lead generation conference we went to, the big one is called LeadsCon. We show up, and everyone's either in a suit or a sports coat with tie and shirts. We're like, what's going on here? They're like, we've never seen this before. I've never been to a conference, like where people aren't in hoodies. Brad Miller (00:54:10): It turns out that the lead generation business had been around long before the internet, right? I mean, guys were sending out flyers in the mail, and there was a bunch of other ways to advertise pre-internet, and lead generation business has been around forever. That's the Genesis of where most of those folks had come from. Whereas our background is much more finance, quantitative analysis, software development, artificial intelligence that kind of thing. Instead of coming at it from a marketing bent, we're coming at it from that data analysis AI bent, which has given us, I think, a competitive advantage in the industry. We use that primarily on our own in-house proprietary systems, as well as when we're hooking up with Google, Facebook, Instagram platforms. Rob Collie (00:55:03): I've never heard that part before. That's new to me. It reminded me of, have you seen Glengarry, Glen Ross? Brad Miller (00:55:10): Not in a while, but yeah. Rob Collie (00:55:11): They now disgraced Kevin Spacey, is holding like a box in his hand and says, "No, these aren't, those other leads, those crappy leads. These are the Glengarry leads. You're not worth it. I'm not going to even share them with you. Because you're so rookie." What a hardcore movie that was really. Yeah. I'm pretty sure that the Glengarry leads did not come from a digital system. I'm pretty sure, they came from some other shady thing. I have no idea what it would be, but he holds them like they're gold. Brad Miller (00:55:44): That's the other thing we love about the lead gen business is that other forms of advertising, whether it's television, billboards, print, whatever it was, it was like, there was no real way to quantify the success of those campaigns. Right? Most of where the money was made was like, you had a good salesman who was taking people out for lunch, and drinking martinis mad men style, and convincing them that their campaigns are working. They were rough numbers. You really couldn't see anything down to any level of detail, obviously, because it's all offline. In the lead gen business, it doesn't require much sales, sorry, Tim, our sales guy, I don't mean to throw you under the bus, but like Kai and I, that's not our forte. Right. In the lead gen business, if the numbers back out, and you're sending the quality, you will get the bigger order, and they will pay more. If the numbers don't back out, they won't. Taking out that salesy middle man was really a feeling to us. Rob Collie (00:56:50): This is a theme across all business, right. Is that in the world of offline lead gen where you just have a good set else, person, you can't lose. If you're the lead provider, if you're the lead consumer, you can absolutely lose. You'll never know it, but it's like, I got you some leads, and your business improved. Well that's me. I did that. Oh, I got you some leads, and your business didn't improve. Oh man. Imagine how much worse it would've been. If I hadn't gotten you, those leads he just got a story for everything. A lot of people are even though no one wants to talk about it or admit it, there's still a lot of very important decisions being made in the business world every day that are basically done like that. There's still a lot of gold rush opportunity everywhere. Brad Miller (00:57:35): Yeah, probably about, I don't know, two or three years ago for the first few years of the company, we were primarily Facebook in terms of our advertising. We were strictly social advertising, Facebook, and then Instagram, maybe two or three years ago, we realized that we had to diversify into Google search for two reasons. One for diversification, two for quality. Facebook is much higher in the funnel. You're hitting someone up in their newsfeed and they're not looking to go back to school at that moment, but something catches their eye, and then they might go check it out. Whereas, Google someone is searching. I want to go back and get my cosmetology license. There's a ton more intent there, much lower in the funnel. The quality of those leads are much higher. We're primarily in the for-profit EDU space. Education vertical, so our customers are names, University of Phoenix, DeVry Art Institute, those kind of schools. They were looking higher quality, and so we transitioned to Google. Now we're about, I'd say 20% Facebook, Instagram, and about 80% Google. We used to be Google AdWords. It's now Google Ads. Rob Collie (00:58:52): You guys have put Sally Struthers out of a job. Remember those old ads on TV where she would go, "Do you want to make money? Sure. We all do." Then she'd run through all of these things that, there was always gun repair, gun repair was always one of the options that was on that she was advertising. You talk about a primitive model, right? How did they ever, that these ads worked, but boy, we saw them for years. Didn't we? Brad Miller (00:59:17): Yeah. We gave you what, like your fifth crack at us, at Agree. We like, come on, Rob, come sell us again on this vision for Power BI. Rob Collie (00:59:26): That's right. I mean, actually, even at Agree, there were multiple failed attempts. We'd given up on the notion of it being a financial transaction, right. It wasn't how it worked. I'd just come out there, and hang out and we'd spend some time and chit chat. We'd both learn that kind of thing. I mean, even once we get to Agree, there's at least two or three totally aborted attempts at trying to get this stuff to take. Then Brad goes off and he finds self this thing called Periscope it's just traditional reporting with a slicker new look on it. You guys brought me out. I remember saying, "Hey, can we replace these Periscope reports with Power BI?" I'm looking at them in like on the average dashboard, like three of the modules on the dashboard we're reporting, oh, sorry. We failed to load the data in time we timed out. Rob Collie (01:00:15): I'm Like, I could probably replace these, but I'm might have a hard time with replicating those performance failures. My stuff's just going to work. All the stuff's just going to return the answers. I suppose we could roll some dice or something behind the scenes, but because you guys now are, are using Power BI very aggressively. Brad Miller (01:00:32): Every day. Rob Collie (01:00:33): Even here I don't get to claim credit for it. It was almost, we had to maintain the joke or something. Like if you guys were going to start using it, we would need someone else to have helped you so that we can say that I still was O for 10 with you guys. Ultimately I think Austin was really crucial and really nudging plus the fact that the matter is, is that just because I was first. I was, I was like the first professional in the world dedicated to what is now called Power BI. Rob Collie (01:01:04): I was the only person that was like saying, this is the only thing I do. My only source of money. I was first. That doesn't mean that would never translate to being best. I was pretty good. I was pretty good, but people we hire are better. I am absolutely the worst at these tools of the people at my company. I think that's great, takes a lot of pressure off me. Austin was definitely one of those. He's better at it. He was able to make progress with you quickly. His goal was to build stuff with you, which I never had time for. Brad Miller (01:01:37): Yeah. Austin was key from your side, and then Sean, our head of BI slash CFO was key from our side, so putting those two together, really helped us turn the corner. Rob Collie (01:01:52): By the way, I love that head of BI slash CFO. That's the future, business intelligence owned on the business side or at least driven from that side, supported from the IT side. For sure. You guys don't have a huge footprint of people, but that's your model you're already running on the model that the world is inexorably heading for. You're seeing so many more of these hybrid titles that you just would never have seen before. Rob Collie (01:02:23): I think that's really exciting, and yeah, Sean has really, really, really, really taken to this stuff. What are the sorts of things that you look at in Power BI? Because you know, Hey, I was going to ask the juicy softball question. I mean, but seriously doesn't Google just provide you with all of the reporting that you need. I mean, AdWords has got a reporting portal. Why do you need Power BI? Kai Hankinson (01:02:45): What don't we look at in Power BI these days? It's really got to the point where we almost exclusively, answer our questions with Power BI. What we found is that, sometimes you'll have these situations where a DBA could bang out a slightly quicker answer with a sequel query, but we've learned the hard way that, if you just put the little bit of extra time, and it's not much, but just a little bit of extra effort it into answer that question with a Power BI model, it will just pay dividends. Kai Hankinson (01:03:20): You will be able to keep asking it questions, and you'll be able to ask the next version of the question, and they get easier and easier to build each time. We are using Power BI for all our BI at this point, and we love it. It's like we're finally living the revolution you promised us back in 2010, it took us a long time to get there. Kai Hankinson (01:03:46): One of the reasons the AdWords portal isn't sufficient is that, we need to stitch that data together with what happens on our side, what happens those ad clicks once they get to our website, which ones drop off, where do they drop off? Which ones become leads, where do they become leads in? Which schools are they matched to? Even then we get the offline data that we have to collect and bring back in and import into Power BI, which is, well, what happened after we sold the lead, which leads applied at the school, which leads enrolled in school, which leads actually started. Kai Hankinson (01:04:21): We get data all the way down to, they pass their first class even. So we need to stitch that full conversion funnel if you will, together. And AdWords only has some of it. Rob Collie (01:04:32): That is the thing so often is that it's so rare that there's a report that's useful that truly helps you. That is sourced from only one system. Anything valuable is almost always going to span at least two systems. You're talking about just in that quick survey, there were at least three systems and one of the systems doesn't even belong to you. Rob Collie (01:04:59): It's your customer's systems that are reporting back to you about like, well, you sold us a hundred Glengarry leads, but they actually felt more like Glen Ross or vice versa, right? So you get a quality feedback on a longer time scale. But, but you need to use that to steer because if you deliver poor quality over time, your customers are going to stop buying from you. Rob Collie (01:05:22): And so if you can't span silos with your BI, if you can't span silos with your reporting, you're all always looking at one part of an elephant going like, "Is this an elephant? I don't know. It's gray. Could be a rhino, no idea.@. Rob Collie (01:05:39): And so I'm really happy that you guys have gotten to that. And you and you operated at a pretty impressive scale. You guys can't even use certain commonly available integration tools like stitch for instance, because they have some cap that I'm sure when they set that cap, the engineering team said to themselves, no one's ever going to need more than this. They said it to be 10 times what we think anyone would ever possibly humanly need, right? It's like 10,000 ad groups or something like that. Is there limitation and you guys are like, please 10,000 ad groups. Kai Hankinson (01:06:19): We have some proprietary technologies that we developed to leverage the AdWords platform. It's driven us to have to come up with some elaborate structures over at AdWords that involve hundreds of thousands of ad groups and campaigns keywords. And in order for them to be able to support the kind of integration with our technologies that are... we live by a philosophy of AdWords whereby we think that the smart bidding technology artificial intelligence is that Google has built is some of the smartest AI in the world. Kai Hankinson (01:06:57): I mean, it is just so, so powerful at doing what we need it to do, which is to go out and bid the right amount on the ads that we run. Bid the right amount for the right user at the right time on the right device, all those things. And we believe wholeheartedly that if you're going to succeed, you have to be leveraging their smarts to the max. Kai Hankinson (01:07:20): But then there's also a component of, there are things that we know, not many, but there are things we know that Google doesn't know about our business. We know that that customer just canceled this other customer, close their calls there for labor day. We know we just got a double size order on our Chicago school. Kai Hankinson (01:07:40): We've set up the mechanisms whereby Google's Smart Bidding will learn those things over time, but that's just it, it's going to take time. So we will elaborate technologies that us to leverage their smarts while still injecting ours. And that's a tricky thing to do because it's easy to clobbered their smarts and be like, we're switching Google onto CPC bidding instead of Smart [TiBO 01:08:03] as bidding or whatnot. Kai Hankinson (01:08:05): But now you've just clobbered their smarts and you're not leveraging the best of both. So, that's been one of our driving philosophies there. That's driven the elaborate structures that have taken us beyond some of the caps in many of the products out there, like stitch being one of them, Stitch data. Rob Collie (01:08:20): I even love the names you guys have come up with for some of your proprietary stuff. It is just as a terminology guy. I got mad respect for things like probabilistic pixel and denominator, even Trillions with capital T. I mean, it's hot. You guys have got good naming game Brad Miller (01:08:41): [Redon 01:08:41], don't forget about Redon. Rob Collie (01:08:43): Ooh, I never heard about Redon. You we'll have to talk about that later. I need to know is PG 13, is it Patrick Swayze? Is it, I mean, just let's leave that. But, one of the things that's impresses me about you guys and there's many things obviously, but you guys are just so into the weeds, into the Twilight zone of all of this. Your intuitions and everything. You've spent so many years building them up and learning the way in building all this technology and proprietary techniques and everything for optimizing your ads, you still keep hiring experimentally. Rob Collie (01:09:19): You keep hiring third party ad agencies, ad optimizing services to see if they can do better than you. I mean, so many times when I visited you, like you've been on the phone and interviewing like just some... Oh this is this other firm we're thinking about trying out, just the humility and also just the stamina that it takes to keep doing that. Rob Collie (01:09:46): And it's worth noting that you're still doing that. You're still trying it. If you could get out of the business of managing ads and just be like the leads clearing house, you probably prefer that in a way, no one's ever come along that's that's been able to do better than you. And so you just keep getting better at your own stuff. Brad Miller (01:10:05): Yeah. I don't know if we would prefer to get out of it. I mean people tend to do what they love. If I was kind of retired and really rich and I wanted to spend a few hours a day doing something like this is probably what I choose to do, right. Is to dive into some huge, powerful platform and start pivoting stuff and figuring out. It's my version of a, that's the way I would spend my hobby time. It's like, but I get to do it at work. Kai Hankinson (01:10:37): I love my job. I mean, one of the greatest things about working at Agree is we're only a team of 12 right now, but we've pulled everyone forward from a prior company. In some cases we pulled them forward from two prior companies. So it's the best of the best that we get to work with. Kai Hankinson (01:10:56): We've probably employed like 600 people together over the years, or maybe more. And these are some of the 12 best. We get to spend all day every day, solving fun problems with good people. It's a dream come true. I've learned over the years with our startups that we both love and do best at a specific type of business challenge. We do best when it's like a high volume of transactions with a quick feedback loop because those are the pro that you can just really work the data to kill. Kai Hankinson (01:11:31): It's like, I need that quick feedback loop or else... is just going to be such a slow cycle of improvement. And I need the high volume of transactions to drown out the luck factor. I learned that at the poker company, I just couldn't to believe I used to love table poker prior to running that poker company. And it almost ruined table poker for me, because we would see these runs, these bad luck runs that would just blow your mind. We would have different versions of our software. So we'd be like running version 53.2 for a week. Kai Hankinson (01:12:07): And kicking, come Monday, all of a sudden what happens? We're losing money. Why are we losing money? We're just spending all day diving to the data. This doesn't make sense. How can we be losing money? Kai Hankinson (01:12:18): We're running the same version that crushed last week, nothing's changed. We're checking all these things. Tuesday get killed again. This doesn't make sense. We made money every day. Wednesday, all of a sudden, after 300,000 hands, our luck turns. It goes back to the whole way of just churning out the profit day after day. Luck is such a factor and you need that high volume of transactions to drown it out. If my success depends on whether or not this one big commercial sale closes this quarter, Ugh. I just don't like sleeping with all that. Brad Miller (01:12:54): To answer your question from earlier, we're currently bidding on 17 million keywords in 2 million ad groups. Rob Collie (01:13:01): Wow. Yeah. 10,000 is cute. Isn't it? It's so cute. Yeah. If you let us have 200 accounts, merge them somehow we can use your service. So let's see if we can check the check boxes. First of all, you guys are like, "Ah, I don't want to be in any sort of hourly business," like that professional services business. So that's something you want to avoid. You want tens of thousands of trials a day and you also need almost a near immediate feedback. You have now described the exact antithesis of our business at P3. Brad Miller (01:13:44): I got one more for you that'll fit in that category too. Rob Collie (01:13:47): All right. All right. Bring it. Brad Miller (01:13:49): When we developed the poker software, we could have licensed it to someone else. Right. And let them play poker with our software, but we decided if it's that good and we're building it, we might as well use it to trade ourselves. Brad Miller (01:14:04): And so up until now, and I know we're working a little bit together with you, but up until now, we're like, "Look, if we're that good at AdWords and Facebook and digital marketing, let's just use it ourselves as opposed to doing it for someone else." Rob Collie (01:14:16): That's right. Yeah. We've been sort of inching our way towards as I say it finding a reason to risk our friendship. That's what you do. When you get into business together. You want to ruin a personal relationship. You really want to ruin it like permanently. I suggest going into business together. And so like moths to flame, we've been slowly inching towards this really slowly, but at least with our eyes open. Rob Collie (01:14:42): So we've been collaborating, we've been providing Agree with additional Power BI support and you all have been helping us by architecting and advising, et cetera, et cetera on our advertising campaigns, which is somewhat unusual for professional services firm. Kai Hankinson (01:15:00): Who did you send to help Agree? Of course... Kai Hankinson (01:15:03): This is firm. Rob Collie (01:15:03): Who did you send to help agree? Of course, your woofer. Kai Hankinson (01:15:05): That's right. We sent Paul. We're like, "Okay, we've got an in here. These are two guys that know how to dumpster dive in Hawaii." Rob Collie (01:15:15): [inaudible 01:15:15] has been great. He's really taken us to the next level. Kai Hankinson (01:15:19): Well, like you said, you've hired 600 people over the years and you're down to like 12, that's essentially like our interview pass rate as well. We're like in the 1%-2% range. Rob Collie (01:15:31): Wow. And so, that's that similarly filtered group. We all say it, we're the only office we would ever go back to. And we're all over the country. With a 1% pass rate, you can't hire in one spot. You can't say, "Oh, it's got to be in Indy, wherever I happen to be living at the moment." People all over the country, full-time employees, no office. And if there were an office with all these people in it we'd actually want to go to it. But yeah, we've been saying, "Hey, let's take your advertising expertise and let's apply it, both to help the P3 business but also as a completely different test case than the industry that you've been in." Rob Collie (01:16:13): Let try it with when there isn't the 17 million keyword volume. There isn't that massive volume. Let's try it. When there isn't instant feedback. We're not running an e-commerce business. It's ultimately a relationship business. We have this thing at our company. We like talk about how people say nobody believed in us. Nobody gave us a chance. I don't know how many times I've been told you can't run a professional services business this way. You can't start relationships over the internet, but we can because once you try us you understand that there's nothing like us. Rob Collie (01:16:50): How do we get people to try us? How do we get more people to try us? And it's been very effective for us, for our business model. It's not been easy. And the whole time we were talking about all these hard lessons you had to learn about bad luck, and needing volume, and all of that. They're really echoing in my head because, I think Brad's favorite way to start a sentence with me is something to the effect of, "Well, you'd think that would be a good idea, but..." Right? This is what I do, I come up with ideas or suggestions and then, way more often than a coin flip would indicate, my ideas are bad. Rob Collie (01:17:28): And so, because that subject matter expertise that you... You're not just looking at spreadsheets, or Power BI reports, and turning knobs like robots. You're having to blend this with, what's basically, a decade and a half of experience being in this digital marketing world. This digital world. It's hard. It is hard. And I know that I take a toll on Brad with my, "Oh, come on." Right? Like, " Shouldn't we do X, Y, Z?" And he's just so exasperated with me. Brad Miller (01:18:05): Yeah. What we spend most of our day doing, or at least what I spend most of my day doing, is trying to figure out what's going on in the black box over at Google. Those guys have built something that's essentially taking over the world. The robots are coming. No one really knows. No one knows what they've built over there. It's taken hundreds of engineers and a bunch of marketing people to write up the documents. But we've done several trips over there and tried to track down someone that has a thorough understanding of what's going on under that hood. And the AI is just, it's taking over. So it's kind of figuring out what it's doing and trying to play nice with it. Rob Collie (01:18:53): I read recently there was like some AI that was able to look at a scan of like a retina or something. Was this something you guys told me about? You look at your retina or something and tell you... It was something ridiculous, it could tell you whether or not you had cancer or something. Kai Hankinson (01:19:10): Gender? Rob Collie (01:19:11): Maybe it was gender. But the point is the human beings looking at it, even though it was deadly accurate, no one could understand what the AI was seeing. Kai Hankinson (01:19:19): Yeah. Rob Collie (01:19:20): The human beings that built it don't know how it's doing it. And this is where Skynet comes from. Right? One day Google becomes self-aware, the AdWords platform, and it starts turning the entire earth into a giant conversion funnel. Then we're all paperclips, all the way down. Brad Miller (01:19:43): It's interesting to note the differences between Google and Facebook. And I don't know if we have time or interest. Rob Collie (01:19:49): Sure. Why not? You do operate on both platforms and they have similarities, but they're of course, very different. Probably more different than similar. What does that end up looking like for you? Brad Miller (01:20:01): It looks like a lot more work on Google. We'd probably spend 95% of our time on Google and 5% of our time on Facebook. Or even less. And that's just based on the design of the two products. They're both so sophisticated and their AI is so advanced, but yet they've implemented in a different way that I would say probably 10 years ago, 5 to 10 years ago, marketing was a very different beast. It was all about, "Okay. I got to figure out who my demographic is. I want to target males 25 to 33, with incomes above X, that live in these zip codes." And whoever could identify their demographics correctly and bid accordingly that was the name of the game that was who won. Now, especially on Facebook, there's none of that anymore. Some of the old school agencies are still doing it that way, just because that's all they know and that's what they've grown up with, but it really doesn't make any sense. Brad Miller (01:21:02): So like at Facebook, what we do now is we advertise, someone clicks on our ads, they come through our flow, we get a conversion, we then send back as much information about that conversion as we can through Facebook. Facebook then goes and figures out who to target using their algorithms and their AI. They then figure out who is most similar to that person that just converted for me. So I have no idea, even, who they're going after and no one at Facebook does either, but based on all the hundreds, thousands, millions of different data points that they analyze, they're able to better figure out who to serve that ad to than I would ever be able to just going after males 25 to 32, when incomes over X. So a lot of what we do at Facebook is just content creation, coming up with new ads, different ad copy, that kind of thing. And then you just kind of throw it in and let Facebook work magic. Kai Hankinson (01:22:08): Can I jump in, there's something about this Facebook thing... Another thing I've learned from you guys that sort of stuck with me, is this notion that if you're going to try something like a new campaign or something like that, and you create 10 different ads that are just 10 different candidates to try to achieve the same thing. You just try 10 different content strategies. And then you play this game with yourself of predicting ahead of time, rank the 10, how well are they going to perform? Brad Miller (01:22:33): And your ability to predict which ones are going to perform or is almost worse than random. Kai Hankinson (01:22:38): Oh yeah. Brad Miller (01:22:38): The end result is closer to the inverse of your guess than it is to your original guess. Kai Hankinson (01:22:45): Yeah. Brad Miller (01:22:45): And just really shows how little we actually know and how little you can know. And the value of experimentation. And, as you said before, the value of letting these massive overlord AIs do what they're good at, which no one understands. Rob Collie (01:23:05): Yeah. The whole notion of AB testing, for example. If Facebook becomes irrelevant because, well, a lot of times where people get tripped up is, let's just say, I'm testing two ads... Well, let me give you a different example. So one of the programs that we advertise for is cosmetology. It's 90% female. And then another one is pick a mechanical engineering, or something like that, it's predominantly male. Facebook, without doing any demographic targeting on our end, Facebook will figure out to serve that cosmetology ad to a female and that mechanical engineering ad to a male within tens of impressions. You'll spend a few dollars and Facebook will already have figured that out. And if, you can imagine, that's kind of an AB test and you were running that you might get a scenario where the cosmetology ad is doing really well. You're showing it to 80% of the people and it's performing well. Rob Collie (01:24:10): The other one is profitable, but it's just not performing as well as the cosmetology ad. So typically, what people would do is they'd call the cosmetology ad a winner and turn off the other one, which is exactly the opposite of what you want to do. Facebook has figured out the subset of people, the minority of people that mechanical engineering ad is best suited for. And that's why you want to leave it running, to show those folks that ad, even though it looks like it lost to the cosmetology ad. Kai Hankinson (01:24:41): Yeah. And this is a very difficult thing to internalize. It's a very difficult thing to build into your intuition. And I can speak to this from my own experience. How many times have I come to you and said, "Hey, this thing we're doing is producing a better return than this thing we're doing. Let's turn off the thing that's doing less well." Brad Miller (01:25:03): Yeah. Kai Hankinson (01:25:04): And pour more money into the thing that's doing better. Brad Miller (01:25:07): Yeah. And the one caveat to that is if you are budget constrained and you can only spend a certain amount of budget, by all means, send it to the better one. But assuming you're perfectly fine spending as long as the spend is profitable and you don't have those budget constraints, then you want to leave both going. Kai Hankinson (01:25:27): Yeah. If you live in California and you work in an internet startup, you're not constrained by budget. The rest of us, out in the middle of the country, little more constrained by reality. Budget constrained but you guys don't worry about budget that's- Brad Miller (01:25:43): Well, I mean, if it's profitable spend, that's the last thing you want to be constrained by. It kills us every time we have a profitable opportunity that we can't throw money at. Kai Hankinson (01:25:54): By the way, folks, you might as well just know Brad never lets me turn anything off. If it's the equivalent of setting a pile of money on fire in the backyard, Brad's like, "Uh-uh, no, keep going. It only gets better from here." Brad Miller (01:26:18): That's how you make a million ad groups. Rob Collie (01:26:18): Yeah. So, that's the Facebook side. So, it's very much "Okay, come up with some new ads." But other than that, it's set it and forget it and let Facebook do its thing. Google, on the other hand, has a million knobs that you could literally spend a lifetime turning. And so, while it's AI is also super powerful, it's just not implemented in the same way that Facebook is because that person is giving you that intent. That person is giving you the signal of saying, "I am looking for cosmetology classes." Whereas you're not getting that signal from the Facebook user. Kai Hankinson (01:26:57): Intent makes it more efficient in a lot of ways. But now, like you say, it's so much more tuneable. Even just the choice of keywords, search strings that should trigger your ad versus not. Which of these actually signals the intent to go back to school as a search string? Probably doesn't mean I want to go to cosmetology school. Probably means what are the back to school sales for my kids and things like that. But how do you know? I don't know. You've got to try it. Brad Miller (01:27:32): Yep. Exactly. Google is not going to get... You're not going to see any statistical significance at Google. that's one of the difficulties, in that if you have 17 million keywords a tiny fraction of those are going to be spending at the levels where you're going to get any kind of statistical significance to say that that is profitable or not profitable, which is the challenge. Rob Collie (01:27:56): Yeah. And so, you've determined that casting a wider net is a good thing. The novice approach to this would be to come in and say, look, "I know, naively, hubristically, I know the five types of searches that people are going to run that I want to advertise on. Going back to school, getting reeducated or something like that. If you think about it naively, you'd think that the number of keywords, essentially, is closer to five than 17 million. I think that's something really, really interesting about the way you operate, that you can even get to 17 million. Brad Miller (01:28:34): Yeah. Rob Collie (01:28:35): But also that there's ROI in doing so. Brad Miller (01:28:38): Yeah. Rob Collie (01:28:38): You guys wouldn't do it, if it wasn't ROI positive, Brad Miller (01:28:41): That's absolutely correct. And also we're forced in doing that because in your example, let's just say round numbers. Let's just say we get paid $30 for every lead that we generate. Some of the best keywords in our industry, things like online colleges. Okay? You can't get at the top of the Google search for less than $50 a click. So if I bought a click for $50 and I'm only getting $30 on a conversion that math is just not going to back out. So the only way to compete is in the long tail. And so, we literally can't compete at a short tail. Rob Collie (01:29:20): And how do you do that? Without giving away any secret sauce, how do you get into that long tail business? Brad Miller (01:29:28): Well, the key at Google is making sure that your ad copy, your search term or keyword, and your landing page are all in alignment. The most important thing for Google is that their users to have a quality experience, they want their ads to be as much like an organic result as possible. So they have what's called a quality score and they will determine your quality score for each of your keywords based on how closely aligned your ad copy, your keyword, and your landing page are. And they will penalize you heavily when those are not aligned. And then that gets fed into you how much you're charged per click. Brad Miller (01:30:13): So it is their way of basically incentivizing that alignment. And so if you're able to take a long tail keyword or search term and come up with finally tuned ad copy and a landing page that aligns perfectly, you're going to get rewarded for that. The trick is how do you do that 17 million times? So it's a combination. And going full circle, taking us back to the poker business, it's a combination of what can I do, programmatically, what can I use the computer for, and what can I use the human for? And so it's figuring out how to draw the line and have the balance between the two where you can get scale, and yet at the same time to get that bump in quality score. Kai Hankinson (01:31:01): Google's investment in the quality of the searcher's experience is enormous to their credit. The quality score varies from one to 10 and they multiply your bid, when it gets inserted into the auction, by that score. So somebody with a quality score of one is getting their bid multiplied by one and someone with a quality score of ten's getting their bid multiplied by 10. Now that means that Google's... Imagine that if we go up against someone that's a one and we're a 10. Let's say we bid a dime a click, or something, and we end up paying a dollar and we win it. And they bid 95 cents and lost. Well, look at what Google just did. Google just sold something for a dime that they could have sold for 95 cents. And that's an extreme example, but that's how wide the quality is score... Kai Hankinson (01:31:53): That's how massive the impact is. And I think that's... Plus quality score is a black box, much like the search engine algorithms because they don't want anyone to be able to game it, but that's why I think it is really important to not take shortcuts and just try to build the entire business and website experience, advertising experience around giving users what they actually want, because that's what Google wants. Rob Collie (01:32:20): It's almost... You listen to it that way, they're willing to give up almost like a 10X revenue opportunity in order to provide a better experience for their users. It almost makes them sound egalitarian, almost like they have our best interests at heart. You could really talk yourself into that, couldn't you? Kai Hankinson (01:32:38): They have their long term best interests at heart. Rob Collie (01:32:43): Honestly, I think that's the best we can hope for, really, from anything is everyone having a long term view. That's probably the closest we're going to get to living at peace with each other is if everyone had a long timeframe. I don't care about what happens to the quality of the air. I'm not going to be around a hundred years from now when it's really bad or whatever. Eventually everything comes around to affect everybody. Hey, I'll take a company, that's got its long term interest and view really clearly at the front of its algorithms over the others. Brad Miller (01:33:21): Yeah. Makes sense. Rob Collie (01:33:22): I just talked myself into Google as a good guy. What do you know? You were here when it happened. To get a little commercial for a moment, because we don't run ads on this podcast so we might as well advertise for ourselves. Rob Collie (01:33:35): So one of the things that we're inching closer towards is combining. We'd be like Hank and Miller, P3 joint offering, bringing the power of our consulting team and our analytics expertise and our ability to span those silos, to provide the kinds of feedback. With you guys, that have basically become like cyborgs that are merged with these advertising platforms, and all the tools that you have behind the scenes as well. Coming soon, whatever soon is. This is one of those things, we're not in any huge hurry. We're going to do this when we're ready, not sooner, but when we're ready. We all believe in it. We're all proud of it. We're going to start offering our joint services to others, help them navigate this world of advertising and the same way that you guys have learned to. Rob Collie (01:34:28): I think that's going to be a really valuable offering for the world. If you look on the Power BI templates gallery, there's an ad words template. It's just so cute. That's not going to solve your problem. It's not even going to come close to solving your problem. It ignores all of your own data on your side, like the things that are happening on your end that the ad platform can't see. It ignores all of this incredibly important tribal knowledge that you've developed over the years like, again, all these counterintuitive things. This ability to go after the long tail, which is like even if you see the opportunity, which most people wouldn't... Even if you see it, it's easier said than done. Kai Hankinson (01:35:11): Yeah. Rob Collie (01:35:11): I'm really excited about it. And, like I said, any opportunity that we have to really hang our friendship out over a cliff, put money in professional fortune stake we got to do that. It's gone so well for so long. Brad Miller (01:35:30): It's time. Rob Collie (01:35:34): Yeah. In closing, when's the next email that's encouraging me to go to Burning Man? I've added a recurring nightmare to my palette, which is that I'm packing for Burning Man and I don't have all my stuff and I got to show up and it is... It takes a lot to add a recurring nightmare to your pantheon. And I've got one now. So how long do I have before we start talking about this again? And I have to start getting terrified and excited at the same time? Brad Miller (01:36:04): Well, Kai and I have been four times now. Our CTO has been, I believe, 17 or 18 times. They canceled it this year, obviously, because of COVID. So if they bring it back next year it is going to be quite the event. It's also my wife's 40th and she loves it more than anything else. So if it does indeed happen next year, we'll definitely be there and- Kai Hankinson (01:36:30): See you at the Thunder Dome at sunset. Brad Miller (01:36:32): Yeah. Rob Collie (01:36:33): We've got some things to settle in the dome, don't we, Kai? Yeah. You're right. Them canceling this year is like the whole Burner community has just been like coiling a spring- Kai Hankinson (01:36:48): Yeah. Rob Collie (01:36:48): For an additional year. Kai Hankinson (01:36:50): Yeah. Rob Collie (01:36:50): Maybe next year is a good one to avoid. I don't know, Brad Miller (01:36:55): Probably, but there's no way I'm getting out of it. So I'll be dragging you guys with me. Rob Collie (01:37:00): Like you say, most guys, when they're the wife goes out of town, that's the time to party. When your wife goes out of town it's when you get a rest. Brad Miller (01:37:07): That's right. Rob Collie (01:37:11): That's a keeper. So, Hey guys, we talked for a long time. I really appreciate you guys devoting so much of your valuable time to this and I hope the listeners do as well. So thanks again. Brad Miller (01:37:26): Thanks for having us. Kai Hankinson (01:37:26): Love you, Rob, take care. Rob Collie (01:37:26): Aw. Announcer (01:37:26): Thanks for listening to the Raw Data By P3 podcast. Find out what the experts at P3 can do for your business. Go to powerpivotpro.com. Interested in becoming a guest on the show. Email Luke P- [email protected]. Have a day to day!