The Data Product Mindset: Beth Lyons

In this episode of Data Product Mindset, Anthony Cosgrove talks with Bethany Lyons, former Chief Product Officer of Kawa, an analytics startup, and now a consultant building managed data platforms at Assured. Beth shares fascinating insights about the distinction between data assets and products, using her experience building trade matching algorithms and liquidity reporting tools at a London hedge fund. She discusses her unique "build it twice" approach to product development and offers thought-provoking perspectives on how the role of data product managers differs from traditional software PMs. The conversation also explores how AI is transforming the technical capabilities required for data product management. This engaging discussion provides valuable insights for anyone interested in data product development and management.

Transcript

Anthony Cosgrove: Hi, everyone. This is episode seven of Data Product Mindset. I'm Anthony Cosgrove, the co-founder of Harbr, and I'm joined by Bethany Lyons, formerly CPO of Kawa, an analytics startup, and now working for Assured, a consultancy that builds managed data platforms. Beth, it's so good to have you on. Thank you for joining us.

Beth Lyons: Thank you so much for having me, Anthony.

Anthony: We always start by asking people to tell us a bit more about what they're up to. I know you're always up to lots of super interesting things. So, over to you.

Beth: So as you mentioned, I came from an analytics startup. We were building like a warehouse native spreadsheet for big data. I actually joined the most promising prospect who never bought our product because I wanted to learn more about their business and what I should be building them. So I joined this consultancy who runs their data platform. They're a hedge fund. And so I've now spent the last four months working full time from a London based hedge fund, building trade matching algorithms and all sorts of weird and wonderful, fun and interesting stuff that we're going to get to chat about today.

Anthony: Amazing. Building algorithms leads very easily into the topic of - is that a data product? How do you think about data products? Can you give us some examples of data products you've built and worked with?

Beth: I'd say 99% of the work goes into building something that is not a data product, and then the last 1% is what actually is the data product, because that's the point at which the customer realizes value.

I spent three months building this trade matching algorithm that connects the buy and sell sides of trades. The problem they have is that their system doesn't expose the link between the buy and sell trades from the FX desk, and without the link, you can't calculate the profit and loss.

The way they were doing reporting today was just at a company level. They'd look at what was yesterday's equity, what's today's equity, take the difference - and that's the P&L. But if you want to break that down by client or any other dimensions, you need to do the P&L at the trade level.

The challenge is there can be many-to-one relationships between a buy and sell trade. It could be that there's 10 trades of 10 each for buying US dollars, and then there's one trade for a hundred dollars to sell US dollars. Because those add up - the buy side adds up to the sell side and they occurred within the same subsecond for the same currency - you should match those.

There was quite a lot of math involved in getting the links to work in the data because the job was to create the identifier essentially. Is that a product? I would say no, because it doesn't enable anyone to do anything. It's just added an ID field, even though that was most of the work.

The product is the final point at which I integrated this data into their client P&L. They had the client P&L across all of the other desks in the fund, and I basically unioned the FX data into that. So one day, they logged in, looked at their credit report, and there was new data in it. It took me two seconds to do that integration, but that's the point at which it became a product.

Anthony: You hit on an interesting point there, talking about how much manufacturing there is, understanding the value, building it out, testing it, checking it. But for you, it feels like it becomes a product when someone gets value from it.

Beth: I think up until that point, it's what you would call an asset. It has the potential to be used for many different types of products. My background is in finance, so I don't have the imagination to think about what else you might use buy and sell links for, but I can imagine there's a million other use cases why that ID field might be useful.

It's sort of like a platform level capability that has the potential to build ten different products off of. And the one product that we've happened to build so far is this report that the credit committee reviews weekly to try and decide, based on the profitability of each client, how much more or less credit they should be extending them.

Anthony: Going back a step to your role as chief product officer within an analytics startup - you must have had quite a unique take there in terms of what's just analytics, what might actually turn into or become a product. Interested to get a sense of some of the different products you might have seen or your take on that difference and when that starts to happen.

Beth: In our platform, which was sort of a warehouse native spreadsheet, most things by the time the data hits our product are what you would call a data product because it's post-manufacturing - the manufacturing has been done and you now want to build a use case for an end user. So I would say most of the use cases that would get built in our product were data products.

Anthony: Do you have any good examples of products you've seen, whether in that context or beyond, that you felt were particularly interesting or good examples of a data product? People always find it quite interesting and you've already gone through this very specific example with the hedge funds and this particular component and then how you might start to productize that in different ways.

Beth: Do you consider a report a product?

Anthony: Well, that's kind of part of the question, right? I think most people probably would consider a report to be a product. It's very much towards that end of the spectrum where you've done all the development work, you've understood what somebody needs, you've pushed it out as a very specific report that probably needs to be managed and maintained on an ongoing basis, unless it's just literally a one-off piece of analysis. So personally, I would consider a report to be a product.

Beth: Can I talk about another product that I'm building right now? This one is a liquidity report that gives the firm visibility into their liquidity position at any moment in time. Based on how much money they have available in the bank, at the moment they can report on what is the liquidity position as of now. But the interesting thing is what's the historic movement of money in and out of the account.

I'm building another product to do that. It's actually a similar problem where you have bank transactions being reported from HSBC, Deutsche, all those big banks. They don't actually give you the timestamp of when the money hit the bank - that's not available in their dataset, which is crazy. So we have to infer the timestamp from when the transaction first appears in the report. 

There's basically a process that continually pulls the transactions from each of these banks and then assigns a timestamp to a transaction at the first moment it appears in the report. We then need to get that timestamp and pull it out of the banking data and put it into the payment voucher data. The payment voucher data just has the timestamp of when the request for the payment was raised, not when the payment actually came through.

We need to get the payment date from the bank data, pull that into the voucher data, and then you're able to report on your liquidity position over time. It's a similar kind of Pareto principle again, where all the hard work is in the manufacturing process. It's kind of bananas when you look at how much work goes into just getting a timestamp into a dataset in the right way, because you have to do all this matching again.

There can be multiple payment requests raised to the same bank and the same client, and then the client just makes a single transaction. So you have to infer that this 200 is made up of 50, 25, 75 - all those transactions together. So I'm building a linear program that does that reconciliation process. And is that a product? Not really. It adds a timestamp to a table.

Anthony: I know that outside of the manufacturing part of that data product, creating a timestamp, creating a new element - outside of that, I know you're very creative and have done lots of super interesting things, which we've discussed over the years, around how you then find market fit for that product, how you understand user demand and user needs. You've spoken about how much work goes into just manufacturing a simple data element, but tell us a bit more about that side of it. I'm really interested to understand how you approach that in terms of something has been manufactured and I'm trying to see additional use cases, or something hasn't been manufactured but there's a problem and I'm trying to work out what to build for them.

Beth: I think there's two schools of thought of how to approach data product creation. One is an infrastructure first perspective of let's get our house in order, build the foundations, make sure it scales, make sure that we're building highly standardized data models that can be repurposed for lots of uses. And then there's the use case first approach of what column do I need to add into what table and then how do I work backwards to build the data foundations to do that.

I think the first approach is like an engineering first mindset and the second approach, the use case first, is like a product first mindset. As a product person, I'm heavily living in the camp of start with the use case and then build out the requirements from there. In fact, one of the approaches I was discussing with my clients and with my firm was that we should actively implement a process where we build things twice.

The first build is the prototype stage where you just are doing the discovery work to figure out what should actually go into building the product. You build a messy prototype and it works and you get some validation, but the code base is nothing you'd ever want to put into production. Then you have a second stage of development where you go back and refactor everything that you built, architect it properly, and then implement it in a way that it's going to be robust, well tested, monitored, and future proof for new incoming data streams.

Both the client and my firm were very receptive to this idea of build it twice. And it's potentially two different people - there's a big difference in the skill set required to prototype a use case versus to put it into production. That's sort of where I see the distinction between where product management ends and where engineering begins. I think a lot of people start with an engineering first mindset because they get a clear output, but then you don't get the right output. You get a column in a table instead of a liquidity report.

Anthony: The idea of “build it twice” and being quite explicit about we are going to build it twice and the reason to build it twice is such an interesting one. A lot of the concerns are around rework and wasted resource. If I've got something very defined, very structured, I can point at it - it exists, there was an output, there's no debate that work happened and something was made. I could build seven prototypes and throw all of them away. None of them could be ready to go into production, to your point. And that probably looks like wasted work or something that needs to be reworked. So that approach is fascinating because you've just been very explicit about the fact that we are going to do it. There's a meaningful reason to do it. And if we don't do it, there's going to be rework anyway, because we risk just building something that exists and it's there and works, but it doesn't generate value.

Beth: I think it's important for people to know that putting something into production might take 10 times longer than it took to build the prototype. The prototype might get you 90 percent data quality, and it takes 10 percent of the work to get there. The last 10 percent can be 90 percent of the work because they're all exceptions that need to be managed individually. So it helps to set the expectations with people about the value cost ratio of these things. And you want to prove the value before you go and adopt all of the costs to take it over the line and get it to be a fully functioning product.

Anthony: In those conversations, it sounds like with your existing client, that went really well and they were receptive to that idea. Is that a well rehearsed conversation that you're used to having now? Or is this something that just has variable outcomes of stakeholders being receptive to it?

Beth: I mean, I've done two projects for one client, so my sample size isn't huge here.

Anthony: Outside of this space in your other product management and management roles?

Beth: My other product management roles were actually quite different. I built data products, but not in the way that you define data products. I built products that do data analysis like Tableau, which is more like a software application than a data product, even though it's a product for building data products. A lot of the product management that I did in Tableau and in Kowa was much more software product management than data product management. So I'm kind of new to data product management, which is interesting because it's helping me figure out what parts of software product management should we bring to data product management and what parts should we leave behind.

Anthony: Have you reached any conclusions on that or some initial thoughts?

Beth: My initial thoughts are that the product manager should do the development in the data product management world, whereas in software development, the PM doesn't do the development work. I was reading this really awesome article this morning by Tristan Handy, the founder of dbt, about how analytics is not an assembly line. Analytics is a highly iterative process where you can separate the work of a data engineer and analysts and the decision maker. You ideally want all three of those processes to happen in a single brain.

The prototype requires the decision point to inform the analysis, which informs the data engineering work. I think a good data product manager can own and do all of those three things themselves to build a fully functioning prototype before handing it over to an engineer to put it into production and maybe rewrite it from Python into Scala or something like that.

The software world doesn't really have an equivalent of that because in the software world, you build a Figma prototype and then hand it over to an engineering team to implement. You're relying on the product management team in the software world to communicate requirements, design and engineering, but you're not requiring the product manager to actually implement the solution. Whereas in the data PM world, the product manager should be implementing the solution.

In terms of where I see the data product space heading, I think AI is making coding accessible to non-technical users. So I see product managers becoming much more technical. I couldn't write a single line of code four months ago. And then when I joined my client, I was thrown a code base and told to make it work. I could never have done that without ChatGPT and Claude and all of these AI models to help me reverse engineer and read the code base, figure out what it does, and then figure out how to evolve it. So I think data product managers are going to become technical data product managers where they build a lot of their own prototypes. That's basically my advice for anyone in the data PM space - become a builder, use AI to help you learn how to code so that you can build these really high value, high impact products.

Anthony: And at that point we had a bit of a technical glitch. Fortunately we managed to get to the end of the interview with Beth and hear all of her experiences and expertise around what she's doing in data products. If you'd like to hear more from Beth, check her out on LinkedIn. And if you'd like to learn more about data marketplaces or data products, check out harbrdata.com. Thanks for listening.