Behavioral healthcare is complex and evolving. How can digital therapeutics improve patient care and outcomes? Originally aired on October 13 at 4:30PST in conversation about how healthcare product managers, designers, researchers and other stakeholders can add meaningful value to their teams. Learn how the best product leaders optimize decision quality for entire organizations, increase development velocity, and navigate the unique constraints and opportunities in healthcare. In this program, we will discuss the ins and outs of building DTx products in the behavioral health space. From evidence-based treatments that enable teens to manage depression, to community-based social support, there's plenty of room for innovation. To learn more about Product in Healthtech, visit us at www.productinhealthtech.com Featured Speakers: Chris Hoyd - Director of Product, Vynyl Davina Pallone - Senior Director, Product Management, Early Product Strategy, Pear Therapeutics Mitul Desai - Senior Advisor, Digital Innovation, Fountain House Gabriel Strauss - Director of Product, Limbix
To learn more about Product in Healthtech, visit us at www.productinhealthtech.com
Featured Speakers:
Chris Hoyd - Director of Product, Vynyl
Davina Pallone - Senior Director, Product Management, Early Product Strategy, Pear Therapeutics
Mitul Desai - Senior Advisor, Digital Innovation, Fountain House
Gabriel Strauss - Director of Product, Limbix
Chris Hoyd:
Welcome everyone. Thanks for joining us for another installment in the Product in Healthtech series, where we explore from a product leadership perspective, what it takes to build great products that add value to businesses and drive positive health outcomes. My name is Chris Hoyd, I'm the director of product at Vynyl, which is a product development and strategy firm that works with a number of healthcare organizations, covering everything from Greenfield product strategy for innovation teams to setting up product departments at early stage companies to large scale digital transformations for incumbents looking to modernize. I am very fortunate to get to host a few exceptional Healthtech leaders today, as we discuss the role product plays in the strategy, design and growth of digital products and interventions in behavioral health.
Chris Hoyd:
I am joined today by Davina Pallone chief product officer at Fruit Street clinic, which is an innovative new tele-health platform that targets the prevention, treatment and reversal of disease by guiding patients from the comfort of their homes. Before joining Fruit Street, Davina built the product team at Neurotrack to launch a digital platform that combined scientifically validated cognitive assessments with personalized cognitive health intervention program. Previously, she scaled the product platform at Pear therapeutics to deliver software that is designed to directly treat disease tested for safety and efficacy in randomized clinical trials, evaluated by the FDA and prescribed by healthcare providers. We've also got Mitul Desai who is a senior advisor for digital innovation for Fountain House. Fountain House is a national mental health non-profit fighting to improve health, increase opportunity and end social and economic isolation for people living with serious mental illness.
Chris Hoyd:
His background includes over 10 years in the health care industry and five years leading teams and products, data services, and digital strategy at a global payments technology firm. His previous experience includes working as a patent attorney at Merck and global law firms and as a healthcare equity research analyst at Piper Sandler. And then lastly, he served in the Obama administration as a senior advisor at the state department where he handled international economic policy and public private partnerships, quite a background there. And then last but not least, we have Gabe Strauss, the product lead at Limbix health, where he's building digital therapeutics for adolescent mental health.
Chris Hoyd:
Prior to Limbix, Gabe worked in product growth and innovation rules in the Healthtech eCommerce and energy industries. And he writes about product management and behavioral Healthtech at behavioralhealthpm.substack.com, which I highly recommend. All right, welcome everyone. Thank you so much for being with me here today. I'm going to kick it off with a question. I'll throw it first to you Davina, your background is so impressive. You've covered so many different areas and so many different kinds of product roles. From your perspective, what are the unique challenges facing a PM or product leader in behavioral Healthtech?
Davina Pallone:
I think one of the most unique challenges, I think it just comes down to learning how to leverage and incorporate your clinical partners in delivering your solution. So if we're coming as product managers or designers from not the health space into the health space, there's just a key new stakeholder that you have to learn how to incorporate into everything you do. I would often say that you, we can't build a product in Healthtech without our clinical partners. And I think that's entirely true. The majority of us coming in to Healthtech as a product manager, as a designer, UX researcher, data science, we do not have clinical backgrounds largely. Some people may, and that's fantastic, but the clinical research that we can leverage is amazing. We're delivering clinical care in a digital format and in order to understand how to do that and what works we just need to become experts at leveraging our collaboration with those clinical teams.
Chris Hoyd:
Excellent. Thank you. And Gabe, what's your take on that?
Gabriel Strauss:
I would say, well, first of all, just to add to that, I think that some people wanting to enter into Healthtech from outside of Healthtech, maybe consumer backgrounds, they think maybe I need a technical co-founder and a business co-founder, but I think really, it's important to have a clinical co-founder or a clinical member of the team from day one, so that we get that input and really embed them into the product development process. And ideally not just have a consultant that's clinical, but really have them as a core part of the product or of the old structure from day one.
Gabriel Strauss:
I think that sort of also adding to this idea of what's different between say a consumer or SaaS product is that we are not the users oftentimes. So we're building intervention for adolescent depression, maybe one person on our team had adolescent depression. But generally none of us are adolescents now. And thankfully we don't have that diagnosis right now and there's some fundamental cognitive differences, for example, differences in engagement. So to that end really incorporating co-design is also a really, really important and really leveraging qualitative research perhaps much more than might otherwise be used. In other industries with product development.
Chris Hoyd:
That's super interesting. I'm going to circle back to that. Mitul, do you have anything to add to those responses?
Mitul Desai:
Yeah, no. Just to build on what Davina and Gabe said, so 100%, agree with what they said and I think to build on Davina's point, really a product person in digital mental health has to hold multiple perspectives at one time when they're building a product. What we're building at Fountain House, for example, is in some ways a three-sided platform. And so it has to serve obviously first and foremost, the patients, but also has to keep in mind as Davina said, how do the providers look at things and really even not just kind of their motivations and their incentives, but really making sure there's product fits into the clinician workflow. If you look at a primary care physician or a psychiatrist, time is the most valuable thing to them, and this has to be really easy and sort of a frictionless plug into their workflow.
Mitul Desai:
And then of course we can't forget the health plan or the payers who oftentimes are paying for these products. And so you have to hold all those different perspectives in mind as you're building the product and to build on what Gabe said about, this concept of for sure for example, I'm working in the space of serious mental illness. And though I have family members who live with it, I myself don't. And so this concept of human centered design, which of course now is kind of common parlance by now, but really doing that and incorporating the patient perspective from day one into these products in a way that builds the trust that you need and the engagement that you need is incredibly important.
Davina Pallone:
I just want to hop in again and double-click on all of that. One thing that can be very different if you're coming from a more consumer tech landscape is definitely some of the regulations. And so if you're coming from FinTech, maybe you're more familiar with the idea of being in a regulatory space, it's not as surprising. If you're not coming from a regulated space, that could be very new to you if you're coming into Healthtech. And another thing that can be a big question mark, if you don't have experience or don't have anybody on your team who can guide you in this direction is how do you find these end-users? And there are ways, but it's not going to usertesting.com for instance. And there's a lot of questions about the sensitivity of how do you then develop a relationship with people who are sharing their personal health information with you?
Davina Pallone:
And oftentimes it could be a very sensitive journey that they have been on with the disease that they're dealing with. So there's a lot of sensitivity both around data around just the emotional intelligence required to interface with people who are dealing with the disease that you are not familiar with oftentimes in a personal way. And so I know at Pear therapeutics, we went through the learning curve on all of this and it's doable, but it's felt very new and unfamiliar for me because I came to Pear therapeutics, not from a healthcare space as a product manager or designer.
Chris Hoyd:
That is so interesting. There's a lot there that I want to unpack as we go. But first I want to circle back to Gabe. You said something interesting that I want to touch on. You said that this role can lend itself to sort of being more reliant on qualitative than quantitative research and approaches. Can you talk a little bit more about that and how do you make sure you do that well, or as well as possible?
Gabriel Strauss:
Yeah. So I think it really depends on what type of Healthtech, but if you're a prescription, digital therapeutic versus a direct consumer behavioral Healthtech product that really impacts how much data you have access to. So on the more regulated side FDA medical device like we are and like Pear was you're probably running relatively small scale clinical trials. And once you get that quantitative data back it can be quite difficult to iterate on it. Particularly once you already have gotten your [inaudible], compare that to direct to consumer product, that's getting tons of data all the time. That's very different. But particularly when the regulated devices really have to rely on a lot of qualitative data co-design sessions, user feedback, and whatnot to do that.
Gabriel Strauss:
The second thing, the other kind of, I guess, difference in quantitative data circling back to that, that I'd like to touch on is in sort of engagement metrics. That's another hot topic, and we can come back to that later, if that was another topic you wants to, but I think it kind of comes in here in a lot of direct consumer and SaaS products you're looking at traditional engagement metrics, like time on site or retention rates and in digital health and behavioral health in particular you really want to look at the engagement metrics that impact clinical outcomes that are leading indicators of clinical outcomes. And they're usually not just going to time and app just retention for example, because people might leave the app because they get better. So how do you find those leading indicators? But again, it can be really difficult to do that. Quantitatively if you have limited data and also in social sciences, as many people might know, it's very difficult to with certainty kind of tease those out. So you often have to be very theoretically driven. And I think that's why a lot of the qualitative feedback can be really helpful in deriving some of those relationships as well and the hypothesis.
Chris Hoyd:
Do you, do you agree with that Davina, With that assessment?
Davina Pallone:
I totally agree about the importance of the qualitative data. And I think that going back to the idea of we who are building the apps are not the ones with the disease. The best way that we can understand what motivates somebody to engage with a therapy for that is by talking directly to them, understanding what, where, and this is the same for non-Healthtech, but it's so unique and I am not able to put myself in the shoes of somebody with schizophrenia, for instance. So I'm going to have to rely on my qualitative interviews with them to understand what is something that would bring you back to this app in a way that feels supportive to you. It's not necessarily because I've been given a guide by a clinician that tells me this is the therapy for schizophrenia Davina please go take this convert it into a digital format.
Davina Pallone:
And we're going to market that. So I would often joke that and if that's the case, you can hand somebody a PDF and say, here's the information. If you read all of this, it will help you. But at the same time, the apps that we're building are competing with things like Instagram, Facebook news apps, something that maybe is more of an escape than dealing with and trying to improve anything around the disease that you have. So the qualitative data that we have, and the ability to conduct various types of qualitative interviews or various studies or what have you, it gives us the information that we need to improve the desire of somebody to interact with this therapy, interacting with the therapy in the way that we understand from brick and mortar or person to person or traditional care.
Davina Pallone:
Usually we can say if a patient interacts with the therapy in this manner face-to-face, we expect to see clinical outcomes like this. And so when we're putting it into a digital format, we need to make it engaging enough to keep them back and doing exactly that. And that's where we start talking about mechanisms of action. And you can gather some quantitative data around that, but again, to Gabe's point, if you're in a prescription digital therapeutic space or a highly regulated space, your ability to gather enough quantitative data, to feel that you have enough information to feel confident about what you're doing you'll get some from a clinical study, but those are all also constructs that aren't necessarily real life. So we have a lot of conversations about this and it's, it's tricky and the sweet spot is going to be different again, to Gabe's point, depending on what your business model is and what disease you're treating.
Chris Hoyd:
Right? So the sweet spot, it sounds like it's somewhere between turning some kind of clinical workflow into a PDF and hijacking dopamine receptors like TikToK or whatever, but somewhere in there. And I would imagine Mitul in particularly for serious mental illnesses, that's sweet-spot is extremely small. Can you talk about some of the challenges you've faced, you mentioned earlier sort of how important it is to build trust with users. And I know your model also relies on some offline interventions and some pretty, I think sophisticated coordination. Can you talk a little bit about your experience with that?
Mitul Desai:
Yeah, sure. Absolutely. And I also want to just build on what Davina and Gabe said. I think two of the drivers of what we're talking about are, first of all we're moving from, so if you talk about digital, mental health, 1.0, it was really focused on access and really driving access, which is incredibly important, but we're moving towards a focus on outcomes, right? And so, as Gabe said, you don't want to just, it's not just time on site, it's just not number of clicks. It's what are the metrics that lead to that outcome which is really important. And the other thing for us, as you talked about is kind of engineering trust into everything we do. If you think about people with serious mental illness, there's a serious lack of trust with the healthcare system and the one-to-one modality talk therapy, or otherwise, frankly, doesn't always work for them.
Mitul Desai:
And so what we have is a community-based model with multiple different touch points from social workers and peer specialists, and that kind of leads to a social infrastructure as well as structured work projects that leads to addressing the symptomology that medication therapy does not address. We fill the social spaces that the healthcare system does not, and you can do that in person, and it has been done for 70 plus years. And then as you take that to digital, you have to A, be really careful about how you're doing this for this population. As Gabe said, it's not just time spent onsite. It's not just number of clicks, it's everything from the acquisition and conversion to engagement, for example, how do you attract folks who may not self-identify as having schizophrenia or chronic depression?
Mitul Desai:
That's an issue, right? And we have some creative ways to do that, and we're testing that. And we think we have some interesting unlocks, but then when you come to measuring engagement. For some folks, episodic engagement is enough. They may have just a need to plug into a social infrastructure or social support system on an as-needed basis, or a couple of times a week. They may not need to be there every day. There may even be we're thinking about, can we build in pathways?
Mitul Desai:
So there's almost a graduation from this. And so if there's a drop off, it's actually a good thing and they can go back sort of into the real world, so to speak and we engage. And so I think people need to really reframe the way they think about these metrics and how they're used. And of course along the way, trust is incredibly important because if we lose that that's kind of one of the most the strongest value props that we have, and there's a number of ways to do that, whether it's co-creating with the folks we call them members, we don't even call them patients or clients. We call people who we work with members because they very much co-create programming. And even in some cases co-employment so really, really important across the board, yeah.
Chris Hoyd:
I'm curious on that last point, because I think we're likely to have some product and design nerds for lack of a better word in our audience. I know you guys want to be a little bit limited in what you can talk about, but are there specific sort of in-app features or nuances or experiences that you've seen work well to help build trust, whether it's an onboarding or just in the nature of the copy, even what have you guys seen be sort of most effective on the products you've worked on and maybe Gabe, I'll start with you.
Gabriel Strauss:
Oh, I mean, I think I'd go back to the importance of calibrating it for your specific audience. What's going to work for an adolescent with depression is very different to an adult with schizophrenia on an older adult with schizophrenia as well. So I think, and it's also interesting that there's a whole sort of taxonomy of different behavior change techniques and whatnot, and the effectiveness can often be all over the place where you look at the data. I think that's because we can implement them well, and you can implement them poorly often the difference is that calibration through co-design and use a feedback. I think one, I guess I can talk well from a trust perspective is this idea of therapeutic Alliance in face-to-face therapy, and that's one of the biggest predictors of people coming back to therapy to adherence and to getting better as well.
Gabriel Strauss:
In fact, it's been said that it's the single biggest predictor and there's this contract of Digital Therapeutic Alliance now, which is really, really interesting. And it kind of does a, I think it's a five item scale, the therapeutic Alliance and kind of map that to developing a relationship with the app itself is the digital program does it feel like a two way kind of conversation with it to build the program of care and so forth? I think I mentioned it in one of my blog articles, if they want to deep dive into it. But so that framework is really, really important. So in our app we have this little character called Limbot, which guides the user through and creates that Digital Therapeutic Alliance. And it's kind of talks about how they have applied that therapeutic modality in their own life to overcome depression and so forth. I could talk more but I definitely pass it off to others to comment.
Chris Hoyd:
Yeah, no I just love that stuff. I have a friend who's a head of design at a digital therapeutic company and she was kind of considering whether having the patient sort of verbally kind of volunteer at onboarding, just describe their experience or what they were going through, if that might sort of help advance that feeling of an Alliance. I just think that those nuances are super interesting. They can help establish that the level of trust that we're talking about here. Davina, I'm curious of what you've seen in your products.
Davina Pallone:
Yeah. I've literally seen the difference that it makes to create a patient panel to work on the language and the tone of voice and quite literally the types of sentences or types of things that one of those patients. So many patients will reach out to external groups of people who are also dealing with the same disease for support. So it's not through the digital app, it's not through any type of a service, but it could even be something like a Pinterest board. So at Pear, we have an amazing lead designer there that began to research sort of the social landscape of a particular patient group. And so she ended up arriving on Instagram feeds and Pinterest boards and noticing the type of uplifting languages and even memes that those patients would post for each other or on their own Facebook threads to just to uplifting.
Davina Pallone:
And so ran a really quick study to understand what if after completing a module or reading something a bit educational about cognitive behavioral therapy or cognitive biases, or just other ways in which we're trying to deliver therapy and when they're done, they get this you might even call it kind of just like a cheesy uplifting meme that says way to go. And after putting that in there, and then running this by our patient panel, so many of them were just like, oh, I really feel like this app it understands me. And it's not high tech necessarily, but it's listening and understanding and empathizing that person connection, but that might be different for one disease versus another, depression versus addiction versus schizophrenia.
Davina Pallone:
You might have very different approaches to what those patients need. So I think you have to develop those patient panels. You have to reach out, you have to involve the people who are actually going to be the end user in designing this. Because again, you don't want to deliver a clinical PDF of what works from a medical sense or a clinical sense. You've got to connect it to what it's like to be a human being in this world today, dealing with that on top of all the other hats that they wear as a person, they could be a mom, a brother, they have a job, they have hobbies, this disease doesn't define people. It's something that they are trying to manage in their lives. And we need to remember that.
Mitul Desai:
So I think that I love that, that this app understands me, right. I think that's, if I wanted to highlight two levers to drive trust is you design for respect and you design for agency. So the rest of society does not respect folks with mental illness just being blind, right? We use this word stigma. I think we should. It's discrimination. And the more serious your illness, the more extreme the isolation, the more extreme the stigma. So again, especially when you're talking about serious mental illness, there's no respect from society and your agency is stripped away. You're just constantly told what to do, take your medication, do this, go there, live here, do this, you can't do this, you can't do this. So if you can design for respect and agency through of course table stakes are like data privacy and security, but things like your programming, your activities.
Mitul Desai:
So yes, onboarding the minute you show up, you feel a warm embrace, and this is a place that understands me, right? Making sure you have pear specialists in there. So it's not just the program, the activities, it's the staff, there's peer specialist who understands me better than folks who have experienced the same condition, and then agency, it should be a place where folks feel empowered to express themselves and empowered to express a point of view. And so I think if you start to build in those features into whatever the app is, you automatically will arrive at a place of trust.
Chris Hoyd:
Some of those points make me wonder as product leaders, we often kind of find ourselves articulating product strategy and marrying that to business strategy or deriving it from the business strategy. So I'm curious if in your explorations of sort of the nuances that we just talked about, have you ever struggled to get buy-in from other executives to prioritize the resources sort of necessary to really nail down with some detail, the nuance of what it might take to make someone feel respected or in Davina your case to find the memes that might make them feel really heard? Has it always been relatively straightforward to get the buy-in from the rest of the organization to invest in that at any given time, or have you guys run into any challenges during that?
Davina Pallone:
Going to happen. Just because I've been in the past seven years at three different Healthtech companies. And so it's three different founders, CEOs, and executive teams. In general, I find that most founders of Healthtech, they do understand, they want to help, right? So you do have a foundation of wanting to help. But there's always been maybe less so, I'm not sure. There's always been a bit of a misunderstanding or maybe for some people, a lack of understanding of how powerful UX research is. For me now if I ever moved to a new role, I say up front, the first hire I'm going to make as a UX researcher. I believe in it that strongly mostly because it's a full-time job and it takes a lot of effort and thought nuance and high, emotional intelligence to build patient panels, to connect, to listen to very hard stories sometimes.
Davina Pallone:
I mean it can be very emotional to hear directly from people what they've been dealing with. And it puts you in a bit of a raw space. But with the right UX research team, and when you can pull that qualitative data in and give that to a team who's building something, most executives, I know the difference is tangible and the stories are powerful. And once you have exposed an executive team or leadership team to what UX research can do for your product, I've never seen anybody decide that they're going to shut that program down. If anything, I usually see at scale.
Chris Hoyd:
I see some heads nodding. Gabe has that been your experience too?
Gabriel Strauss:
Yeah. I definitely echo everything that Davina said particularly around it being a full-time hire. I think that in the early days, especially when you're a small startup everyone has to wear a lot of hats, but I think UX research is one of those high impact a little urgency things that so often gets pushed aside, unless you have someone that's actually dedicated to it. And that's a big mistake to push that to the side. So that's why I think it really needs to have a full-time hire early on. So I just would, would echo the, prioritizing it. The reality is that there are a lot of people, a lot of founders outside of Healthtech that get into Healthtech because they care.
Gabriel Strauss:
But they also just might not get some of the unique things about Healthtech, particularly around the importance of qualitative data early on. And then there's also all the older aspect of commercialization and understanding egoics and proceeds of the payer system that you can't always just common sense your way through, you could be a really, really smart founder, but there's just fundamental things that you can't arrive by from necessarily from first principle no matter how smart you are, you've got to learn about it. And that's a whole lot of discussion, we can have that.
Chris Hoyd:
Awesome. What about you Mitul? What's your experience been like?
Mitul Desai:
Yes, I would just make two points. So one I'm lucky to be working with an organization where this model, this community-based recovery model was actually developed 70+ years ago by individuals living with serious mental illness. So our Genesis is based on human centered design, and that's something that we hold very near and dear to everything that we do. I think the other macro driver that will be supportive of a more nuanced approach is, again, this move from access to focus on outcomes. You can't get outcomes if you don't build products that the patient sees as trusted and as engaging and meets all the things we've been talking about. So I think the overall industry is moving in a place that will be supportive of product managers, arguing for those resources.
Chris Hoyd:
We've touched on a few topics now that I'm going to circle back to. One, earlier Davina you used the term mechanism of action. So while we're still somewhat on the topic of sort of in-app experience and what it can actually do to improve outcomes with your guys' experience with digital health products and interventions, are there certain mechanisms of action that you've seen that have struck you as particularly sort of effective or insightful or anything that's really resonated with you? I just want to give us a chance to kind of highlight any of those things. So Davina, maybe I'll start with you.
Davina Pallone:
I'm going to point to Gabe.
Gabriel Strauss:
I guess the talking point, I think they've spoken about a lack of [inaudible] that it's really unique to the intervention. So mechanism of action. I think that terminology often comes from a therapeutic mechanism of action, which is going to be unique to a therapeutic, right? The mechanism of action in antibiotic is very different from an SSRI. So it's hard to say that this is the mechanism of action that has worked in my Healthtech, my behavioral health product and it's going to work for you.
Gabriel Strauss:
I can talk about frameworks for finding a mechanism of action in your product which I think we've talked about a little bit and it starts theoretically. What are the active ingredients in your specific therapeutic? So in behavioral activation therapy, it's understanding the relationship between mood and behavior, and then tracking that experientially and then actually scheduling activities to impact one's mood. So that's how mechanism of action goes through things. And we measure those as metrics of meaningful engagement. That's going to be very different from probably the meaningful engagement metrics and making some actions for Davina in diabetes prevention program and same for serious mental illness program for schizophrenia. So that's why I'd go with that.
Mitul Desai:
Yeah, by the way, mechanism of action. You're taking me back to my days when I was a patent lawyer at Merck talked about, as Gabe said, it's kind of used in the therapeutics world, for sure. I think one thing I I'd like to highlight is loneliness and socialization. I think now people are recognizing that this is something that's unfortunately a cross cutting issue. It certainly is a massive problem for people with serious mental illness. If you think about the extent of social and economic isolation, that folks with serious mental illness face it's not just that their friends have abandoned them. Oftentimes their families have abandoned them. They're usually not employed. They're usually not in school. Those are all the ways we get socialization and they don't have any of those ways.
Mitul Desai:
So that's extreme isolation, but even folks with milder, mental health conditions do have loneliness and socialization challenges. And so I think that is something that any digital therapeutic that can address that and measure that there are validated scales for loneliness. There's actually a couple and measure that and keep fine tuning, what is the thing that is actually breaking that isolation? And we're just really refining that kind of that one metric that matters and the activities that drive that metric, I think can be really interesting. And it's an unlock for a lot of other positive following behaviors. The way we talk about it at Fountain House, we create a place where your absence is felt right. That's so powerful. It's so powerful.
Chris Hoyd:
Wow. Cool. All right. Well before I jump to the next question, Davina was there anything you wanted to add to those responses?
Davina Pallone:
I think to Gabe's point, yeah. The mechanism of action really depends on the therapeutic you're delivering. And it just kind of came to my mind Mitul, as you were speaking and Gabe that once you have launched your digital therapeutic and you are able to gather more quantitative data, it can be a fun process to go through, to begin to chug that data and look for any of the leading indicators that you can find engagement wise that you have in the app that can redirect people that seem like this is the type of activity that we're seeing that we recognize is either funneling people towards that mechanism of action, or it's encouraging usage of that mechanism of action. But in general, it's leading to the health outcomes that you want. And so you'll start your first iteration and you'll get that out there.
Davina Pallone:
And you have some assumptions, you have some assumptions based off of classic healthcare in this disease space. And you try to digitize that and you try to direct people towards the engagement and the behaviors that you want. Before you're able to get a lot of users in, into that app, you have a hypothesis that you've built something that's going to direct them towards the types of behaviors that lead to that outcome. And now once you have patients in the app or participants or members, and you're actually able to see how they're interacting and you're able to get some more longitudinal data begin to give that to your data science team really play with that, try to find what might be predictive of that. Then you can go back through the design process and you can try to maximize interaction with those parts of your experience and measure and test it again.
Davina Pallone:
And it's incredibly uplifting for our team at Fruit Street every time we get the success stories. Anybody who can tell our coaches that they've been able to reduce or eliminate their medication, that their A1C levels have dropped, that they've lost weight, that they're able to more easily spend time with their family is the one that always just really gets to me. And so, you know that you want to continue completely different than commercial applications. You're not trying to create addictive behavior in the app, but you're trying to reward good change, which is a very hard thing to do, but you're very motivated as a product and design and even an engineering team to make the app better to drive that not because you're going to get more clicks or you're going to get more money, but because the better you can help this resonate with the patient population, the better the chances of their outcome and the better their lives are for those of us who did not go into healthcare, it's an amazing experience.
Gabriel Strauss:
Going to put you on the spot. But I'm genuinely curious of when you send that data to your data science team, were there any sort of counter intuitive or well sort of leading indicators that came up that you never would have considered otherwise, but it came up and wow, this is a really good leading indicator and now has really shifted, maybe informed the feature that wouldn't otherwise being built for some theoretical basis?
Davina Pallone:
I'm trying to think of a good example. Sometimes it can be a bit surprising, we'll do small things like AB testing, email campaigns, who does it come from? And you learn if it comes from the product or the company versus if it comes from a coach or a person with a name that's more personal and maybe that's not terribly surprising, but we see email open rates that would make any marketer jealous. And I think part of that points to how much people... This wouldn't be the same if somebody was sort of pre-contemplative and wanting care, but if somebody really wants the care and you're offering it we've found it's interesting whether or not they get a message from the company, are they feeling that something very official is the thing that would drive them to come back or are they looking for that one-on-one touch? Are they looking for a message from somebody with a name and it's very warm and welcoming?
Davina Pallone:
And sometimes it's not exactly what you would think but really what I'm talking about with the example I have in mind is when people are starting to go down the unhappy path. And so we have lots of ways to trigger an outreach to them, to try to get them to come back. And we'll do testing around that. And we all go in to user testing with our assumptions. I largely find my assumptions are not true that it just further reinforces the fact that I'm not the patient. And I have to go out there and test it. There's probably better examples. We're in the early days at Fruit Street of really playing with this data. And it's fun. It's like a playground. And especially when you know that what you're trying to do has such a good goal.
Gabriel Strauss:
Definitely thanks. And I know I kind of put you on the spot there, it'd be easy to, I'm sure after the fact you'll think of other ones, but I agree that there are a lot of sort of bread and butter engagement techniques that are really effective. For example for us notifications, push notifications are highly impactful and there's a lot of going into it. We had some people that had worked at Facebook, Instagram is like, oh, don't give to them. And we came with the idea that don't send to me notifications, you're going to burn people out.
Gabriel Strauss:
And so we really just sort of eased into it, but over and over again, we hear the feedback, I have depression, I'm forgetful, I want more notifications, please send me notifications. So I remember to do these schedule activities, I remember to use the app. And we've seen that I think 43% of our sessions start within 30 minutes of the notification being sent. So it's clearly having a huge impact and we haven't yet reached that sort of maximum where we start to decrease engagement by going too fast. So it's definitely interesting in some of these techniques that can be used for good as well rather than notifications being something that's bothersome.
Davina Pallone:
One thing that brings up for me is just to note that we're always looking for those leading indicators in the sense that we want to understand what type of... So we started early in this talk talking about, well, there's, sometimes non-engagement is fine when you're talking about Healthtech, right? So maybe it's that somebody is feeling better. So how do you know if somebody is disconnecting because they feel better versus, they're disconnecting and this disconnection is actually not looking good for them. And so figuring out what that difference is. So if people start dropping off towards the end of the program, maybe they're really feeling like they're successful. Maybe if we have a measure of their outcome and we know that it's looking good and now they're sort of not around so much, that could be fine, but we can also look, we also understand now that early engagement in the program is really key to success.
Davina Pallone:
So the onboarding process make establishing that trust early. But the warning signs of somebody not showing up, or in our case at Fruit Street you take photos of your food and it's a food blog. And then you upload those photos of everything you eat and your coach comments on all of them. So participants who are not doing that, who are not engaging in that way, is that okay? Or is that actually an indicator of somebody who might drop out of the program and we should kick into that unhappy path, try to re-engage them and what type of re-engagement works. And so we want to try to understand the behaviors that indicate risk for that person in not achieving their health outcomes. And those could vary widely, depending on people's circumstances, their real lives. And especially with the last 15 months have been a doozy for everybody, right? So that's the sort of thing that we're looking at because we want to try to catch people as early as possible, re-engage them and understand what it is that is disengaging them and what would bring them back.
Chris Hoyd:
So I want to ask now about hiring in this space, we talked a little bit earlier about what makes the role tricky, right? Mitul you mentioned you have to be able to hold a different sort of stakeholder viewpoints in your head to prioritize well and help ship the product effectively. Gabe you mentioned you need to come in with a willingness and ability to co-create. Davina, you mentioned a couple of times the importance of emotional intelligence. So the role is demanding. It's not necessarily easy to find people who are ready for this combined with the fact that I think none of us here would be surprised to learn that mental health is one of the hottest sort of areas in tech right now from a funding and growth perspective, right? So there's the talent wars are on. So Davina as a product leader at a few different Healthtech startups over the last several years someone who has been responsible for building teams, how have you gone about hiring effectively? What do you look for in candidates? What's the process like what are some challenges that you've faced?
Davina Pallone:
So first of all, I cast my net wide. I'm not particularly looking for somebody who is coming from a health, a [inaudible] or a design and Healthtech space. I'm not even really looking for anybody who's coming from a regulatory space. First of all, a candidate who is really excited moving into Healthtech is important. So I need to feel that tangible excitement. I need to understand what's compelling them to move into the space. And largely it's mission-driven, I really have not found many candidates who want to get involved in Healthtech that don't really come back to wanting to help people. Entrepreneurial attitude, because there's still a lot of learning that's happening in this space. I feel a bit more have more experience now, but I very much remember being a senior product manager at Pear therapeutics, and it was all very unfamiliar to me.
Davina Pallone:
So, you see this in job descriptions all the time must be comfortable in a fast moving startup, with ambiguity and none of it is unique necessarily. I'm trying to think of what really pulls a candidate up higher than others. And a lot of it does come down to that emotional intelligence. And I think it's because my opinion is that it's harder to teach emotional intelligence than it is prioritization frameworks or how to write a good story or how to use a particular tool. And so I need to feel energized in my discussions. I love it if I learn something when I'm talking to somebody that's the best, right? Because I want to be around people who are bringing something to the table and that can keep me entertained also at work. I definitely, I often hire people who are way smarter than I am. And I think just because somebody reports to me doesn't mean that I'm not the student. And so I think that me learning something in an interview is always a really great win for any candidate.
Chris Hoyd:
What a great answer. That's awesome. Mitul, what about you?
Mitul Desai:
Yeah. Davina, you sound like an incredible boss, I love that. So like Davina, I look for non-teachable skills. Do they, have things that you can't learn in textbooks? You can learn regulatory issues, you can learn technical skills, you can learn even some of the basic analytics stuff, but what you can't teach someone is empathy, creativity, and really are they in love with the problem, right? We're all working on a really hard part. Healthcare is hard. Behavioral health care is even harder. Digital behavioral health care, even harder. So you have to be in love with the problem. This is not something where you're going to get some kind of overnight success, although maybe if you get lucky, but really, it's going to take years. And so you just have to love cranking away at this problem.
Mitul Desai:
And the good news is we are in a moment where actually, I think we're making transformational changes and we are actually moving the needle. And so I think that's one, I think the empathy piece is huge and you don't necessarily need lived experience. You just have to be willing to put yourself in the shoes of someone who maybe lives with a mental health condition or even providers, right? Not traditional providers. We work with social workers and peer specialists and the like, and they look at the world differently than psychiatrists and psychologists. And to understand the provider perspective really, really important. The last thing, creativity, you just have to be creative in terms of how do you get access? We talked about human centered design; co-creation so important. How do you get access to patient population? So maybe traditional partnerships maybe traditional vendors are not the model. Maybe I need to partner with non-profits. Maybe you need to partner with community care clinics. And driving that creativity throughout every step of the way, super, super important.
Chris Hoyd:
I think those two responses captured pretty much everything I look forward so I'd be surprised Gabe, if you have anything to add to those great answers, but I will kick it to you.
Gabriel Strauss:
Yeah, I definitely echo what's been said already. I think both of you did a great job in describing that. I think, yeah it's interesting for product managers, there's clearly a product managers, skillset that's agnostic to the industry. And then there's, I guess, a love for the industry and a passion for the problem. And I found that having that passion for the problem and coming to work and seeing this particularly in product and sort of the aligned areas is the way that you want to have your impact on the world to give your gifts to the world is what really attracts people at least to our company. And I think to the industry and to stay in the role I think the people that maybe came for other reasons have found other opportunities that have better fit with someone that's specifically interested in deep, deep tech, the tech we're using is relatively, vanilla it's, it's not deep machine learning or something like that.
Gabriel Strauss:
So I look for obviously there's the product management skill sets, but one of the things I really look for is in addition to that, that's really important is someone who... There's people out there that really have a yearning to work in this role. And they want to come to work every day, this is the way they're going to impact the world and leave the world a little bit better for them being there. That's the kind of person that I want to work with and that I feel energized with, it's the person that's going to read about it and think about it outside of office hours and get out of bed and be excited. And I think there are a lot of people out there that are looking for this kind of opportunity. So it's definitely an exciting place to work in.
Chris Hoyd:
Agreed.
Davina Pallone:
I do want to say just one more thing as Mitul and Gabe were talking, I was thinking most candidates would probably say, oh, yes, yes, I'm fine working in a highly regulated space. I think that if you're looking for a job in this industry and you haven't been in it do think about the business model of the company that you're interviewing with because there's a big difference between a prescription digital therapeutic space, where you have a regulated, like a class two medical device and more of a direct to consumer disease prevention space that is going to have to snap to things like GDPR and HIPAA, but it's not going to be as tightly regulated as a medical device. And so if you are interested in the more tightly regulated spaces, you absolutely need to be the type of person who thrives in being creative within constraints.
Davina Pallone:
The example that always comes to mind for me is nothing to do with this space. But when I was in art school, we had assignments that were given to us. And it was like you will paint a canvas that's seven foot wide, and you can only use these colors or something like that. And I was always amazed at how it would unlock your creativity when you were given such tight bounds within which to work. And I think that people who can understand that and can say that, yes, we can't change the copy, we can't change this. We have to do that after the clinical study. And after we get our DiNovo, then we have to jump through all of these fire hoops in order to make a particular change, or we really have to educate ourselves about where the boundaries of the playing field are, and then be super creative in there. If you like that, that's fantastic. If you are just saying that you like that, but you're not sure, you may find the job is well constraining and not creative, right. So that's a really important thing to remember when you're in this space, because it could be frustrating or it can be energizing in its own weird way.
Chris Hoyd:
That's a great point and a good segue to another topic I don't want to touch on, which is kind of the commercialization aspect of this role. I think you all have made some great points around sort of the development and design and co-creation of what goes into an effective product. But as product people, does not necessarily all we have to think about, right? We have to think about sort of the business side and the commercialization model and the challenges that come with that. And I know Gabe in particular, you've written some incredible I think articles on sort of the commercialization challenges facing this field. So maybe can you take a minute and just kind of talk to us a little bit about what you've seen and what you've written about and maybe some ways that we might improve those challenges from a systemic level.
Gabriel Strauss:
So first off, I think it's, you really understand that the user is not the payer. So people that are maybe in a complex B2B product, understand that it's very, very much a complex sale. So you have your personas and the patient personas, but you need to understand the provider personas particularly if it's a digital therapeutic and that's your pathway and then the payers themselves and what is the value proposition to the payer. And in one of the articles that I wrote, I map out all the different types of value propositions that you can have. So probably the strongest is bottom line cost saving and you really need to impact large line items. So for example and I talk about the first order cost and the second order costs, a little bit as well in mental health, but one of the hot areas now is in mental health conditions that are comorbid to chronic illnesses because when someone has comorbid depression or anxiety plus diabetes or COPD or heart disease, those medical conditions get exacerbated a lot.
Gabriel Strauss:
And just understanding how payers think about that. Second value proposition is sort of benefits differentiation. So payers, health plans always trying to attract more members. And particularly right now silver lining of COVID is that mental health is very much on everyone's mind right now. And access is a massive issue. So a lot of members and like employers, self [inaudible] that thinking about who's going to administer their plans. They're going to choose the network that has the best mental health access. So even though having mental health product might not decrease their bottom line, it might increase that top line. I say those are the two biggest ones that you want to think of. A couple of the other ones think about provide a happiness examples.
Gabriel Strauss:
So payers want to attract providers to the network. And if you're offering coverage that providers really want to be able to offer to their patients, it might make them more likely to be part of the network. And the fourth is mental health parody, as well as, is the law requires payers to provide mental health access and there's ways to get around that, but increasingly regulators are clamping down. So it's kind of this idea of the eight there's, the Amazon kind of elastic cloud, but for mental health. So can you help payers sort of expand and contract to meet the demand?
Chris Hoyd:
Excellent. Thank you, Gabe. And I know Mitul, you have some thoughts on ways we might leverage our positions as sort of innovators to at least advocate for maybe some change. So can you maybe talk a little bit about how you think about that?
Mitul Desai:
Sure. Yeah. I mean, this gets to your earlier question around some of the traits to look for in a PM. And this is creativity in all aspects, not just product dev, but business strategy and understanding how your product will be commercialized. The good news is at least based on our customer discovery, talking to payers across the board, whether Medicaid, Medicare, private, they all are saying that serious mental illness gap for them. And they want new solutions in this space. That's a good thing. And specifically they want digital solutions to help meet members where they are. But to Gabe's point, the proof is in the pudding in terms of what does the contract look like? What is it? Is it a PM-PM? Is it a pure value base?
Mitul Desai:
Is there resharing? What does that mean? Probably you're not going to start with a pure value based contract, just given the fact that in behavioral health, there's not that many models and templates to follow. We're getting there, but unlike in physical health, where there are many models and templates to follow, there aren't that many, certainly at least not in serious mental illness. So what are proxy measures and metrics that you can prove to the payer that actually leads them to what Gabe was talking about, cost reduction, demand aggregation, and even things like customers, member satisfaction, I think not just kind of staying in touch, but staying in touch and keeping their members happy, NPS scores and things like that. So you will need to maybe bring payers along in a journey, even the ones who are open and are interested, and especially the ones that are open to this, you want to be able to be flexible and sort of maybe have to do a pilot, start with a pilot but eventually get them to a place where it can be maybe a multi-year contract that's really meaningful for you.
Mitul Desai:
Now, there are other payers who, again, even if they're open to a reimbursement contract and digital SMI solution, even value based-ish type contracts, they may be aren't open to new models and frameworks. And so you have to kind of fit your approach and your contracting mechanism, and your reimbursement contract into the current templates and forms and engagement models. And so that's a completely different approach and can maybe take a longer time period. But what I'm seeing is, again, the good news is that payers by and large are open to this, at least the ones that we're talking to. And so you should come with a plan, not just a plan for your product, but a plan for what the contract will look like in a way that speaks their language so they understand what you're talking about and sets you up for eventual value-based contracts. And there are multiple different ways to think about that.
Chris Hoyd:
That is some great advice. Davina, do you have anything to add here?
Davina Pallone:
I think just one thing is the proof of concepts or pilot studies is a really great way, especially if you're a young Healthtech startup to not only get some users or some patients in your app, but also as Mitul was saying a great way to begin having a deeper conversation and showing what you're capable of delivering. It's a fantastic opportunity for every startup that I've ever seen. And I highly recommend exploring any type of pilot that you can get your hands on more data.
Chris Hoyd:
All right. Well, I think we're about at time. I want to thank Davina, Gabe and Mitul so much. I really admire you guys and the work that you do. This was a pleasure and a privilege for me to host you guys here today. I think you made some incredible points and I look forward to talking to you guys again soon. Thank you.
Mitul Desai:
Thank you.
Davina Pallone:
It was a pleasure.