Colloque : Conference on the Economics of Innovation in Memory of Zvi Griliches
Conférence du 23 mai 2024 : Causes and Consequences of Innovation
Chair: Dietmar Harhoff
“Who Drives Digital Innovation: Evidence from the U.S. Medical Device Industry”
Intervenants : Ariel Stern, Iain Cockburn
Retrouvez les enregistrements audios et vidéos du cycle et son texte de présentation :
https://www.college-de-france.fr/fr/agenda/colloque/conference-on-the-economics-of-innovation-in-memory-of-zvi-griliches
Chaire Économie des institutions, de l’innovation et de la croissance
Professeur : Philippe Aghion
Retrouvez tous ses enseignements :
https://www.college-de-france.fr/chaire/philippe-aghion-economie-des-institutions-de-innovation-et-de-la-croissance-chaire-statutaire
Le Collège de France est une institution de recherche fondamentale dans tous les domaines de la connaissance et un lieu de diffusion du « savoir en train de se faire » ouvert à tous.
Les cours, séminaires, colloques sont enregistrés puis mis à disposition du public sur le site internet du Collège de France.
Découvrez toutes les ressources du Collège de France :
https://www.college-de-france.fr
Soutenir le Collège de France :
https://www.fondation-cdf.fr/faire-un-don
Suivez-nous sur :
Facebook : https://www.facebook.com/College.de.France
Instagram : https://www.instagram.com/collegedefrance
X (ex-Twitter) : https://x.com/cdf1530
LinkedIn : https://fr.linkedin.com/company/collègedefrance
[Music] [Music] brilliant um thank you thank you everyone um I will do my best to to do my part here uh so um a few sort of notes before I launch into it so this paper is called who drives digital Innovation evidence from the US medical device industry um for those of you who’ve known me for the last 10 years you’ll know that I’ve been at HBS I’m in the process of catching me during the last few weeks of having two affiliations I’m in the process of moving to the house of plaa Institute which is also my apology to my patient discussion uh Ian who not only received a paper late but but not fully baked so thank you Ian again for being for being patient with me um and also a brief advertisement which is I will be hiring postdocs who do stuff at the intersection of innovation and Healthcare um over the coming months and years so if you if you are or know of wonderful people who want to do things uh related to healthcare and innovation digitization in particular um drop me a line so with that um I’m only gonna as everyone knows I do Innovation but really only in the healthcare setting um and I want to give you a quote from this this great book by Bob wer called the digital doctor this is his vision of the future of healthcare the heart failure patient will have his state of hydration and Vital Signs monitored through sensors embedded in a watch a wristband or a stick on device the diabetic or the kidney failure patient who needs periodic blood monitoring will be able to prick a finger at home assuming that test still even requires a drop of blood many of today’s tests will be replaced by sophisticated sensors the specimen will be processed in seconds through a smartphone attachment and the result will automatically be entered into the electronic record so this is sort of the vision anyone who’s actually interacted with a healthcare system on either side of the Atlantic recently is well aware that we’re still quite far away from that nevertheless uh we are really seeing the rise of a set of of both novel and not so novel I’ll talk about that in a bit later uh digital devices in healthcare so um this thing on the left is in fact such a stick on sensor that can measure things um like blood sugar levels through sweat um this is of course a digital insulin pump right here that a diabetic patient might wear um and who knows what the thing on the right is you can see for scale it’s the size of a dime it’s not in the ear it’s a good guess because the ear devices are quite small uh it is a card mems device it’s an implantable heart failure monitor so this device is actually implanted in the patient’s aorta it measures blood pressure within the heart there’s a precipitous change in blood pressure um up to 24 hours before a cardiac event happens for a patient with heart failure and so if you have one of these devices you can actually remotely monitor a patient um the research that has come out since we started this project is increasingly positive about this device um it is cost-effective it saves lives you can actually uh prevent major cardiac events and uh statistically deaths uh by giving patients these things so quite exciting uh what these things have in common is that they’re all regulated medical devices and that’s what I’ll be talking about um some quick definitions less necessary for this audience um so when we talk about digitization uh or in Europe with the extra syllable digitalization um we’re really just thinking about things that that generate Digital Data um or allow to visualize things digitally um digital transformation however many of of you think about um is really about these industry- level changes that come about as as a result of digitization or digitalization um and so what I’m going to do in this project is take advantage of Digital Data from the regulatory documents describing uh newly approved medical devices in fact basically all of them to characterize the digital transformation of the medical device industry um with a brief disclaimer this is still work in progress and it’s super micro so relative to some of the papers we’ve seen recently this is sort of really it’s just going to be the Micro Data so um if you’re here for that great if not then um I’ll be able to see if you’re sleeping but uh feel free to to check out um you know in terms of sort of how to think about this paper I will say that um that in terms of the the sort of Flavor of the research I’m about to talk about uh it’s it’s most similar to the work that you already saw from my HBS colleagues uh kareim and Shane sort of thinking about both characterizing digitization um uh through like a bunch of descriptive statistics as kareim did and then I think there are many similarities with Shane’s wonderful presentation from earlier this morning so uh Shane I at least expect some sort of comment from you afterwards um so how much digitization are we talking so these are the questions that I’m hoping to answer uh at least in part today for you how much digitization are we talking about here who are the actors who are the firms actually doing this I’ll spend less time on that today what types of resources seem to be most important for digital innovation in this setting and then does technological innovation seem to be enabling the rise of new entrance or reinforcing incumbent advantages this is an age-old question but we don’t talk about this much uh in the regulated setting and and digital transformation is a cool way to think about that so um so softwar driven Innovation and Healthcare previous Studies have highlighted the importance of software and digitization determining both from Innovation and performance outside of healthcare many of you know and have written these papers um here we have this opportunity and I think what I what I like about studying um the medical device industry in addition to knowing the data and this setting quite well is this is an opportunity really to observe the wholesale digital transformation of an industry through micro dat and the reason is it is literally illegal to Market medical devices without regulatory approval so if you have and and all the regulatory data is public so if you have all the regulatory approval data you you can necessarily see the universe of products that ever came to Market um in general uh that said we know very little about the impacts of software and digitization and Healthcare there are a few notable exceptions studies of electronic health records uh studies of things like genetic privacy laws um but otherwise we actually don’t have tons of Empirical research at the intersection of sort of healthcare and Innovation thinking about digitization um and recent years there’s been of course massive growth in the number of connected uh devices fitness trackers medical equipment and the medical device industry itself is $70 billion per year uh industry in the United States alone it’s about double that worldwide so it’s of consequence um so what am I going to show you uh diar’s going to keep me on time so if I run out of time then uh at least you’ll know where this ends up so far um we’re going to use text anal analysis very much in the spirit of what kareim and his colleagues did perhaps a bit less sophisticated to characterize over 30,000 new medical device product summaries um we’re going to document substantial digital innovation in medical technology with pretty clear heterogeneity across different medical specialty areas it won’t surprise you but it’s nice to have these data I should just mention if you ask the US FDA how many digital medical devices did you approve last year they would not be able to tell you because that’s not something that’s RA in the regulatory data so you need people like me who are fascinated by regulatory data sets by Innovation to actually go scrape product summaries and tell people like you about it um but we’re also uh doing some work communicating this to Medical audiences of course um we’re going to examine the relative importance of measures uh that will seem very familiar things like with INF firm experience Financial Resources do you have VC funding how much VC funding do you have um and geography and predicting new digital product development we’re going to find that different types of products or different types of firms are going to be contributing to to digital new product development differently um in terms of really novel Innovation and this kind of has flavors of of Shane’s presentation earlier um both publicly traded firms and those with with Venture Capital at all and greater amounts of venture capital are going to be more likely to commercialize truly novel digital devices these are products in categories that did not exist until a digital product came to Market um but access to Capital or these measures of access to Capital I should say seems to play very little role in predicting the digitization of analog product lines and what I mean by that is like slapping a digital monitor onto a blood pressure cuff we’ve been measuring blood pressure with regulated medical devices for decades and decades um actually since before our current medical device regulations existed in the United States we’ve been able to measure blood pressure cuff blood pressure with blood pressure cuffs now they’re all digital um that is digitization but that’s not really innovation in the sense like we could measure blood pressure before um and so these measures of access to Capital don’t predict basically the digitization of existing analog products or developing follow-on Innovations um what does is is these different measures of experience so these are things like if you have experienced developing clinically similar products if you’ve developed a lot of cardiovascular products you are going to be marginally more likely to develop followon digital devices in the cardiovascular space um both clinical experience as well as software experience predict the development of these followon digital products and in contrast um we actually don’t find that any form of this of experience whether it’s experience with regulated products experience with a specific medical specialty area or experience with digital products none of those things predict Who develops truly novel digital products so these measures of experience tell us sort of who does various flavors of follow on Innovation as such um we so far have concluded that the digital transformation of the medical device industry has afforded new and distinct opportunities for industry incumbents versus new entrance um and that sort of slices differently of course depending on how you think about things like Capital so medical devices super quick um this is a heterogeneous category of of medical products that you don’t ingest or inject um these can be analog so these can be things like a simple catheter they can be relatively complex things like a you know drug alluding coronary artery stent or they can be digital they can be insulin pumps or implantable heart failure monitors which you learned about earlier um what I’m going to do is consider all of the devices that are subject to an FDA regulatory approval process this will include all of the high-risk moderate risk products out there as a quick reminder uh this will include the first two categories so this is class three devices these are devices of very high risk these are implantable and or life sustaining if something goes wrong it will go horribly wrong and could cause permanent damage or kill you moderate risk devices uh you don’t want anything to go wrong it could cause damage but it’s unlikely to kill someone is basically the rule of thumb these are things like blood pressure monitors hearing aids by the way just about every diagnostic or treatment algorithm I have some extra work separate work on artificial intelligence and regulated medical devices I’ll show you one picture from that later um just about all of the AI products that the FDA regulates are in this category so just in case anyone’s wondering um what I won’t tell you about are the lowest risk medical devices these are not subject to a formal regulatory clearance prod process these are things like latex examination gloves stethoscopes um tampons condoms these are things we want to be manufactured in a clean and certified facility but we’re not going to actually regulate uh the product itself because it’s so low risk okay we’re going to have three data sets uh we have Regulators administrative data from the FDA two recent Decades of medical device approvals something we’re working on uh right at this moment is getting another two years of data which I had hoped to have for this week and don’t but um it’s coming um unstructured text Data describing all of these devices so we’re going to actually scrape product summaries I’ll show you a screenshot I think on the next slide um and then we’re going to match this with firm level financial data um at and leading up to the time of medical device commercialization from various sources that you’re probably familiar with as well as data about VC financing and then we’re going to account for Acquisitions through a process that you don’t want to know how the sausage is made but took two months to get right um but basically that the firm that is responsible for bringing a device to Market is given credit for that for that um commercialization and then we’re going to uh pull some additional Geographic data on things like expertise and experience um using e using firms own histories of product commercialization from the FDA as well as measures of like how many software Engineers do you have in your state uh to figure out if that’s a a friction or predictive here um and we’re going to be able to uh consider about 30,000 new devices so this text database um I’ll show you the screenshot now on the next slide so what is this so it’s part of the application for clearance or marketing that’s sent to the to the FDA at the time of product approval it’s published online the day after a new device is cleared or approved for marketing it’s in a standardized format 98 % of these documents are machine readable with off-the-shelf optical character recognition software um and there are no sort of funny concerns you should have about that 2% there’s no concentration uh over time or in Medical Specialties um it seems to just be sometimes things were randomly scanned in at a diagonal um one of the other things that we’re doing in this data update is using better OCR software and that number will go up uh don’t worry about that piece uh we’re going to end up with 30,000 computer readable text documents um and uh we’re going to do some text analysis now it’s really next slide uh so uh method one is actually I’ll give credit to Dan Gross who told us about um the National Library of medicine’s uh mesh descriptors we’re going to do what Dan did um it’s the off an off-the-shelf algorithm um it’s used all the time it’s rigorous it’s third party but it’s uh it’s not transparent so you can’t actually see the algorithm itself you just get the output of the algorithm when you send the text files to the National Library of Medicine so we don’t really know what’s in there um the simplest thing that’s super transparent but but homegrown is this supervised document classification that we do where we basically focus on keywords from a glossery of computer terminology and this looks quite a bit uh like what kareim and his colleagues did we’re going to scan for keywords that are unambiguously associated with software Andor product digitization words like software and computer um the nice feature of this we had raas check nearly 200 of these I think it was 180 these words are only used if the functionality of the device actually involves these things so there’s no reason like you wouldn’t write in one of these product summaries oh by the way this hip replacement device does not include any software like that there’s no there’s no reason to mention software if there is no software um we found a in the the 180 that we looked at we found a 0% rate of false positives so um you do with that what you will um and we’re going to focus on multiple terms it turns out just looking for the word software actually does the trick so this is the screenshot I promised um we’re going to have a summary of safety and Effectiveness data for this device this is a metronic continuous glucose monitor what you can see is just there’s a whole section about software this thing is clearly a digital product and it talks about its software a lot um I don’t I don’t want to bore you with the tables but we’re basically going to you know have the sample that you expect I have um we have these different flavors I I just want to take a moment to tell you about these different flavors of new devices so there’s a new digital product code that’s what we think of as novel Innovation this is a product that add itss a product type that add itss in section was digital uh these product codes you can think of as things like basically product categories these are things that are that are uh excellent subst stitutes for one another these are things like there’s a product code for drug alluding stance there’s a product code for um pacemakers so these are things they’re not perfect substitutes but they’re basically excellent substitutes for one another um and then you can there are various ways in which you can do follow on Innovation you can take something that’s historically been around for a while that’s been analog you can put some you can put a digital display on it that’s the first digital product code in an existing the first digital product in an existing product code you can have a followon digital product and then you can have a digital product update which is a firm that already has a digital version of a product in a product code comes up with another one so that’s just like a product line update that’s not even I mean that’s innovation in some sense but not novel Innovation okay I’m going to go through this quickly just in the interest of time I’ll show you some pictures this is easier to digest than tables so you can see the share of newly approved digital devices is growing over time this isn’t ASM toting this isn’t doing what you know Kareem’s pictures did uh what AI does also in this setting but you know this is sort of digitization lots of heterogeneity that Top Line is Radiology we’re now at a point where like 96% of all Radiology devices that come to Market uh have or interact with software um we can look at the cumulative number of product types that have been digitized that’s going up over time we can look at the cumulative number of firms doing this that’s going up over time we can look at the cumulative number of new digital product codes this is that novel type of innovation that’s going up over time uh if anything there’s a lot of action in cardiovascular devices uh which is the the solid navy blue line um we can look at the cumulative number of digitized previously anal analog product codes so this is this like digitizing and analog thing also going up and to the right over time we can look at the cumulative number of product codes with followon digital devices these are the four different flavors I told you about of of new product commercialization going up over time um and then these are the updated digital devices this sort of mechanically only has to go up over time but um it is so these are again this is when a firm that already has a digital device in a specific product code comes up with another one so all of these uh are and the axes are different in all these so we’re going to run some quick regressions um I’m looking at the time I’m going to uh skip over this um I I’ll get to the conclusions I definitely won’t boil you with this table which is far too small you’ll see uh See I get to this in a minute I just want to say one just quick word about ongoing work in next steps um something that we haven’t done yet is actually look at how these things change over time the the graphs I just showed you look pretty smooth but um we have some reasons to believe that if you if you run some of these regressions on different subsets of years the coefficients change a little bit I want to understand that a bit better I can’t tell you much about it now but thinking about what’s going on over time in particular what’s going on in the context of the introduction of artificial intelligence and machine learning devices um do the Innovation profiles for these products differ from you know quote unquote regular digital devices digitization of course is a necessary but not sufficient condition for artificial intelligence and machine learning uh this graph is figure two from uh from chapter four of the NBR volume on uh the economics of artificial intelligence in healthcare which came out inch March um where uh this top line is just a version of the pictures I just showed you which is the share of software devices the bottom line which is on a different axis and this is important is the is the um number or not the not the share sorry the number of of software devices in the the yellow line is the is the number of AI devices um which uh looks much more like the pictures kareim showed you earlier and um I think there’s a question maybe Ian you have some thoughts on like how can we kind of incorporate things like this um is this an instrument for something um next steps so in summary I have two minutes right this is I’m doing okay this my my last slide uh we see significant growth and digital medical devices with a lot of heterogenity across Medical Specialties I I think there’s more to do there um we have this method for using off-the-shelf document classification tools to analyze the contents of new product descriptions um resources matter for commercializing novel digital products um both publicly F traded firms and those with more VC funding are more likely to innovate um but exess to Capital doesn’t seem to matter for all of these different flavors of followon innovation um experience in developing similar products both clinically similar so again within the same medical specialty area or softwar driven products is associated with a greater likelihood of digitizing analog product lines but not with developing novel digital devices um and so as I said earlier you know the the digital transformation if we can call it that of the medical device industry um does really seem to be creating different types of opportunities for different types of players and I welcome any thoughts on um what we should do next this is still work in progress um I think I I think I like saved us a minute so now we’re only nine minutes behind uh yes that’s that’s right and uh in 5 seconds you should get your final minute ring but we can proceed uh and hear the speaker thank you it’s a pleasure to to read and think about this paper as it invariably is with Ariel’s papers um don’t want to rehearse word for word the the principal findings but uh let me just offer some observations what I think was is learnable here and why we care um you know first point I think is that medical devices is a grossly neglected and understudied part of the Innovation economy uh Ariel is one of the few people working on it uh you know it’s it’s a couple of hundred billion dollars a year in the United States put that in context Pharmaceuticals is you know maybe 700 billion uh it’s very large um it has these kind of interesting distinctive features which make it I think a useful place to go testing and thinking about our ideas about how we think the Innovation sector Works somewhat like the pharmaceutical business Market access is largely controlled by a safety regulator uh unlike Pharmaceuticals it’s an area where there’s this kind of rapid and continuous incremental Innovation that looks a lot more like ICT and Silicon Valley in that respect um it’s also an industry where uh you know there’s an interesting and different cast of players including but not limited to you know I think it’s approximately true that every single orthopedic surgeon has their own little stainless steel thing want to to drill into your spine there’s a lot of you know artisanal physician level inventors working on this and they think they go surprisingly up far up the the hierarchy of of complex and expensive devices um so that’s all kind of interesting and different um you know this is a very gesan paper as I say that I’m reminded of the the years I spent sitting in a Carol outside V’s office where there’s a parade of people coming in with every possible mispronunciation of his name I think my they will come and knock on the door and say things you know Professor grilled cheese that was my favorite grilled cheese but uh you know grickles grickis Grill you um at the end of which you’d always you know in the course of this discussion I couldn’t help overhearing there’d be a comment of well you know I think you got some problems with this data uh so this is a very Grill aesi in paper in the sense that it’s you know involves some heroic data collection I’m reminded of what Jim Heckman was saying yesterday about administrated data sets you know they’re large they’re comprehensive they’re very interesting they’re often you know difficult to deal with so hats off to what the authors of this paper have done in digging into you know all of this public data which is actually a substantial barrier to entry to getting into it and understanding it uh and linking it out um you there is I think here a very useful distinction drawn uh between adding digital capabilities or add-on functionality to existing products you know versus doing something which is completely new and is in and is of its Essence something digital um here’s my set of pictures of of medical devices one of the top left there is an implantable pacemaker interestingly enough you know those have been around since since the late 60s and in them they have a computer in the late 60s it was kind of an analog computer but what it does is you know monitor what’s going on in terms of electrical activity and if it looks like you’re going into you know atrial fibrillation then it’s you draw some power from its battery and zaps your heart and tries to get it going again the current state of the art obviously does not in you know there’s not a little tiny circuit board with you know valves on it transistors um but you know it’s a enely a little computer sitting in there you know hopefully running the Linux operating system and not you know windows but the top right is you know where I think the the state of the art is here now here’s the here’s the dreaded spot on the scan which has been found by a machine for many years they used to track digitization by asking my MBA class if you call an Uber and it arrives and there’s no driver are you willing to get into it so eight or 10 years ago the answer was about 10% now it’s over 50 but I’ll just let’s get some audience participation in the after lunch session hands up if you were be perfectly happy you felt a bump and the doctor came back and said well the machine says it’s X and I’m going to prescribe you drug y who’s willing to take the machine with no human radiologist 20% yeah then there then there are these things you know the the the continuous glucose monitor uh and then lastly and I think this is a really interesting thing this is supposed to be a picture of a know clinical decision support software where Ai and algorithms are going to try to force the the physician’s hands in what they how are they going to diagnose or treat you each of these I think are kind different and interesting classes of of uh of progress in this area um so I have a few you know questions and suggestions here um let’s start with this question of what can you identify the descriptions of new devices as being digital and in what respect I’m sure there’s not many false positives from the approach that the authors have taken here even though as I understand it from the paper they’re using a pretty restrictive set of keywords um the false negatives you know I’m not quite so comfortable with um so I have a couple of suggestions here one of which is the FDA has published this list of what 882 AI machine learning medical devices that would be a good you know validation exercise to see what your rate of false positives and negatives is against that you could also think about extending the set of keywords uh by going to things like abstracts related patents and so forth where I’m a bit worried about the validity of this approach is in thinking about changes over time you know spare a thought for the regulator here you know back to that cardiac pacemaker you know it’s kind of like the problems you have with console software for the Boeing 737 right there’s an algorithm in there it’s running you can try to validate the software if it goes wrong it’s going to go very badly wrong and so the FDA since the arrival of more and more of these digitally uh based devices has been really struggling to come up with a regulatory framework so that suggests to me that is the possibility for some unevenness over time and particularly depending on the categorization and the amount of FDA wrangling you know there’s a clear incentive for for some device for some of the applicants in this process to try to you know weave and Dodge and and try to find their way through this regulatory system I don’t think this is likely a big problem but it’s something I’d want at least to think about another thing I think you could usefully do here with these data now the FDA and other International their International equivalence is starting to you know try to draw a hard distinction between software in a medical device so that’s like the pacemaker example or CT scanners doing computerized tomography versus uh software as a medical device which are the AI clinical decision tools and then so forth I think splitting the data that way might be might be helpful um the the paper starts you know after coming through this data it goes through some interesting observations about or initial observations about how they might think about IO and competition in this industry um I’m not too surprised that you don’t find much truly novel digital stuff coming from new firms because the problem is that the incumbents have a lot of intellectual property right pacemakers have a couple of thousand patents covering them right so it’s going to be very hard to well I want to take dear’s exciting new device and going to add another feature on it cuz if deepar doesn’t license me I can’t get into the market so I’m not even going to try um I’m also a little uh concerned here about how to think about the demand side here you know this this is not the the place to worry about it in depth but you know the healthc care sector has been 20 studious steadily 25 years behind the times compared to the rest of the economy and uptake of digital anything for a lot of complicated reasons you know I wonder whether that impacts incentives for innovators to to think about getting into this a minute one minute um unfortunately sales data are hard to find and very expensive it would be great if you could do things like reweight the findings based upon you know is it a Mickey Mouse little thing that never got anywhere versus versus something exciting uh we can’t do that um I’ll wind up here by making the observation I think there’s at least two and possibly three papers in the inside the one that Ariel sent me to read one I think is a measurement paper which now I think is is important and really useful and you know you know there’s clearly further Avenues to go there the other is this kind of incumbent entrant what’s the nature of competition what’s the role of of incremental versus radical Innovation there um and thirdly Ariel didn’t uh emphasize it but there’s this angle about what’s it got to do with uh uh Jo geography uh so I encourage authors to keep working and I look forward to reading all three of those papers as they emerge thank you thank [Applause] you great uh we have time for questions and comments but first a response from yeah thank you I learned my lesson this morning if you’re discussing a paper have to get up again don’t sit like smack dab in the middle of the row um so I made it up quickly this time uh so Ian absolutely uh this is wonderful and I was I was delighted uh that you were going to be discussing this paper because I think you you also are one of the few people who knows about the medical device industry Visa things like Pharma and I think that there are some good but also some very bad analogies that get made in Innovation research in that setting um you know in short uh so I agree with sort of the excitement around studying the medical device industry I think the false negatives point is really important I think the only thing I want to say before we open up for questions is um there are always measurement challenges when you’re trying to come up with new measures and so um I welcome uh perhaps not now but in the next coffee break uh any suggestions about things very much in the spirit of what you suggested I think we’re we’re doing some work we’re actually do using a new database and trying to kick the tires a bit around what we’re actually measuring I think there’s an argument to your point about sort of drift over time of actually like what if we just focus on the last decade um because 20 years of data sounds sounds really cool and sexy Until you realize that like people talked about things differently in 2003 versus 2023 so I I agree completely in fact we’re seeing glimmers of that already so youve you nailed something that you didn’t know we I already believe is a weakness of the data set so uh and the sort of split of software in versus as a medical device I think is increasingly important um and that’s a great suggestion we’ll do that um so that’s all Shane has a mic already yeah great [Music]