The AI Masterclass was delivered by the iheed CEO, Dr Tom O’Callaghan and Dr Mark Sujan. Dr Sujan is a Chartered Ergonomist (C.ErgHF) and Managing Director of Human Factors Everywhere. Dr Sujan is also a visiting academic at the University of Oxford, a visiting Fellow at the University of Loughborough, and honorary Associate Professor at the University of Warwick. He is a Trustee of the Chartered Institute of Ergonomics and Human Factors (CIEHF) and chairs the Institute’s special interest group on Digital Health & Artificial Intelligence.
Afternoon good afternoon or depending on where you are in the world good morning or good evening uh my name is De M branck I am director of academic Affairs here at iheed and absolutely delighted to welcome you all to this master class on the deployment of AI in health care
Uh joined by two esteemed colleagues um Professor mark sujan from the University of war and Dr Tom mallahan CEO and founder of iheed and they’ll introduce themselves in a minute and get into the actual Master Class itself this is one of a series of master classes run across
2023 and 2024 um some are publicly available some are only for uh students enrolled on our programs and what we are trying to bring is a a series on some of the most topical areas in healthcare AI of obviously being a very prominent one at the moment future ones will include
Social media Finance for non-financial professionals Refugee health patient engagement healthc care compliance and a whole whole range of uh a whole range of subject areas that will hopefully Empower you to improve health systems and ultimately deliver better patient care throughout the next hour we will welcome as many questions as you have um
Obviously with so many people on the webinar the voice is disabled for everyone and as is the chat but the Q&A uh is where we would love to get as many questions as you have so you’ll see the Q&A tab uh down the bottom we will
Answer those either in real time or at the end of the session and your question you can you can ask it publicly or privately and again we we may answer something that’s relevant to the group uh publicly so the session is being record it and you’ll get a copy of the
Recording afterwards uh to rewatch and with that I’m delighted to hand you over to Dr Tom mallahan thanks very much thanks deerman thank you very much I’d really like to welcome everybody to this important discussion I know that many of you joining are practicing clinicians like myself H and others are leading
Within your Healthcare organizations as you think and plan for the future H development within those organizations and as many of you who are already in the IE digital Community know our team are very passionate about providing you with access to the education new value that allows you to deliver better care
To your patients that enhances your career and improves your Global mobility and as you know we’ve a suite of over 12 postgraduate diploma and masters programs H with our leading Global Partners like University of wari world College of Physicians London southback University Irish College general practitioners and others to support you on that
Journey and also by joining any of our programs you become part of a family which I believe we are who share a common purpose and a common ambition H to improve patient care across all our communities in over 130 countries where we support you doctors nurses and others
Working in the healthcare field right now and embedded in each of our programs is a really sharp focus on Innovation and in this fast changing world particularly using and adopting the latest Technologies and tools and systems to help you in your daily work and today we’re talking about one of
Those which is artificial intelligence and whether you count yourself as an AI evangelists which some do and others Dooms dares one thing is clear we’re all at this critical inflection point in the evolution and Adoption of artificial intelligence with large language models like chat gbt Google’s Beed and the
Acceleration of many new products and services that will further transform Healthcare which are coming off the back of these these recent developments and we’re told it’s going to profoundly change our world and how we do everything and our lives will be better by technologists but they would
Say that wouldn’t they and I’d be interested to hear people’s views on that in Mark Anderson’s essay why AI will save the world and I will share a link with this and people can read it later he imagines a future where every child will have an AI tutor with infinite patience compassion and
Knowledge and every individual will have a similar assistant Mentor therapist trainer to maximize our output every scientist will have an AI collaborator to expand our ability to research and enhance our achievements and every doctor and nurse will have one of these two and there will be new Industries new
Jobs faster scientific breakthroughs and therapies and medicines we’ve all read about it in the news and he points to AI chatbooks Bots being already more empathic than their human counterparts and I know Mark will refer to this he believes rather than making the world harsher and more
Mechanistic he believes that we will be infinitely more sympathetic and patient and I wonder what our views are on that and in healthcare we’ve seen some of the existing deployments of AI over the last few years in medical imaging decision support Predictive Analytics virtual assistance drug Discovery teley medine
Robotics Etc so I suppose how do we prepare ourselves for what the opportunities are here and what are the risks what are the real life issues for us as practicing clinicians and those working in healthcare and how should we be thinking about these things and focusing on them Professor Mark suan is
A key faculty member at the University of wari and teaches as faculty on our Healthcare leadership Masters and Mark is a deep expert and passionate believer in human factors research and I’m delighted to hand you over to Mark we look forward to his thoughts and afterwards we’ll regroup for questions and answers thanks
Mark right thank you Tom and welcome everyone so my name is Mark sujan um I think it’s probably quite useful to give you a bit of insight into my background so that you understand how I look at AI because by background I’m a chared ergonomist so for those of you
Who don’t know what that is I study people at work I study how they work I study how we organize work and I try to inform the design of tools and Technology like AI but also the organization of work and the assurance of the safety and the well-being of
People at work and so already from that description you can see I’m neither clinician nor actually an AI expert um so I will talk about the the use more than the actual technological details of AI where actually you know many of you will have much more insight than
Myself um in terms of my day job I work part-time in the NHS at the Healthcare Safety investigation branch what we do is we investigate patient safety incidents um but not to blame people or blame technology or anyone for that matter but to identify systems-based lessons and learning for
The NHS in order to make the NHS a better and a safer place for staff and for patients also I’ve got so many different hats but one of my other hats is I’m uh leading a special interest group at the chared ins Institute of ergonomics and human factors where we have a community
Of factors around digital health care and artificial intelligence which is a really friendly and welcoming place where we debate current trends um and how we can help designers and users of AI technology and digital Technologies in healthcare just to design them in a better way better meaning here with a
View to staff well-being and and patient safety so that’s my my background and the take I will give you on AI in healthcare right so the I mean as Tom already said the you know the hype around AI these days couldn’t be any bigger I think um but also some of the
Fears um people are afraid that their jobs may be lost um that they will be replaced and then there more extension fears uh around Ai and it’s quite common now to start almost any presentation with a reference to chap GPT or any of the other uh competitors um and as many other people
Have done I just asked chat GPT you know tell me how to deliver a master class on AI and Healthcare it’s quite informative really and you know you can have a little play around what I will do is different from what chat chat gbt suggested but still quite quite an
Interesting tool to play around with but if you look on Twitter and elsewhere every day there are so many developments so it’s actually impossible to keep up with what’s going on um and so you know that really is one of the main reasons why people uh in regulatory roles but
Actually beyond that are somewhat concerned um about whether there there’s too much power in the development uh of these Technologies and whether we are actually understanding what these Technologies are about um and it’s quite natural that you know even even though chap GPT is not designed specifically for healthcare
That in healthare we start playing around with it of course we do it’s an interesting tool it can be potentially useful um it’s good for publications of course and so uh I’m on the editorial board of bmj uh Health and Care informatics and so we receive a lot of
Um papers now I just put one up there uh on the left which was recently published in the journal which looked at whether chat gbt could be used to to classify uh brain tumors and to recommend a treatment regimen um it was a small study and what the study found was
Actually the classification was quite poor but suggestions around treatment uh afterwards were encouraging now that’s not to say that we should be using the tool straight away what it does however hint at is the power that these tools have both on our imagination but then also for the application in
Healthcare um the New England Journal of Medicine just recently had uh or started a Series this year on AI in healthc care um and I picked up one one of the papers here but I make reference to others later which looked at the benefits and you know the limits and the risks of
Using chat gbt and other kind of chat Bots uh in healthcare and so you it’s well worth a read um I mean there were there were kind of things which were quite interesting around you know we could use it for example for medical note taking but there are peculiarities such as
Sometimes it imagines things which it didn’t even hear and so it integrates non-existent information into into notes it’s quite good as a Q&A kind of for checking medical background um but actually again when you want to use it in consultation it can come up with things uh which sound highly plausible
But are actually wrong and that is then very hard to identify and so you know people are highlighting both the benefits but also the the risks which are as yet probably un underexplored and this is this is another classic example of a phenomenon uh associated with chap GPT and another BS which is
Referred to as hallucination and so depending on how you talk to it or how you write to it um it comes up with sometimes very surprising um things which are just not true so it makes up stuff basically in this instance uh you know it was asked
How do you know so much about I think it was diabetes management and it said well I’ve got um a master’s degree or something like that and I study a lot whatever um which obviously you know is complete rubbish but even more interesting you can then feed that back
Into chat jpt and ask it to give its opinion on what it’s heard and then it will identify actually this is nonsense an AI chat Bo doesn’t have master’s degrees so there are really peculiar patterns of behavior and interaction emerging here and we are just really at the beginning of seeing these
Understanding these and knowing what they mean if they were to be used in clinical practice and you know that that kind of level of concern and the speed of development and the lack of Regulation and understanding of risks and lack of societal consensus I suppose around these issues has
Prompted uh you know the um the future Life Institute I think it’s called and the number well the significant number of uh researchers and celebrities to call for a pause in the development of large scale uh AI research until or deployment until we have really understood a bit more the
Risks and how to manage these probably futile but what it does do is it kind of fuels a bit you know a larger societal and and political debate about these issues so as I mentioned I’m I’m really no expert in AI Technologies but it’s probably quite useful to have a couple
Of slides about um what we mean when we talk about AI uh and in this instance machine learning specifically just to understand why we might be somewhat concerned or what the peculiar things we need to look out for are and so in terms of machine learning
There are three major kind of um ways of getting the AI to do what we want it to do one is uh So-Cal supervised learning so in essence you Pro you present lab data sets to uh the AI tool usually a neural network and then um you have an
Algorithm which calculates an error function and it tries to reduce the error function and so it’s in essence a classifier based on previously labeled data and so the labeling of data is really important but then there are unsupervised uh learning algorithms where in essence you just present uh
Lots of data to the AI and you ask to identify patterns which are meaningful to the AI and it might identify patterns which uh are not meaningful to people or which were unanticipated because it is unsupervised so you leave it to its own devices and the third class is the
Socalled reinforcement learning where you specify a reward function and depending on what the AI does it can try to maximize or optimize its reward so if it does something which moves it further away from its goal it would get penalized If It Moves closer to the ultimate goal it would be rewarded and
It self-organizes in this way so the key really here um as simplistically as I put this really is that machine learning is datadriven so it learns from the data we pick the data we choose the data we pick the learning algorithms but we do not hard code or program what the actual
Algorithm then in its execution looks like so the machine learning algorithm will learn self self-organized in way based on the data which it sees and largely um you know machine learning these days is built on artificial newal networks and so you know in in the way in the way they were
Developed they try to mimic or simulate or take inspiration from the human brain so largely about neurons and the interactions uh between the neurons but so my knowledge of AI is back in the 1980s I I did the Masters in artificial intelligence but that was in the 80s and
90s we used um one one so-called hidden layer and a standard back propagation algorithm that is where I kind of stopped in my development but of course these days we have and you can see that here on the r socaled de learning uh neural networks which have loads and
Loads and loads of hidden layers which do different things and are much more sophisticated and that is really why we are now seeing these significant advances but bearing in mind you know we are seeing these significant advances largely because of the advances in computing power and in you know memory
And and storage availability and in the cost which has gone down significantly and so you can see it here um you know the really incredible development uh in Hardware uh which underpins the uh the recent advances and the breakthroughs which we are seeing in artificial intelligence which are really
Enabling in many ways to feed uh you know the AI tools with all the huge amounts of data which they need and so you know still these images are from the New England Journal of Medicine uh from that special issue series um so applications in healthare
I’ve got a few examples uh in a moment but you know just looking at how people have used it maybe the you know the most successful and the most prevalent application examples are around image analysis so analysis of X-rays and CTS and so on and so for that plays into the
Strength of machine learning which can classify things quite well so these are probably the most mature applications in during co uh Public Health examples became quite interesting um a company whose name now I forgot was credited with identifying uh Co Forest based on analysis of social media uh traffic and
Communication so the AI picked up on recurrent themes around chess problems and cuffs and so on and so forth in the community and was then ID able to identify hotpots um which is you know quite interesting use of AI then there is also of course um less contentious if
You want to put it like that use of AI back room use of AI for scheduling and logistical purposes um which of course holds a lot of promise but you know the the breadth of AI application is endless um and of course you CI applications in all areas of Health and Social
Care so I promised a few a few examples one of the really early studies uh of deep learning were Google Health’s mography study and I’ve put that on the slide because it was a very good one in terms of its scientific approach and I’m not commenting on the tool at all but
The scientific approach was good because it Illustrated something quite forcefully even though the research just didn’t highlighted it to the extent that it merited in my view but so what they did is they uh trained um their tool um to identify um areas of concern from from mammograms based on a large
Data set procured from the UK as well as a sim a smaller much smaller data set procured from the United States and so when they trained the algorithm on the UK data set tested it on on UK data it performed really well once they started testing it on the American data set the
Performance dropped significantly and that is ever so important because we need to bear in mind that the AI learns based on the data which you feed it if the data is representative of one context but not representative of another context we are in serious trouble and I’m mentioning
That for the reason which is largely around um well health inequalities I would say you know if you think about it a big promise uh which AI gives us is to support low-income countries um where there are you know shortages of trained technicians and so on and so forth and
We’re saying actually AI could make a significant difference in these countries but where does the data come from you usually these algorithms are trained and developed in wealthy countries representative maybe of a more white population but also not just population C istics also technology characteristics technology where these
Images are are taken um and so the operating context in the deploying countries if we’re talking about lowincome countries might be very different and the risks might be quite significant in the UK we’ve got um Chiron medal um who have the mography intelligent assistant which um um you
Know I mean I I kind of had the the pleasure of working with them on a on a kind of side project but um so I learned a bit about their approach and you know it was really quite interesting um both their kind of aspiration as well as the
Approach they’re taken in terms of evaluating the technology having clinical trials now in a number of European countries but so they make the point that in the UK as well as elsewhere we have a significant shortfall in our diagnostic Workforce capacity and so the current practice of
Double readers uh of a mamogram for example so independent double reading is just not sustainable and so what they’re proposing is well if we can substitute one reader with an AI tool so we can have a human reader and then independently the AI reader um we can significantly reduce the diagnostic
Backlog and that would be ever so important um because we had one not just because of Co but before Co but Co has made it even worse and so people need to wait way too long to have their investigations done AI could make a significant difference well it’s not just clinicians
Uh you know who can use AI um of course uh patients well everyone really uh can have it on their smartphone I’ve um picked up this this little tool here not because I think it’s particularly impressive representative whatever just because I attended the present presentation where
Someone gave away a free code for it so I downloaded it and tested it well the reason I’ve got it here is actually it’s quite cheeky in a way so this is for detection of skin cancer and I kind of had a little spot on my arm so I took a
Photo and then it says yeah you don’t need to worry about it but what I want to highlight is more you know in order to get this done I’m not sure whether you can see it on the slide it’s quite small for me here but you need to sign
Over your data for not further specified research and health purposes so actually what I’m thinking is you know we need we need a bit more dialogue about what happens with our data when we when we feed it into these applications you know how transparent are the developers and you
Know why why are they forcing us to consent to this um you know why why can’t we use the tool without consenting to our data being used for whatever purpose they feel fit so you know I think there is a lot of work still to be done you know around privacy and you
Know in any case how health data is used so but you know um machine learning AI is is of course already here um so ideally we would go down the rout of having it uh regulated and certified as a medical device and there were a couple of papers looking at
The number of devices using machine learning on the market now an earlier one looked at number of devices uh having certification between 2015 and 2020 but actually more recently I think it was published not too long ago um this year at least so they found 8 unique machine learning based medical
Devices already on the market um so you know that will grow exponentially um and so you know that is already with us and it’s probably fair to say The Regulators are really having trouble to to catch up here with the developments and so we need to kind of
Almost take a step back and think about the hype uh and the reality of AI in healthcare the hype is always in the news and of course technology developers need to push for their tools they need to justify the funding and so on and so
Forth and often what we see is kind of the opposition in a way of you know people and AI where the headline is doctors are being outperformed by AI or put it the other way around AI outperformance doctors AI is better at people at doing XY and and Zed which is
Very narrow and restrictive in a way and actually when we look at the evidence the scientific evidence to support these claims it’s quite weak it’s quite patchy um there’s nothing to justify you know these claims really and one of the big problems is we have very very few
Prospective studies so all of these studies which make these claims tend to be retrospective studies so they will look at nicely procured data sets in isolation but not in a clinical setting and that is what is really missing here and so that phenomenon has really been been captured quite nicely uh by
What is referred to as the The Last Mile in a way so a lot of focus on you know data uh but then you know also building the model testing the model on data but retrospectively and then where it all falls down is once we start integrating
It into the mes messy world of real life health care so that is really where our Focus needs to be from the outset really you know it’s understandable that technology developers have a focus on the technology but there must be someone somewhere or the agency that sets it out
Quite clearly you need to have the implementation at the Forefront of your thinking right from the outset so that is where we come in uh kind of um you know being systems thinkers let’s say within our profession so um at the charted Institute of organ human factors I’m kind of shamelessly
Making adward for the membership body would be published white paper uh on human factors and ergonomics and healthare Ai and what we are saying there quite quite simply is focus so far has been on the technology it was all about the AI itself about data quality about the accuracy of the algorithms
Whether there’s any bias in the data all important kind of attributes but actually the bigger picture is really equally or more important and so we arguing for a systems perspective and so you might you know ask what on Earth you mean by a systems perspective really a systems perspective
Just means AI is simply a tool it is not an end in itself and so you know there will be people who have to interact with the AI so we need to understand people and we need to understand how people interact with technology and AI specifically but the AI as well as
People might be interacting with other tools so we need to you know understand how does the AI interface and integrate into our existing it infrastructure um people as well as AI will never work in isolation clinical systems are full of other people other purposes and so we need to understand
What does it mean for teamwork for relationships among people and for the organization of work really and then let’s not forget the patients of course you know what are the what’s what is the impact on P patients and on actual patient care AI will not deliver patient
Care as such and definitely not in isolation and so we need to really understand what happens when we start to introduce AI into a messy busy complex clinical system and so uh you know one one of the project I was working on was um with the Welsh Ambulance Service um who were
Approached to to uh well test and ideally purchase um an AI tool which can support call handlers in the recognition of cardiac rest calls so cardiac rest calls are frequently being missed um and of course every minute of delay uh in getting help to the person in kardak
Arrest will reduce your chances of survival significantly so you know there’s justification for such a tool now such a tool has been developed it has been tested retrospectively so the tool can listen in in the conversation and then can identify uh whether or not this is a cardiac rest call so the retrospective
Study and I apologize is probably again too small to to see the detail um the retrospective study found that actually the AI tool was better than the human call handlers at identifying card rest calls and it was quicker as well so that’s a great starting point but as I
Was saying retrospective evaluation once the same tool was then implemented into an urban call center I believe it was in Copenhagen and tested prospectively uh you know what the researchers found was actually the joint system of call Handler supported by AI did not perform any better than the than
The call handlers just by themselves right so this is soing um the evaluation did not go into the detail of why that happened they kind of speculated but it didn’t go into actually collecting evidence around that but that is precisely what we need we need to know
What happens when you integrate AI in the technical system and more importantly we need to already design it with the tech with the source technical system in mind so I something which apparently was Noel or you know kind of unexpected um I asked call handlers what makes the recognition of cardiac
Rest calls difficult you would have thought that this would be the first thing that people do but no the first thing which people do is let’s have the data let’s put it into the AI and let’s check whether the AI is better than people so there was uh interviewing call
Handlers what makes it difficult quite interestingly they said actually a card arrest call is among the easiest calls which we can get so okay but how do we then explain that so many calls are missed well so they were saying actually there are a few things first of all CEST
Is really kind of challenging situation for the caller to be in they might be agitated they might be in denial and when you ask them is the patient conscious are they breathing you might not get an honest answer you will you you will get wishful thinking as an answer and so
It relies a lot on your personal interpersonal skill to get the right answer to get to the truth of what’s happening but then there are things like poor phone connection um where you know the caller might be on a mobile phone and the phone connection keeps dropping
Out um then there are situations where uh the caller is the patient themselves and they have kind of slurred speech or something really hard to understand and so you know they highlighted these kind of challenges to what otherwise would be a fairly simple decision which they can
Make and so that prompted me then to think you know what’s the underlying design metaphor here so technology developers have substitution as their metaphor let’s get the AI to do what the people do not to replace people but you know it’s the design metaphor let’s let’s you know let them recognize
Cardiac Arrest let them do what what people do but actually talking to people if you wanted to support people maybe they need something quite different maybe they don’t need anything as know that would generate the sexy headlines of AI outperformance people maybe they need something to make slurge speech
More intelligible and these tools can be developed for sure maybe they need something to enhance uh you know phone phone audio quality to support the technology Developers not as exciting not as sexy but I can’t say but I can maybe bet on it maybe more successful than just having the standard substitution design
Metaphor the other thing which we found is organization Readiness or lack of organizational Readiness um so organizational Readiness refers to whether an organization is willing to adopt something a change a technology or machine learning in this instance and whether they have the ability to do so welge Ambulance Service um definitely
Very willing super willing super Keen to adopt the technology and I hope I don’t offend anyone if there’s a member of West on on the call but probably not yet so able um and so there were lots of things kind of you know an understanding
Of what AI is really about uh what it can do what it can do what questions to ask of the developer um what organizational mechanisms structures do we need to have in place to procure data for example how do we even share data with a developer where is data going to
Be hosted can we share data do we have a legal basis for sharing audio data of call Handler or of calls coming into the call center these kind of things no one no one had any answers to that um in addition the safety culture um was very
Individual focused and you know if you think about it the AI is supposed to be a support tool and the developer was very quick to say you know it won’t replace people but you know you need a lot of psychological safety to say actually I’m going against the AI
Recommendation if you are living in a culture which tends to blame the individual so in the interest of time I’ll just move a bit further one thing which um which I found really interesting and it’s often neglected is the impact uh on patients and so I interviewed um patients who
Were previously on intensive care about their attitudes and perceptions um around the use of artificial intelligence in in their Care by and large you know everybody was very positive and they said yeah of course you know if the NES is using it then it’s going to be great if it helps me
It’s going to be great and you know positive thought regard of age really young young and old patients but what struck me was something which they were saying you know but I don’t want kind of to lose the uh the personal contact with the nurses um you know we have such a
Special close Bond it’s such a traumatic experience in intensive care especially and so you rely on that human connection on that human touch and you can say from a technology devous perspective of course you know the AI won’t impact that no the a I will free up nurses to spend
More time at the bedside that’s great but you can also see the business perspective you know the finance department saying hang on we now have this fancy new AI technology do we really need nurses kind of you know uh on a onetoone basis patients or now that
They have the technology can they have two patients maybe or three or maybe in five in in a couple of years five 10 patients so what What patients and I don’t want to see is that nurses become almost carers for AI technology rather than caring for patients that impact on the personal
Relationship is super super important so um what can we do um but the answer is first of all we need to of course think very carefully about what’s going on here but we need support and so the uh the British standardization Institute with BSI um just recently concluded development of a new standard
Um which proposes an auditable validation framework for healthcare AI what it really tries to do is distill all the diverse set of guidance and standards and recommendations into one simple auditable framework it won’t do everything for for everyone but at least it brings together a lot of the
Information in one place and you can audit yourself against that so that will be published fairly soon the paper explaining the bashal has already been published and so you can see for example you know I I managed to get human factors ergonomics in there and so there are a few kind of
Sub Clauses which you then need to demonstrate that you are considering around usability around understanding the user requirements uh for training for example how you manage the potential for automation bias and so on and support and so that standard is something which organizations should demand in my opinion of Technology developers who
Come and approach them and say actually would you like to purchase our uh smart new II tool um in the Future Organization should ask well uh are you being compliant with BS 30440 and more widely and just concluding this is my last slide with the New England Journal of Medicine
Again um quite nicely um you know kind of suggested uh kind of safeguards for AI development and use so rather than just looking retrospectively you know is the AI or the machine learning is it accurate on the data set um we really want to promote real world evaluation um
Which also means from the outset understanding the clinical system within which it will be operated we need urgently uh useful post- deployment monitoring there are things which we unclear about such as you know how do we collect data um who owns that data uh which we need to to look at does it
Belong to the patients does it belong to the healthcare organization does it belong to the manufacturer we don’t know um issues about transparency in the development and the reporting um again things that are Curr being debated but for which there are not yet um proper Solutions so you know that is something to
Consider and so just a shout out again to the charted Institute um it’s just bit of self-promotion but I think the white paper which is free of charge is useful I think for the community because it helps us to think a bit about the systems and maybe helps us to ask the
Right questions right so thank you very much and Tom back to you I stop Mark you’ve given some you know a really good and balanced view of some of the issues that are facing all of us as healthc care practitioners and there are some themes coming up in the questions really around
Bias practical applications governance accountability equity and I think they’re all areas you’ve touched on and are all areas we could spend lots of time on I have one specific question just to start the questions um many computer scientists focing focusing on AI ethics have really pointed out the
Near-term harm that could come from perpetuation of biases and the potential for misinformation that seems to be what we’re seeing a lot in the media what’s your view of that Mark I was muted yes no I I I completely um agree um I would say a number of
Things so first of all there are lots of examples where that is already the case um and so for example I mentioned you know where data sets come from um and how AI tools which have been traded on trained on certain data sets do not perform as well for other uh population groups
Um and so you know that that has been documented there’s an interesting book which um people might find find useful to go to it’s called weapons of mass destruction um which I I quite enjoyed reading um yes and so I think um ethics is important it is not necessarily my
Area of expertise but lots of people are already working on that and so the Alan churing Institute for example in the UK have uh published a report the European Union has guidance on this so there is a lot of active discussion and debate and awareness uh around that
Issue yeah I saw a very good description of it said AI you should treat it like a child that it needs a mom and a dad to supervise it at all times and I think you know it’s just a very simple but a good description some of the other questions
Coming in um one question should should the wealth of anonymized NHS NHS patient data sold to large tech companies like Google for research with AI I know I know you’ll have a view on that well that is that is a a really really challenging question to answer what I
Can say is Google got itself into trouble um so Google Deep Mind um they were developing an AI tool with the Royal free I was I believe if I’m not mistaken and so the Royal free shared a lot of patient data for the development of AI and testing and Google and the
Royal free um both contested that you know in order to develop safe AI we need to have actual data um but there is no legal basis for the sharing of patient data for the purpose of development and testing and so again I am not an information governance specialist but I
Can probably say from previous experience there is a massive hole currently in information governance practices as as far as data sharing for AI are concerned and so yeah so there’s gdpr so the European uh act um and so you know as well in in the UK we’ve got the the common law of
Confidentiality or whatever it is called again I’m not I’m not an expert on that but so you need to establish the legal basis and that is far from from simple I’ll give you the example of the ambulance service the ambulance service initially thought well we’re just de
Identifying the uh the calls and then we can share it but no it’s not so simple because even audio data which has been deidentified carries your voice and when it carries your voice it can carry indicators of your ethnic origin of your Social cultural background these are actually classed as
Identifiers yeah so anonymization is super super tricky very tricky and in truth the reality is Apple and Google already have your data um would be my view um so there’s a good follow-up question to this do you think um who who do you think should be at the table when laws
Start to be put forward for AI to protect patients people potential for Health Care Systems well a range of stakeholders but I would just focus on one often forgotten the public yeah and so what I mean by that is so there’s a concept which is gaining traction uh and it’s
About responsible uh research and Innovation and what it really says is that regulation first of all needs to be much more based on a societal dialogue um we need to involve people uh you know in the discussion we need to make these discussions available to people we need
To be responsive to their concerns um and we cannot consider that we are able especially with AI and machine learning to have one set of regulations that we just fit the bill and you know we’re good to go it needs to be a continuous cycle of reflection really and so
Everybody needs to buy into this Regulators as well as technology developers need to be more responsive uh and more responsible as well yeah and I I think also what we’re seeing as you dis intermediation of Health Care you know and of the healthare system that a providers are are kind of going around
Uh regulation and Truth um and being able to to access your data another question here junior doctor how can I learn how to implement AI in my daily job I mean it’s a very good question I think it’s kind of we’d all be struggling with how how is this really
Going to be implemented in my daily job or how active have I to be or do I do I wait for it to be implemented by others around me yeah no I mean that’s a good question so as I was saying I’m not a clinician so I can’t really say um yeah
What I would say is you know take the examples of chap GPT and and other kind of things as reminders of the dangers and the risks that that might be there so that’s not to say you know never never use it never try it but be mindful
Of the pitfalls and if I just extend that one step further I think organizations uh need to encourage AI literacy um among their staff among among themselves organiz ganizations as well as deploying clinicians or using clinicians need to be able to ask the right questions and what I mean by that
Is if you think about an infusion pump for example in the olden days you were told you know which button to press and how to operate an infusion pump and that was it and you knew whether it was working or not with machine learning it is very difficult because it’s a context
Dependent you know you need to be told what are the boundaries of views of the ml tool where where can it perform and where is it not safe to perform anymore and actually that paper which I showed previously about um you know 500 medical devices using ml have already been
Certified they looked at the reported incidents a significant um proportion of incidents leading to Patient harm were because clinicians misinterpreted the design boundaries of the ml this this is not to blame the the clinicians at all what I want to say is the developers and the deploying
Organizations need to ensure that we as users understand the design boundaries and that these design boundaries are very clear to us yeah there are a number of questions marked too around Radiology um the future of radiology I suppose it’s been interesting it’s been an area where AI
Has already developed fairly sign um one of the questions is really around um who’s who’s responsible responsibility and if if if something’s misinterpreted um who’s accountable and and who for a discrepancy yes well so let’s start with accountability as as a word and a concept you will have a legal
Perspective and I I I really cannot comment on the legal perspective but I I mentioned so you know I work at the Healthcare Safety investigation Branch where we investigate incidents and we will of course in the future investigate incidents involving machine learning now accountability in in our kind of
Interpretation is not about who do we blame or who gets the sack it is more about who can give an account of what happened with whom do we need to talk to understand what happened why did it make sense to you know act in particular way
In the situation so that lessons for the for the wider system can be identified so you know that is probably not as simple an answer as uh you know the question might might be getting at but you know it is a difficult um undertaking but I think we need to
Distinguish between the legal interpretation and then you know the safety investigation for the purpose of learning yeah yeah just maybe we’ll start to wrap up Mark but one one last question if you were to look at the whole landscape now as somebody who’s been involved in human
Factors if you were to pick your top three areas where you feel that AI will have the biggest impact in the next five years next decade what would you think they are H that is a a good question and it’s probably even beyond my imagination and hence I will conclude on the three
Things which I think we definitely need to generate that impact and I think first of all I alluded to it organizational Readiness we need to support organizations to understand their organizational Readiness and identify the gaps we need to build an Education and Training infrastructure we used to have health education England
They have been merged into enes England we used to have NS digital they have been used into NS England who is going to provide that train training at hsib we try here within this forum we try but we need that training infrastructure and lastly we need a systems approach people
Need to be aware of what is the systems approach and that needs to be integrated from the outset and we need to insist on it yeah it’s a it’s a very good place to finish I think ultimately if we don’t H as doctors and healthc Care Professionals and people who work within
The sector and who run and manage Healthcare infrastructure if we don’t get involved in the discussion and take a leadership position then what happens is technologists Implement Solutions upon us that don’t really work for us or for our patients so with that thanks million mark on everyone’s behalf I’ll
Thank you most sincerely for H today and hand you over to deid and thank everybody for joining Mark uh to to Echo Tom’s thanks that was an absolutely fascinating discussion and I think looking at the looking at the questions that have been coming in that there there’s very real concerns over how AI
Is going to be governed regulated and what what it practically means for people but also that question of accountability and that question of how do you govern something so powerful and you know we’re we’re at the very start of this journey is what it feels like to
To a lot of the uh the people on this call um I suppose I’d I I’d finish on a note that this this type of debate and discussion is the type of debate and discussion at the heart of our programs and an awful lot of people on the call today
Are are here because you you have expressed an interest in or have been a a student honor and alumni of one of our programs so what we want to do very very briefly is we we’ve launched a poll if you are interested in hearing about any
Of the specific programs so Mark is a Cher on our masters in healthcare leadership we have a portfolio of programs across all of the main kind career enhancing areas of uh health care so areas like education clinical research um Public Health leadership Etc and then a range of very clinically
Specific courses um these are all wholly online courses they are generally interprofessional we’re very strong Advocates of that the The Faculty on these programs uh are as you’ve seen from Mark absolutely worldclass and um you know the insight and information and confidence is uh is what we get um as
Feedback from our alumni and the I think the the ability of Education to Spur the imagination and make people think differently irrespective of the particular program they on is is huge um we you know IED we are absolutely proud to be one of the world’s largest providers of postgraduate Healthcare
Education and we’re extraordinarily proud of the impact that our alumni are having within their health system so fantastic to see uh so many people filling out that poll so quickly um so our team of Education advisers would be delighted to talk to anyone about the programs any of the information that you
Want uh you’ll get on IH heed. org and I’ll just Tom if I hand over to you for the last uh thanks to everyone but absolutely amazing uh session today just to thank everybody and if you feel that there are areas for future program development that you’d like us to focus
On we’d love to hear from you or if you’d like to join us uh as faculty or supporters of the organization in any way H we’d love to hear from you please reach out to us and thanks everybody for joining and have a great weekend thank
You very much bye thank you so much everyone thank you goodbye all