• DDPS Talk date: January 19, 2024
• Speaker: Jan Christoph (UC San Francisco, https://cardiacvision.ucsf.edu/people-jan-christoph)
• Description: Heart rhythm disorders, such as atrial or ventricular fibrillation, are caused by abnormal electrophysiological wave activity which propagates through the heart muscle and induces irregular contractions. Heart cells are electrically excitable, and they contract upon excitation. They can be described as nonlinear oscillators and the wave activity correspond to nonlinear waves of electrical excitation, which can produce complicated spatio-temporal patterns known to emerge in reaction-diffusion systems. Presently, this abnormal wave activity is still incompletely understood, as it lacks imaging technology that can penetrate the heart muscle tissue and resolve the three-dimensional electrophysiological dynamics within the heart walls. Consequently, diagnostics are limited, and it remains challenging to develop better therapies. In this talk, I will discuss how artificial intelligence (AI) can be used in combination with high-speed fluorescence and ultrasound imaging to overcome these limitations. I will show how it is possible to compute or predict the heart’s electrical dynamics from its mechanical deformation, and I will demonstrate how we train AI ex vivo with isolated hearts in order to be able to make the predictions.
• Bio: Jan Christoph is an assistant professor at Cardiology and Bioengineering in University of California, San Francisco. He also serves as head of Cardiac Vision Laboratory. Jan obtained PhD in Biophysics at University of Gottingen, Germany in 2014. He did his a postdoctoral study at Max Planck Institute for Dynamics and Self-Organization from 2014 to 2018 and served as a staff scientist at Department of Cardiology in University Medical Center Gottingen, Germany before he come to UCSF. He has earned NIH New Investigator Award DP2 in 2022.
DDPS webinar: https://www.librom.net/ddps.html
💻 LLNL News: https://www.llnl.gov/news
📲 Instagram: https://www.instagram.com/livermore_lab/
🤳 Facebook: https://www.facebook.com/livermore.lab
🐤 Twitter: https://twitter.com/Livermore_Lab
About LLNL: Lawrence Livermore National Laboratory has a mission of strengthening the United States’ security through development and application of world-class science and technology to: 1) enhance the nation’s defense, 2) reduce the global threat from terrorism and weapons of mass destruction, and 3) respond with vision, quality, integrity and technical excellence to scientific issues of national importance. Learn more about LLNL: https://www.llnl.gov/.
IM release number is: LLNL-VIDEO-859499
All right. Welcome everyone to the DDPS seminar. Before we introduce our invited speaker, let’s go over some rules and logistics. First, please mute yourself during the talk. But if you have questions, you’re welcome to unmute and ask questions. Or
You’d have to post your questions so that we can address them in q&a session at the end. Second, today’s GDPR seminar is open to external audience. Therefore, no classified discussion is allowed. Finally, the talk today will be recorded and uploaded in our YouTube channel. Okay,
Let me briefly introduce our speaker today. It is an honor to host Yan Christopher, who is an assistant professor at cardio cardiology and bioengineering in University of California, San Francisco, please close by from Livermore where I am. He also serves as head of cardiac
Vision laboratory in UCSF, and then obtained a PhD in biophysics at University of Goettingen. In Germany in 2014. He did his a postdoctoral study at Max Planck Institute for Dynamics and self organizations from 2014 to 2018, and served as a staff scientist at Department of
Cardio Cardiology in University of Medical Center going on in Germany, or before he comes to came to UCSF. He has on NIH new investigator awards and TP two in 2022. Okay, today, Yan will talk about predicting heart rhythm disorders from spatial temporal imaging data
Using artificial intelligence. Please enjoy and expect a wonderful talk. Now, let me ask a couple of questions. If you don’t mind me, and just to get to know you better and then make this seminar somewhat informal for first question is, obviously you are passionate about
Hurt or medical field in general, is is there any moment in your life when you realize that? Wow, this is it? This is the field I would like to commit my career to do you have that money? So the question was, when was that moment when I want
It? Yeah, yeah. Yeah. And if you can describe that moment? Yeah. Yeah, I think, um, one thing that I need to say and before is that I never wanted to become a scientist, it was not not a lifelong dream of mine. When I was younger, I never imagined to
Be a professor or scientists later on in my life. I think I did a PhD out of curiosity, because I wanted to spend time on on working on quantitative biology, I think there was my interest somehow. And then spending six or seven years, almost eight years on this
One important publication that defined my career that came out in 2018. In nature, even opened up, I think the way for me to even consider becoming a professor. So it was a pretty late decision. So I was already way into my postdoc and published an impactful paper and then realized, oh, wow,
I can actually do this. If I want to, I can, I can try and get a tenure track faculty position and, and keep doing this for the rest of my life. Maybe even. And I think before that, I didn’t have
Necessarily the plan to do that. I see. It was a thing. At that point, I was definitely hooked. I was really intrigued by this type of research that has had built up over the years. And then,
You know, in that moment, it became clear. Okay, I want to do that. That was roughly in 2018. Wow. Well, I didn’t know at some point in your presentation today. Maybe you can share. What was that? Paper? Yeah, I will have one slide about that paper. And it’s awesome. It is
Kind of the foundation for for the last, the second half of my talk. This paper I see as well. Okay, the I have one more question. If you don’t mind. Obviously, you’re using artificial intelligence for your research. When were you first exposed to artificial intelligence and decided to use that in your research?
So I think the first time I heard about Convolutional Neural Networks was in the cafeteria at Max Planck in 2012. And 13 You know, when Alex net and all that came out and people talked about it, but for many more years, I didn’t get my fingers on it. Yeah. It was actually only
During the pandemic, when our Institute’s shut down, when I sat at home. And my student and I, we both kept talking about how we wanted to progress and make it or not improve certain computational methods that we have developed and talked about auto encoders in unit. And,
You know, I just tried it out and it worked right away. And it was amazing. And I will have in my talk, I will have a few slides about exactly that. Awesome. Awesome. Thank you. Thank you, and I’ll state is yours. Okay,
Thank you, Sue, for the invitation and the introduction. Again, my name is John Kristof. I work at the Cardiovascular Research Institute. I’m appointed in the Division of Cardiology here at UCSF. My background is in physics, I obtained a PhD in physics. And let me briefly introduce my
Lab. And you will notice here that most people in my lab also have a quantitative background, either computer science, physics, or electrical engineering. And the pictures here are kind of weighted by how many contributions are how important these lab members are, or were
Over the past years. And you can notice here young labor is a central figure in my lab. He was also a PhD student and getting an A in Germany and came down with me to set up the lab here at UCSF. And
A lot of the work that I will talk about today was actually done by him. Then we were also able to recruit Trey last year, who has a background in physics and computer science worked as a software engineer and he is interested in computer vision. So he has built important numerical methods for
Us. And then on the right side, Daniel, he is now at Caltech, he was an undergraduate student who made very important work last year together with Yun, and on the bottom left here tarnish who has an interest in you know, everything deep learning and in particular diffusion modeling and
He he did the diffusion modeling work that I will also show so, what what can we do as physicists, mathematicians, or engineers in cardiology, and if you are interested in this topic a bit further, I can recommend this podcast that came out recently in quantum magazine that’s that’s a really nice
Magazine I I often read, and Steven Strogatz, who’s an applied mathematician from Cornell, he interviewed Flavio Fenn was a colleague of mine from Georgia Tech, and he spent many years leads a group there, and they research questions such as How can we predict what the heart does next?
And how can we intervene and apply defibrillation pulses in order to stop heart rhythm disorders, and they talk about that at length. And we do very similar research with a slightly different focus, we try to focus on imaging itself. And we try to understand the interplay between
Mechanics and electrophysiology in the heart. So communities is small, but very persistent and very active. And very interdisciplinary. We have, you know, cardiologists, with an interest in machine learning, we also have a background in engineering. We have mathematicians, fight mathematicians, engineers, physicists, and they all have their different views and approaches.
But we together we meet quite frequently, at least once a year, this was, for instance, a meeting in 2018, which also I think helped me to make this decision that I want to stay in academia and that
I want to give it a try to keep pursuing these these projects. And this this research topics that I’m currently working on. It was at the PRDP at Santa Barbara. And I would say that these people like us who have a quantitative background, are very small group in this huge arena of
Cardiologist and biologists and physiologists who research heart rhythm disorders or heart disease in general. And there’s roughly an even distribution between you know, mathematicians, physicists, engineering computer scientists how to how to say who dominates this field, say, last year, we organized a meeting or mini symposium at Siam conference, and this year again,
We will meet at the same conference. And this is also something that we we often participate in or organize. And the people who come there are regular people who who’ve done have come to these meetings since many, many years they keep showing up. So the heart is an electromechanical
Pump. That is something that I need to introduce first, the most important property of cardiac muscle cells and you see here one on the left side is that they are electrically excitable they are the transmembrane potential undergo was variations over time that serves that that’s a
Way of communicating for these cells. And these changes are referred to as action potentials. You see there on the right side, three curves. The first one, the green one is the action potential. As soon as cardiomyocyte experiences such an action potential, it opens intracellular storage
For calcium and the calcium is then released into the cell and the calcium kind of fuels the contraction of the cell. So the contraction follows the action potentials is elicited by the action potential. Then, this electric excitation overall on the global scale, the heart triggers
And orchestrates the heartbeat, we have in our heart, multiple centers, the sinoatrial node and backup node. And in there are cells that beat autonomously, roughly one time per second, and then they shoot out electrical excitation through a cable, and then through through a branch
Of other cables into the main heart muscle tissue that is shown here. And this heart muscle tissue consists of roughly 5 billion cardiomyocytes. And once this electrical excitation reaches the end of these so called purkinje fibers, they exit at these sites excite the first cardiomyocyte
Or small group of cardiomyocytes, and they pass on this excitation to the neighboring cells. And so via diffusive process, this excitation then is propagated through the entire cardiac muscle and this excitation propagates through the muscle tissue at a speed of one meter per second. So
It’s relatively fast. Then atrial fibrillation and ventricular fibrillation are the two most severe forms of heart rhythm disorders. There are many different flavors of heart rhythm disorders, but those are the most prominent ones, because they either affect many, many people or they are
Highly deadly, so atrial fibrillation affects the pre chambers of the heart, the atria, that can be a chronic condition that you can have for many weeks or months before you have your elective procedure to get rid of atrial fibrillation. And the atrial fibrillation itself is very problematic
Because it can cause heart failure or induced ventricular fibrillation, which then affects the main chambers and is immediately life threatening, however, then trigger fibrillation can also occur on its own. Without atrial fibrillation, it can occur spontaneously. And you often have,
Sadly, these reports here that you can do, I can update the slide every three, four months, another celebrity dies from sudden cardiac death, it is very common, and it affects really, people at all ages, can sometimes affect people who do sports, especially in professional sports.
At a very young age, it can affect people with genetic defects at a young age. But, of course, it becomes more and more prevalent, the older you become. Physicists describe these electrical wave phenomena with partial differential equations. And we’ll have a few slides about
That. And here are some reviews about that and which nicely summarize the work that has been done in our field. In short, you can see here an electrocardiogram at the top and that describes a sudden onset of ventricular fibrillation, so the most severe, most deadly form of heart rhythm
Disorder. On the left side, you can see the QRS complex here, which corresponds to a heartbeat, the C corresponds to the atria contracting this to the ventricles contracting and GC are three heartbeats. Before all of a sudden, out of nowhere, this ventricular fibrillation sets on.
And we associate with this episode here three computer simulations that describe the spatial temporal dynamics of the electrical waves that determine the heartbeat doing these three stages here. So the normal heart rhythm is caused by a plane wave that really originates down there with
The Purkinje fibers inject the excitation into the heart muscle, then this wave propagates upwards, and needs to be re initiated by the sinus node. And here are what happens here is that this wave becomes perturbed and undergoes a transition into this spiraling or rotating state and we refer to
This wave of form here as a square wave scroll because it’s it’s curled up like that. And this square wave can even further degenerate into spatial temporal chaos, which is only on the right side. And these computer simulations are supposed to illustrate what we think
Happens inside of the heart. muscle during these different stages, we can model cardiac cells in that sense as relaxation oscillators or nonlinear oscillators. And you see here an example of a quite old mathematical model for such a relaxation oscillator that Fitzhugh Nagumo model and in in
Such a model you have two equations one stands for the excitation one for the refractoriness and the kind of compete with each other whenever the excitation goes up the refer to nets also go excitation and in phase space you then have this behavior here that most of the time the
Cell is in the fixed point, but if you perturb it slightly if the cell receives excitation from one of the neighboring cells, then you perturb this oscillator here over this separatrix and you see an excursion through face space, which corresponds over time to an action potential is a mathematical
Modeling of of the electrical activity can in principle be divided in either this synthetic bottom up approach, where you measure all ions in cells or ion channels and in their contribution to these transplant their potential and an ion fluxes to the cells on analytical top
Down or phenomenological approach such as the Fitzhugh Nagumo approach, where you kind of mimic the behavior of action potentials using these relaxation oscillators. And of course, over the years, this was back then in the 50s. Many many different models were developed and we
Kind of distinguish them into either these ionic, very detailed models, which are computationally expensive, or phenomenological models, which are cheap and, and easy to implement, but also are less realistic, or any hybrid version between these two extreme forms.
And this is kind of the sweet spot. For instance, defendant comma model is one model is used widely and there are many other models which are, which consists of fewer equations and fewer parameters, but sufficiently many parameters so that you can recapitulate the most important dynamics
Of the heart. So, if you didn’t connect all these oscillators in a tissue, you have them nonlinear waves of electrical excitation, which propagates through the tissue and here, this is a cellular automaton. And you see if, if you excite one cell passes on the excitation
To neighboring cells, and eventually, this wave dies out at the boundary, because it’s followed by this refractory wave that prevents further re excitation of the tissue. And this is important, this is the most important property of our heart that the excitation is supposed to occur once a
Second and that it’s also supposed to die out so that the heart can relax again. The big analogy is for instance, fire front you have excitable tissue that is maybe dry grass on the excitation wave
Itself is a fire front and the refractory powders the fuel has has burned and has gone into the fire can turn around and re excite or re ignite the forest. Now, in a continuous medium, we can nicely
Simulate this with a PDE s. And in this case, this is a special form of a reaction diffusion system you can see here again, this is a diffusive term, these are local reaction, local reaction terms that describe the excitability of the tissue. So again, you have a state and a threshold. And once
You perturb this oscillator over the threshold, you have this excursion the action potential and then due to the diffusion, you have here this wave that propagates outwards and then dies out at the system boundaries. Now cardiologists they try to measure these wave phenomena in patients, however,
There are many limitations in terms of how practical it is to access the tissue to measure and obtain recordings from from patients. The gold standard currently is to use on so called catheter mapping. So catheters are inserted into a patient this procedure is called minimally invasive,
But nevertheless it is invasive. And then under anesthesia, cardiologist required here electrical activity from these tips of these catheters now many different designs and shapes. But what it is in the at the end of the day, it is a point by point measurement. And it’s a lengthy measurement,
It’s time consuming because they have to reposition the catheter. And you have to assume that there’s reason that this patient is in a stable as soon as it is irregular and chaotic. They cannot obtain reliable measurements. So it is really only applicable in patients that have
A stable, abnormal rhythm. Nevertheless, you can obtain very nice maps from patients see a patient that is an A procedure and has been tricky attack Ikaria you can see here, two maps one was obtained
On the on the epi card, the outside of the heart and one was obtained on the undercard of the heart and the inner surface. And these two recordings were obtained kind of simultaneously and and were synchronized in time. You see this kind of rotating wave pattern and this is a wave pattern
That shouldn’t be that it is abnormal, and that causes irregular beats or irregular contractile function. And it’s it’s circulating around this area here where there’s presumably a scar in this patient. So this patient had a myocardial infarct. And then what can happen is that these waves get
Caught there and they kind of circulate around these infarcts. However, they can also be much, much more complex rhythms. And you can see here an excised Picard on the left side that was excised at the Masters University in the Netherlands. And they you can see very nicely how chaotic.
This is hard contracts during ventricular fibrillation. And on the right side here, this is a rabbit hole that we have imaged in our lab during ventricular fibrillation that we induced electrically. Now we’ll slow down this video for now, and I hope it plays smoothly,
You can see now overlaid action potential waves in pink, which correlate with this mechanical contraction at the heart exibit trim and trigger formulations. So here we have a measurement of these chaotic leaf parents obtained on the surface. Another example, how fast and how
Irregular the contractions can become. And this is a very extreme example of a technology regulating, there’s almost no contractor activity and more. This is why venture preparation is so dangerous, there’s no blood flow any longer. So we can in a laboratory setting excise hearts and keep
Them alive for roughly one day. And do imaging experiments with these hearts, we have fused them with liquid called tyroid solution, which is essentially water plus salts plus sugar and oxygen to keep the pH level at a level where it’s supposed to be. And then we apply something that
Is called retrograde perfusion, we pump blood, this solution yeah retrograding into the heart, it closes the valves and it goes into the coronary vessels and perfuses the heart and therefore can be kept alive for many, many hours. So this is our imaging setup that we have built
Here at UCSF. You can see the centerpieces here this imaging chamber, inside of the chambers, this liquid and the intact heart. The heart can also be seen here in 12. video images, these images are acquired simultaneously at very high speeds 500 images per second by these cameras
That are here on this table. And on the left side, you can see a commercial ultrasound system that is also integrated into the immune system. And its ultrasound system records three dimensional volumetric images of the heart of this place where you can see here, a rabbit heart beating in our
Tank. This is the same rabbit heart and this is the ultrasound transducer here at the bottom. It’s a 2d matrix array transducer that produces between 25 and 100 volumes per second. Again, this is a drawing here of of these different camera images, these cameras, cameras are calibrated. And we were
Able to reconstruct the entire surface of the ventricles with 12 cameras that are synchronized and calibrated. And then we cross register the optical reconstructions that we can make with the cameras with the ultrasound data so that we have structural imaging data plus functional imaging
Data that we acquire optically, because optically we do not only image the surface of the art, but we have recent dice and these resin dice are then excited with this green light on the left side, you can see here 48 LEDs are also integrated into this imaging setup. And they illuminate the heart
From all sides evenly and excite fluorescent dye which then emits a particular wavelength one the red, and this this initial light and carries information that we can measure and from which we can extract action potential waves. And this year now is one such measurement during the normal
Heartbeat sinus rhythm. You can see these are 3d reconstructions with relatively high resolutions to heart contracts. And you will also notice if you watch closely, that there’s a slight decrease in intensity, or in slow this video now Don, again we have added voltage sensitive fluorescent
Dye. And this dye indicates action potential waves or voltage changes. I don’t know if you can see that on Zoom, you may or may not see it, I can see it pretty well on my screen, there’s a slight decrease in I will now amplify this decrease. This is
The routine processing step that we do we extract the signal by simply amplifying it over time. And you can see here the action potential wave front in black saw this activation occurs in multiple sites in the ventricles due to this Purkinje fiber system that injects regexes excitation and several
Sites. We can also measure them accordingly mechanical strain, which you can see here is the area change in red that is a decrease in area change, and blue is of dilation or increase. So that we are able to measure this is the result of many many years of work. This technique is
Called optical mapping and it is almost 50 years old. In the 70s. The first measurements were done with was voltage sensitive fluorescent dyes and people demonstrated that it is possible to measure actual potential waves. And then this technique really had a had its heydays in the 90s,
Where people using a single camera were able to provide the first measurements of ventricular fibrillation and these wave phenomena in hearts and isolated hearts. However, they performed these measurements with pharmacologically contraction, inhibited hearts and in the past 10 years, I and
Other people have contributed or developed either with with which it is possible to track the heart and extract the sickness from beating hearts. And so there are I would say 99% of all publications that were done many 1000s of publications were done with these pharmacologically contraction
Inhibited hearts. And since 2016 17, roughly, there are now more and more publications coming out where this compound called blebbistatin is stepped out of the experiment. And we can now study for the first time, mechanics and electrics at the same time. And also,
What is kind of rare in our field is that people use multi camera systems such as our system in order to reconstruct a 3d surface of the heart. So right now, as far as I know, this measurement that
I showed to you is the only one of its kind. This is now a measurement of ventricular fibrillation, you can see that the heart is moving very slightly, almost not at all. And you can appreciate he also these chaotic wave parents that we can observe now on the across the surface
Of the house, they are spiraling they’re rotating. And that now begs the question, what what is this? What what are these dynamics? What is cardiac fibrillation? This is a very old question that has fascinated many people for many, many years. And in the 60s, chemists, physicists and chemists
Who I also worked in gutting and in Germany, came up with this hypothesis, okay, I know that in the so called blues of simultaneously you chemical reaction, I see these spiral waves and these virus must be related to cardiac fibrillation. Spyros are ubiquitous in nature. There are many,
Many different examples and either biology or in chemistry. This is an example why I saw them here in the wild in California. And there are many, many other examples where you can observe spiral bass in nature. And these are our reaction diffusion systems. Turns out,
They have all very similar equations, different compounds that become depleted or regenerated and that creates these these patterns. So how can these rotating parents be created? Or how do they emerge in the heart in particular, they were very early experiments roughly 100 years ago, where
Somebody cut out a piece of a heart and stimulated in such a way that they were able to see rotating waves circulating in this in this ring of cardiac muscle tissue. And that was for a very long time
Than the central hypothesis, there must be the circuits in the heart that sustain this persistent activation. But since Valentin Krinsky We also believe that this happens here in this tissue. So if you activate cells in such a fashion here that there’s an open end of the wave, that can emerge
In the real heart. Or if you keep this part here temporarily refractory activate this part and this following happens. This wave is able to rotate around its open end and this is a spiral wave and and the central mechanism in the heart It drives hardware disorders, tacky arrhythmias. This is a
Cardiac cell culture where you can appreciate that this wave pattern really exists in 2d at least. And this is a recording made by Flavio Fenland, you also look at Georgia Tech, they really see here, the voltage and the calcium rotation of wave that are stationary and then degenerate
Into more complex wave patterns over time. These wave patterns are surprised surprisingly robust, they can be sustained and highly an isotropic and very complex geometries, you can see despite the in the atria, there’s varying fibers and complex geometry, but this rotating wave can live on such
A substrate. Depending on the parameters, and depending on your genetic makeup, and so on and so forth physiological conditions, these waves can have all sorts of different shapes and as illustrated here in this three variable model, depending on these two parameters alone, you
Can create various different dynamical regimes of these waves, then the mechanisms how these waves can break up is is is manifold, there are many different mechanisms. But one is for instance, that these waves can interact with themselves, this leading edge here crashes into the tail
Of its of its own tip arm and then can break up this wave. And this is also shown here. They are these breathing instabilities that can last and further down the road cascade into chaos. This is another illustration this wave propagates through tissue that is damaged, the white tissue is as not
Supporting this wave that will end it accordingly it can lead to heterogeneity and breakup. So heterogeneity in general can promote our revenues. And now in 3d, it becomes a little bit more complicated. But even in entirely homogenous media without any heterogeneity in the substrate, you
Can have a route into chaos. You can see here a scroll wave and the blue line is a filament of the scroll if that’s the center around which this wave rotates. And due to certain instabilities, these these filaments can interact with themselves and break up and then cause such a complex turbulent
Wave pattern, which is also the underlying mechanism for this wave pattern be on the right side. So now to the part where AI and data driven modeling can help us to better understand or diagnose problems and disorders. We have applied various generic deep learning techniques, such as
Unit, for instance, to solve certain problems in our field. One application, for instance, is that people are interested in localizing these regions here around which these waves spiral or rotate, because you can imagine that if you would remove this point, then this, the
Center here would stop shooting out these waves. And accordingly, this part of the heart would go maybe eventually back into silence. So getting rid of these points of these facing polarities is a way of liberating art. And therefore, automatically tracking these points is important
And it was already shown in the late 90s, that you can automatically identify these points, you embed this temporal signal using a Hilbert transform. And then in these resulting face maps, you can compute the gradient of the face and whenever you compute a circular integral over
The gradients, around such a face singularity, you get a discontinuity. And these discontinuities can then serve as a detection mechanism to track automatically facing layers through or across the tissue. However, in practice, it is not easy to do that, even with high resolution data due
To noise and other artifacts. It is sometimes you end up with spurious face similarities or measurements that are simply not robust. So you can see here in a computer simulation, this works really nicely. But if you imagine you work with Kathy data mapping, and you have sparse
Measurements, only here on these points or among these points, then the underlying measurement is far from the ground rules and accordingly the trajectories of these facing block points that can deviate quite significantly here shown on the right side from from the grassroots.
So we thought this is the perfect problem for units autoencoder or LSDM version of a unit to process such data. We trained such a network here or encoding decoding convolutional neural network with sequence of video of such data. So this see on the left side are voltage sensitive
Measurements. And then on the right side, we either target discrete points directly, or we compute face maps and in this face maps then face similar points. And we saw that this network can do this easily with only five snapshots, five measurements, from the activity, even
Sparse measurements can provide you a face map. So it can successfully cross predict these face maps from electrical measurements. And they are not far away from the ground truth. And they can do that with less than 10% of the spatial information. And so that showed us immediately how powerful
New networks can be for certain applications in our field. And you can come compare that previous success saw measurement with maybe this example here. Once you’ve seen this energy on the right side, it will also be much easier for you to anticipate what this sparse measurement of that
Image might be. And then what the network here does is it has seen so many examples of these dynamics that it can easily recover the underlying dynamics from it. The overall prediction accuracy, of course, goes down, there’s sparser and the more noisy the data gets. But overall, we would still
Be able to identify hear these facing lock points. And for these rotation of parents, across the surface of of tissue, we’ve then applied it to our optical mapping recordings, you can see a ventricular fibrillation and a Picard. And we can train this network either
On this data or on simulation data and then instantly compute these facing loved ones. If we were to compute these fashionable points for from our recordings, we would need a very long, much, much longer recording, at least maybe one second was he over here, only a few milliseconds
Are sufficient to compute these points. Most interest, interestingly to us was that we were able to train it on simulation data as shown on the left side, and then apply it to experimental data as shown on the right side. Even though this data here are these dynamics that are shown here
Are three dimensional dynamics. And they are, what we see here is the surface wave pattern of a three dimensional wave pattern that evolves within the heart muscle over here on in contrast to the simulations that we performed are really only two these that completely different dynamics kind of,
And of course, these computer models also not not fitted to this real wave pattern here. But nevertheless, it’s sufficient for the network to perform relatively well. So you can see here that if we train on simulation data and apply to simulation data, we have a very high accuracy.
And we can train on pick data and drive data, meaning that we used isolated hearts from either pig or rabbit still get a very high accuracy. And if we only train on simulation and applied to pick a rabbits, we still get a decent Angular accuracy. The next problem that we then tried to
Address with the following, we can image currently with our present eyes, the 3d wave pattern, or the projection of this wave pattern on the surface of the heart, but these dyes do not really penetrate. They do penetrate into the heart muscle, but the light coming out of that muscle cannot reach
The surface because the hardest to thick, and therefore we don’t really know what’s going on underneath the hard surface. And we do not have measurements of ventricular fibrillation in 3d, we have computer simulations, but not real data. And we tested now in these generic toy models
Scenarios, how well, convolutional neural network would be able to extrapolate the three dimensional wave pattern from two dimensional surface measurements, you can see on the left side, a 3d Scroll wave pattern that shoots out waves towards the surface on the right side, you see,
Whatever we can see on the surface. So we’re kind of pretending this is our experimental setting, and then trained a neural network to predict from 2d surface measurements. So this is the surface layer of this bike peeled off and fed into the neural network. And we used five time
Steps of five of these snapshots. And the target then was the full 3d parent. And this year is a reconstruction so you can see that works pretty well. When we look at it from above you see, this super superficial layers, they are all reconstructed pretty well. But then if we zoom
In and look at it in more detail, we see that the construction accuracy decreases with time. I saw the surface layers here they are sharp and crisp and also accurate. But the deeper you go, the more
It fizzles out and becomes less accurate. We use diffusion models in the same way, training them to predict either from only the surface top surface or from the bottom surface or from both surfaces, the three dimensional wave pattern and saw that they perform similarly to a simple unit
Architecture. However, if you provide them with more data, they can perform much better than unit. So we did a mistake kind of we we tried to reproduce the same results that we obtained with the unit and with unit we saw that as saturation, if you provide unit more than five frames, let’s
Say 10 frames 20 3050, the the network doesn’t become better at it. And we then also stopped with the diffusion model was five friends because we thought it didn’t get any better with the unit model. But here, Stan and I were able to show that if you actually use more data and provide
More data to the diffusion model, and with more data, I mean, you know, these five samples that we extracted were five snapshots of one rotation of such a spy wave, forcing you to take into account
How fast do these waves propagate to the surface, and so on and so forth. And here, if you have much, much more data that goes maybe beyond one period of the spiral wave, then you can actually
Decrease this error quite a bit. Nevertheless, you can see that the deeper you go, the greater the uncertainty and Arabic columns. And therefore this is maybe not the best approach to try to extrapolate these complicated chaotic dynamics from surface observations alone. And therefore,
Another approach is needed. And this is something that I would discuss now in the second part of the talk. Before I jump into that, I wanted to mention that I told him he in 1903 came up with this invention of the electrocardiogram and since then, the electrocardiogram has had tremendous success
In cardiology, everyone uses it, it’s very cheap, it’s very effective. And you can immediately even at home yourself can assess the state of your heart, you can say, okay, my heart is beating normally or all something is something is wrong, I need to see a doctor. And it’s very cheap. So
Therefore, I think people have never really dare to come up with alternatives to this. The problem was the electrocardiogram and all electrical measurements is what I just showed to you, you can not access the entire heart, and you cannot really visualize what’s going on beneath the surface of
The heart. And this focus on the electrocardiogram has maybe kind of blended us and prevented us from from looking at what’s already there, and the heart beats the heart contracts, and it’s visible, you can use an ultrasound machine, we can use MRI, and we see the contractions of the heart, and we
Can analyze them. And so therefore, we could now ask the question, can we actually calculate the electrophysiology or the action potential wave parents that trigger the contractions of the heart from the motion of that, and this is, you know, illustrated in these three idealized ventricular,
By ventricular electromechanical simulations, you can see on the left side, this plane wave here induces a very distinct, contractile pattern that is distinct from the contractor pattern that is induced by to rotating waves or by five or more rotating waves. So it should be possible
To compute something like that. But we didn’t have the measurement data that could actually give us this this certainty or this experimental evidence that would support this hypothesis. And here, in our setup, we have no such data that motivates this even further, you can see here, this wave
That propagates across the ventricles of a rabid heart, and this weight induces a contraction that is highly correlated with the action potential where it’s not that everything contracts at once. It is really that the contractor activity sets on and and follows the action potential. And then the
Two questions that we had initially and we still have, and I’m trying to address this, is this sufficient information to compute electrical activation, if you looked at this mesh here, is that enough? And is this mapping between electronics and mechanics unique?
These three images here show you how closely correlated that actually is the action potential wave here propagating from the left to the right. And users contractor activity over there, because the contracts over there which was shown here and the strain rate in which tissue is being pulled
Towards the wave, and that tissue dilates as a concept plants and this is a very characteristic phenomenon that we observe over and over now in our experiments. And this weight frontier between dilating strain and contractile strain is following the action potential wavefront. There’s
A small delay that is typical for cardiac tissue and it propagates in the same direction. So, there is really experimental evidence that justifies this approach. And there are more observations that we made in the past. So here you can see waves with your this is a recording
That we made with the phone, mechanical waves propagate through the heart of the solid was treated with cardioplegic solution. And therefore, the activity was pretty slow down and what you can see the space this now is the paper that I mentioned in the beginning, I studied this
Mechanical deformation pattern in the ventricular was in computer simulations and then also made recordings measuring hear voltage calcium and strain and you see here a clockwise rotating pattern and all of these three quantities. So down here, we I was able to compute phase singularities
That indicate the rotational centers of these waves for voltage, calcium and strain. And there are there are we published in two papers that one is a physics based approach one is a deep learning based approach in which we try to compute now electrical waves from mechanical deformation.
And I will briefly talk about this year, mainly because I wanted to highlight how much better deep learning works actually. So the first approach is a data simulation approach. We say system one is the system that we want to measure. System two is a virtual replication of that system, we introduce
Sensors in the system that we want to measure and controllers in the virtual replication of that system and drive the system with the measurement data that we have available eventually from system one. So we can see here to come excitable media excitable systems, system one exists system two,
And there’s a coupling term here and this coupling term is really only difference between the states of these oscillators over here versus here. And because we do sometimes explain that much better than 1000 words, you can see that here, this computer simulation is coupled to the psychiatry
Measurement over here. And just constantly receiving input from this from these sensors over here by by the controllers that are placed in this domain. And you see that this simulation code evolves with the cyclotron. And we adapted this principle and and modified it a little bit. We
Have we know that based on our measurements, these electrical waves and use also mechanical waves, therefore it might be possible to do exactly this, slightly modified instead of having in your control term, a comparison between the two electrical states, you can now compare
Either an electrical and mechanical state or the mechanical state of system one, the mechanical state of system two, and modify the electrics in such a way that the mechanics of the second system match the mechanics of the first system. So that’s shown here we have sensors in our drive system
Are the first system that we want to measure which measure mechanics strain, for instance, strain rate, and they modulate the controllers and the electrics of the second system. So here we have a spiral wave that induces mechanical contractions, red is contraction, glues dilation,
We added some noise to it and then observed that and induced electrical excitation in these locations here, wherever contraction occurred was a negative electromechanical delay. And you can see that over time, this system becomes synchronized with the first system and therefore
We have a reconstruction of the dynamics in the first system based on a mechanical measurement. And then we were relatively happy that this also works for very complex wave parents. So you can see here comparison of the reconstructed wave pattern from mechanics with the original wave
Parent to apology is very similar. However, there were quite significant differences too. So we were happy about an 80% reconstruction accuracy. But all of that was kind of killed with deep learning because it worked right away much better using simply an autoencoder for this
Problem. So we trained an autoencoder and a unit with pairs of mechanical deformation patterns with electrical excitation patterns that cause six deformation patterns. You see a spiral wave pattern that induces together with multifiber anisotropy this anisotropic spiral wave like
Mechanical deformation pattern, the Folker wave pattern, a more complex wave pattern and we use 20,000 samples of such pairs to then estimate to electrical waves from mechanics with unseen data. So the network architecture again was a simple encoder decoder, we analyze displacements,
X and y displacements in 2d which could either be with ref with respect to an undeformed, mechanical state or any arbitrary mechanical state, or the previous mechanical state of the system, and saw that this works pretty well, after a short period of training. And after a few epochs,
This is an example of 3d dynamics, very, very turbulent dynamics. These dynamics induce tiny mechanical deformations here, which were analyzed by the network. And this is the reconstruction. It’s visually indistinguishable from the original dynamics. So, the complexity of the wave parents
Really no limitation. And you can see here on the right side, those are wattics filaments that were computed either from the original wave parent or from the reconstructed wave parents in black and gray and perfectly overlap. So now the question is, how is this applicable in patients can we use
This to reconstruct electrical activity in a patient who is there in in the clinic and doing such a procedure here you can see, this heart is beating abnormally, this patient has ventricular tachycardia. And at the center is the tip of a catheter. So this person performs
This procedure here of mapping the patient, and spends many, many minutes maybe a full hour to obtain a 2d map of the electrical activity. But since we are measuring this at the same time, this ultrasound recording, we could instantly provide this activation map if we are able to
Compute it from the mechanics. So we perform computer simulations, again, idealized by our trickier computer simulations, which retain muscle fiber architecture that is necessary in electromechanical simulations. This massive fiber architecture determines how the heart contracts, but also how the waves propagate. Then one issue was that we needed roughly at least 10,000,
Or maybe ideally 50,000 training samples. How do you get that many training samples and computer simulations that are usually traditionally very expensive. So we use sphinxes, which is smooth particle hydrodynamics. framework, because it provided us sufficient speeds, and we were able to
Compute one episode here in less than 10 minutes and could therefore over the course of a few days, simulate enough training samples. And then this problem becomes the following. Here we have three dimensional point cloud of three dimensional motion vectors, which describe the movement,
The formation of the ventricles, the target is a static voltage wave pattern. Again, we have many many different examples of these pairs, and the unit that we then use as a sparse res unit, we use this Minkowski engine here to process unstructured grid data and convert it into a
Format that this unit understands. The inputs are five frames, describing five states of the mechanical information state of the heart, that could in principle be extracted from ultrasound recordings, MRI recordings, and the target is then the electrical morphology of the wave pattern. In the computer simulations, we varied for different electrical models,
Which are produced very distinct electrical wave parents with four constitutive laws for the mechanics. And then randomized initial conditions model parameters, the geometries of the tissues can either induce focal or reentrant base, you can see here, the different anatomical geometries that also have an influence on the fiber architecture, and created 1000 different
Geometries, plus five architectures and then varied our initial conditions. So we for instance, stimulated the heart at the bottom at the front or the back in random positions, or induce these different spiraling wave parents also in random positions. And you can see here how these
Different electrical models produce very different qualitatively different way quantitatively different wave patterns. Again, this these are four examples. This is maybe a more realistic simulation that would be more comparable to what’s what’s happening in a patient who has a BT episode, and this is a more idealized computer simulation. We also in space,
Vary the model parameters accounting for spatial heterogeneity, and overall varied randomized More than 15 I think parameters and also included scar tissue or heterogeneity. So here in red, these parts of the tissue did not contract or did not conduct. And then we were happy to see that
This works, we can actually train such a network. This is the ground truth electrical wave pattern that induced this mechanical deformation pattern, they are shown on the left, this was the input to the network. And this is the prediction that works for focal wave patterns, we can compute now,
These activation maps here from these a sequence of predictions of electrical wave patterns. This is an activation map that was directly computed from the sequence of electrical snapshots. And then from this mechanical deformation computer, this sequence down here and this activation map,
And it’s very similar. And you can now compute from the mechanical information of the heart these electrical activation maps, which are important for cardiologist because sometimes they’re interested in where in these ectopic beats these additional undesired beats come from. It
Works for reentrant wave activity, also pretty well. And these are the different accuracies that we achieved, how much more time do I have? Do I need to run through the remaining slides? Or how many how many more? Slides? 11? No. And I might run out of time.
Yeah, maybe you can wrap up. Okay. I’d like to write a write up next. Five minutes. next five minutes. Yeah. Okay, good. So this is an example where the heart includes also scar tissue. And the prediction nevertheless, works quite well. This is very important, because most patients actually do have
Myocardial infarct and accordingly, heterogeneity, and scars. And if this method wouldn’t work in the presence of scars, and it would be useless, probably. So now, how do we translate that to the clinic, we showed in computer simulations that this works, but that can also doesn’t
Mean that it needs to work with patients, we have tried to apply the network that was only trained on simulation data on such data on the right side, and that obviously did not work. And our approach now is to generate training data in our experimental setup with isolated parts,
And then see how well that could perform maybe on the patient data or fine tune model subsequently, further down the road with with all of these different types of data. So this is the experimental data that we have is really high quality, electromechanical data of parts,
We know us stimulate the hearts in such a way that we have many, many different stimulation sites and the corresponding mechanical deformation pattern. This is just the optical recording the red vectors indicating mechanical information, but we also have, of course, transmitter with mechanical data from the ultrasound. And this is shown here,
This is raw ultrasound data segmentation that deforms based on the 3d motion tracking where we perform, there’s a corresponding target than on the surface, at least of the electrical activity. And we are currently in the process of fully automating these these experiments,
In order to be able to generate 50,000 training samples, or many, many 1000s of training samples. So that means that we need to be able to go into these recordings, pick certain frame and extract the frames, reconstruct it in 3d, cross the interest of these two recordings,
And extract here the corresponding data and fuse the data together. We’ve seen in other examples that in terms of generalization, we can train the sorry, we can train the model and simulation domain well, and leave out one of the electrical models and it still works pretty well. We’ve
Applied it to significantly different geometries. It also works in those we have borrowed data from from collaborators who use finite element modeling instead of SPH modeling that we use to train the model and it also works efficiently when I think that if we improve our of our methodology,
It should be possible to reconstruct overall excitation waves from mechanical deformation. Now, the big question is, how do we get that to work in our ex vivo imaging setup? And let me jump through the nebulous slides here we showed that for instance, in these two dimensional cell cultures,
We can apply a model that was trained only on simulation data on these experimental measurements and get we constructions that make us hopeful that this could actually work in practice. As an outlook, of course, we would like to use this data not only for predicting one thing from another,
But also maybe to evolve and obtain a data driven model of of the heart based on these dis experimental data that we’re able to generate now in our setup. So we’ve used the fusion modeling and saw that fusion modeling can be used to evolve project dynamics, we can generate shapes
And waves that look like ventricles and reentrant circuits. And I’m sure that we will also be able to generate beating hearts, maybe even driven by the data that we have now in our experimental setup. So that adds now a third pillar to our set of tools that we have the conventional
Computer simulations. And now maybe, hopefully soon, also, data driven modeling of curricula for mechanics. So and I hope I didn’t exceed too much. Thank you very much for your attention. Very nice. Very nice. Thank you so much for the interesting talk. Ian. enjoyed it a
Lot. Let’s have a q&a sessions. So Well, I first want to introduce you to rob Robert Blake at Lawrence Livermore. Rob, are you still there? Yeah, I’m still here. Oh, cool. So Rob has been working on the computational model for cardiovascular system. We have some computational tools. Rob, do you want to introduce?
Yeah. Hi, Yan. I worked with Natalia for a long, long time and wrote. So we have cardioid here that can do a whole bunch of cardiac simulation stuff you might be interested in. I also have melody
That lets you take anything off cell ml, and quickly generate code for it to run in chased or whatever simulator you want. And I have a really cool data set that you’ll want to get access to. We did a whole bunch of simulations of just tricky colors, and then recorded pseudo ECGs.
And that was able to train a neural network to go from the ECG to transmural maps on the Hga 16. Nice. Okay, yeah, we should definitely talk. Yeah. Awesome. All right. Let’s have a q&a sessions. If you have any questions, yeah,
Please don’t hesitate, just unmute yourself and ask questions. I can I can start. So, you know, you said that the measurement can be only done on the on the surface, right. And in order to fill the gap, you relying on the computational model, but every patient has
A different shape of the heart? Probably. How do you deal with that? I mean, it’s building those computational model. One of them will take some time. And if you have a patient with a different shape of the part, do you quickly do you parameterize, the heart computation model,
So that quickly change the shape of the heart and run the run the heart model? Good question, I think I don’t know. I would assume that from imaging data would have the shape of the heart. And the question would also be How sensitive is the solution to the
Shape of the patient? Is it possible to use a generic shape that represents most patients, so that adapts slightly to the imaging data? I’m not sure how, how the anatomy would influence potential solutions, those imaging data is going to give you the shape of the internal
Structure of the heart as well for each patient. Yeah, yeah, sure. We don’t have MRI data or generally have MRI data or ultrasound data. Even though ultrasound has a pretty low spatial resolution, it’s possible to to extract at least
The global shape of the ventricles from it with the Atreides it’s a bit more tricky, but that’s definitely possible you wouldn’t be able to extract you know, final structures is especially on the endo card. And it’s also sometimes not easy to delineate the organ boundaries,
But overall, we can definitely see differences in in adult anatomy across patients and ultrasound. One of the techniques you you showed was, you know, only the using the sun Stata. And well, I mean, of course, do you use the computation, 3d computational model and then generate the
Data and train the unit, for example. And then once the unit is trained, you just, you know, get the measurement in sparse way, you had a particular shape of the measurements and the location in the space on the surface. One was the uniform is distributed,
And the other one was the sun. How should I say, you know, the straight line to the center? Yeah, I think you mean the Kathina mapping data, right, that lets us there was a study that couldn’t go back. Yeah, if you can go back. See.
This slide. Basically, my question is, how how do you? Right? Like that sucks about last zoom in this the star shape at the bottom right? The right one? Yeah, exactly. How do you choose that location is it’s somewhat flexible to choose those locations. So all of the measurements
In this is completely determined by the manufacturer of that catheter. So this study here, by by Ronnie at eye, she there I think, either chose generic representations of wood, like this geometers are similar to these catheters, or they are directly somehow
Created from the classical, I don’t know. But there are either these Penta re catheters, or these lots of catheters, and then also basket catheters, or just different types of catheters that are on the market. Some of them have up to 64 I think electrodes, the basket catheters, not
All of them measure something at the same time, but they can sense which ones contribute to the measurement. And there are often many different episodes that that recorded the same time. So that’s where it comes from. And in our study, we just used something that looks like a star. Got
It? And I think in another point of your presentation, you talked about the training the unit, and then use the training unit to you know, get the information about the insight, not just the surface, I mean, the measurement, the standards on the surface, but also the internal. Yeah,
Yeah, this year. Yeah. Right. So this is, we pretend that this is a piece of the ventricular wall. It’s, of course, heavily over simplified. But we were simply interested is from our from a dynamic systems perspective, is it possible to reconstruct very chaotic wave parents in 3d in
3d from only surface measurements, which is a situation which we have in our lab, we can currently really only measure the outer surface eppicard. But in principle, you can also measure from the inside the undercard at the same time. So you have two cameras, for instance, facing each
Other and in between is the hard wall. And in that scenario, you have two measurements, like here on the slide. And then whatever happens within the hardball, the question was, can we recover that from only these two surface measurements? On what we what we’ve learned in these two
Studies here Standard Oil and and labor the die from 2020 we it is possible to some extent, in this scenario like a one step your network the neural network is trained after that is trained for the inference stage, do you assume that the whole surface
Image is available as an input or only the sparse data on the surface is available? Here we assume that the entire services available because the scenario that we tried to recapitulate was optical mapping. So optical measurement, we added some noise to it
And saw it also works with noise, no problem, but we didn’t try out a sparse measurement. So that would be the next step to move to more realistic geometries and add sparsity Okay, great. I mean, we have some technology of you know, you know, enabling that actually,
You know, once you train the like auto encoder or you net and then only we get the sparse data on the surface and then try to reconstruct the whole 3d No output data. Maybe I can introduce to you after the talk. Yeah, okay. Well, that’s my question. I don’t want
To take whole time of the q&a sessions. Is there any questions from audience over billion questions, but it’ll go weeds. Only one or two allowed? How much? How much luck have you had with face singularities? Every time I’ve tried to track face
Singularities in cardiac tissue? Like, it’s super messy? To do? Absolutely. Yeah, we just gave up and started with like, just tracking wave fronts and measuring them as like a 40 surface instead. Yeah. But yeah, like, under so much. And depending on like, little tiny things are very sensitive to
Air, and they kind of go whoo, wobbly all over the place, and then disappear and reappear, like continuity and fade singularity has always been really difficult for us. Yeah, absolutely. I agree. So there might protect ourselves that influenced that.
So the noise and motion artifacts and optical recordings, at least they can cause, you know, Basinger’s to jump or disappear or something, you have spurious very similar results are not mere, that are just a result of the filtering. So we, we have a protocol, I can send it to you if you want,
That works relatively well when the signal quality is good enough. So you can process video data in a way that you focus on the wavefront, rather than the wave itself. Both way works, and you have to slightly smooth the data. And then you remove outliers, from the normalized
Data. So what you want to do is you want to normalize each time series, to amplify the waves, and then use the Hilbert transform or whatever you need in order to embed the signal and compute the
Face map from it. And then what is very important is to smooth the face maps using a complex filter. So you convert the face maps into a 2d vector field with with a normalized amplitudes. Like a complex number, and then filter that. And with that technique we had, were quite like,
It’s pretty robust. Of course, it doesn’t prevent you from detecting face singularities that are, you know, further down further upstream in the processing pipeline created there, or just simply the fact that you’re looking at a projection of a 3d wave pattern. And these some
Of these spirals and meander really strongly they they have a linear core rather than a point around which they rotate. And that can also cause issues. And there’s some work by a group in Belgium that came out recently where they look at that in more detail. But it can get very messy. Yes.
Have you ever seen an arrhythmia without a face singularity? Either linear core or point core? I know data from Flavio Fenton where he has shown that I think, have I seen that in my experiments?
I don’t think so. Okay, because we I’ve been we’ve been like we saw it all the time and stimulation, it would happen, especially around scar structures. And like, the whole face singularity stuff just really kind of broke down because we’re like, can’t find and we have to
Do integrals. And we’re like half the heart. We can’t find the in the face singularities. Yeah, the linear core is really an issue. And that has also raised concerns with this concept of facing law even makes sense. So the people in Belgium, Hans Dirks, and his people, they publish
Something last year thing where they expand this concept of a face singularity into different lines. And it gets also really complicated highlighting that it is a complicated topic, more complicated than at least just points looking at points. I think it’s, it’s a nice
Concept too, too, too, to look at, but I would not overemphasize it. To describe fibrillation, it’s in the end, it was supposed to reduce complexity, if you look at a three dimensional wave pattern that changes quickly in time. So for the wave pattern, it can become very, you lose track of it.
It’s just complicated and difficult to understand. And therefore I thought back then and 2000s, it was thought to help people to understand what’s going on and that way it helps because you can count the number of waves you can determine the density of these waves. You can see how waves
Interact with each other these filaments when they touch each other and break up. That’s that’s a nice concept. But other than that, it’s I think we also need other to was to look at corroboration. When I see that you you’re using unit or autoencoder type of approach,
That is nice because it gives you some reduced latent space dimensions and, you know, bring the data size effectively reduced to the smaller dimensions and but have you tried to actually interpret that any physical meaning on that latent space,
The reduced dimension later in the middle of the unit or the middle spacing in the auto encoder? Do you mean if you’ve ever used visualization tools of what’s going on in the latent space or physical? Yeah, physical principle later, I mean, your underlying physics, they say electrical
Mechanical systems themselves, you know, I saw you present his some partial differential equations associated with it. And, you know, of course, the data is, you know, if you use the simulation, that data will be really high dimensional. So after the auto encoder, it is reduced,
And it’s, it’s quite unknown, what physical meaning those latent space has. Maybe you can put some meaning to it, physical meaning to it, associated with those electrical, mechanical, electric, electric mechanical system. You have? Have you tried that? No,
We, that we we try to kind of verbally, you know, talk about what what, what happens in the latent space, maybe here, in particular with these facing polarities, which is a simple, I think, a much simpler problem where you only have a scalar valued field that you transform into a
Complex field. You have these two numbers that define the phase angle? They are, I would say, the latent space, what is it? It’s that conversion, you know, that you do with the hero transform somehow, locally, this is much more local, I think,
In the case of the electromechanics, you have long reaching elastic effects. So, global motion of I don’t know how it translates different modes of the contraction of the of the deformation of the heart that are in the latent space somewhere. I mean, it’s pretty amazing when you look at these
Simulations here that you can convert something like this vector field on the left into a scalar, varied, filled with complicated geometry, relatively complicated geometry. It’s amazing and very interesting. Very interesting. Yeah. What happens to us? Right? Well, you know, our group, we tried to, there’s a framework we developed called elastic
Latent space dynamics, identification, meaning that so once once you project those 3d motions, vector field into the latent space through the auto encoder, and you get some dynamic signal in the middle of latent space, and we don’t know, what’s the governing equations for that
Trajectory data, so we try to associate that with a some sort of differential equations, and then try to understand what that differential equations, what kind of meaning it has. So yeah, maybe I can introduce that to you as well. Okay, so I have a to do list.
Sounds great. Yeah. I would love to talk about that. Awesome. Awesome. All right. Any other questions from audience? Well, it’s now let’s, let’s thank our speaker yarn for the wonderful talk and interesting talk. I learned a lot and hopefully we can, you know,
Further stimuli stimulator. The collaboration between Lawrence Livermore and UCSF. Yeah, would be great. I was very happy when I found out about this seminar. So it was recommended to me by Michael Gregory. Awesome. Awesome. Thank you. Thank you. Thanks. All right. Transcribed by https://otter.ai