ARTSOUNDSCAPES SEMINAR SERIES. 8 MARCH 2024. SPEAKER: Brian Katz: (Sorbonne University, CNRS) TITLE: Experimental virtual archaeological acoustics: bringing together physical, computer and social science researchers SEMINAR ABSTRACT: Research in historical musical acoustics has traditionally focused on instrument fabrication, while musicological and theatre studies have explored notation, performance styles, and theatrical elements. However, scant attention has been paid to crucial acoustical conditions for human-to-human communication in historical contexts. Recent advances in computational acoustic simulations offer ecologically valid reconstructions of sites like Notre-Dame de Paris and the Palace of the Popes in Avignon. VR tools such as the Anaglyph binaural renderer and RoomZ spatialiser aid in immersive audio rendering and dynamic movements within simulations. These tools facilitate live performances in virtual historic venues, allowing real-time adaptation of audio and performer movements. Research into the impact of acoustic variations on performance and listener interpretation contributes to understanding historical contexts comprehensively. Moreover, these VR models can be used for public communication and artistic endeavours through mediums like VR performances, cinematic reconstructions, podcasts, and musicographic installations. ABOUT THE SPEAKER: Brian F.G. Katz heads the “Sound Spaces” research team, investigating spatial 3-D audio rendering and perception, room acoustics, HCI, virtual reality, and applications to acoustic cultural heritage. With a background in physics and philosophy, he obtained his Ph.D. in acoustics from Penn State in 1998 and his habilitation (HDR) in engineering sciences from the Université Pierre & Marie Curie in 2011. Before joining CNRS, he worked in acoustic consulting at Artec Consultants Inc., ARUP & Partners, and Kahle Acoustics. He also worked at the Laboratoire d’Acoustique Musical (UPMC), IRCAM, and LIMSI-CNRS. Brian is a fellow of the Acoustical Society of America (ASA) since 2018.
Uh thank you very much uh for um attending this um ound skapes um um seminar and um I’m going to first um present uh the project and um the speaker uh the the project is an EC project that um started uh in 20 8 so it should have finished by now but because
Of covid we asked and obtain the permission um to uh continue for one more year so we are now in our last um uh year of the project uh as you may know some of you definitely know because you are in the project some others um you perhaps you
Are not that familiar with um um uh this is is uh a project that has several research lines uh it is based on um brout so I am an archaeologist um and uh the the archaeological basis is from toart uh in fact in the application I said five to
Se five to se lines that didn’t include Rock so that’s why I this the last one which we now call call research line zero but anyway the first one uh was physical Acoustics characterizing the acoustic properties of Rocka Landscapes then uh we have a research line two that was pycho Acoustics uh the
Pycho Acoustics of um proad the third one that was uh neuros pychology the Acoustics U or neuro um Acoustics um if you want then we had another one of anthropology anology um sort of looking at information in um the anthropological sources ethnological ethnographical sources and then the fifth one was putting everything
Together really um so um looking at how all that uh could combine so here it says the analysis of sacred emotion in Ro at um sses right um and in fact we have been working in all those research lines um let me give you some highlights of the
AR soundscapes project because we have been we started organizing many many seminars at the start so that was one of the ways in which all these resarch lines could combine and we understood what all the others were U talking but in the last U few um in the last couple
Of years we have organized several seminars and media has been very active in this but um not as many as we used to so what has happened um we have uh a p recently thisis should say that submitted um this week early this week so we are really delighted with is in
The psycho Acoustics of rock of of rock Landscapes then um she will be moving now to a postop position then Nas was successful in a mar l k application and um is now in boo in new lands um we have two new researchers that is Natalia here
And Yoshua um Natalia working on Mexico and the brat Landscapes of Mexico and Joshua musical instruments in um South um uh in sou Africa we of course haven’t included her here because uh you know we have also so s who is mar working on medieval that s say hello uh medieval um Monastery
Landscapes U and then yeah we were 2023 we did tons of we we successfully published many many different um things so you know you can see how in the end everything worked out out and and it was really successful here in terms of applic of um Publications and have also
A look at our blocks I think it’s fun you know what is going on in the project really um so at the moment uh there the these are um the ones who are um here who um are in the project although Samantha is in fact has now moved to
Their new Post stop but that’s and um let go to the next one and um so let’s um see then uh our uh finally we arrived to our St uh Brian right uh Brian um I think that half of these uh um you know people audience know exactly
Who Brian is the other ones probably as well uh one of the most Renown International specialist in acoustical measurements and and Analysis and uh virtual reality simulations uh background in physics and philosophy wow that I didn’t know that so the vi was also included philosophy but then he um moved on to definitely
Acoustics um and um Han State University he was working in the US and then moved to Europe about the turn of the century and has been working as a consultant and then uh now in the cnrs um and he now Lees this s spaces research team
At and um yeah um as he will be talking to us about um part of the research that there are is leading um there investigates special 3D audio rendering on perception room Acoustics HCI uh which you will tell me now okay great virtual reality applications um to acoustic cultural
Heritage and is a fellow of the acoustical Society of American um then if this moves like um so um today no I have this one um he is successful in many many many projects if you go to that web page you will see the amount of projects that uh his research
Team is uh running it’s absolutely impressive um anyway so we have the honor of having him here today to talk about um experimental virtual archaeology archaeological Acoustics bring bringing together physical computer and social science researcher researchers and uh the idea of the question is how human to human communication worked in historical
Context that what he’s going to try to answer and try to convince us that computational acoustic simulations can help to answer this question with virual reality tools um uh how doing for example uh live performances in Virtual historical v um if I am incorrect he will um tell
Me or tell us later on but um so virtual reality modeling uh allows two type of different things that I think this um key uh what is going to tell us about first research and then Public Communication and artistic projects so research what is the impact of this acoustic variation on performance and
Listen interation in particular historical periods um and regarding Public Communication is how virtual performances um uh can help this communication cinematic construction forcast music musicor graphic installations so anyway so the floor is um yours uh Brian Brian and um you know now it would be fantastic to listen to
Know what you have to tell us so I will uh stop sharing and um allow you to then share your uh PowerPoint so thank you very much there we go yeah not that okay um thank you very much for coming uh thank you very much for inviting me uh the date’s wrong because
I think was when the seminar was supposed to be before we mooved yeah um it’s very weird seeing my own slides up before we try do that um so what I want to do is more of an overview of some of the different projects that we’ve been
Doing um that have kind of touched on social science research history research to give you an idea um and hopefully spur some discussion more than explaining in detail how anything has been done so well I don’t need to go into the detail of who I am because my already
Presented who I am um but the the key bits are kind of what my core research has been in and it’s in spatial hearing and psycho Acoustics so really how do we hear where sounds are coming from uh room Acoustics in terms of measurements and simulations and then in the context
Of virtual reality looking at interactive uh scenarios in uh audio and spatial audio um we’ve done some research on spatial auditory navigation and simulations for the Blind and the most recent uh kind of big uh direction that I’ve been working on is in this cultural heritage uh historical acoustic
Simulation and kind of how I like to to begin in the beginning is so room acoustics already on its own is an interdisciplinary subject uh so we look at the physics of the building we can look at the cycle Acoustics of what it’s like to be in the space it also impacts
Music and architecture if we now look at the Heritage aspects um then we also bring in historians art historians musicologists archaeologists and sociologists to see how those all interact and if we look specifically kind of at music and music performance there’s already kind of a a direction of research in musicology called
Historically informed performance where musicians try to use historic instruments or historic reproductions of instruments uh and tried to perform as they did at the time to recreate those pieces of music uh but very little has even has really been looked at the spaces where they’re playing so you can
Have a historically informed performance in a 1960s concert hall and so some of the questions we’ve been asking are well if you actually did that same performance in the building that it was intended um would the music actually change uh so that kind of brings us into
That idea of looking at the Acoustics of performance or the acoustic spaces of human communication and how do those impact to those Communications um and then so basically and I now I’m assuming everybody here knows this idea the Acoustics of a space is kind of the intangible result of all
Of the tangible physical uh decisions in making a building the room Acoustics impacts effects and completes any sound that’s made inside that space um and kind of the same way if I’m doing graphic visualizations if there’s no light source in my visualization I don’t see anything if there’s no Sound Source
In my acoustic simulation there’s nothing to listen to um so we’ll kind of get into what sounds you put into spaces uh when you’re kind of evaluating them um this is a a small little exercise that that I like to do so this is um kind of giving the main characteristic
That we use to quantify the Acoustics inside spaces is the reverberation time and this is just to give you an idea of the same sound uh under different reverberation conditions and I want you to think about what is the best reverberation what do you think is the best reverberate uh reverberation in
This condition so it’ll play with kind of zero one 3 and six uh is a Reverb Time The Oh If I took a general poll which one do you think or which one do you find is is the best reverberation time this one yeah one one which one the Dr one the zero yeah so the zero to me sounds like a like an accordion um and one three three sounds like a
C yeah yeah and six and six like a cathedral with very bad but you’re far away from um yeah so and if you so yeah three is kind of the one that most people uh kind of tend somewhere between one and three depending on the piece of music um so we
Did test with kind of the same the same Acoustics but different pieces of music so on the same organ playing a more modern very kind of conceptual piece then people prefer zero or one and then if you play something very long very kind of traditional for an organ then it
Can work at six um and if you repeat the same thing for example with speech you know you don’t get it all the same so the the idea is just to think that the the notion of best Acoustics is very context dependent in the chat oh there
They said 3 three three then someone zero and then someone weia says six is a mess um and if you look at kind of big Gothic Cathedrals we’re up at like nine and 10 so notan is 10 to 12 as an example so it’s but nobody says UD the Acoustics is terribly sated
Public um so it’s yeah it’s just think in terms of the the context of the Acoustics and avoid this idea of what is the ideal reverberation time kind of the idea there we go um so how do study the Acoustics of a space and I think everybody here kind of already gets that
Idea so we can do modeling and numerical simulation so we can look at the impact of form and material uh on different spaces we can also do measurements uh inside spaces which uh the team here has done a lot of and that gives you a way to kind of
Quantify the Acoustics of the moment to kind of take a snapshot of it um but it also lets you calibrate your acoustic model so if I make an acoustic model of to today and then I do a measurement of today um then I have confidence in the
Model and then I can start playing with the model and change it to different parts uh periods in history you want me to just there we go I’ll let people in when um so with numerical simulations right we can look at kind of the impact of specific propagation effects look at
The impact of different kind of here it’s looking at reflectors here it’s looking at the impact of some diffracting elements up on the ceiling um and we can do simulations in 2D and we can do simulations in 3D um but what’s important of it’s not only the geometry
But we have to know what the materials are um and we have to know a lot of the times what the materials are constructed of so you can’t just say o this wall is wood because it could be 1 millimeter of plywood or it could be you know 10 cm of
Solid oak um and those aren’t going to have the same acoustic condition so it’s not just a visual inspection usually we need to know a bit more about what’s behind it um and how things are I don’t know that should kind of go away on
Yeah I think if I don’t touch it for a little while it’ll go away um oh okay it’s going for good no let’s see that’s fine so some of the examples that we’ve been doing recently uh to try and categorize the Acoustics of uh historic materials because a lot of the databases
Don’t really have those um was to go to a couple museums and textile archives and look at some textiles from the Middle Ages from the Bro period um so we did a series at the Lou uh with um in collaboration with Angelo freeness team
Uh in Italy uh we went to kind of the National Furniture storage um and got some curtains and this is a huge carpet that actually covers the whole choir section of not to them that only comes out every 500 years um and it luckily was being
Restored um during the fire uh so it kind of got saved to that because they found it had bugs in it so they took it out um so now it’s in a a big state of restoration because the hope is to bring it back uh for part of the opening uh
But in this giant hanger we were able to get access and and do some measurements on it as well um so how do we kind of experience uh these kind of room acoustic simulations um so this is kind of an approach that that we’ve been using so
We can have dry recordings of sound sources in ano rooms uh we have our computer simulation which is an offline calculation then we can convolve these two and create the acoustic uh impression we have a visual rendering uh which is usually more detailed than the acoustic rendering and then through some
Kind of real time usually game engines uh then we can actually link these two all together and then we’re allowed to move around and see and hear at different places in the environment but kind of before using all of these computer simulations in in a in scientific study I kind of have broken
Down into kind of three levels uh of confidence that I think we should have before actually using them in science so the first is can we trust the acoustic Fidelity of our simulations and that basically means if I measure something in reality and I measure something inside the computer simulation
Do I get the same thing and then I can look at it as more of an auditory level so if I give you uh two recordings one is made in the simulation and one is made in the real space do I hear the same thing and last one is can I make a
Convincing immersive environment which is really looking at human behavior so if I stick you into uh a historic place and I let you sing or speak and then I put you into the computer simulation do you behave the same way so that’s kind of the progression of how I think we need to
Have kind of very high quality simulations to actually do acoustic science with them as opposed to doing a museum exhibit or or something uh kind of for the general public am I going the wrong Way I need the thing back again that there we go I don’t know what I did no I think that brings up like a menu if I uh so a way to kind of test those different levels uh we kind of came up with a uh methodology for calibrating models and creating models we did a test on three different rooms uh so we have a kind of small Italian style opera house
Uh theater um a small Cathedral and then the large notam Cathedral we did measurements um all over uh in each of them created computer models um here we have the kind of the measurement hello can you hear us yes is that better now oh yeah yeah it’s it’s coming
Back sorry please continue so here you can see in the three different rooms uh kind of the the quality we’re able to get in terms of the simulations okay yeah um so in the center line you have the average of the measurements uh the little gray bars are the
Petual variance that we can notice I think or the standard deviation and then the ones with the dots are what we’re able to do with the computer setes so we’re able to kind of reproduce quite well the the local room Acoustics u in all of those spaces and then the next idea is
Actually to compare with a listening test so we can take the the measurements in the spaces the simulations in the spaces um and then have people listen to different Productions in each of those and judge for example um which one is more reverberant which one has a higher clarity
Um sense of distance tonal balance coloration plausibility uh Apparent Source width which is kind of how big a Sound Source sounds um and the sense of envelopment and doing this is a big blind study and comparing between the measured and the simulated we see that we’re pretty much
50/50 between all of these parameters so we can have confidence on these questions that the simulations are just as good uh as the reality which means we can now start modifying those historical simulations um here I can give an example we’ll see how well it sounds on
This I think this is the one that sounds not very loud so I to turn this up um so this is a concert recording inside notam uh the microphone sitting over the conductor’s position and you’ll hear kind of a 30 second extract uh of an orchestra performance and then we can
Switch to the simulation version with the microphones in the exact same place with all the same musician playing the exact same thing but in the virtual space um and the idea is to kind of see how similar do you find the two and let’s see if I can do This The people at home have it easier because they can because this is meant to be in voral but Hopefully I’m not sure if it comes across very well reconstruct it was better it sounded it can’t be better because that’s real yeah so it can be okay it can be clear clear yeah and that was one of the things that it end I think our impression of at least on that
One was that the the vocalist who’s singing two meters to the left of the conductor is has a higher Clarity in the simulation than in the in the real and then we worked on things afterwards that’s the general impression um there we go um so one of the things
That sorry may I ask you something sure sure uh when you did the yeah when you did the reconstruct um not the virtual uh thing uh did you first have like a dry recording and then or so uh so for that concert it was recorded by uh the national Conservatory with kind of
Semi close micing okay of the musicians in not so what you hear in the real one is actually just sitting at the choir’s position during the performance and then the virtual version is taking the microphone recordings at each of the instrument groups and then putting those
Into the rooms of so it’s not perfectly dry um but I think other than a little bleed through between neighboring microphones it works I think works pretty well um the worst is the kettle drum because when the kettle drum goes really loud all of the microphones pick it up um so then you
Get a a change a real change in the sense of how big the kettle drum is between the two um but for example all the singers uh I think the singers had were wearing head microphones um and I think each section of the orchestra had a microphone so there was like 45
Microphones um media commented that the sound um with the headphones that the sounds both of the examples were very similar that’s the goal um so one of the things that we notice is uh kind of what’s important is the directivity of the instruments and even even more so the directivity of the
Voice and that’s probably why it was more clear um so we’ve been doing some work on fine analysis of voice directivity uh so this is kind of an example in the 2 and a half thousand uh Herz region if you look at the directivity of the human voice as a
Function of vow or phonm um you get quite different patterns um so the idea is your simulation has to take that into account um so this kind of giving example between an A and an M uh and you see kind of the side radiation being
Quite stronger for an a than for an M uh but you can also find frequencies around 6 khz where all of the sounds are kind of collapsing to the same dur activity and there’s less uh differentiation so this is something that we’re looking at now uh how to integrate that into the
Simulations and also what is the perceptual limits of how fine we have to model that and also if we allow people to move so in those examples nothing is moving um but in current examples if I’m doing say a simulation in this environment every time I turn I want to
Correct for my voice directivity at the same time other things that we’ve been uh kind of paying attention to in in kind of reconstructing complex environments uh in addition to kind of movements is the background noise um because a lot of times when you do simulation you end up recreating this ideal environment
Where nothing else is happening which gives you kind of a loss of perspective of dynamic range and naturalness so if I wanted to recreate kind of this environment right now I want to make sure to have the water cooler um because that’s kind of giving you a real life
Reference of how loud things are and masking subtleties that you may hear in simulations uh but you wouldn’t hear naturally um so kind of an example that we have of that is uh on the theater project um was to to to film some actors
That we’re doing a bit of a piece uh so that’s using a depth camera uh so we have a not a high high resolution so it’s not kind of virtual avatars but we don’t have to worry about animation and we get all of the natural movements and
Gestures um and to some degree facial expressions um on the actress and then so they can move around and it’s all taken into account in one recording uh we can then extract the head orientation position through the entire piece and then actually have the voice uh simulation follow where that is in the
Model um and the first time we did that and we were working with a group of theater historians looking at how vocal usage and the room uh Acoustics had changed and they all complained that it felt like they were watching a rehearsal and not our performance and they
Couldn’t judge it in the right way uh so then we had to go and create a whole audience in our simulation um so we kind of went for something somewhat simplified uh that was easier to animate so it was kind of wooden mannequin um giving animations where they could you
Know scratch and cough and move around um and got into a lot of issues with how to randomize those kind of actions because if 20 people all of a sudden start scratching simultaneously it’s a really weird feeling um or or copy uh so we did a
Bunch of work on that um and the result is kind of this simulation which was I think it’s now 10 years ago um so I think in this extract it’ll kind of bounce between about three positions in the theater um while they’re performing on stage um and then what we and we also
Did kind of a scientific study on seeing how um The Voice directivity being included in the simulation impacted the perception um and the results were basically uh taking into account improved the plausibility of the rendering the sense of natural distance uh to the actors and the feeling of development when all that
Is correct um so this won’t show changes in it uh but it will show a bit when she’s walking around um it’s a bit dark and the resolution is not great but it was originally designed for an Oculus s DK2 um so everything is kind of tinted
That way and and simplified at that resolution of the time and we’ll see what Happens For spee Speee Again it doesn’t render very well over this but if you listen to it in Bor you can really get the sense of while she’s moving around and when she’s turning and talking uh that comes across quite well um so this was uh like 10 years ago so that’s a connect one
Um so we’ve been able to redo kind of similar things with the connect two and multiple connect 2 so you can get a much higher kind of graphic resolution uh but at the time we were trying to get the we were trying to get the theater historians to focus on the Acoustics not
On the graphics so it was kind of a a Middle Road oops going to do it again there we go um uh so kind of so with those studies which were mostly at Le proof of Concepts on modern day spaces uh once we have confidence in the the plausibility
And the realism of the simulations then we can start changing things and modifying uh the computer simulations to create lost acoustic conditions and what that lets us do is either reexamine hypothesis from social science um so for example there are hypothesis of what was the best Acoustics for Gregorian ch
At the time and what was the impact of some decoration and most of that research is coming out of literature kind of reading and reflecting but not a lot of kind of experimentation um so this kind of lets us redo those or do new experiments to answer those questions as well as to
Come up with new questions that sociologists had never even thought about because they could never imagine being able to listen to kind of these different environments um so if I give a couple examples of kind of interdisciplinary projects uh the first big one um is
Looking if you give me an idea of time I don’t remember what time we started I can go fast or slow I’m realizing I’ve been talking rather um you’ve been talking for about 30 minutes I guess that because we started late so okay yeah um so this is
A historical building in the science of Acoustics um so this is a a semicircle lecture hall that has domed arches everywhere um and kind of the end of the the 19th century the Acoustics was really bad uh the president of the University contacted the physical department and said you’re a young guy
Smart guy can you fix it um and what that Professor did was basically spent three years doing measurements in this room trying to understand what was happening and that was uh Wallace sing Who then invented room Acoustics as a science out of that study um sadly though he wasn’t able to fix the
Acoustics um and so over uh several decades the room was modified constantly in the hopes of trying to improve it uh so this is the original then they made it smaller U then they raised the podium and then they even added a kind of a canopy in
It um and then it was demolished in 1973 pretty much um and I ran into this old architect who had actually got in there the day when the bulldozer showed up and did some measurements in the room right before the building got wiped out
And the idea was to kind of see could we recreate the building that room Acoustics was invented in um using kind of these procedures so we can create a version of the the room at the time when they did the measurements we can look at their documentation all of the photos
They took and the measurements that they took to calibrate this room and then work backwards to arrive at the original room that saving was in um so it’s kind of a history of architecture and a history of science project and the idea was to make that available to
Acoustic uh students so you can try and figure out what happened what it would have been like what he was confronted with um and we found a lot of evidence of what he did he went to court because he got sued because the Acoustics wasn’t good um he then counter sued because it
Was because the architect who would let him do the changes he would make like that but this was kind of a an interesting historical statement uh the second one that we tried that on is the small Cathedral um and that was mostly looking at what is the Acoustics today and then
What was the Acoustics in the 16 17th century when it was still quite an active space um but at that time kind of this whole choir area was enclosed uh with stalls and much more richly decorated with carpets and tapestries and things like that and also
It was a it was a monaster it was a monaster is that right yeah an Abby um so basically the monks signed for themselves they didn’t care what other people heard when they came toing um and the result ended up if you if we looked at the computer simulations is that you
Have a much higher Clarity and a very easy time scene in that choir space um but everything that’s kind of public and there places they didn’t care about the Acoustics was quite bad and that’s quite a start difference to what it is now when it’s all open um so kind of a art
History architecture study uh this was the theater uh project the reason why we did those simulations and that was really looking at how the Acoustics of this one famous theater in Paris changed over about a hundred years because it got renovated about every 20 years um and also looking at because it’s the
Origins it’s at the start of recordings so they actually have recordings in the theater um not great recordings um but they can look at how the people’s use of voice changed over history um um and the question was are people singing differently and speaking differently because of the Acoustics or because of
The kind of other techniques and vocal usage and what was trending at the time so a more of a performing art study uh using simulations um this is the Palace of pad which was built uh the world fair before uh the Eiffel Tower kind of across the
The center from the Eiffel Tower it had absolutely horrendous Acoustics it’s this kind of big round space um and you’re kind of before the emergence of a of a solid understanding of how room Acoustics is working but there are quite a number of Notions and people inventing their own
Ideas uh so the architect here uh tried to understand it in terms of Echoes um and actually built this whole device uh as an echo detector so it could send sound at a very specific uh um directivity and would have people in the audience and you would raise your hand
If you heard the Echo and the idea was to try to identify the different uh surfaces that were important and then he tried to correct them they still didn’t work so as in the other building and uh the mid-30s they tore it down and built a new one um but through the simulations
We were able to not only simulate what the Acoustics was but also to reproduce his experimental tools and try to see really what was he finding and also what was he missing and why was his approach not working um and basically because he could only understand first order Reflections he couldn’t understand
Second order Reflections and almost all of the problems in this room are second and third order because of the curvature of the spaces um so it was kind of interesting to see uh to kind of understand why what they did didn’t work um this is a project more of a kind
Of a digital humanities project so this is a a group of students uh that work in uh digital reconstruction of Heritage spaces um and they’ve been the school has a link with the chat de ver and every year they do a different room and they try and recreated um and this year
Uh is a collaboration we got them to look at the Queen’s music chamber uh which was a room where chamber music was actually written for and performed in and the idea was to kind of give a reconstruction of that um and I can give you a quick example so all of the images
Reconstructed uh On Foreign So again that’s all everything we’re doing here is is is made for headphone listening uh but it’s just kind of show how the the simulation can have a context I have a question for the musicians you made the an recording and are they any way like fed the Acoustics
Of the place that they made they were fed un acoustic but not this one okay um they were actually recorded as part of another project okay um so we kind of created what would be an average ideal of go so they would have some Reverb um but it wasn’t on a specific space
Okay okay U but they weren’t playing dry because that doesn’t work very well I guess the muss also like especially for classical music they tend to adjust like how they play according to the place that they’re playing and what they yeah yeah exactly um and this was uh so this
Is a database that we made with four musicians um playing I mean like four very different styles from jazz to MH kind of this uh more chamber music um and there was a whole routine basically where they would play together um in the acoustic that we were
Giving them for I don’t know 15 20 minutes um that was recorded and then they played one by one hearing the recordings of the other three from the previous set um so that we could be sure to have a a perfectly kind of isolated recording of each of them MH
Um but yeah but that was that was not interactive with the The um so at the same time uh as part of that simulation uh we actually did studies with real musicians um in a virtual Acoustics and in the real Acoustics um and had them play um I think we had six to eight musicians playing historic instruments
Of the period Apparently one of them had actually already been actually been played at valide um this woman had a very old Baro instrument um and the idea was to recreate it in our kind of immersive environment and allow them uh to kind of play in both and to see what
The difference between this room was and a more 1960s uh uh chamber music Performance Hall of about 200 seats I think um and see how the playing style differ between those two um I think this is the last one that I have to present which is uh kind of
The biggest project recently uh which has been on notm um everybody knows it was a big fire roof fell in and things like that um originally we were involved uh with the idea that we didn’t know how they were going to restore it and we could use our computer
Simulations to help the Architects um evaluate a lot of proposals they were having there were a lot of really crazy proposals of how it would be renovated um and then president maon came out and said no we’re going to build it the way it was before which kind of took a lot
Of the getw work and scariness out of what the future would be so we were able to kind of turn our project upside down and say well let’s look more at the history um and if we kind of look at the history of not uh we have kind of this is the building
That was there before notam was there that was active and it was torn down to make uh this is how notam first opened uh while the construction was going on in the rest of the bid then the whole party opens and and then about 50 years
Later they they add all of these chapels on the side so you have this kind of evolution over I think 200 and something years of this building and it’s co iding with the Evolution of honic Music um which is also coming out of kind of the school of
Not that I’m singing which is also attached to the cathedral and so there were questions of what is the relationship between that singing style and the changing architecture um because it’s all happening at the same time so we could say is there an architecture for which
This polyic music sounds the best or is there an architecture for which it’s easiest to sing in and the idea was to kind of evaluate that um so we’ve done a number of studies with recording uh choirs that was a children’s choir we’ve also done two four male choirs and
Placing them in those different Acoustics and singing music uh kind of monophonic music so what was before polyic music then polyic music it was emerging and then something from I think the the 14th or 15th century to really have a variation and see how does their singing and interaction style change as
A function of the cuses um at the same time uh yeah I think we were also looking at uh intelligibility and how the changes in decoration and architecture uh affect the sermon intelligibility and how when we switch from Latin to French uh how does that become important um and one of
The other things is how does the Acoustics change depending where you are in the cathedral uh so this was a production that we did based on that same Orchestra recording um where you can we actually constructed the whole piece and you’re on a like a magic carpet and you can fly
Over the whole Orchestra uh and through the cathedral and here the idea was to convey how the Acoustics is a function of where you are uh during Cathedral [Applause] to Um so this just a a number of the different ways we’ve tried to kind of share this research mostly on the the notam with the public so there is that uh that one video that I just showed in excert up so it’s about five minute video of the piece is Ghost Orchestra
And that’s on a 360 YouTube video so you can hear it in binaural uh over the computer or over a smartphone um during covid uh for the anniversary of the fire we decided to release the whole length version of that concert um but more as a radio
Production uh so it’s a web radio um because we didn’t think people were going to sit with you know VR glasses on for an hour and a half um but you can actually listen to the entire concert at different places uh in the audience and even I think the conductor’s position uh
We had an installation at one of the science fairs um where you we just had basically I think four yeah a quad system set up with two microphones and basically just invited people to come in and you know sing in notes for them and see what it
Was like um Julia headed that one um and that was pretty well so for two three days uh that was up and running and it was from little kids coming to I mean some people who were I would say professional singer um that he one guy
Came looked like he was a rapper and just started singing like he was in the children choir and it was quite good um so that was quite a fun experience um we’ve been trying some things at more organized or kind of higher quality Productions uh so the first one was uh
The idea was to make a podcast uh radio fiction and we contracted a company and basically said uh you can create a story um that uses all of our scientific output and kind of integrate all our Acoustics and and musicological findings into a fiction uh that would be
Interesting to people uh so they came out with looking for not or which is a fictional story about Victor Hugo who’s having writer block while he’s writing his Cathedral novel and has interactions with people and B he’s visited by a muse who takes him back in time to experience different
Acoustic events um and and so that’s a a free oneh hour uh piece on Audible um and then right now we’re in the ends of a of an audio guide production uh that focuses more really on conveying the actual results of the research and some historical
Elements uh and here it’s more of a a GPS guided audio tour around the outside of notm and at each of the spots as you go around um you get some more information uh and some more historical elements I think the idea is 16 pieces 12 18 well 12 right there’s 12
Pieces plus six interviews with researchers um and that should be coming out uh in mid April um and so that’s free on uh Android and iOS um you can also you don’t have to be in open end to listen to it but it’s more fun if you’re
On site um because a lot of references to what you see uh from the different spaces uh positions um and let’s see uh so the podcast is in English and French the audio guide is in English French and Spanish there you go um and then the latest one is a production that we’re
Looking at for the end of the year kind of with the reopening of note for them uh very much music focused is 11 pieces of music that are kind of tracing the kind of the musical history of notam and the Acoustics of not to them where each piece of music is played in
The corresponding Acoustics uh over the course of an hour with a little inter um and that’s going to be kind of a web streaming and then also a virtual reality excerpt of about that’s it um that’s a number of sites if you want to get more information um I’m missing the one on
The audio guide which is Whispers ND PES years um but yeah so that should come out uh in mid April and then there’s some of the publication in case anybody had a question where all that’s coming from there you go thank you if you have questions or more questions lots of questions
Thoughts things that you want to do you can also make it an open t of how would you use this kind of ideas in your own research so I guess that the public in um or the the authorities in Paris for example with this notam are very
Interested in this type of research or how how have you found um the reception of this just um it’s it’s hard to say it’s it’s hard to know how to judge that um if I judge it on interviews um because we’re part of the uh the acoustic scientific committee
On the restoration um so we kind of get contacted by the press for um most of the press that calls us is not FR I’ve done more interviews uh in for the Germans for Spanish for Italians Canadians Swiss um I think at the moment the French are much more focused on the
Building and the Acoustics is kind of a a side thing uh very much hoping that the audio guide and once it’s no longer a question of is it going to fall down um and and what it will look like that there’ll be a lot more
Yeah that way I’m sure it it will um it will it will raise a lot of interest because um everything that has to do with um with the senses is very much um um raises a lot of interest by the general public so I would expect it to
Be I’m hoping and and the fact that we can we’re presenting not just the Acoustics today but kind of over eight n00 years and I think it’s the only real uh research examples that are able to do that I mean a lot of the the other visual reconstructions don’t do it in
The same way and once it reopens um I think there’ll be even less interest in how it was before um but the idea is that the audio guide should run for five years uh minimum it was see even the case for it afterwards um let me try to open because
Um there are some comments because we from using let’s see whether I can allow them to talk or you can read the comments or I can read you want me to stop sharing and put that up or yes perhaps um Eva S said than for very interesting presentation unfortunately I will have
To leave I have been Pi in a similar but a smaller project of medieval Swedish site of B sta ABI uh where we recreated the Church of 1470 uh and it’s soundscape uh yet there’s no much written in English but um I’d like to share some links uh with you the
Publication on the project will come um up this year on sketch uh fa and YouTube uh what you can see in these links are preliminary versions the results will be presented to your visitors in b as well uh and that was just and then Camille Copic um says
Sorry no Cameron I can’t really say my Nick right now thanks for a great presentation I’d like to ask if you published your results of the studies on Fidelity and immersive VR versus real thing yes um um no I think it’s this one I think Camille you might be able to it’s the
Second one there I think I don’t have we turned off the highlighter so I can’t write them so second one oh there we Go one of um so that was the the study actually comparing the perceptual difference between the simulated and and and the real says thanks a lot I’ll check it out and Eva has also the the links um but I don’t know how to share this with
Everybody perhaps if you put uh uh the uh the the the chat comments oh don’t do that okay that one that One yeah so you can see uh there um I’ll copy them I I have a question but it’s I don’t know like it’s I just want to know your opinion also like it’s like I guess you call like philosophical uh many of the important buildings or like what they consider as
Monuments right now they went through change over the centuries depending on the needs like of the century like they did extensions they Chang things sometime there were like war fire Etc but there always there was actually less less I I couldn’t call the necessity but interest in like keeping
Things as is but they kind of like evolved over time like and became what they are now and I mean I was I always thought that’s like they tried to keep the Acoustics of nraam the same I kind of like it was a nice thing for me especially working in the field of
Acoustics I said like okay like that’s it’s interesting that they have like importance to this and like someone working on this and they showing an effort which is nice because normally it’s just like discarded in like some other cases but also now like seeing your presentation I also like I don’t
Know suddenly get like we also like lost opportunity to like evolve and like go further and I mean like further is might not be the same word but like just do something that like more like depending our times but still looking like anthropologically back to like what it
Was and still make something like even like I don’t know like hurder out of it like rather than just like sticking no we’re just going to do it exactly as is which is sounds like a more nostalgic than the as as it was um and that was kind of
The the big part of of what our research was as it was includes a lot of things yeah I mean when they say as it was they mean the day before the fire yeah yeah this is what I thought so it’s also like it’s an opportunity to like maybe if we
Weren’t in a like such a rush to reconstruct it we could actually have like a like open conversation and it’s like so what do we do with this because most of the important buildings their construction went on through like centuries and it wasn’t just like something that okay like let’s do it as
Fast as possible but like it was like evolution of that building and then like I I find like now like I feel like they we missed an opportunity like to do something like they could have like propose something that okay like times have changed and also like this is open
Like a big monuments like we could do something while like looking at the like past it was but also looking at the future rather than like let’s build it like as it was yesterday yeah so we’ve been use I mean so so the idea was this whole study on how the Acoustics changed
Over the centuries was a way to start that discussion um the the way that the building is being reconstructed there’s the Architectural Stone elements M by the state and then gets handed back to the cathedral to you as a user um which means everything that’s decorations is in the later part so our
Idea was well if you’re not changing the stone and you’re not changing the form of the vaults this I mean there were proposals of a glass ceiling and things like that so the idea now is to get more into discussions with the the people who run the building and say well you know
This is how the the Acoustics has changed this is how it affects singing this is how it affects you know preaching where where do you want to be in kind of that Acoustics for the uses that you have now um so I’ve already had a couple
Discussions with uh the head of the the choir and the music and the cathedral kind of looking at that um the idea that before it reopens we want to get back in and do some measurements and already be able to compare before and after the fire um because we’re expecting it to be
More reverberant now than it was before with all of the the plaster and the painting being kind of cleaned and and redone um so then we can talk with and they’re also expecting much more visitors and much louder visitors because they’re all coming to see the rec construction they’re not coming to
Pre um and how to mitigate kind of all these external noises they’re going to be coming in um so those two things I think will be part of kind of this year but the later half of this year and then into the following year discussions on
What can we do with the Acoustics to help them with their uses um and I think it’s more on the they are more interested on the the preaching side but definitely the background noise side than the music side um because from their point of view uh when
Not to them was an active Cathedral you had acquire of 50 or 100 normally now they have 10 um so they said we don’t care about the Acoustics we want a new sound system um so there’s a big effort to rebuild the the sound system that they have with
Very directional speakers to kind of get the benefits of a good Acoustics but through Electro Acoustics um so it’s all phased arrays and things like that um so that’s their solution knowing that the second they leave the primary singing position it doesn’t work any point so any if all of
A sudden the singers start moving around or or at the organ side or are walking that whole system kind of falls apart and the issue is for them to come to grips and the realization of how it can and can’t be used um but yeah kind of the electro acoustic solution into that
Um the other is the choir organ so the the grand organ is a classified Monument it wasn’t damaged it was just Dusty so it’s being cleaned and then put back in so there’s nothing with that uh the choir organ uh is not a historic monument and it got totally destroyed by
Kind of water D and they’re very much looking at what do we want the new organ to be and to better represent the use of the um so uh before it’s in the choir choir and right historically kind of the the choir organ is there pointed
That way which is great when it was originally put there because it’s put there for the people that are here singing um so they have some ENC compan but here it sounds horrible so at the moment they actually have speakers that have miked that organ that they turned
It down and hopefully you can’t tell that it’s coming from that um but it’s totally poorly suited to the today’s use which is concerts and and the general public um so they’re looking at expanding the organ because it was quite small um almost doubling it in size and
Also thinking about where but they put it differently to better uh project into the rest of the hall um so we evaluated I think three proposals from uh the competition for the organ Builders and put those into the acoustic model to see how each of those would
Address kind of the new needs of of the cathedral so they are thinking in terms of that but uh but that’s on the Cathedral side not on the Reconstruction side um and it’s a it’s a very strange kind of divide between the two but I think those things will happen
Once the building gets handed back over to the cathedral yeah and you can go ahead and you can listen and then you can say oh we need to fix it and we’re at least hoping to be able to be there to kind of help them in that direction
Um uh I mean I’ve been thinking about variable Acoustics you know we could actually bring in you know the the 14th century acoustics with you know banners or things like that that could be very well done visually to represent what the historical kind of tapestries were at
The time and I think that would be more interesting then if they came in and carpeted the place which is what they did in the 90s when all of a sudden there was too much uh noise from the public they just put a carpet and there
There was a carpet in there that we can have measurements before and after the carpet and we can see what the effect is it’s a pretty ugly solution um and not at all kind of integrated into any kind of historical so I think that’s kind of the next phase is because we’re not
Changing the the form or the construction material that’s now into decorations and I think that’s where uh we’re hoping to be much more involved in the next it’s kind of a shame because because as you show the example of like also like there are some things that are very much depending on
The like also like arit part it’s much more difficult later on come and to take an acoustician and say like solve it yeah yeah and then we’re like too late um I mean I think in in at least in the context of not to Cathedral the the the acoustic
Difficulties you’d have to tear the building down I mean it’s not the things that make it the things that are unique in how the Acoustics moves in that space to fix it you’d have to rip out the tripon and that’s there’s no I mean that’s just crazy
Um so it’s really about the decorations to there’s no like focusing effects or weird domes or or things like that it’s the fact that the trans is huge and sound on one side doesn’t make it very well to across it I think had a question perhaps you can put your camera on hello
Everyone yeah I have aou of questions um thank you first for the this very nice presentation right yeah the first thing uh going at the very the beginning of your presentation I’m curious about the experience you have had with the you know this impedant scan to measure the textiles in the in the
Lra not a good experiment that’s why I’m asking that because I Tred to use that few years ago and it was very difficult to get reliable results yeah so we did I mean so uh we did a lot of measurements with it and then we did some comparative measurements between that system and
Kind of more traditional methods um uh so Julian was working on that so I mean we took a one of the velour curtains from the from the museum that they let us have for a couple weeks and measured it in our anaco room uh took it to a Testing Lab where they
Measured it in like an impedance tube umber Reverb chamber and a Reverb chamber so we were all the kind ofal measurements um and right now still kind of going through the data of how they how similar are they and if there’s issues with that impedance probe is
There any way to correct it after the fact and that’s where we are at the moment um at the moment I’m not very happy about it because there are things that we only measured with the impedence probe like in the Lou that is kind of a lost
Opportunity if we can’t recover the data um but at the moment it doesn’t it doesn’t seem easy uh and it seems quite unstable but I’m hoping that if we kind of put our heads down we can figure something out about it oh it would be great to heard about that when you
POS very depressing yeah I understand um a second question was um in one of the pictures um I saw you two recordings in the anake chamber of the um singers with a Celia microphone yeah am I right um it was one of the microphones yeah I mean they each
Had a had a headworn microphone so they had individual microphones plus we put a zilia in to kind of as a backup remember if the zelia was what was used maybe in the in the audio guide piece or I I was just curious how you integrate how you use this uh special uh
Recording uh in the the different ways to to get a better isolation of each singer um to have different tracks for each singer so we have individual mics but there’s some kind of cross talk oh I see them uh and we we thought the zil would be an you know the amasic
Mic is another alternative where you could do a beam forer each one uh but I think the headworn mics work so well that we haven’t use the Z microphone so now it’s it’s kind of I guess it’s more unused if you didn’t use it well we nice interesting and the last one for
My side um in the in the simulation uh where in which you integrate the is um voice directivity of the source M for the theat one um do you use a self-developed software to do that or you integrate that in a commercial software um so uh we published it a couple years
Ago so we we made a a plugin uh that lets you decompose a source into a number of individ of independent directions um so imagine instead of a like a if you have a do deedin Source if each speaker is perfect and only sends sound in its
Direction um then you do a numerical a computer simulation for each of those speakers and then in changing the the level of each of those speakers you can create a directivity pattern uh so we have a plugin that does that yeah so we’ve done it I think with
12 we did we did one the first example for the theater was with 12 different directions and then the latest version is 20 directions uh you can have quite an accurate reconstruction of the directivity pattern oh right so they yeah I the idea of using ambisonics is um uh similar to
The load speaker we have in the project the the mimo load speaker oh it’s not sharing anymore okay um but yeah so uh so if you look at the third reference yeah so there’s a we we presented that as a as a plugin and and
That I don’t know if we have a a website for it now that I think about it um but yeah that’s something that I have a look that we can share quite easily as it and it’s a a VST plugin uh that you use with
Right now we use it with basically a set of simulations that you run in cap uh kind of a predefined directional Source that’s optimized thank you very much oh yeah back to the so there’s something I I sometimes wonder about and I think that this is a perfect perfect space to ask this
Question so um I was wondering how much these sacred spaces like churches or um or the rocks or the caves uh were intentionally created with a specific form to enhance certain acoustic experiences or how much these experiences are like a side product of of the form so it’s like a chicken and
Egg problem right like it’s like would the people who like caveman where they like checking the cave and asking oh this cave looks nice but I don’t like the Acoustics because we cannot make our like event here or was it the other way around yeah I don’t know just just I always
Wonder how how much I mean on on our side uh if there’s no documentation of someone saying it uh it’s very hard to attribute intention um I mean there and they did I mean like with know them there’s a lot of writing about what Gothic architecture was meant to do and in
Cruus wasn’t one of them I mean it’s kind of a byproduct or analogies that if you really push it you can kind of Imagine um but the understanding of kind of how Acoustics worked um I don’t I have a hard time saying they did it because um rather than saying oh it was
Like this and look what they could do with it so that’s one of the interesting questions I think about the polyic music is did it come out because they all of a sudden had a performing space that it worked like if you tried to say sing the same way in the building was
Before was it harder and it’s only because of the new building that they said oh now we can actually hear each other in a way that we’re able to sing this new style um and things like that also evolve faster um I mean not so much in
Rockard but it’s a cathedral it takes 200 years from when they started building it to when it was mostly finished you know any kind of human cultural style evolves much faster than that so it’s much easier easier for me to imagine the culture adapting to the architecture than the other
Way I think cart it’s a bit different yeah um like kind of the similarity between I mean I know there studies on you know you know there’s great Acoustics every place there’s a big drawing well yeah but it’s also the big drawing is on the big place which is in
A big cavern that’s the Acoustics you get if you know rightly no ugly you know Cavern like that so is it is it more of a byproduct I mean and and we are in the worst position um you know people who are working in medieval or post medieval
Times because we don’t know the music we have no idea of the melody how we might know about some instruments but we don’t know how they were played or in what occation really so what we are looking is at patterns so even a place with a
Landscape with BR out uh we have places that have been painted and some others that haven’t um are the Acoustics telling us that they were selecting the places uh but what they were doing there we will never know we can say this place has very good ration of not we so we can
Say this place would be good for that for this particular reason but we cannot go much Beyond um that and we are looking for patterns so it’s not enough that the space is fantastic with or the that the reation is fantastic in a particular painting in a cave or no
Because one is not doesn’t tell me anything can be just you know by chance so I’m looking for patterns there but of course we are in a worse position than you know but you get go different places yeah yeah we have nice time going to
Places but yeah that that kind of said I mean there’s some things that I think are naturally at least you notice like an echo and a focusing and then you can say well you know were things chosen because they had actually do we have groups of let’s say architectural changes in stes by
Then yes they were leaving yeah no no they were modifications at time we are talking here about hunter gatherers and early agriculturalists so there they were not interested and in fact in most places um I would say that the Acoustics were not keen U for them
Uh and the reason for that I was explaining before and I’ve mentioned this in couple of talks is that um in fact what you want in a group when you in a small group um and you sing together you do things together it is that sort of community things that works
Really well in order to make U uh groups for um you know to create that sense of group um when you have so and we here we are talking about a small groups so Acoustics might be important in some cases but in most or in many uh cases is
Not um essential it could be important for communication purposes um because you don’t have the speakers so you really needs even if you know when Hunter GS uh get together in an aggregation size and you have a large group you really need to be able to communicate um what you are saying if
You have everybody there together so we are not talking here about 20 people but perhaps 200 or 100 and then needs to be a able to communicate um but once you go into State societies that you have large large numbers of people and you start and you start having
Constructions then I think that’s when Acoustics start to be important because you control emotion and once you have you are controlling emotion but you can control it in many different ways so there’s no one solution in the same way that acoustic is not the same in the
Church and hasn’t been I mean there have been changes throughout time but it’s not the same um in mosque or a synagogue or a church or a Buddhist temple everyone is using Acoustics in a different way and even that Chang is through out yeah I mean we were looking
At how how the situation changed when the the preaching shifted from Latin which is only spoken by the clergy to the local language and all of a sudden now it became important for those ex hundreds of people to actually understand what you’re saying well now you can’t have
The the preacher all the way at the end anymore in his little golden area you have to move into the middle of the people because the Acoustics doesn’t let it be intelligible if you’re too far away and you can kind of compare how how intelligible it was depending on where
You are and like that’s that’s why the podium is stuck where it is is so that you could actually hear said so you can imagine the same thing in caves where it’s you’re not necessarily looking for kind of religious spiritual Echo but you’re looking for I want to
Talk and I want everybody to be able to hear me so the choice of acoustic can also be very different from the I I never thought about this like before like like even if the ceremonies like took place especially in like semiopen placees if the ceron took place during
The night or during the day like different Acoustics effect on The fact is that we don’t know yeah exactly Start theic the oldest instruments that we have um are dated in the upper pic but perhaps josa want you want to explain about this you might know more I mean the oldest um I will say this and then can um uh add uh the oldest musical instruments are the fluts that have been
Found in Germany in tapes there might be others but um uh you know research in Europe is much more uh developed than in other parts of the world and because it was very cold people were doing many things in caves so preservation is better okay uh but uh working in human
Evolution say that music came probably at the same time that language and the this the evolution is with hormone so there it can be very old uh but we don’t have any evidence um because it’s bones and so on but they are parts of the uh
Brain um that uh so this broker area that is reflected in the cranium in the skull and then um um says that language is very old in hor but you know it’s sort of secondary evidence of what we have musical instruments um per is 40,000 in this B is and what’s the name
Of the other one something something in German but that’s limited but that’s bone because bone survived right I was but if it was a tree Trum that you made a drum out of yeah exactly there be no trace of it anymore so it’s I mean vocal music because at the end of
The day the voice is an instrument but so it could have started much much earlier and that’s what people working in human evolution say that language and and music may come at the same time that we’re sort of parallel some people say first language uh other one says first mus
Probably at the same time um Danny has a question Danny can you check if you can activate audio Yeah I think I think I can right yeah great so perhaps you can put the camera on if you want to go ahead uh no I actually I
Can’t uh I don’t know why but I I’m not allowed to like use my camera Okay but well in any case thank you Ryan for such an inspiring presentation and I would like to know um in regards to the ER the simulations at Notre Dam uh
How do you actually do it because I assume that I mean the performers are the same but when you record in the cathedral are you I mean did you guys uh do it in mono in stereo with a particular microphone configuration and then you rep replicated in the ano chamber I mean had
You uh managed to preserve like the right or the same configuration in such different places uh so for the the orchestra performance uh is that I think that’s the one that you mean um that had uh close mics on all the music sections over the entire Cathedral so there’s 45
Microphones spread out everywhere capturing loly what each one is is doing and then in the computer simulations uh for each of those recordings we create an acoustic Source in the model at the right position with a directivity of that instrument section okay but then when you recorded the the
Because I guess the audio was recorded as well right in aamber so then it’s the audio then gets fed into the impulse response of of that musical instrument at that position in the model all right and when you record in the an chamber you use like approximately the
Same chamber is a it’s kind of a different use that’s looking really at um recreating the Acoustics for the musicians a coic room while they’re playing um so for that for example with the the four singers uh we do a simulation in notam of the four singers where they would be
Standing with the directivity of uh The Voice uh for each of them we simulate the acoustic conditions of of each of them separately which is then used with the the real feed from their microphones that’s then played back to them over headphones okay and the the result uh in both cases
Is in estereo or IM mon um we do mostly bormal okay um but we’ve also it could also be in we usually do either ambisonics or binaural okay I see if uh like for the singers um because there’s four of them all simultaneously it’s kind of too computationally
Intensive uh to do ambisonics for each of them um and also they don’t move that much so head rotation from the singers isn’t that important instance it’s in binaural uh in the orchestra recording where you’re flying around it’s all in third order ambisonics that again then gets transformed to
Binaural when you listen to it okay as a function of like on the the player like the the YouTube player does its own conversion from ambisonics to voral you’re looking so those are kind of the two extremes but the to do something in real time uh we’ve only
Really done the ambisonics when it’s one musician uh not sure for all right and it was done I I assume no before the the fire right yeah yeah the the concert was recorded I think in 2013 okay uh it was for the 850 anniversary no to them concert all
Right and just out of curiosity where was the like the conductor’s position or location in the cathedral just off of the trans Set uh um if you if you look online and and you look for the ghost Orchestra notam uh we have a whole web
Page that kind of breaks down uh all the technical aspects of how how many weeks the simulations took uh there’s a there there’s a good presentation on that okay all right I’ll check it out thank you so much thank you I have question you said you put like some inside of virual environment
To and I’m curious about what was their feedback as musicians play EnV and um see there was any due to the fact that they were exposed to these visuals in of them don’t change the or turn off visual no they had nothing oh in the the veril one yes
Um the vers one was one of the first interactive ones that we did um and while we usually like to I prefer to use headphones um because I you get a better quality um and less chance of feedback um some of the instruments don’t work with headphones the same way we couldn’t use
Virtual Reality glasses because they couldn’t see the sheet music anymore um so we went for a speaker installation and the projection uh and the biggest problem we had was that uh we ended up getting some feedback between the the recording microphone and the reverberation MH uh for some of the
Instruments basically we tested it with voice and we sang and it was fine and then the flu came in and went and got a very bad feedback so we had to turn it down because of the screen no because it’s the speaker the speakers on the microphone and we had it
Just at the limit of where it was okay and then because you had to mic the the flute closer and because it plays so much louder it it made a feedback um so they started complaining about the coloration of the system and it took us a little while to figure out what that
Was um but I think the the other instruments that that weren’t in that situation uh said it sounded very natural and comparable to how it had been um we didn’t get to the next phase which was starting to swap Acoustics and visual because that was that’s the kind
Of the advantage of the virtual space is I can give you the Acoustics of the verside and the visuals of the modern conert Hall and back and forth um so I think that’s still in in a next stage to see how much are the musicians basing their impression on what they
See we need to leave this room in 15 minutes um so I don’t know whether you have any other questions been quiet over there um right Rael was saying Rael could you see whether you can now use your mic I put up a question yes exactly that’s why
I’m saying if ra is able to do it that um right now perhaps she’s in a place where she says thank you very much um thank you for the presentation unfortunately I can’t use my NE as I in the share office here my background in Psychology my my background is in
Psychologist I’m intrigued by the potential application of your research in my field uh I recall attending a congress on sounds in intensive care units where there was a discussion about making machine related sounds less traumatic for the hospitalized individuals by transforming these sounds into more soothing
Ones um like piano keys you you you research or your research somehow reminded me of this I can imagine that when confined to spaces it must be beneficial to immerse oneself even if virtually in the environments of cultural significance and uh significance and openness or at least
The spaces that are more open than hospital rooms do you know of any applications of your research in this context if not uh I think it would be an interesting thing to test uh well that covers a lot of things doesn’t it um I mean the the equipment that’s all
Beeping isn’t for you it’s for the medical staff right it’s not for the patient to hear the beeping it’s for the nurses and the doctors to hear the be yeah but the they should put so I mean already just to be able to wear headphones and go somewhere else I mean
I know there’s been work on redesigning all of the sorry we we we should talk yes I know sure we’ll figure something we’ll figure something out thank you for thanks for coming I want interrup um I mean I know there was already like a big push to redesign the instruments because I think
They did a study that the the loudest annoying instrument was the least significant thing that we’re measuring at the moment um and that they there should be some Readjustment to what’s important and how loud and how distracting all of the different instruments are um I we did go on a competition with an
Architect uh to make a relaxing space in a hospital um so there would be a corner where basically patients who can’t leave um can go and pretend to be like on a beach or or in a in a very open space using that kind of virtual reconstruction so there’s definitely in
Terms of relaxation there’s there’s a way to do that right um I think that um only I mean I’m more interested in like technical stuff like um maybe more more of the also the visual side like you mention game that you do for simulation I’m thinking about the possibili of a
Work you know like that have you thought about maybe how would you be able to for I for like more like real time applications instead of simulations and all that uh I mean right now uh I mean we’re using Unity mostly uh we used to be using
Blend we used to use Blender Game engine but it it doesn’t exist anymore um so you’ve been using unity and communicating between unity and an audio engine an external audio engine so max MSP or something like that um when we want high quality rendering um so then you have kind of
The two running in parallel and they can talk back and forth um if you have to do if you have to do it self-contained in the game engine uh then Things become much more limited yeah you need to pick I mean there’s a number of different middleware
It kind of depends what really you want to do um we’ve been doing a lot on the the quest now um so trying to find a way to get kind of ambisonic audio into a quest we haven’t used the microphone yet um so it’s not interactive
Um but not using wise but using some other ways um so we have third order ambisonic simulation well playback so basically you create a movie and then you can play it back um but there you can use third order aonics on the quest with the head tracking build in and
They’ve had pretty good success with that so that’s quite doable um but I don’t I haven’t we haven’t at all tried getting a microphone involved in in that situation and those we really it’s better to have something that’s high performing and dedicated yeah than you there’s no reason
To try to integrate audio into the game G engine when you’re running on a desktop machine when you can actually have higher performing audio rendering engine and the communication is pretty simple yeah can imagine I’ll say pretty simple the audio engines and all that always like compliment to to the visual usually
Like industry ging is more like always I mean I I think there are the tools that are in there um and and wise let you do some things and de let you do some other but they’re kind of limited in ways that that I we prefer to have kind of our own
Tools and know what it’s doing and be able to to update and modify quite quickly and so it’s just easier the game manage says you’re here you’re looking that way and you’re in state three and then the audio engine just mates them I think we should probably stop here we
Been almost two hour two hours here so I guess it’s time so thank you very much for this fantastic talk and this fantastic discussion and so anyway so let’s see where how how all these takes um us in the future so thank you very much