ABC classic FM has recorded the live performance of the Interior Design.
You can download the podcasts of the 6 pieces here:
A few different ways of ‘listening in’ on the brain as it’s processing sound have appeared recently, both using EEG technology (electrodes measuring brain electrical activity, either from the scalp or inside the brain itself).
Firstly there is the ‘frequency following response,’ which is generated in the brainstem. The brainstem is a ‘low level’ part of the brain, responsible for sending signals from sensory nerves (from the ears, skin etc) to cortical areas for further processing. However, some types of auditory processing fo occur in the brainstem – things like direction, basic pitch processing, and basic loudness processing occur in the brainstem (although pitch and loudness also require cortical input). Anyhow, the frequency following response (FFR) is an electric signal generated in the brainstem by repeated short sounds (like a word or short musical note). In order to measure this response, it’s necessary to present 1000’s of repetitions of exactly the same sound, and average the recordings. By specially analysing the resulting electrical waveforms, it’s possible to determine the pitch, timbre, and other features of the original sound that was heard by the participant in the experiment. In fact, if you play back the FFR recording of a word over a loudspeaker, you can actually understand the word!
This review paper by Nina Kraus in Physics Today includes some auditory examples (scroll to the bottom of the article).
http://dx.doi.org/10.1063/1.3603917
I’m not sure how I forgot to post about this. Late last year a producer from SBS’s pay-TV arts channel “STVDIO” came and shot some footage around the institute. He was making a documentary on all of the Synapse residency artists. As well as the project here at the Institute, I think there were around 5-6 other projects where artists of all types were made ‘residents’ at various scientific institutions.
Check out some of the other projects on the STVDIO site here:
http://www.stvdio.com.au/arts-news/art-meets-science-in-the-australia-council-synapse/
Or watch the movie about us below (if my embedded movie works).
Well, it’s been a long time between posts! One of my New Years resolutions was to try and be more consistent with posting here, as it’s a great link between the volunteers who contribute their time and efforts into doing research with us, and the final results.
So, here are a few ‘final results’ from the data we’ve collected over the last year or so. If you’ve been a research participant here at the Bionics Institute over the last two years, the result of your listening and button presses will be reflected in one of the little dots on a graph in one the papers below! At the moment we’re looking for more volunteers to participate in our research – if you’re interested, get in touch by emailing music@bionicsinstitute.org.
Overall, our research so far has focussed on the ability to hear multiple ‘streams’ of sound. We believe that, in addition to accurately hearing pitch, loudness, and other low-level auditory cues, the enjoyment of music is largely affected by the ability to hear the different streams in music. This could take the form of different instruments playing in an ensemble, or even of different lines of melody played by the same instrument. In normal hearing, the ability to hear these different streams is based on different types of acoustic differences between the streams (loudness, pitch, instrument timbre etc). The brain can then use these cues provided by the ears and auditory system to disentangle the different streams of sound. When using a cochlear implant or hearing aid, however, the basic perception of these cues is changed, and so therefore is the perception of different ‘streams.’
Our research so far has been based on understanding how different hearing devices effect these auditory ‘streaming cues.’ Our first published study found that visual cues also affected the ability to hear separate streams of sound. When a visual cue was provided, most participants found it easier to hear two mixed-up streams of sounds. In addition, we found that no special training was needed in order to interpret the visual cues (although musically-trained people were better at the tasks overall). This research was published in ‘PLoS ONE’, an international general science journal, and can be downloaded free of charge at this address (look for the ‘PDF’ link on the right hand side):
http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0011297
Our next paper was also focussed on stream segregation, and found that visual cues were also able to make the streaming task easier for people using cochlear implants. In future, it may thus be possible to devise ways to make music more enjoyable using visual cues, either in a live setting, pre-recorded music with pre-made visual aids. Listening to music live, where visual cues can be provided by the musicians themselves, may also be more enjoyable than listening to the radio or CDs at home. This research was also published in PLoS ONE, and can be downloaded here:
http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0029327
We have also been analysing further data from the same experiment that compares the different types of acoustic cues against each other, to determine which cues have the greatest effect, especially for people using cochlear implants and hearing aids, though that data is not published yet. We just received notification that the paper we wrote has been accepted into a journal called ‘Music Perception’ though, so that should be available soon.
We are also just about to publish the results from a large concert that was held last February at the Arts Centre Melbourne (more info here: https://musicalbionics.wordpress.com/interiordesign/) . In that event, six new works were composed especially for CI users, and survey data was collected after each of the six pieces. So far, the results are very promising – there were big differences between the appreciation of each piece, and between each survey item, but on most of the survey items there was no difference between the responses of the CI users and normally-hearing listeners in the audience. So although some pieces were liked more than others, both normally-hearing and CI users had similar responses!
In other projects, Dr Tom Francart, a visiting scientist from Belgium, is also developing a system specifically designed for a cochlear implant combined with a hearing aid in the other ear, to improve sound quality, and improve the ability to localise sound sources. The system also improves loudness relations between different sounds, to make soft sounds better audible and loud sounds more comfortable. Tom is working with Cochlear to implement the results of his research in real devices. Examples of other results from research at the Institute and partners are the SmartSound Beam function to reduce background noise and ADRO.
Mohammad Marefaand, a new PhD student in our team, is also working on improving music perception for people using a hearing aid and cochlear implant together. His approach will be to discover the essential elements of music that are necessary to transmit to hearing devices in order to best enjoy music. Watch this space!
PS, here is our new sound treatment in the booth! This is where some of the testing will be held, looks cool hey!
Tonight a few interesting sounds from around the world.
Irish accents AND foghorns, all in one place! This delightful program from RTE radio in Ireland documents the disappearance of the sound of foghorns from around the Irish coastline. The best bit is when the foghorn afficianados put together a fund-raising night called “blast from the past”… Ho ho ho.
http://www.rte.ie/radio1/doconone/the-curious-ear-doconone-foghorns.html
A soundmap of the Parvia River in Portugal (I’m really interested in making one of these soundmaps one day!) It would be good to be able to see a picture of where each sound was recorded I think.
http://www.mapize.com/soundmap_paiva/
Music made from recordings of cargo ships bumping and grinding on the River Waal in the Netherlands, from Esther Venrooy
There’s a sample here: http://www.entracte.co.uk/project/esther-venrooy-e104/
http://www.esthervenrooy.net/music/releases/
And if I were in the UK, I’d be going along to one of these gigs at Tryptic, a whole festival of Eliane Radigue in London.
The festival: http://soundandmusic.org/projects/triptych-music-eliane-radigue
In this video she mentions how the kinds of sounds she uses inn her work can act as a kind of mental mirror… so true. The venues look amazing too.
The Melbourne International Jazz Festival has some really interesting stuff this year.
First of all, the Cave late nightclub venue this year is the KELVIN CLUB. The Kelvin Club is presumably named after Lord Kelvin, who devised a reference-free scale for temperature or heat energy based on previous ideas that there could be a hypothetical “absolute zero” temperature, at which the vibrations of molecules which give an object heat energy stop. There’s a review on the Kelvin Club from 2009 on a site called “Mel: Hot or Not,” which deems the Kelvin Club to be “Hot.” Anyway, I’ve always been a little bit fascinated by the Kelvin Club – it has a big shiny painted door deep down a dingy alley off Russel Street, and is one of those members-only clubs that I imagine are full of stuffed couches and paintings with huge gold-leafed frames. And this year, it’s going to be full of free music!
Secondly, there are going to be evening sound-walks around Melbourne hosted by the Australian Forum for Acoustic Ecology. If you’ve never been on a soundwalk, I highly recommend going, especially if you have a hearing impairment or use a hearing aid or cochlear implant! I don’t have a hearing impairment, but I’ve found that going on sound-walks, especially when they’re guided by someone who knows some good locations, can really change how you listen to the world of sound around you, and can re-define what you consider to be ‘noise.’ It’s also a perfect opportunity to be taken around on a free tour of your city 🙂
Sound walks (click picture for details):
A quick post as I’m on my “PhD day”, which means I’m not supposed to be doing anything work-related, but I thought this was an interesting read:
http://funnyoldlife.wordpress.com/2011/04/16/bouncing-into-a-hearing-life/
Especially interesting (and relevant if you’re using a CI) is the part about the mapping process, and adjusting sounds to “comfortable” levels rather than painful levels.
Secondly there is an interesting snippet about some research being done in the UK where it seems they are looking into how pitch perception can affect speech intelligibility scores.
Always interesting reading! Now back to PhD work. Bye till Monday!
Excitement builds! Our concert project is in the (very) short list of projects in the FutureEverything Festival Award! The FutureEverything Festival is a big conglomeration of Art, Music and Ideas held for the last 15 years in Manchester in the UK.
Fingers Crossed!
“INTERIOR DESIGN: Music for the Bionic Ear” jumped out at me as the obvious choice for my second year research project: as a musician and student audiologist, it combined my two interests perfectly. So what if there was some data entry required? How much could there possibly be?
Famous last words.
So many surveys...
Faced with the daunting task of ploughing through over 400 feedback surveys, I recalled that sage advice from Alice’s Adventures in Wonderland: “Begin at the beginning, and go on till you come to the end: then stop.” So, beginning at the beginning, over the course of a few weeks, I slowly made my way through the box of surveys from the first concert of the evening, tackling along the way such obstacles as ‘unique’ handwriting (to put it kindly), and the dreaded ‘ambiguously circled number’. Having reached the halfway point, it seemed an appropriate place to take a break and give an overview on what we’ve found so far.
I’ve enjoyed reading the comments along the way: whether positive, negative, confused, angry or delighted, it’s exciting that the majority of the audience seems to have been able to hear enough of the music to be able to form some opinion about it. Some interesting trends have emerged from the data so far. When comparing data from cochlear implant users and normal hearing listeners, I’ve been pleasantly surprised to find that their responses tend to mirror each other—indeed, for some of the pieces, the responses almost overlay one another, suggesting there was little difference in the way the two groups of listeners interpreted and enjoyed the pieces of music.
This is encouraging because one of the main goals is to create a meaningful dialogue between hearing-impaired and normal hearing listeners by introducing new musical ideas accessible to both groups. Whatever their reaction, good or bad, it means they’re listening to—and, more importantly, hearing—the same piece of music.
I thought I’d end with some nice comments I came across today, from two cochlear implant users:
“I am very keen to take up music again. Listening to the concert has shown me that I should experiment with what instruments I decide to pick up. This has made me realise there is more to music than strictly classical – which is what I grew up with!”
“Thank you so much for all your time and dedication and for all your work to make the most of our hearing. I am very touched by it all and today was fascinating and vastly enjoyable.”
First box done!
Photos courtesy of Hamish
It’s been over a month now since the concert, time to write a bit about how it went!
Overall the response has been quite positive. We had about 700 people turn up on the night (in two shows, one at 5:30 and one at 8:00, and one lecture at 7:00). We have had reviews in The Age (click here and here), and we even made the 7 o’clock news that night (click here)!
We have had quite a bit of feedback, some overwhelmingly positive, some bewildered, and some negative. Even the negative feedback is interesting though, as so far it has all been a question of taste, rather than of being able to “hear” the work. It has certainly got hearing impaired and normal-hearing people talking, which was one of the main aims of the project.
In the meantime, we have two large boxes of feedback forms to get though. These forms contain answers to a standard set of questions we asked about each piece, and will ultimately help us decide what kinds of musical sounds are well and not-so-well transmitted by the cochlear implant. A new audiology student has just embarked on the task of entering all the data and corralling it into shape, and we will then work with Kate Stevens at the University of Western Sydney and Emery Schubert at the University of NSW to analyse and interpret the data. We’ll try and keep you posted about how that data looks as we start to look at it.
Here are some pictures of the night (taken by Anne-Sophie Poirier):
Hi from the Arts Centre! We’ve been here since 8 bumping in the show tonight. Two shows, 5:30 and 8pm, with a lecture explaining the processes behind the show at 7 pm. Come along, there are still tickets left but they’re selling fast!
Here’s Jeremy putting the final touches on his lecture slides…
Music in cochlear implants Mulitsensory integration in cochlear implants
ness
hamishib
mazoro
Hearing Organised Sound 