Monday, 7 August 2017

A revolution in speech animation?

I have always been fascinated by animated characters.  We know there is more to speech than simply words.  Facial expression adds significantly to our understanding.  As a deaf person I also know too well how precise movement of the lips and face help in my understanding of the spoken word. 

Forming speech is complex. About a hundred different muscles in the chest, neck, jaw, tongue, and lips must work together in forming speech. Every word or short phrase that is physically spoken is followed by its own unique arrangement of muscle movements.   No wonder then that animations can often appear flat and characterless. 

New research from the University of East Anglia (UK) could revolutionise the way that animated characters deliver their lines.

Animating the speech of characters such as Elsa and Mowgli has been both time-consuming and costly. But now computer programmers have identified a way of creating natural-looking animated speech that can be generated in real-time as voice actors deliver their lines.

The discovery was unveiled in Los Angeles at the world’s largest computer graphics conference - Siggraph 2017. This work is a collaboration which includes UEA, Caltech and Carnegie Mellon University

Researchers show how a ‘deep learning’ approach – using artificial neural networks – can generate natural-looking real-time animated speech.

As well as automatically generating lip sync for English speaking actors, the new software also animates singing and can be adapted for foreign languages. The online video games industry could also benefit from the research – with characters delivering their lines on-the-fly with much more realism than is currently possible – and it could also be it can be used to animate avatars in virtual reality.

A central focus for the work has been to develop software which can be seamlessly integrated into existing production pipelines, and which is easy to edit.

Lead researcher Dr Sarah Taylor, from UEA’s School of Computing Sciences, said: “Realistic speech animation is essential for effective character animation. Done badly, it can be distracting and lead to a box office flop.

“Doing it well however is both time consuming and costly as it has to be manually produced by a skilled animator. Our goal is to automatically generate production-quality animated speech for any style of character, given only audio speech as an input.”

The team’s approach involves ‘training’ a computer to take spoken words from a voice actor, predict the mouth shape needed, and animate a character to lip sync the speech.

This is done by first recording audio and video of a reference speaker reciting a collection of more than 2500 phonetically diverse sentences. Their face is tracked to create a ‘reference face’ animation model.

The audio is then transcribed into speech sounds using off-the-shelf speech recognition software.

This collected information can then be used to generate a model that is able to animate the reference face from a frame-by-frame sequence of phonemes. This animation can then be transferred to a CG character in real-time.

‘Training’ the model takes just a couple of hours. Dr Taylor said: “What we are doing is translating audio speech into a phonetic representation, and then into realistic animated speech.”

The method has so far been tested against sentences from a range of different speakers. The research team also undertook a subjective evaluation in which viewers rated how natural the animated speech looked.

Dr Taylor said: “Our approach only requires off-the-shelf speech recognition software, which automatically converts any spoken audio into the corresponding phonetic description. Our automatic speech animation therefore works for any input speaker, for any style of speech and can even work in other languages.

“Our results so far show that our approach achieves state-of-the-art performance in visual speech animation. The real beauty is that it is very straightforward to use, and easy to edit and stylise the animation using standard production editing software.”

Thursday, 3 August 2017

Could disability awareness end supernatural myth-making in Africa?

Many people in rural African communities still believe that disability is caused by supernatural forces, curses and as ‘punishment’ for wrongdoings – according to University of East Anglia research (UEA, UK). 



The resultant stigma leaves disabled people vulnerable to neglect and abuse - with sexual abuse reported by 90 per cent of people with learning difficulties.

Many disabled children are kept ‘locked up’ at home – often for their own safety. But the more that communities come into contact with disability, the more awareness and understanding grows.

Meanwhile medical explanations for disability are beginning to emerge, with increasing numbers of families seeking medical advice for children with disabilities rather than consulting a witchdoctor.

The ‘Preparation of Communities: Using personal narratives to affect attitudes to disability in Kilifi, Kenya (Pre-Call)’ project was set up to promote disability awareness in small communities in a rural part of Kenya, by encouraging a process of reflection and education.

The research team at the Kenya Medical Research Institute (KEMRI) led focus groups to find out how cultural beliefs and knowledge shape people’s understanding of disability.

The discussions involved 21 community groups located across the five constituencies of Kilifi County, bordered one side by the Indian Ocean coast and stretching into the rural interior. A total of 263 participants were involved who observed Christianity (70 per cent), traditional religious practices (20 per cent) and Islam (10 per cent).

Lead researcher Dr Karen Bunning, from UEA’s School of Health Sciences, said: “Information on the medical causes of childhood disability are not widely available across communities in low-income countries and understanding is generally poor.

“In Namibia for example albinism is explained by the mother having sex with a white man or a ghost. And in Guinea-Bissau, epilepsy is widely thought of as being caused by evil spirits, or sometimes as a punishment for wrongdoing.

“We found that disability is often explained by things like extra marital affairs invoking a curse, witchcraft, supernatural forces such as demons or ghosts affecting the child, and the will of God.

“Curvature of the spine or limbs represented the effects of a curse, saliva production was linked to demons and ill-gotten financial gain.

“The different explanations represent a real mixture of traditional, religious and biomedical beliefs,” she added. “And while biomedical factors such as inherited conditions or antenatal care were increasingly talked about, these explanations did not negate other culturally-based accounts.”

The findings reveal that underpinning all of these explanations is a desire to make sense of disability and, particularly for carers, to improve the given situation. And where an explanation of wrongdoing or the presence of an evil force might result in a visit to a local witchdoctor, a medical attribution might be followed with a visit to a medical centre.

The project also looked at the challenges faced by people with disabilities and their carers.

Dr Bunning said: “What tends to happen is that these types of cultural beliefs affect how individuals with disabilities view themselves and how other people see them. Attributing the child’s condition to some form of malevolent preternatural force by reference to demons, evil spirits and witchcraft contributes to the view of disability as both undesirable and unacceptable.

“People with disabilities in Africa have poor access to health provision, low school attendance, limited employment rates and low wages. More extreme consequences include neglect and abuse – with sexual abuse reported to occur at some time in the lives of 90 per cent of the population with learning difficulties. People with communication difficulties are at a high level of risk because they are less able to report abuse.

“We found that children with disabilities are often kept apart from the local community in restricted environments – contributing to the social distance between them. Although in many cases this is to protect them from abuse and keep them safe.

“The burden of caring for family members with disabilities also led to discrimination by association,” she added. “And the stigma associated with people with disabilities is so great that it also extends to anyone trying to help. The implication being that anyone offering help would also give birth to a disabled child.

“But we found that the more people come into contact with disability, the better their understanding. Real life encounters with people who have disabilities can be a really positive step, so raising disability awareness in small communities can really help.”

‘The perception of disability by community groups: Stories of local understanding, beliefs and challenges in a rural part of Kenya’ is published in PLOS ONE on August 3, 2017 . The research was led by UEA (UK) in collaboration with the Kenya Medical Research Institute (Kenya), the University of Oxford (UK) and the London School of Hygiene and Tropical Medicine (UK).