Cross-lingual prosody evaluation through transfer learning

access_time 07 de fevereiro de 2019 às 11:00 até 07 de fevereiro de 2019 às 13:30
place INESC-ID, Room 336

The importance of automatic prosody assessment has been acknowledged, as it provides relevant information about the speaker, the languages, the pragmatics and paralinguistics of speech. Nevertheless, available data sets are often insufficient for the tasks aimed at, namely when it comes to the usage of DNNs, which require great amounts of data. Transfer learning is a state-of-the-art technique being used for several tasks and proven to be very informative, as training with diverse data sets and testing on distinct ones can assure robustness and cross-lingual analysis. In this case, we investigate whether transfer learning can be applied to L2 learners, either exclusively in learning contexts or in e-health ones too, which is a very challenging task.The main data set we use was built for an intonation imitation task with native speakers of Portuguese, whose assessment relied on a DTW algorithm only. Building upon previous work, a nativeness assessment task of L2 speakers of English, we applied a DNN for feature extraction. We have considered features with no temporal dependency and features with temporal dependency, corresponding to an LSTM layer.

local_offer Tópicos de Investigação
person Candidato: Mariana Dimas Julião Nº 94140
supervisor_account Orientador: Prof. Alberto Abad Gareta / Dra. Helena Moniz