Adversarial Machine Learning Against Voice Assistant Systems

WINLAB Summer Internship 2022

Group Members: Matt Kokolus, Catherine Mathews, Raymond Huang, David Man

Project Objective

This project aims to study the security of voice assistant systems under adversarial machine learning. Adversarial learning algorithms can generate adversarial audio samples to serve as the input of voice assistant systems, so as to fool the machine learning models in the system. In this project, students will focus on the white-box attack in the digital domain by generating adversarial samples using adversarial machine learning algorithms to attack a speaker recognition system based on X-Vector. The students will learn Python with Tensorflow Library.

Please see "Final Research Poster" attachment at the bottom of this page for more information.

Week 1

• Defined project goals
• Created plan of action for future weeks

Week 2

• Read and studied paper: 'Practical Adversarial Attacks Against Speaker Recognition Systems'
• Changed project objectives to focus on automating drone flight and testing with the ultimate goal of automating drone flight to potentially fly with voice commands
• Became familiar with Yuneec Tornado H920 Pro Drone

Week 3

• Began Udemy course: 'Complete Guide to Tensorflow for Deep Learning with Python'
• Learned how to set up the drone (charging batteries, calibrating GPS, etc)
• Practiced flying the drone

Week 4

• Continued working on Udemy course to learn Tensorflow
• Changed primary drone to Holy Stone hs700d

Week 5

• Completed Udemy course
• Read three papers to understand audio preprocessing: 'Real-time, Universal, and Robust Adversarial Attacks Against Speaker Recognition Systems', 'Adv Pulse: Universal, Synchronization-free, and Targeted Audio Adversarial Attacks via Sub-second Perturbations', and 'Enabling Fast and Universal Audio Adversarial Attack Using Generative Model'

Week 6

• Researched MFCC feature extraction
• Worked on voice recognition model based on Google's Mini Speech Commands dataset and the convolutional neural network (CNN) architecture

Week 7

• Read and studied 'Hidden Voice Commands' paper
• Learned about the adversarial workflow for generating hidden voice commands
• Worked on code for inverse MFCC feature extraction

Week 8

• Worked on code to add noise to an input audio
• Recorded our own voice commands compatible with a Google Home
• Created hidden voice commands from our recorded commands and experimented with them on Google Home

Week 9

• Conducted experiments with Amazon Alexa and Siri
• Started working on final poster and presentation