ALIF: Low-Cost Adversarial Audio Attacks on Black-Box Speech Platforms Using Linguistic Features

We select five sentences and generate audio samples with varying blurring components for each. The audio samples in the second column are created by adding linguistic perturbation only. The subsequent three columns feature audio samples generated using a single blurring component. The final column includes audio samples produced using all blurring components, except for linguistic perturbation.

We perform a user study (6 volunteers who evaluated 12 ALIF samples on a 0-4 intelligibility scale and transcribed them). Results show an average intelligibility score of 0.84 and a 78% CER, confirming their incomprehensibility.

We ablate blurring components and gather volunteers' opinions on the audios. The results confirm that perturbations on linguistic features are the most crucial factor that affects human perception.

We utilize 200 event noise instances to evaluate the ASRs' robustness, resulting in a similar result to the white noise case. Google is the most vulnerable one.

We choose 10 new commands ("Clear notification", "Play music", "What's the weather", "Take a picture", "Where is my car", "Tell me a story", "How old are you", "Sing me happy birthday", "Good morning", "Where is my home") and generate the attack examples under the setting of α=0.3, β=22Hz, γ=1 and η=0.1 (η is only for OTA examples). Then we attack APIs (Amazon, Azure, iFLYTEK, and Tencent) in the digital domain and attack voice assistants (Amazon Echo and Microsoft Cortana) over the air, and the results are shown below. Attacks belonging to the xxx-online and xxx-offline columns are digital-domain attacks. Attacks belonging to the Echo and Cortana column are over-the-air (OTA) attacks. Commands used in OTA attacks are generated with the online method.

★ Note: The online terminology (generating one command requires at most 50 queries) means in the iterating progress of command generation, we inquire about the target API with the audio given a new smaller loss is obtained. In contrast, the offline terminology (generating one command requires only 1 query) means we only query the API once at the last iteration round during command generation to check whether it is valid.

We assess the impact of ambient noise, revealing that our examples perform well at an SNR of ~20, akin to soft speech, but decline with higher ambient noise. ( α=0.3, β=22Hz, γ=1 and η=0.1, distence between the speaker (Marshall) and the VA is 15 cm)

In addition to the downsampling method presented in the paper, we conduct further tests on the other three possible defense methods: applying a 4000Hz low-pass filter, a 5000Hz low-pass filter, and a 500 Hz high-pass filter to the audio. The results in the following table show that all these three methods can effectively defend against our attack. Specifically, the 500Hz high-pass filter exhibits the best performance, with only 1, 3, 2, and 0 attack commands being accurately recognized by the corresponding ASR.