Attacking ELECTRA-Small: Universal Adversarial Triggers for Reading Comprehension

Date

2023-12-14

Journal Title

Journal ISSN

Volume Title

Publisher

Abstract

New question answering computer programs are pretty good at their job. They perform well when tested on certain datasets, like SQuAD (Rajpurkar et al., 2016) However, these datasets might not fully represent the challenges that come up in real-world situations where understanding what you're reading is important.Even the most advanced models can be affected by disruptions and adversarial text that is meant to decrease a model’s performance. In our paper, we gener- ate adversarial text examples to lower accuracy for the task of reading comprehension testing against the ELECTRA-Small model (Clark et al., 2020) trained on SQuAD. We suggest using a method called "beam search" to create what we call "universal triggers." We then test these triggers on different parts of the SQuAD dataset, with different vocabulary lists. The adversarial trig- gers produced by our beam search are able to effectively reduce a model’s accuracy on the SQuAD dataset by 33% for ’Why’ questions.

Description

LCSH Subject Headings

Citation