Many new products nowadays were made possible by advances in machine learning, in particular image recognition and automatic speech recognition. Examples are spam filters, virtual personal assistants, traffic prediction in GPS devices, or face recognition. Real-world machine learning applications see more widespread use and enjoy ever increasing accuracy on common benchmark datasets. Unfortunately, it has been observed that virtually all models return incorrect results if data is fed to the model that is not from common benchmark datasets, data that was purposefully but imperceptibly altered, and plain garbage data. In this blog post I will talk about the purposeful, imperceptible input modifications, so-called adversarial examples. I will present possible kinds of modifications in computer vision and requirements for successful attacks before discussing attempts to improve models. Finally, I will make two predictions about the future of machine learning models. The appendix contains remarks about the misinterpretation of the largest eigenvalue of a weight matrix; the modulus of this value is almost meaningless without considering the smallest eigenvalues, too.

Introduction

The goal in machine learning is having a computer perform specialized tasks by fitting a statistical model to a set of previously observed pairs of inputs and outcomes. For example, a popular task is digit recognition where 32x32 grayscale images of handwritten digits have to be mapped to the digits zero to nine. Key challenges in machine learning are acquisition of many representative samples, selection of an appropriate statistical model, and fitting the statistical model to the given samples; model fitting is usually a hard optimization problem. Ideally, the fitted model would prove equally accurate for samples it was not trained on.

Around 2013, researchers first observed that the most accurate image recognition models of the time were susceptible to minuscule perturbations in the input. Specifically, given an image and a model, the researchers devised a method to compute perturbations leading to misclassification with high confidence [1, 2]. Since these perturbations were (almost) imperceptible by humans, these perturbed images were dubbed adversarial examples. The reference [3] contains examples for semantic segmentation, pose estimation, and automatic speech recognition. Nicholas Carlini's website has audio adversarial examples that one can listen to.

More research quickly improved existing approaches to compute perturbations (e.g., [4]) and to find uncountable new ways to fool image classifiers. The adversarial modifications can be

small in norm over the whole image [5, 6],
individual pixels (one to five pixels suffice) [7],
synthesized from scratch [8],
output from harmonic functions [9] (like from shade for example),
rotations and translations of the original image [10],
small colorful patches (so-called adversarial patches, [11]),
an elephant [12], and
changes to hue and saturation (semantic adversarial examples, [13]).

The adversarial examples above exploit that image classifiers may compute completely different classifications for similar images. The same classifiers were also shown to compute the same classifications for completely distinct images [14]. It is also possible to find a single perturbation small in norm causing all images to be misclassified, so-called universal adversarial perturbations [15, 16]; such a perturbation can also be used to improve classification accuracy [17]. Finally, adversarial examples can be used to re-purpose existing models for classification tasks they were not trained on [18].

Adversarial examples transfer well [19] meaning an adversarial example for a given model will in all likelihood also cause misclassification on

the same model architecture with different weights as well as
completely different model architectures, e.g., classification trees or support vector machines [20, 21].

Originally, the algorithms computing adversarial examples needed access to the model. In particular, these algorithms needed to be able to calculate the gradients for given inputs. This restriction was soon lifted and recent methods to compute adversarial need only observe probabilities for attacker-provided input [22, 23] or the classification of attacker-provided input [24]. Adversarial examples can also be computed if the objective function is non-differentiable [3]. As stated above, adversarial examples transfer well. For this reason it is also possible to craft adversarial examples on a substitution model.

Model inputs that are not from one of the classes the model was trained on are called out of distribution, the opposite being in-distribution data. Ideally a model should assign (approximately) equal, low confidence to all possible labels in response to out-of-distribution input. Fooling inputs [25] are examples for out-of-distribution input receiving high confidence labels.

The Implications of Adversarial Examples

Early into the research on adversarial examples, the question arose if adversarial examples carry over to the real-world. Initially, some researchers observed that if they printed adversarial examples on paper, photographed them, and fed to them to an image recognizer, then the classifier would still compute incorrect classifications most of the time [26]. Next, 3D adversarial print-outs followed [27] and adversarial attacks were attempted on models recognizing street signs [28] and camera lenses were tampered with [29]. Another group of researchers succeeded in adversarially modifying 3D meshes used for rendering; the rendered objects would then be reliably misclassified [30]. In March 2019, researchers succeeded in having a Tesla drive itself into oncoming traffic by fooling the lane assistant with an adversarial attack [31].

Adversarial examples are not specific to computer vision. For example, these attacks can also be performed on natural language processing models [32], malware detection models [33], automatic speech recognition model with sound recorded by a microphone [34, 35, 36], and medical models [37]. Examples for non-speech input that is interpreted as speech can be found on Nicholas Carlini's website.

In summary, machine learning reached a state where models are widely deployed in safety-related capacities. At the same, an attacker can easily craft malicious model inputs with or without access to the victim model. These malicious inputs go undetected by humans, they can rarely be detected by algorithms, and they can be passed directly (say, as image) or indirectly (played over air, after photographing) to the victim model. This is a recipe for disaster (imagine terrorists dropping confetti with adversarial patches on a highway in the rush hour).

In Search of Robust Models

As soon as adversarial examples had appeared, attempts were made to defend against them. Most defenses were quickly shown to be ineffective including actual defenses [38, 39, 40, 23] and methods to detect adversarial examples [41, 42]. Seven out of eight defenses submitted to a dedicated competition at the International Conference on Learning Representations 2018 (ICLR 2018) were broken before the actual conference took place [43].

There is growing evidence that adversarial examples are unavoidable in high-dimensional spaces because of the peculiarities of geometry [44, 45, 46]. For example, the effects of "high" dimensionality can be measurable with a number of dimensions as low as ten, under certain conditions the distances between the nearest and the farthest neighbor of a given point may then converge towards one, and the unit hypersphere is completely contained by the unit hypercube only for dimensions one to four, cf. [47, 48, 49]. In this scenario, feature engineering and dedicated feature detectors (e.g., for eyes in case of face recognition) would help. Some authors also pointed out the connection between noise robustness and robustness to adversarial examples [50], the connection being the geometry of the decision boundary between classes [51, 52, 53]. For the analysis of this link between noise and adversarial robustness, the intrinsic dimensionality of the input is again relevant.

Additionally, it was shown that training a model robust to adversarial examples requires much more training samples than a model with the same accuracy without this robustness [54]. Furthermore, it is possible that models robust to adversarial examples must exhibit lower accuracy than a non-robust model [55]. It is certain that models with 100% accuracy on the test set must be susceptible to adversarial examples [45].

Early on, it had been discovered that adding adversarial examples to the training set may convey some degree robustness; this is called adversarial training. The currently most effective adversarial training method is based on the reformulation of the optimization problem during model fitting [56]. The standard formulation is

Modify the model parameters so as to minimize the classification error for all training samples.

Usually though, one wants that similar model input is classified similarly resulting in a problem statement from the field of research called robust optimization:

Modify the model parameters so as to minimize the classification error for all training samples and map all points near a given training sample x to the same class as x.

This approach turned out the be the only working defense submitted to ICLR 2018. It increased the perturbation size by about factor four for a successful attack computing provably minimal perturbations [57]. This robust optimization procedure will defend only against the subset of adversarial examples created by perturbations small in norm; it will not protect against, e.g., adversarial image rotation and translation because the perturbed input is in the mathematical sense not "near" the original sample. The defense also fails as soon as the perturbation norm is slightly larger than the maximum perturbation norm used during model training and it does not work against attacks that minimize the Hamming distance (i.e., single pixel attacks).

Machine learning practitioners usually apply a different basic rules for neural networks than for other machine learning model architectures, e.g., models are routinely overparameterized and individual weights are not driven to zero unless model size is critical (the state-of-the-art ImageNet model AmoebaNet has 557 million parameters and there are approximately 1.3 million training samples). In response, some researchers contend that current machine learning models are routinely overparameterized [58] with adversarial examples being one consequence. To test this claim, a research group trained smaller models with stronger weight and they found their models [59, 60]

were as robust to adversarial examples as models using adversarial training,
possessed intuitively interpretable gradients, and
assigned labels with less confidence to out-of-distribution inputs.

Conversely, adding out-of-distribution data to training has a marked regularizing effect [61].

Initially, it seemed that a combination of adversarial training, stronger weight decay, smaller models, and reduced accuracy of weights as well as activations might lead to robust models. Unfortunately, a group of researchers soon pointed out that the models using one of the techniques above had become susceptible to invariance-based adversarial attacks. That is, the labels do not change when they actually should [14].

Perturbation-based adversarial attacks prove that machine learning models may change labels in response to input modifications when they should not and invariance-based adversarial attacks demonstrate that labels do not change in response to input modifications when they should. Thus, models are not properly classifying in-distribution input. The high-confidence labels for fooling inputs are evidence that models cannot tell in-distribution and out-of-distribution data apart. In short, the models are clearly not working if you look closely.

Achieving adversarial robustness exerts a whole new set of demands on practitioners then. Adversarial training requires one backward propagation for every gradient computation, there is at least one backward propagation for every perturbation, and there is one perturbation for every adversarial example added to the training. Thus, training becomes much more expensive in terms of computing time. Strong weight decay confers adversarial robustness but does not work with some models, e.g., Wide ResNets [59], and decreases clean test set accuracy. Smaller models are more robust and speed up training but this is contrarian to the model growth in computer vision. Additionally, more robustness against perturbation-based attacks makes models more vulnerable to invariance-based attacks.

The Future

Conjecture: In general, adversarial examples cannot be detected.

Some authors try to detect adversarial examples [41, 42] and in my opinion this approach is doomed to fail except for specific types of adversarial examples (say, single-pixel perturbations) because the existence of adversarial examples is a symptom of a problem inherent to state-of-the-art machine learning models. Consider for example that every misclassified sample in a given training set can be thought of as an adversarial example: the input is indistinguishable from a benign sample of this class but still misclassified. Also, adversarial examples may improve accuracy [17] meaning detection would decrease accuracy. Finally, adversarial examples can be synthesized from scratch [8]. In [62] the authors attempt to detect adversarial examples by exploiting the observation that noise usually does not change the label of genuine input whereas noise added to adversarial examples easily does [63]. I think this observation only holds for attacks minimizing the perturbation norm (e.g., projected gradient descent [56] or the Carlini-Wagner attack [42]). Consequently, the expectation-over-transformation attack [27] should be able to craft adversarial inputs avoiding detection.

Prediction: In ten years, at most five percent of all deployed machine learning models will be robust to what were state-of-the-art adversarial attacks at the time of model training.

The major reasons for this are:

Adversarial robustness is hard to pitch to stakeholders.
The economics of adversarially robust models put them at a disadvantage.
It requires a paradigm change.

Model accuracy is a single number that can be pitched to anyone, be it a friend, a team leader, a manager, a CEO, an investor, or a customer -- the higher, the better. Training an adversarially robust model for a given task immediately leads to a drop in accuracy in comparison to a non-robust model and makes it necessary to balance a whole set of contradictory requirements, e.g., robustness against perturbation-based as well as invariance-based attacks. Finally, since models are usually resilient to (small) random perturbations, one actually needs to convince the stakeholders that an adversary should be considered.

To add to that, training adversarially robust models comes at a significant cost. The training itself becomes much more expensive because of repeated gradient computations and a larger number of training samples. Some real-world models do take up to a month to train [64, 65] on modern hardware and an increase in training cost would be unbearable. Furthermore, if the procedure outlined in [66] does constitute the bare of minimum of steps required to achieve adversarial robustness, then model development cost will rise markedly: threat models have to be designed, transferability must be tested, and a large number of attacks must be run with tuned hyperparameters. For researchers the document recommends to publish the source code and trained models but this is still not happening on a regular basis and won't happen if the genesis of a large model in the article The Machine Learning Reproducibility Crisis is accurate. Even worse, the guideline document does not consider the invariance-based attacks in [14]. (Besides, in my experience determining the state-of-the-art with respect to adversarial examples is in itself challenging currently.)

For some practitioners and researchers the quest for higher model accuracy on standard benchmarks has become their only goal. This sentiment is perfectly expressed by researchers after training the possibly largest and most accurate image recognition models so far in the following statement:

Our work validates the hypothesis that bigger models and more computation would lead to higher model quality.

Apparently, quick training, fast inference, or a small memory footprint are not traits of high-quality models. Likewise, image recognition models were touted as being as "more accurate than humans". On the contrary, popular image recognition models see a notable drop in accuracy with input that is not from the test set [67] and they react much more sensitive than humans to noise [68]. Consequently, a commitment to training adversarially robust models is an acknowledgment that current machine learning models cannot be assumed to match human perception, that there are severe shortcomings in conventional training methods, and that accuracy numbers on popular benchmark datasets are inflated. For corporate machine learning practitioners, a commitment to training adversarially robust models requires an acknowledgement that the model may have a negative impact on the safety of the user but this constitutes a liability for companies. Companies will fight this liability. Case in point: Tesla wrote in a press release that the adversarial attack making a Tesla steer itself into oncoming traffic [31] is "not a real-world concern".

Appendix: A Note on Matrix Singular Values and Eigenvalues

In the machine learning literature, many authors examine the modulus of the singular values or eigenvalues of the weight matrices in neural networks but the modulus is meaningless if it is not put in relation to the smallest (non-zero) singular values. Weight matrices are used for linear mappings (for matrix vector multiplications) inside a neural network and it holds that $A (c x) = c (A x)$ , where $c$ is a scalar, $A$ is a matrix, and $x$ is a vector. If certain eigenvalues of this matrix $A$ are deemed too large, one can just scale the matrix; this will affect the output of non-linear activations and the softmax but it will the leave the direction of the vector $Ax$ unaltered. Instead, the relevant measure must be either the largest singular value divided by the smallest singular value (the so-called condition number) or the largest singular value divided by the smallest non-zero singular value (the condition number of the full rank part of the matrix). Obviously, an analysis of only the largest singular values ignores the rank of the matrix.

In this context I want to mention Parseval networks [73]. During training, Parseval networks penalize weight matrices that are not approximately orthogonal (a matrix $A$ is called orthogonal if $A^T A = I$ ) and this constrains every entry of the matrix $A$ . To see why, consider that the diagonal entries of $A^T A$ are sums of squares of all entries of $A$ and consider that the diagonal entries of $A^T A$ must be equal to one, the effect being that the Euclidean norm of the weights is equal to the square root of the matrix dimension. In comparison, weight decay drives the individual matrix entries and consequently all eigenvalues toward zero. Moreover, Parseval networks as regularizers interpret weight matrices really as matrices and not as a collection of real values like weight decay does. Ignoring this property will destroy any structure of the matrix (e.g., symmetry, normality) and lead to severely rank deficient matrices. Parseval networks on the other hand guarantee full rank matrices, i.e., the smallest eigenvalues are non-zero.

References

[1] C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus, Intriguing properties of neural networks, 2014.
[BibTeX] [Download]

@misc{SzegedyZS2013,
author = {Christian Szegedy and Wojciech Zaremba and Ilya Sutskever and Joan Bruna and Dumitru Erhan and Ian Goodfellow and Rob Fergus},
title = {Intriguing properties of neural networks},
year = {2014},
url = {https://arxiv.org/abs/1312.6199},
archivePrefix = {arXiv},
eprint = {1312.6199},
}

[2] I. J. Goodfellow, J. Shlens, and C. Szegedy, Explaining and Harnessing Adversarial Examples, 2014.
[BibTeX] [Download]

@misc{GoodfellowSS2014,
author = {Ian J. Goodfellow and Jonathon Shlens and Christian Szegedy},
title = {Explaining and Harnessing Adversarial Examples},
year = {2014},
url = {https://arxiv.org/abs/1412.6572},
archivePrefix = {arXiv},
eprint = {1412.6572},
}

[3] M. Cisse, Y. Adi, N. Neverova, and J. Keshet, Houdini: Fooling Deep Structured Prediction Models, 2017.
[BibTeX] [Download]

@misc{CisseAN2017,
author = {Moustapha Cisse and Yossi Adi and Natalia Neverova and Joseph Keshet},
title = {Houdini: Fooling Deep Structured Prediction Models},
year = {2017},
url = {https://arxiv.org/abs/1707.05373},
archivePrefix = {arXiv},
eprint = {1707.05373},
}

[4] F. Croce, J. Rauber, and M. Hein, Scaling up the randomized gradient-free adversarial attack reveals overestimation of robustness using established attacks, 2019.
[BibTeX] [Download]

@misc{CroceRH2019,
author = {Francesco Croce and Jonas Rauber and Matthias Hein},
title = {Scaling up the randomized gradient-free adversarial attack reveals overestimation of robustness using established attacks},
year = {2019},
url = {https://arxiv.org/abs/1903.11359},
archivePrefix = {arXiv},
eprint = {1903.11359},
}

[5] N. Papernot, P. D. McDaniel, S. Jha, M. Fredrikson, B. Z. Celik, and A. Swami, "The Limitations of Deep Learning in Adversarial Settings," CoRR, vol. abs/1511.07528, 2015.
[BibTeX] [Download]

@article{PapernotMJ2015,
author = {Nicolas Papernot and
Patrick D. McDaniel and
Somesh Jha and
Matt Fredrikson and
Z. Berkay Celik and
Ananthram Swami},
title = {The Limitations of Deep Learning in Adversarial Settings},
journal = {CoRR},
volume = {abs/1511.07528},
year = {2015},
url = {http://arxiv.org/abs/1511.07528},
archivePrefix = {arXiv},
eprint = {1511.07528},
timestamp = {Mon, 13 Aug 2018 16:48:42 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/PapernotMJFCS15},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[6] L. Robinson and B. Graham, "Confusing Deep Convolution Networks by Relabelling," CoRR, vol. abs/1510.06925, 2015.
[BibTeX] [Download]

@article{RobinsonG2015,
author = {Leigh Robinson and
Benjamin Graham},
title = {Confusing Deep Convolution Networks by Relabelling},
journal = {CoRR},
volume = {abs/1510.06925},
year = {2015},
url = {http://arxiv.org/abs/1510.06925},
archivePrefix = {arXiv},
eprint = {1510.06925},
timestamp = {Mon, 13 Aug 2018 16:47:30 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/RobinsonG15},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[7] J. Su, D. V. Vargas, and K. Sakurai, "One pixel attack for fooling deep neural networks," CoRR, vol. abs/1710.08864, 2017.
[BibTeX] [Download]

@article{SuVS2017,
author = {Jiawei Su and
Danilo Vasconcellos Vargas and
Kouichi Sakurai},
title = {One pixel attack for fooling deep neural networks},
journal = {CoRR},
volume = {abs/1710.08864},
year = {2017},
url = {http://arxiv.org/abs/1710.08864},
archivePrefix = {arXiv},
eprint = {1710.08864},
timestamp = {Mon, 13 Aug 2018 16:46:37 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/abs-1710-08864},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[8] Y. Song, R. Shu, N. Kushman, and S. Ermon, "Generative Adversarial Examples," CoRR, vol. abs/1805.07894, 2018.
[BibTeX] [Download]

@article{SongSK2018,
author = {Yang Song and
Rui Shu and
Nate Kushman and
Stefano Ermon},
title = {Generative Adversarial Examples},
journal = {CoRR},
volume = {abs/1805.07894},
year = {2018},
url = {http://arxiv.org/abs/1805.07894},
archivePrefix = {arXiv},
eprint = {1805.07894},
timestamp = {Mon, 13 Aug 2018 16:47:28 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/abs-1805-07894},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[9] W. Heng, S. Zhou, and T. Jiang, "Harmonic Adversarial Attack Method," CoRR, vol. abs/1807.10590, 2018.
[BibTeX] [Download]

@article{HengZJ2018,
author = {Wen Heng and
Shuchang Zhou and
Tingting Jiang},
title = {Harmonic Adversarial Attack Method},
journal = {CoRR},
volume = {abs/1807.10590},
year = {2018},
url = {http://arxiv.org/abs/1807.10590},
archivePrefix = {arXiv},
eprint = {1807.10590},
timestamp = {Mon, 13 Aug 2018 16:47:59 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/abs-1807-10590},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[10] L. Engstrom, D. Tsipras, L. Schmidt, and A. Madry, "A Rotation and a Translation Suffice: Fooling CNNs with Simple Transformations," CoRR, vol. abs/1712.02779, 2017.
[BibTeX] [Download]

@article{EngstromTS2017,
author = {Logan Engstrom and
Dimitris Tsipras and
Ludwig Schmidt and
Aleksander Madry},
title = {A Rotation and a Translation Suffice: Fooling CNNs with Simple Transformations},
journal = {CoRR},
volume = {abs/1712.02779},
year = {2017},
url = {http://arxiv.org/abs/1712.02779},
archivePrefix = {arXiv},
eprint = {1712.02779},
timestamp = {Mon, 13 Aug 2018 16:48:14 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/abs-1712-02779},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[11] T. B. Brown, D. Mané, A. Roy, M. Abadi, and J. Gilmer, "Adversarial Patch," CoRR, vol. abs/1712.09665, 2017.
[BibTeX] [Download]

@article{BrownMR2017,
author = {Tom B. Brown and
Dandelion Man{\'{e}} and
Aurko Roy and
Mart{\'{\i}}n Abadi and
Justin Gilmer},
title = {Adversarial Patch},
journal = {CoRR},
volume = {abs/1712.09665},
year = {2017},
url = {http://arxiv.org/abs/1712.09665},
archivePrefix = {arXiv},
eprint = {1712.09665},
timestamp = {Mon, 13 Aug 2018 16:46:21 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/abs-1712-09665},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[12] A. Rosenfeld, R. S. Zemel, and J. K. Tsotsos, "The Elephant in the Room," CoRR, vol. abs/1808.03305, 2018.
[BibTeX] [Download]

@article{RosenfeldZT2018,
author = {Amir Rosenfeld and
Richard S. Zemel and
John K. Tsotsos},
title = {The Elephant in the Room},
journal = {CoRR},
volume = {abs/1808.03305},
year = {2018},
url = {http://arxiv.org/abs/1808.03305},
archivePrefix = {arXiv},
eprint = {1808.03305},
timestamp = {Thu, 04 Oct 2018 20:06:33 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/abs-1808-03305},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[13] H. Hosseini and R. Poovendran, "Semantic Adversarial Examples," CoRR, vol. abs/1804.00499, 2018.
[BibTeX] [Download]

@article{HosseiniP2018,
author = {Hossein Hosseini and
Radha Poovendran},
title = {Semantic Adversarial Examples},
journal = {CoRR},
volume = {abs/1804.00499},
year = {2018},
url = {http://arxiv.org/abs/1804.00499},
archivePrefix = {arXiv},
eprint = {1804.00499},
timestamp = {Mon, 13 Aug 2018 16:46:08 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/abs-1804-00499},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[14] J. Jacobsen, J. Behrmann, N. Carlini, F. Tramèr, and N. Papernot, "Exploiting Excessive Invariance caused by Norm-Bounded Adversarial Robustness," CoRR, vol. abs/1903.10484, 2019.
[BibTeX] [Download]

@article{JacobsenBC2019,
author = {Jacobsen, J{\"{o}}rn{-}Henrik and
Jens Behrmann and
Nicholas Carlini and
Florian Tram{\`{e}}r and
Nicolas Papernot},
title = {Exploiting Excessive Invariance caused by Norm-Bounded Adversarial
Robustness},
journal = {CoRR},
volume = {abs/1903.10484},
year = {2019},
url = {http://arxiv.org/abs/1903.10484},
archivePrefix = {arXiv},
eprint = {1903.10484},
timestamp = {Mon, 01 Apr 2019 14:07:37 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/abs-1903-10484},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[15] S. Moosavi-Dezfooli, A. Fawzi, and P. Frossard, "DeepFool: a simple and accurate method to fool deep neural networks," CoRR, vol. abs/1511.04599, 2015.
[BibTeX] [Download]

@article{Moosavi-DezfooliFF2015,
author = {Moosavi-Dezfooli, Seyed-Mohsen and
Alhussein Fawzi and
Pascal Frossard},
title = {DeepFool: a simple and accurate method to fool deep neural networks},
journal = {CoRR},
volume = {abs/1511.04599},
year = {2015},
url = {http://arxiv.org/abs/1511.04599},
archivePrefix = {arXiv},
eprint = {1511.04599},
timestamp = {Mon, 13 Aug 2018 16:47:14 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/Moosavi-Dezfooli15},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[16] S. Moosavi-Dezfooli, A. Fawzi, O. Fawzi, and P. Frossard, "Universal adversarial perturbations," CoRR, vol. abs/1610.08401, 2016.
[BibTeX] [Download]

@article{Moosavi-DezfooliFF2016,
author = {Moosavi-Dezfooli, Seyed-Mohsen and
Alhussein Fawzi and
Omar Fawzi and
Pascal Frossard},
title = {Universal adversarial perturbations},
journal = {CoRR},
volume = {abs/1610.08401},
year = {2016},
url = {http://arxiv.org/abs/1610.08401},
archivePrefix = {arXiv},
eprint = {1610.08401},
timestamp = {Mon, 13 Aug 2018 16:48:58 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/Moosavi-Dezfooli16},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[17] Y. J. Yoo, S. Park, J. Choi, S. Yun, and N. Kwak, "Butterfly Effect: Bidirectional Control of Classification Performance by Small Additive Perturbation," CoRR, vol. abs/1711.09681, 2017.
[BibTeX] [Download]

@article{YooPC2017,
author = {Young Joon Yoo and
Seonguk Park and
Junyoung Choi and
Sangdoo Yun and
Nojun Kwak},
title = {Butterfly Effect: Bidirectional Control of Classification Performance
by Small Additive Perturbation},
journal = {CoRR},
volume = {abs/1711.09681},
year = {2017},
url = {http://arxiv.org/abs/1711.09681},
archivePrefix = {arXiv},
eprint = {1711.09681},
timestamp = {Mon, 13 Aug 2018 16:47:19 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/abs-1711-09681},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[18] G. F. Elsayed, I. J. Goodfellow, and J. Sohl{-}Dickstein, "Adversarial Reprogramming of Neural Networks," CoRR, vol. abs/1806.11146, 2018.
[BibTeX] [Download]

@article{ElsayedGS2018,
author = {Gamaleldin F. Elsayed and
Ian J. Goodfellow and
Sohl{-}Dickstein, Jasha},
title = {Adversarial Reprogramming of Neural Networks},
journal = {CoRR},
volume = {abs/1806.11146},
year = {2018},
url = {http://arxiv.org/abs/1806.11146},
archivePrefix = {arXiv},
eprint = {1806.11146},
timestamp = {Mon, 13 Aug 2018 16:46:25 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/abs-1806-11146},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[19] F. Tramèr, N. Papernot, I. Goodfellow, D. Boneh, and P. McDaniel, The Space of Transferable Adversarial Examples, 2017.
[BibTeX] [Download]

@misc{TramerPG2017,
author = {Florian Tram\`{e}r and Nicolas Papernot and Ian Goodfellow and Dan Boneh and Patrick McDaniel},
title = {The Space of Transferable Adversarial Examples},
year = {2017},
url = {https://arxiv.org/abs/1704.03453},
archivePrefix = {arXiv},
eprint = {1704.03453},
}

[20] N. Papernot, P. D. McDaniel, I. J. Goodfellow, S. Jha, B. Z. Celik, and A. Swami, "Practical Black-Box Attacks against Deep Learning Systems using Adversarial Examples," CoRR, vol. abs/1602.02697, 2016.
[BibTeX] [Download]

@article{PapernotMG2016a,
author = {Nicolas Papernot and
Patrick D. McDaniel and
Ian J. Goodfellow and
Somesh Jha and
Z. Berkay Celik and
Ananthram Swami},
title = {Practical Black-Box Attacks against Deep Learning Systems using Adversarial
Examples},
journal = {CoRR},
volume = {abs/1602.02697},
year = {2016},
url = {http://arxiv.org/abs/1602.02697},
archivePrefix = {arXiv},
eprint = {1602.02697},
timestamp = {Mon, 13 Aug 2018 16:49:06 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/PapernotMGJCS16},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[21] N. Papernot, P. D. McDaniel, and I. J. Goodfellow, "Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples," CoRR, vol. abs/1605.07277, 2016.
[BibTeX] [Download]

@article{PapernotMG2016b,
author = {Nicolas Papernot and
Patrick D. McDaniel and
Ian J. Goodfellow},
title = {Transferability in Machine Learning: from Phenomena to Black-Box Attacks
using Adversarial Samples},
journal = {CoRR},
volume = {abs/1605.07277},
year = {2016},
url = {http://arxiv.org/abs/1605.07277},
archivePrefix = {arXiv},
eprint = {1605.07277},
timestamp = {Mon, 13 Aug 2018 16:48:28 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/PapernotMG16},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[22] N. Narodytska and S. P. Kasiviswanathan, "Simple Black-Box Adversarial Perturbations for Deep Networks," CoRR, vol. abs/1612.06299, 2016.
[BibTeX] [Download]

@article{NarodytskaK2016,
author = {Nina Narodytska and
Shiva Prasad Kasiviswanathan},
title = {Simple Black-Box Adversarial Perturbations for Deep Networks},
journal = {CoRR},
volume = {abs/1612.06299},
year = {2016},
url = {http://arxiv.org/abs/1612.06299},
archivePrefix = {arXiv},
eprint = {1612.06299},
timestamp = {Mon, 13 Aug 2018 16:46:18 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/NarodytskaK16},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[23] P. Chen, Y. Sharma, H. Zhang, J. Yi, and C. Hsieh, EAD: Elastic-Net Attacks to Deep Neural Networks via Adversarial Examples, 2017.
[BibTeX] [Download]

@misc{ChenSZ2017,
author = {Pin-Yu Chen and Yash Sharma and Huan Zhang and Jinfeng Yi and
Cho-Jui Hsieh},
title = {EAD: Elastic-Net Attacks to Deep Neural Networks via Adversarial Examples},
year = {2017},
url = {https://arxiv.org/abs/1709.04114},
archivePrefix = {arXiv},
eprint = {1709.04114},
}

[24] W. Brendel, J. Rauber, and M. Bethge, "Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Models," in 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings, 2018.
[BibTeX] [Download]

@inproceedings{BrendelRB2018,
author = {Wieland Brendel and
Jonas Rauber and
Matthias Bethge},
title = {Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box
Machine Learning Models},
booktitle = {6th International Conference on Learning Representations, {ICLR} 2018,
Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings},
year = {2018},
crossref = {DBLP:conf/iclr/2018},
url = {https://openreview.net/forum?id=SyZI0GWCZ},
timestamp = {Thu, 04 Apr 2019 13:20:09 +0200},
biburl = {https://dblp.org/rec/bib/conf/iclr/BrendelRB18},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[25] A. M. Nguyen, J. Yosinski, and J. Clune, "Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images," CoRR, vol. abs/1412.1897, 2014.
[BibTeX] [Download]

@article{NguyenYC2014,
author = {Anh Mai Nguyen and
Jason Yosinski and
Jeff Clune},
title = {Deep Neural Networks are Easily Fooled: High Confidence Predictions
for Unrecognizable Images},
journal = {CoRR},
volume = {abs/1412.1897},
year = {2014},
url = {http://arxiv.org/abs/1412.1897},
archivePrefix = {arXiv},
eprint = {1412.1897},
timestamp = {Mon, 13 Aug 2018 16:48:10 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/NguyenYC14},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[26] A. Kurakin, I. J. Goodfellow, and S. Bengio, "Adversarial examples in the physical world," CoRR, vol. abs/1607.02533, 2016.
[BibTeX] [Download]

@article{KurakinGB2016a,
author = {Alexey Kurakin and
Ian J. Goodfellow and
Samy Bengio},
title = {Adversarial examples in the physical world},
journal = {CoRR},
volume = {abs/1607.02533},
year = {2016},
url = {http://arxiv.org/abs/1607.02533},
archivePrefix = {arXiv},
eprint = {1607.02533},
timestamp = {Mon, 13 Aug 2018 16:48:46 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/KurakinGB16},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[27] A. Athalye, L. Engstrom, A. Ilyas, and K. Kwok, "Synthesizing Robust Adversarial Examples," CoRR, vol. abs/1707.07397, 2017.
[BibTeX] [Download]

@article{AthalyeEI2017,
author = {Anish Athalye and
Logan Engstrom and
Andrew Ilyas and
Kevin Kwok},
title = {Synthesizing Robust Adversarial Examples},
journal = {CoRR},
volume = {abs/1707.07397},
year = {2017},
url = {http://arxiv.org/abs/1707.07397},
archivePrefix = {arXiv},
eprint = {1707.07397},
timestamp = {Mon, 13 Aug 2018 16:48:27 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/AthalyeS17},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[28] I. Evtimov, K. Eykholt, E. Fernandes, T. Kohno, B. Li, A. Prakash, A. Rahmati, and D. Song, "Robust Physical-World Attacks on Machine Learning Models," CoRR, vol. abs/1707.08945, 2017.
[BibTeX] [Download]

@article{EvtimovEF2017,
author = {Ivan Evtimov and
Kevin Eykholt and
Earlence Fernandes and
Tadayoshi Kohno and
Bo Li and
Atul Prakash and
Amir Rahmati and
Dawn Song},
title = {Robust Physical-World Attacks on Machine Learning Models},
journal = {CoRR},
volume = {abs/1707.08945},
year = {2017},
url = {http://arxiv.org/abs/1707.08945},
archivePrefix = {arXiv},
eprint = {1707.08945},
timestamp = {Mon, 20 Aug 2018 13:55:57 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/EvtimovEFKLPRS17},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[29] J. Li, F. R. Schmidt, and Z. J. Kolter, Adversarial camera stickers: A physical camera-based attack on deep learning systems, 2019.
[BibTeX] [Download]

@misc{LiSK2019,
author = {Juncheng Li and Frank R. Schmidt and J. Zico Kolter},
title = {Adversarial camera stickers: A physical camera-based attack on deep learning systems},
year = {2019},
url = {https://arxiv.org/abs/1904.00759},
archivePrefix = {arXiv},
eprint = {1904.00759},
}

[30] D. Yang, C. Xiao, B. Li, J. Deng, and M. Liu, "Realistic Adversarial Examples in 3D Meshes," CoRR, vol. abs/1810.05206, 2018.
[BibTeX] [Download]

@article{YangXL2018,
author = {Dawei Yang and
Chaowei Xiao and
Bo Li and
Jia Deng and
Mingyan Liu},
title = {Realistic Adversarial Examples in 3D Meshes},
journal = {CoRR},
volume = {abs/1810.05206},
year = {2018},
url = {http://arxiv.org/abs/1810.05206},
archivePrefix = {arXiv},
eprint = {1810.05206},
timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},
biburl = {https://dblp.org/rec/bib/journals/corr/abs-1810-05206},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[31] Tencent Keen Security Lab, Experimental Security Research of Tesla Autopilot, 2019.
[BibTeX] [Download]

@misc{Keen2019,
author = {{Tencent Keen Security Lab}},
title = {Experimental Security Research of Tesla Autopilot},
year = {2019},
url = {https://keenlab.tencent.com/en/2019/03/29/Tencent-Keen-Security-Lab-Experimental-Security-Research-of-Tesla-Autopilot/},
}

[32] M. Alzantot, Y. Sharma, A. Elgohary, B. Ho, M. B. Srivastava, and K. Chang, "Generating Natural Language Adversarial Examples," CoRR, vol. abs/1804.07998, 2018.
[BibTeX] [Download]

@article{AlzantotSE2018,
author = {Moustafa Alzantot and
Yash Sharma and
Ahmed Elgohary and
Ho, Bo{-}Jhang and
Mani B. Srivastava and
Chang, Kai{-}Wei},
title = {Generating Natural Language Adversarial Examples},
journal = {CoRR},
volume = {abs/1804.07998},
year = {2018},
url = {http://arxiv.org/abs/1804.07998},
archivePrefix = {arXiv},
eprint = {1804.07998},
timestamp = {Mon, 13 Aug 2018 16:46:19 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/abs-1804-07998},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[33] K. Grosse, N. Papernot, P. Manoharan, M. Backes, and P. D. McDaniel, "Adversarial Perturbations Against Deep Neural Networks for Malware Classification," CoRR, vol. abs/1606.04435, 2016.
[BibTeX] [Download]

@article{GrossePM2016,
author = {Kathrin Grosse and
Nicolas Papernot and
Praveen Manoharan and
Michael Backes and
Patrick D. McDaniel},
title = {Adversarial Perturbations Against Deep Neural Networks for Malware
Classification},
journal = {CoRR},
volume = {abs/1606.04435},
year = {2016},
url = {http://arxiv.org/abs/1606.04435},
archivePrefix = {arXiv},
eprint = {1606.04435},
timestamp = {Mon, 13 Aug 2018 16:47:14 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/GrossePM0M16},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[34] N. Carlini, P. Mishra, T. Vaidya, Y. Zhang, M. Sherr, C. Shields, D. A. Wagner, and W. Zhou, "Hidden Voice Commands," in 25th USENIX Security Symposium, USENIX Security 16, Austin, TX, USA, August 10-12, 2016., 2016, p. 513–530.
[BibTeX] [Download]

@inproceedings{CarliniMV2016,
author = {Nicholas Carlini and
Pratyush Mishra and
Tavish Vaidya and
Yuankai Zhang and
Micah Sherr and
Clay Shields and
David A. Wagner and
Wenchao Zhou},
title = {Hidden Voice Commands},
booktitle = {25th {USENIX} Security Symposium, {USENIX} Security 16, Austin, TX,
USA, August 10-12, 2016.},
pages = {513--530},
year = {2016},
crossref = {DBLP:conf/uss/2016},
url = {https://www.usenix.org/conference/usenixsecurity16/technical-sessions/presentation/carlini},
timestamp = {Wed, 14 Jun 2017 15:21:24 +0200},
biburl = {https://dblp.org/rec/bib/conf/uss/CarliniMVZSSWZ16},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[35] D. Iter, J. Huang, and M. Jerman, "Generating Adversarial Examples for Speech Recognition," Stanford University 2016.
[BibTeX] [Download]

@techreport{IterHJ2016,
author = {Dan Iter and
Jade Huang and
Mike Jerman},
title = {Generating Adversarial Examples for Speech Recognition},
year = {2016},
url = {http://web.stanford.edu/class/cs224s/reports/Dan_Iter.pdf},
institution = {Stanford University},
}

[36] Y. Qin, N. Carlini, I. Goodfellow, G. Cottrell, and C. Raffel, Imperceptible, Robust, and Targeted Adversarial Examples for Automatic Speech Recognition, 2019.
[BibTeX] [Download]

@misc{QinCG2019,
author = {Yao Qin and Nicholas Carlini and Ian Goodfellow and Garrison Cottrell and Colin Raffel},
title = {Imperceptible, Robust, and Targeted Adversarial Examples for Automatic Speech Recognition},
year = {2019},
url = {https://arxiv.org/abs/1903.10346},
archivePrefix = {arXiv},
eprint = {1903.10346},
}

[37] S. G. Finlayson, I. S. Kohane, and A. L. Beam, "Adversarial Attacks Against Medical Deep Learning Systems," CoRR, vol. abs/1804.05296, 2018.
[BibTeX] [Download]

@article{FinlaysonKB2018,
author = {Samuel G. Finlayson and
Isaac S. Kohane and
Andrew L. Beam},
title = {Adversarial Attacks Against Medical Deep Learning Systems},
journal = {CoRR},
volume = {abs/1804.05296},
year = {2018},
url = {http://arxiv.org/abs/1804.05296},
archivePrefix = {arXiv},
eprint = {1804.05296},
timestamp = {Mon, 13 Aug 2018 16:48:58 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/abs-1804-05296},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[38] W. Brendel and M. Bethge, Comment on "Biologically inspired protection of deep networks from adversarial attacks", 2017.
[BibTeX] [Download]

@misc{BrendelB2017,
author = {Wieland Brendel and Matthias Bethge},
title = {Comment on "Biologically inspired protection of deep networks from adversarial attacks"},
year = {2017},
url = {https://arxiv.org/abs/1704.01547},
archivePrefix = {arXiv},
eprint = {1704.01547},
}

[39] N. Carlini, "Is AmI (Attacks Meet Interpretability) Robust to Adversarial Examples?," CoRR, vol. abs/1902.02322, 2019.
[BibTeX] [Download]

@article{Carlini2019,
author = {Nicholas Carlini},
title = {Is AmI (Attacks Meet Interpretability) Robust to Adversarial Examples?},
journal = {CoRR},
volume = {abs/1902.02322},
year = {2019},
url = {http://arxiv.org/abs/1902.02322},
archivePrefix = {arXiv},
eprint = {1902.02322},
timestamp = {Fri, 01 Mar 2019 17:14:17 +0100},
biburl = {https://dblp.org/rec/bib/journals/corr/abs-1902-02322},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[40] N. Carlini and D. A. Wagner, "Defensive Distillation is Not Robust to Adversarial Examples," CoRR, vol. abs/1607.04311, 2016.
[BibTeX] [Download]

@article{CarliniW2016a,
author = {Nicholas Carlini and
David A. Wagner},
title = {Defensive Distillation is Not Robust to Adversarial Examples},
journal = {CoRR},
volume = {abs/1607.04311},
year = {2016},
url = {http://arxiv.org/abs/1607.04311},
archivePrefix = {arXiv},
eprint = {1607.04311},
timestamp = {Mon, 13 Aug 2018 16:49:17 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/CarliniW16},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[41] J. H. Metzen, T. Genewein, V. Fischer, and B. Bischoff, On Detecting Adversarial Perturbations, 2017.
[BibTeX] [Download]

@misc{MetzenGF2017,
author = {Jan Hendrik Metzen and Tim Genewein and Volker Fischer and Bastian Bischoff},
title = {On Detecting Adversarial Perturbations},
year = {2017},
url = {https://arxiv.org/abs/1702.04267},
archivePrefix = {arXiv},
eprint = {1702.04267},
}

[42] N. Carlini and D. A. Wagner, "Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods," CoRR, vol. abs/1705.07263, 2017.
[BibTeX] [Download]

@article{CarliniW2017,
author = {Nicholas Carlini and
David A. Wagner},
title = {Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection
Methods},
journal = {CoRR},
volume = {abs/1705.07263},
year = {2017},
url = {http://arxiv.org/abs/1705.07263},
archivePrefix = {arXiv},
eprint = {1705.07263},
timestamp = {Mon, 13 Aug 2018 16:46:30 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/CarliniW17},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[43] A. Athalye, N. Carlini, and D. A. Wagner, "Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples," CoRR, vol. abs/1802.00420, 2018.
[BibTeX] [Download]

@article{AthalyeCW2018,
author = {Anish Athalye and
Nicholas Carlini and
David A. Wagner},
title = {Obfuscated Gradients Give a False Sense of Security: Circumventing
Defenses to Adversarial Examples},
journal = {CoRR},
volume = {abs/1802.00420},
year = {2018},
url = {http://arxiv.org/abs/1802.00420},
archivePrefix = {arXiv},
eprint = {1802.00420},
timestamp = {Mon, 13 Aug 2018 16:48:14 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/abs-1802-00420},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[44] S. Dube, "High Dimensional Spaces, Deep Learning and Adversarial Examples," CoRR, vol. abs/1801.00634, 2018.
[BibTeX] [Download]

@article{Dube2018,
author = {Simant Dube},
title = {High Dimensional Spaces, Deep Learning and Adversarial Examples},
journal = {CoRR},
volume = {abs/1801.00634},
year = {2018},
url = {http://arxiv.org/abs/1801.00634},
archivePrefix = {arXiv},
eprint = {1801.00634},
timestamp = {Mon, 13 Aug 2018 16:48:10 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/abs-1801-00634},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[45] J. Gilmer, L. Metz, F. Faghri, S. S. Schoenholz, M. Raghu, M. Wattenberg, and I. J. Goodfellow, "Adversarial Spheres," CoRR, vol. abs/1801.02774, 2018.
[BibTeX] [Download]

@article{GilmerMF2018,
author = {Justin Gilmer and
Luke Metz and
Fartash Faghri and
Samuel S. Schoenholz and
Maithra Raghu and
Martin Wattenberg and
Ian J. Goodfellow},
title = {Adversarial Spheres},
journal = {CoRR},
volume = {abs/1801.02774},
year = {2018},
url = {http://arxiv.org/abs/1801.02774},
archivePrefix = {arXiv},
eprint = {1801.02774},
timestamp = {Mon, 13 Aug 2018 16:46:17 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/abs-1801-02774},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[46] A. Shafahi, R. W. Huang, C. Studer, S. Feizi, and T. Goldstein, "Are adversarial examples inevitable?," CoRR, vol. abs/1809.02104, 2018.
[BibTeX] [Download]

@article{ShafahiHS2018,
author = {Ali Shafahi and
W. Ronny Huang and
Christoph Studer and
Soheil Feizi and
Tom Goldstein},
title = {Are adversarial examples inevitable?},
journal = {CoRR},
volume = {abs/1809.02104},
year = {2018},
url = {http://arxiv.org/abs/1809.02104},
archivePrefix = {arXiv},
eprint = {1809.02104},
timestamp = {Fri, 05 Oct 2018 11:34:52 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/abs-1809-02104},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[47] C. C. Aggarwal, A. Hinneburg, and D. A. Keim, "On the Surprising Behavior of Distance Metrics in High Dimensional Spaces," in Database Theory - ICDT 2001, 8th International Conference, London, UK, January 4-6, 2001, Proceedings., 2001, p. 420–434. doi:10.1007/3-540-44503-X_27
[BibTeX] [Download]

@inproceedings{AggarwalHK2001,
author = {Charu C. Aggarwal and
Alexander Hinneburg and
Daniel A. Keim},
title = {On the Surprising Behavior of Distance Metrics in High Dimensional
Spaces},
booktitle = {Database Theory - {ICDT} 2001, 8th International Conference, London,
UK, January 4-6, 2001, Proceedings.},
pages = {420--434},
year = {2001},
crossref = {DBLP:conf/icdt/2001},
url = {http://users.informatik.uni-halle.de/~hinnebur/PS_Files/icdt2001b.pdf},
doi = {10.1007/3-540-44503-X\_27},
timestamp = {Wed, 24 May 2017 15:40:45 +0200},
biburl = {https://dblp.org/rec/bib/conf/icdt/AggarwalHK01},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[48] K. S. Beyer, J. Goldstein, R. Ramakrishnan, and U. Shaft, "When Is ``Nearest Neighbor'' Meaningful?," in Database Theory - ICDT '99, 7th International Conference, Jerusalem, Israel, January 10-12, 1999, Proceedings., 1999, p. 217–235. doi:10.1007/3-540-49257-7_15
[BibTeX] [Download]

@inproceedings{BeyerGR1999,
author = {Kevin S. Beyer and
Jonathan Goldstein and
Raghu Ramakrishnan and
Uri Shaft},
title = {When Is ``Nearest Neighbor'' Meaningful?},
booktitle = {Database Theory - {ICDT} '99, 7th International Conference, Jerusalem,
Israel, January 10-12, 1999, Proceedings.},
pages = {217--235},
year = {1999},
crossref = {DBLP:conf/icdt/99},
url = {https://members.loria.fr/moberger/Enseignement/Master2/Exposes/beyer.pdf},
doi = {10.1007/3-540-49257-7\_15},
timestamp = {Wed, 14 Nov 2018 10:56:23 +0100},
biburl = {https://dblp.org/rec/bib/conf/icdt/BeyerGRS99},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[49] R. Rojas, "The Curse of Dimensionality," Freie Universität Berlin 2015.
[BibTeX] [Download]

@techreport{Rojas2015,
author = {Ra\'{u}l Rojas},
title = {The Curse of Dimensionality},
year = {2015},
url = {https://www.inf.fu-berlin.de/inst/ag-ki/rojas_home/documents/tutorials/dimensionality.pdf},
institution = {Freie Universit\"{a}t Berlin},
}

[50] N. Ford, J. Gilmer, N. Carlini, and D. Cubuk, "Adversarial Examples Are a Natural Consequence of Test Error in Noise," CoRR, vol. abs/1901.10513, 2019.
[BibTeX] [Download]

@article{FordGC2019,
author = {Nic Ford and
Justin Gilmer and
Nicolas Carlini and
Dogus Cubuk},
title = {Adversarial Examples Are a Natural Consequence of Test Error in Noise},
journal = {CoRR},
volume = {abs/1901.10513},
year = {2019},
url = {http://arxiv.org/abs/1901.10513},
archivePrefix = {arXiv},
eprint = {1901.10513},
timestamp = {Sun, 03 Feb 2019 14:23:05 +0100},
biburl = {https://dblp.org/rec/bib/journals/corr/abs-1901-10513},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[51] A. Fawzi, S. Moosavi-Dezfooli, and P. Frossard, "Robustness of classifiers: from adversarial to random noise," CoRR, vol. abs/1608.08967, 2016.
[BibTeX] [Download]

@article{FawziMF2016,
author = {Alhussein Fawzi and
Moosavi-Dezfooli, Seyed-Mohsen and
Pascal Frossard},
title = {Robustness of classifiers: from adversarial to random noise},
journal = {CoRR},
volume = {abs/1608.08967},
year = {2016},
url = {http://arxiv.org/abs/1608.08967},
archivePrefix = {arXiv},
eprint = {1608.08967},
timestamp = {Mon, 13 Aug 2018 16:47:18 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/FawziMF16},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[52] A. Fawzi, S. Moosavi-Dezfooli, P. Frossard, and S. Soatto, "Classification regions of deep neural networks," CoRR, vol. abs/1705.09552, 2017.
[BibTeX] [Download]

@article{FawziMF2017,
author = {Alhussein Fawzi and
Moosavi-Dezfooli, Seyed-Mohsen and
Pascal Frossard and
Stefano Soatto},
title = {Classification regions of deep neural networks},
journal = {CoRR},
volume = {abs/1705.09552},
year = {2017},
url = {http://arxiv.org/abs/1705.09552},
archivePrefix = {arXiv},
eprint = {1705.09552},
timestamp = {Mon, 13 Aug 2018 16:49:15 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/FawziMFS17},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[53] S. Moosavi-Dezfooli, A. Fawzi, O. Fawzi, P. Frossard, and S. Soatto, "Analysis of universal adversarial perturbations," CoRR, vol. abs/1705.09554, 2017.
[BibTeX] [Download]

@article{Moosavi-DezfooliFF2017,
author = {Moosavi-Dezfooli, Seyed-Mohsen and
Alhussein Fawzi and
Omar Fawzi and
Pascal Frossard and
Stefano Soatto},
title = {Analysis of universal adversarial perturbations},
journal = {CoRR},
volume = {abs/1705.09554},
year = {2017},
url = {http://arxiv.org/abs/1705.09554},
archivePrefix = {arXiv},
eprint = {1705.09554},
timestamp = {Mon, 13 Aug 2018 16:48:18 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/Moosavi-Dezfooli17},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[54] L. Schmidt, S. Santurkar, D. Tsipras, K. Talwar, and A. Madry, "Adversarially Robust Generalization Requires More Data," CoRR, vol. abs/1804.11285, 2018.
[BibTeX] [Download]

@article{SchmidtST2018,
author = {Ludwig Schmidt and
Shibani Santurkar and
Dimitris Tsipras and
Kunal Talwar and
Aleksander Madry},
title = {Adversarially Robust Generalization Requires More Data},
journal = {CoRR},
volume = {abs/1804.11285},
year = {2018},
url = {http://arxiv.org/abs/1804.11285},
archivePrefix = {arXiv},
eprint = {1804.11285},
timestamp = {Mon, 13 Aug 2018 16:46:50 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/abs-1804-11285},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[55] D. Tsipras, S. Santurkar, L. Engstrom, A. Turner, and A. Madry, Robustness May Be at Odds with Accuracy, 2018.
[BibTeX] [Download]

@misc{TsiprasSE2018,
author = {Dimitris Tsipras and Shibani Santurkar and Logan Engstrom and Alexander Turner and Aleksander Madry},
title = {Robustness May Be at Odds with Accuracy},
year = {2018},
url = {https://arxiv.org/abs/1805.12152},
archivePrefix = {arXiv},
eprint = {1805.12152},
}

[56] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, Towards Deep Learning Models Resistant to Adversarial Attacks, 2017.
[BibTeX] [Download]

@misc{MadryMS2017,
author = {Aleksander Madry and Aleksandar Makelov and Ludwig Schmidt and Dimitris Tsipras and Adrian Vladu},
title = {Towards Deep Learning Models Resistant to Adversarial Attacks},
year = {2017},
url = {https://arxiv.org/abs/1706.06083},
archivePrefix = {arXiv},
eprint = {1706.06083},
}

[57] N. Carlini, G. Katz, C. Barrett, and D. L. Dill, "Provably Minimally-Distorted Adversarial Examples," CoRR, vol. abs/1709.10207, 2017.
[BibTeX] [Download]

@article{CarliniKB2017,
author = {Nicholas Carlini and
Guy Katz and
Clark Barrett and
David L. Dill},
title = {Provably Minimally-Distorted Adversarial Examples},
journal = {CoRR},
volume = {abs/1709.10207},
year = {2017},
url = {http://arxiv.org/abs/1709.10207},
archivePrefix = {arXiv},
eprint = {1709.10207},
timestamp = {Mon, 13 Aug 2018 16:45:59 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/abs-1709-10207},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[58] A. Rosenfeld and J. K. Tsotsos, "Intriguing Properties of Randomly Weighted Networks: Generalizing While Learning Next to Nothing," CoRR, vol. abs/1802.00844, 2018.
[BibTeX] [Download]

@article{RosenfeldT2018,
author = {Amir Rosenfeld and
John K. Tsotsos},
title = {Intriguing Properties of Randomly Weighted Networks: Generalizing
While Learning Next to Nothing},
journal = {CoRR},
volume = {abs/1802.00844},
year = {2018},
url = {http://arxiv.org/abs/1802.00844},
archivePrefix = {arXiv},
eprint = {1802.00844},
timestamp = {Mon, 13 Aug 2018 16:48:06 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/abs-1802-00844},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[59] A. Galloway, G. W. Taylor, and M. Moussa, "Predicting Adversarial Examples with High Confidence," CoRR, vol. abs/1802.04457, 2018.
[BibTeX] [Download]

@article{GallowayTM2018,
author = {Angus Galloway and
Graham W. Taylor and
Medhat Moussa},
title = {Predicting Adversarial Examples with High Confidence},
journal = {CoRR},
volume = {abs/1802.04457},
year = {2018},
url = {http://arxiv.org/abs/1802.04457},
archivePrefix = {arXiv},
eprint = {1802.04457},
timestamp = {Mon, 13 Aug 2018 16:48:30 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/abs-1802-04457},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[60] A. Galloway, T. Tanay, and G. W. Taylor, "Adversarial Training Versus Weight Decay," CoRR, vol. abs/1804.03308, 2018.
[BibTeX] [Download]

@article{GallowayTT2018,
author = {Angus Galloway and
Thomas Tanay and
Graham W. Taylor},
title = {Adversarial Training Versus Weight Decay},
journal = {CoRR},
volume = {abs/1804.03308},
year = {2018},
url = {http://arxiv.org/abs/1804.03308},
archivePrefix = {arXiv},
eprint = {1804.03308},
timestamp = {Mon, 13 Aug 2018 16:46:27 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/abs-1804-03308},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[61] X. Zhang and Y. LeCun, "Universum Prescription: Regularization using Unlabeled Data," CoRR, vol. abs/1511.03719, 2015.
[BibTeX] [Download]

@article{ZhangL2015,
author = {Xiang Zhang and
Yann LeCun},
title = {Universum Prescription: Regularization using Unlabeled Data},
journal = {CoRR},
volume = {abs/1511.03719},
year = {2015},
url = {http://arxiv.org/abs/1511.03719},
archivePrefix = {arXiv},
eprint = {1511.03719},
timestamp = {Mon, 13 Aug 2018 16:46:50 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/ZhangL15e},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[62] K. Roth, Y. Kilcher, and T. Hofmann, "The Odds are Odd: A Statistical Test for Detecting Adversarial Examples," CoRR, vol. abs/1902.04818, 2019.
[BibTeX] [Download]

@article{RothKH2019,
author = {Kevin Roth and
Yannic Kilcher and
Thomas Hofmann},
title = {The Odds are Odd: {A} Statistical Test for Detecting Adversarial Examples},
journal = {CoRR},
volume = {abs/1902.04818},
year = {2019},
url = {http://arxiv.org/abs/1902.04818},
archivePrefix = {arXiv},
eprint = {1902.04818},
timestamp = {Sat, 02 Mar 2019 16:35:40 +0100},
biburl = {https://dblp.org/rec/bib/journals/corr/abs-1902-04818},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[63] P. Tabacof and E. Valle, "Exploring the Space of Adversarial Images," CoRR, vol. abs/1510.05328, 2015.
[BibTeX] [Download]

@article{TabacofV2015,
author = {Pedro Tabacof and
Eduardo Valle},
title = {Exploring the Space of Adversarial Images},
journal = {CoRR},
volume = {abs/1510.05328},
year = {2015},
url = {http://arxiv.org/abs/1510.05328},
archivePrefix = {arXiv},
eprint = {1510.05328},
timestamp = {Mon, 13 Aug 2018 16:47:48 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/TabacofV15},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[64] E. Variani, T. Bagby, E. McDermott, and M. Bacchiani, "End-to-End Training of Acoustic Models for Large Vocabulary Continuous Speech Recognition with TensorFlow," in Interspeech 2017, 18th Annual Conference of the International Speech Communication Association, Stockholm, Sweden, August 20-24, 2017, 2017, p. 1641–1645.
[BibTeX] [Download]

@inproceedings{VarianiBM2017,
author = {Ehsan Variani and
Tom Bagby and
Erik McDermott and
Michiel Bacchiani},
title = {End-to-End Training of Acoustic Models for Large Vocabulary Continuous
Speech Recognition with TensorFlow},
booktitle = {Interspeech 2017, 18th Annual Conference of the International Speech
Communication Association, Stockholm, Sweden, August 20-24, 2017},
pages = {1641--1645},
year = {2017},
crossref = {DBLP:conf/interspeech/2017},
url = {https://research.google.com/pubs/archive/46294.pdf},
timestamp = {Tue, 16 Jan 2018 11:21:54 +0100},
biburl = {https://dblp.org/rec/bib/conf/interspeech/VarianiBMB17},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[65] W. Zhang, X. Cui, U. Finkler, B. Kingsbury, G. Saon, D. Kung, and M. Picheny, Distributed Deep Learning Strategies For Automatic Speech Recognition, 2019.
[BibTeX] [Download]

@misc{ZhangCF2019,
author = {Wei Zhang and Xiaodong Cui and Ulrich Finkler and Brian Kingsbury and George Saon and David Kung and Michael Picheny},
title = {Distributed Deep Learning Strategies For Automatic Speech Recognition},
year = {2019},
url = {https://arxiv.org/abs/1904.04956},
archivePrefix = {arXiv},
eprint = {1904.04956},
}

[66] N. Carlini, A. Athalye, N. Papernot, W. Brendel, J. Rauber, D. Tsipras, I. J. Goodfellow, A. Madry, and A. Kurakin, "On Evaluating Adversarial Robustness," CoRR, vol. abs/1902.06705, 2019.
[BibTeX] [Download]

@article{CarliniAP2019,
author = {Nicholas Carlini and
Anish Athalye and
Nicolas Papernot and
Wieland Brendel and
Jonas Rauber and
Dimitris Tsipras and
Ian J. Goodfellow and
Aleksander Madry and
Alexey Kurakin},
title = {On Evaluating Adversarial Robustness},
journal = {CoRR},
volume = {abs/1902.06705},
year = {2019},
url = {http://arxiv.org/abs/1902.06705},
archivePrefix = {arXiv},
eprint = {1902.06705},
timestamp = {Sat, 02 Mar 2019 16:35:38 +0100},
biburl = {https://dblp.org/rec/bib/journals/corr/abs-1902-06705},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[67] B. Recht, R. Roelofs, L. Schmidt, and V. Shankar, "Do ImageNet Classifiers Generalize to ImageNet?," CoRR, vol. abs/1902.10811, 2019.
[BibTeX] [Download]

@article{RechtRS2019,
author = {Benjamin Recht and
Rebecca Roelofs and
Ludwig Schmidt and
Vaishaal Shankar},
title = {Do ImageNet Classifiers Generalize to ImageNet?},
journal = {CoRR},
volume = {abs/1902.10811},
year = {2019},
url = {http://arxiv.org/abs/1902.10811},
archivePrefix = {arXiv},
eprint = {1902.10811},
timestamp = {Mon, 04 Mar 2019 15:54:38 +0100},
biburl = {https://dblp.org/rec/bib/journals/corr/abs-1902-10811},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[68] R. Geirhos, C. M. R. Temme, J. Rauber, H. H. Schütt, M. Bethge, and F. A. Wichmann, "Generalisation in humans and deep neural networks," CoRR, vol. abs/1808.08750, 2018.
[BibTeX] [Download]

@article{GeirhosMR2018,
author = {Robert Geirhos and
Carlos R. Medina Temme and
Jonas Rauber and
Heiko H. Sch{\"{u}}tt and
Matthias Bethge and
Felix A. Wichmann},
title = {Generalisation in humans and deep neural networks},
journal = {CoRR},
volume = {abs/1808.08750},
year = {2018},
url = {http://arxiv.org/abs/1808.08750},
archivePrefix = {arXiv},
eprint = {1808.08750},
timestamp = {Sun, 02 Sep 2018 15:01:55 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/abs-1808-08750},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[69] B. Biggio and F. Roli, "Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning," CoRR, vol. abs/1712.03141, 2017.
[BibTeX] [Download]

@article{BiggioR2017,
author = {Battista Biggio and
Fabio Roli},
title = {Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning},
journal = {CoRR},
volume = {abs/1712.03141},
year = {2017},
url = {http://arxiv.org/abs/1712.03141},
archivePrefix = {arXiv},
eprint = {1712.03141},
timestamp = {Mon, 13 Aug 2018 16:48:36 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/abs-1712-03141},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[70] N. Akhtar and A. Mian, "Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey," CoRR, vol. abs/1801.00553, 2018.
[BibTeX] [Download]

@article{AkhtarM2018,
author = {Naveed Akhtar and
Ajmal Mian},
title = {Threat of Adversarial Attacks on Deep Learning in Computer Vision:
{A} Survey},
journal = {CoRR},
volume = {abs/1801.00553},
year = {2018},
url = {http://arxiv.org/abs/1801.00553},
archivePrefix = {arXiv},
eprint = {1801.00553},
timestamp = {Mon, 13 Aug 2018 16:48:49 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/abs-1801-00553},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[71] J. Rauber, W. Brendel, and M. Bethge, "Foolbox v0.8.0: A Python toolbox to benchmark the robustness of machine learning models," CoRR, vol. abs/1707.04131, 2017.
[BibTeX] [Download]

@article{RauberBB2017,
author = {Jonas Rauber and
Wieland Brendel and
Matthias Bethge},
title = {Foolbox v0.8.0: {A} Python toolbox to benchmark the robustness of
machine learning models},
journal = {CoRR},
volume = {abs/1707.04131},
year = {2017},
url = {http://arxiv.org/abs/1707.04131},
archivePrefix = {arXiv},
eprint = {1707.04131},
timestamp = {Mon, 13 Aug 2018 16:46:53 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/RauberBB17},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[72] I. J. Goodfellow, N. Papernot, and P. D. McDaniel, "cleverhans v0.1: an adversarial machine learning library," CoRR, vol. abs/1610.00768, 2016.
[BibTeX] [Download]

@article{GoodfellowPM2016,
author = {Ian J. Goodfellow and
Nicolas Papernot and
Patrick D. McDaniel},
title = {cleverhans v0.1: an adversarial machine learning library},
journal = {CoRR},
volume = {abs/1610.00768},
year = {2016},
url = {http://arxiv.org/abs/1610.00768},
archivePrefix = {arXiv},
eprint = {1610.00768},
timestamp = {Mon, 13 Aug 2018 16:47:06 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/GoodfellowPM16},
bibsource = {dblp computer science bibliography, https://dblp.org}
}

[73] M. Cisse, P. Bojanowski, E. Grave, Y. Dauphin, and N. Usunier, Parseval Networks: Improving Robustness to Adversarial Examples, 2017.
[BibTeX] [Download]

@misc{CisseBG2017,
author = {Moustapha Cisse and Piotr Bojanowski and Edouard Grave and Yann Dauphin and Nicolas Usunier},
title = {Parseval Networks: Improving Robustness to Adversarial Examples},
year = {2017},
url = {https://arxiv.org/abs/1704.08847},
archivePrefix = {arXiv},
eprint = {1704.08847},
}

Christoph Conrads' Blog

The Fundamental Importance of Adversarial Examples to Machine Learning