The Fundamental Importance of Adversarial Examples to Machine Learning

Many new products nowadays were made possible by advances in machine learning, in particular image recognition and automatic speech recognition. Examples are spam filters, virtual personal assistants, traffic prediction in GPS devices, or face recognition. Real-world machine learning applications see more widespread use and enjoy ever increasing accuracy on common benchmark datasets.  Unfortunately, it has been observed that virtually all models return incorrect results if data is fed to the model that is not from common benchmark datasets, data that was purposefully but imperceptibly altered, and plain garbage data. In this blog post I will talk about the purposeful, imperceptible input modifications, so-called adversarial examples. I will present possible kinds of modifications in computer vision and requirements for successful attacks before discussing attempts to improve models. Finally, I will make two predictions about the future of machine learning models. The appendix contains remarks about the misinterpretation of the largest eigenvalue of a weight matrix; the modulus of this value is almost meaningless without considering the smallest eigenvalues, too.

Continue reading The Fundamental Importance of Adversarial Examples to Machine Learning

Faster RANLUX Pseudo-Random Number Generators

RANLUX denotes a class of high-quality pseudo-random number generators (PRNGs) with long periods, solid theoretical foundations, and uses in computational physics. These generators are built on top of add-with-carry and subtract-with-borrow PRNGs which allow customization through three user-provided parameters yet there are only two variants in use today working internally with 24-bit and 48-bit arithmetic; this comes at a cost because modern computers do not possess instructions for 24-bit or 48-bit arithmetic. In this blog post I will present the theory behind RANLUX PRNGs, a list of possible parameters for 16-, 32-, and 64-bit RANLUX generators, criteria for selecting generators as well as benchmarks comparing the most competitive RANLUX variants with the RANLUX generators in the C++11 standard library, the Mersenne Twister, and xoshiro128+.

Some of the new RANLUX flavors are easily implemented in C++11 (sometimes in only two lines of code), they are  faster than the existing RANLUX flavors, they retain all desirable properties of a RANLUX generator, and they pass all DieHarder and TestU01 BigCrush PRNG tests.

The programs used to compute the RANLUX parameters are called ranlux-tools. They are free software under the terms of the Mozilla Public License 2.0 and can be found online:

Continue reading Faster RANLUX Pseudo-Random Number Generators

Rademacher Floating Point Library Released

The Rademacher Floating Point Library provides a C++11 class computing random uniformly distributed floating-point values in a given half-open interval [a,b). In comparison to every other existing software, this library

  • computes values that are actually uniformly distributed,
  • draws randomly from all floating-point values in range,
  • avoids round-off errors causing out-of-range values,
  • uses only integer arithmetic, and
  • works with built-in C++ float types as well with custom floating-point types.

The Rademacher FPL is licensed under the Mozilla Public License 2.0 and therefore free software ("free" as in "free beer" as well as "free" as in "freedom"). The source code can be found online:

Hylas and the Nymphs

Between January 26 and February 3, 2018, the Manchester Art Gallery (MAG) removed John William Waterhouse' painting Hylas and the Nymphs. In a press release, the gallery states the reason for its action:

This gallery presents the female body as either a ‘passive decorative form’ or a ‘femme fatale’. Let’s challenge this Victorian fantasy!

The removal was filmed and is part of an exhibition by Sonia Boyce.

The painting shows the moment before Heracles' lover Hylas is abducted by the Nymphs. Neither the painting Hylas and the Water Nymphs by Henrietta Rae showing the very same scene nor The Sirens and Ulysses by William Etty were removed by the MAG.

You can find all paintings mentioned inside this blog post.

Continue reading Hylas and the Nymphs

Wake-Up Call

Partyzone Berlin: So gesichert wie nie (which translates roughly to Party zone Berlin: Secured like never before) is  the title of a newspaper article about the heightened security measures for the New Year's Eve party in Berlin. Expecting up to one million visitors, there will be 900 policemen (150 more than the year before), a ban of backpacks as well as big bags, and a fenced-in exclusion  zone around the event area. One year later, the expected 200.000 attendees of the New Year's Eve Party on December 31, 2016, will be protected by 1.000 policemen armed with submachine guns and the access roads to the exclusion zone will be blocked by armored vehicles, trucks, and concrete blocks (source with photos).

To appreciate the degree of desperation, you need to know that in the last two years, only two out of formerly 17 shooting ranges (five out of formerly 73 shooting lanes) were available for Berlin's police force for some time. On average, police officers had two minutes of shooting exercises per year and about 700 policemen were unable to complete their obligatory one-time training per year (source, source, source, source).

After the September 11 attacks, I started to see groups of policemen patroling airports, one of them being armed with a submachine gun. Next, policemen across Germany started wearing ballistic vests. On New Year's Eve, I was at the Berlin city center less than one kilometre away from the German parliament and most policemen carried submachine guns; there were even three policemen with submachine guns guarding the club where I celebrated New Year's Eve. Although Christmas season is over now, you can see policemen in Berlin's government district with submachine guns every day. It is planned that Berlin's police will upgrade its existing ballistic vests and acquire 6300 new ballistic vests, 12.000 new pistols, as well as new submachine guns for 8.8 Million Euros (there are 17.000 policemen in Berlin).

When I visited New York in 2009, I was dumbfounded seeing two policemen patroling the streets wearing body armor and carrying machine guns. In 2016, heavily armed police in Germany is a common sight. Something is amiss in Germany and militarizing the police will only suppress the symptoms but not cure the disease.

Why I Chose the Mozilla Public License 2.0

Selecting a software license requires a conscious effort  because of the various rights and obligations that come with it. For me, the most important aspects of a license were

  • using an existing open source license,
  • commercial use clauses,
  • patent protection,
  • the obligation to disclose changes,
  • the ability to revoke the license as the sole copyright holder, and
  • compatibility with BSD-licensed code.

In this blog post I elaborate on why I chose the Mozilla Public License 2.0 for my eigensolver DCGeig based on these features.

Continue reading Why I Chose the Mozilla Public License 2.0

The Discretized Laplace Operator on Hyperrectangles with Zero Dirichlet Boundary Conditions

In this blog post, I present stiffness and mass matrix as well as eigenvalues and eigenvectors of the Laplace operator (Laplacian) on domains (0, \ell), (0, \ell_1) \times (0, \ell_2), and so on (hyperrectangles) with zero Dirichlet boundary conditions discretized with the finite difference method (FDM) and the finite element method (FEM) on equidistant grids. For the FDM discretization, we use the central differences scheme with the standard five-point stencil in 2D. For the FEM, the ansatz functions are the hat functions. The matrices, standard eigenvalue problems A v = \sigma v, and generalized eigenvalue problems K w = \tau M w arising from the discretization lend themselves for test problems in numerical linear algebra because they are well-conditioned, not diagonal, and the matrix dimension can be increased arbitrarily.

Python code generating the matrices and their eigenpairs can be found in my git repository discrete-laplacian.

Continue reading The Discretized Laplace Operator on Hyperrectangles with Zero Dirichlet Boundary Conditions

CMake: "Errors occurred during the last pass"

You executed

ccmake -G ninja /path/to/source-code

on the command line and after pressing

[c] to configure

ccmake presents you an empty window with the message

Errors occurred during the last pass

in the status bar. The cause of this problem is the misspelled generator name on the ccmake command line (argument -G): the generator is called Ninja with capital N as the first letter.