In this paper, we examine the association between contact with migrant populations and support for the populist radical right (PRR) in Switzerland. Building on group threat and intergroup contact theories, which offer opposing predictions, and drawing on Appadurai¿s thesis of the `fear of small numbers¿, we propose a new theoretical framework to explain this association. We predict that the relationship between the size of the migrant populations and PRR voting is nonlinear: a small but noticeable minority triggers the formation of anti-immigrant attitudes, which soften as the minority grows and people start having meaningful interactions with foreigners. To test these theories, we combine individual-level data with municipality-level information. Mixed-effects multilevel models confirm that individuals in municipalities with a moderate proportion of foreigners are more likely than those with fewer or a greater number of migrants to cast their vote in support of PRR parties; this is particularly so for certain stigmatised minorities. We further explore the effect of perceived immigrant threat in moderating these relationships.