The smartphone revolution has placed powerful, multipurpose devices in the hands of youth across the globe, prompting worries about the potential negative consequences of these technologies on mental health. Many assessment tools have been created, seeking to classify individuals into problematic and non-problematic smartphone users. These are identified using a cutoff value: a threshold, within the scale range, at which higher scores are expected to be associated with negative outcomes. Lacking a clinical assessment of individuals, the establishment of this threshold is challenging. We illustrate this difficulty by calculating cutoff values for the Short Version of the Smartphone Addiction Scale (SAS-SV) in 13 Spanish-speaking samples in 11 countries, using common procedures (i.e., reliability, validity, ROC methodology). After showing that results can be very heterogeneous (i.e., they lead to diverse cutoff points and rates of addiction) depending on the decisions made by the researchers, we call for caution in the use of these classifications, particularly when researchers lack a clinical definition of true addiction-as is the case with most available scales in the field of behavioral addictions-which can cause an unnecessary public health alert.