Representativeness two ways: an assessment of representativeness

Abstract

Vulnerable populations like at-risk adolescents are often difficult to study, yet the data they provide are invaluable to researchers in a number of fields, including, but not limited to health and education. Because these populations are difficult to study, some argue that any studies of these populations (or samples from these populations) have an inherent selection bias, suggesting that the results may not be generalizable to the populations studied. These arguments are made stronger when random sampling techniques are not used to identify a sample of at-risk adolescents, for example. This study examines the representativeness of a sample of at-risk adolescents in a community-based longitudinal study (the Mobile Youth Survey, or MYS) of poverty and adolescent risk in Mobile, Alabama. Further, this study examines the missing data patterns that exist in 10 waves of data collected in this study to determine which missing data mechanisms exist in this dataset, in order to conclude whether these missing data are ignorable. With over 20,000 data points, and items measuring developmental, behavioral, and psychosocial constructs, the MYS can be used by researchers in several fields, but only to the extent that the data are of high quality, that is, representative of the population in terms of demographic (grade level, gender, race, free lunch eligibility status, neighborhood type) and functional (cognitive and behavioral) characteristics. Results show that while there are concerns about the demographic representativeness of the MYS sample to the population, overall, these results are not alarming, and in fact, are somewhat expected. Further, these differences suggest that perhaps the population should be re-defined. Overall, these results demonstrate that (a) there is not an inherent sampling bias in studies of vulnerable populations; (b) in the MYS, while demographic characteristics may not always be representative of the defined population, there are no consistent differences between the sample and population with respect to functional characteristics once demographic factors have been statistically controlled; and (c) missing data can be studied as it relates to representativeness.