Research and Publications - Department of Gender and Race Studies

Permanent URI for this collection

https://ir.ua.edu/handle/123456789/2843

Browse

Now showing 1 - 16 of 16

Incorporating Criterion Ratings Into Model-Based Rater Monitoring Procedures Using Latent-Class Signal Detection Theory
(Sage, 2017) Patterson, Brian F.; Wind, Stefanie A.; Engelhard, George, Jr.; University of Alabama Tuscaloosa; University of Georgia
This study presents a new criterion-referenced approach for exploring rating quality within the framework of latent-class signal detection theory (LC-SDT) that goes beyond commonly used reliability indices, and provides substantively meaningful indicators of rater accuracy that can be used to inform rater training and monitoring at the individual rater level. Specifically, this study illustrates a flexible application of restricted LC-SDT modeling, in which restrictions can be specified for the true latent classification to reflect the unique characteristics of a particular assessment context. While the LC-SDT modeling framework provides immediately useful characterizations of raters' behavior, the restricted LC-SDT offers complementary evidence to further support the monitoring of rater behavior by bringing criterion ratings to bear. This study uses ratings from a large-scale writing assessment, and findings suggest that the criterion (i.e., restricted) LC-SDT provides useful information about rating quality for operational raters relative to criterion ratings, which may ultimately inform rater training and monitoring procedures.
The Use of Multivariate Generalizability Theory to Evaluate the Quality of Subscores
(Sage, 2018) Jiang, Zhehan; Raymond, Mark; University of Alabama Tuscaloosa
Conventional methods for evaluating the utility of subscores rely on reliability and correlation coefficients. However, correlations can overlook a notable source of variability: variation in subtest means/difficulties. Brennan introduced a reliability index for score profiles based on multivariate generalizability theory, designated as G, which is sensitive to variation in subtest difficulty. However, there has been little, if any, research evaluating the properties of this index. A series of simulation experiments, as well as analyses of real data, were conducted to investigate G under various conditions of subtest reliability, subtest correlations, and variability in subtest means. Three pilot studies evaluated G in the context of a single group of examinees. Results of the pilots indicated that G indices were typically low; across the 108 experimental conditions, G ranged from .23 to .86, with an overall mean of 0.63. The findings were consistent with previous research, indicating that subscores often do not have interpretive value. Importantly, there were many conditions for which the correlation-based method known as proportion reduction in mean-square error (PRMSE; Haberman, 2006) indicated that subscores were worth reporting, but for which values of G fell into the .50s, .60s, and .70s. The main study investigated G within the context of score profiles for examinee subgroups. Again, not only G indices were generally low, but it was also found that G can be sensitive to subgroup differences when PRMSE is not. Analyses of real data and subsequent discussion address how G can supplement PRMSE for characterizing the quality of subscores.
An exploratory decision tree analysis to predict physical activity compliance rates in breast cancer survivors
(Routledge, 2019) Paxton, Raheem J.; Zhang, Lingfeng; Wei, Changshuai; Price, Daniel; Zhang, Fan; Courneya, Kerry S.; Kakadiaris, Ioannis A.; University of Alabama Tuscaloosa; University of Houston; Amazon.com; University of North Texas Denton; University of Alberta
Background: The study of physical activity in cancer survivors has been limited to one cause, one effect relationships. In this exploratory study, we used recursive partitioning to examine multiple correlates that influence physical activity compliance rates in cancer survivors. Methods: African American breast cancer survivors (N = 267, Mean age = 54 years) participated in an online survey that examined correlates of physical activity. Recursive partitioning (RP) was used to examine complex and nonlinear associations between sociodemographic, medical, cancer-related, theoretical, and quality of life indicators. Results: Recursive partitioning revealed five distinct groups. Compliance with physical activity guidelines was highest (82% met guidelines) among survivors who reported higher mean action planning scores (P < 0.001) and lower mean barriers to physical activity (P = 0.035). Compliance with physical activity guidelines was lowest (9% met guidelines) among survivors who reported lower mean action and coping (P = 0.002) planning scores. Similarly, lower mean action planning scores and poor advanced lower functioning (P = 0.034), even in the context of higher coping planning scores, resulted in low physical activity compliance rates (13% met guidelines). Subsequent analyses revealed that body mass index (P = 0.019) and number of comorbidities (P = 0.003) were lowest in those with the highest compliance rates. Conclusion: Our findings support the notion that multiple factors determine physical activity compliance rates in African American breast cancer survivors. Interventions that encourage action and coping planning and reduce barriers in the context of addressing function limitations may increase physical activity compliance rates.
Examining the Impacts of Rater Effects in Performance Assessments
(Sage, 2019) Wind, Stefanie A.; University of Alabama Tuscaloosa
Rater effects such as severity, centrality, and misfit are recurrent concerns in performance assessments. Despite their persistence in operational assessment settings and frequent discussion in research, researchers have not fully explored the impacts of rater effects as they relate to estimates of student achievement. The purpose of this study is to explore the impacts of rater severity, centrality, and misfit on student achievement estimates and on classification decisions. The results suggest that these three types of rater effects have substantial impacts on estimates of student achievement and on classification decisions that impact the fairness of rater-mediated assessments. Accordingly, it is essential that researchers and practitioners evaluate ratings across all stages of rater-mediated assessment procedures, including rater training and operational scoring.
Evaluating the Fit of Sequential G-DINA Model Using Limited-Information Measures
(Sage, 2020) Ma, Wenchao; University of Alabama Tuscaloosa
Limited-information fit measures appear to be promising in assessing the goodness-of-fit of dichotomous response cognitive diagnosis models (CDMs), but their performance has not been examined for polytomous response CDMs. This study investigates the performance of the M-ord statistic and standardized root mean square residual (SRMSR) for an ordinal response CDM-the sequential generalized deterministic inputs, noisy "and" gate model. Simulation studies showed that the M-ord statistic had well-calibrated Type I error rates, but the correct detection rates were influenced by various factors such as item quality, sample size, and the number of response categories. In addition, the SRMSR was also influenced by many factors and the common practice of comparing the SRMSR against a prespecified cut-off (e.g., .05) may not be appropriate. A set of real data was analyzed as well to illustrate the use of M-ord statistic and SRMSR in practice.
A Sequential Higher Order Latent Structural Model for Hierarchical Attributes in Cognitive Diagnostic Assessments
(Sage, 2020) Zhan, Peida; Ma, Wenchao; Jiao, Hong; Ding, Shuliang; Zhejiang Normal University; University of Alabama Tuscaloosa; University of Maryland College Park; Jiangxi Normal University
The higher-order structure and attribute hierarchical structure are two popular approaches to defining the latent attribute space in cognitive diagnosis models. However, to our knowledge, it is still impossible to integrate them to accommodate the higher-order latent trait and hierarchical attributes simultaneously. To address this issue, this article proposed a sequential higher-order latent structural model (LSM) by incorporating various hierarchical structures into a higher-order latent structure. The feasibility of the proposed higher-order LSM was examined using simulated data. Results indicated that, in conjunction with the deterministic-inputs, noisy "and" gate model, the sequential higher-order LSM produced considerable improvement in person classification accuracy compared with the conventional higher-order LSM, when a certain attribute hierarchy existed. An empirical example was presented as well to illustrate the application of the proposed LSM.
Estimating Cognitive Diagnosis Models in Small Samples: Bayes Modal Estimation and Monotonic Constraints
(Sage, 2021) Ma, Wenchao; Jiang, Zhehan; University of Alabama Tuscaloosa; Peking University
Despite the increasing popularity, cognitive diagnosis models have been criticized for limited utility for small samples. In this study, the authors proposed to use Bayes modal (BM) estimation and monotonic constraints to stabilize item parameter estimation and facilitate person classification in small samples based on the generalized deterministic input noisy "and" gate (G-DINA) model. Both simulation study and real data analysis were used to assess the utility of the BM estimation and monotonic constraints. Results showed that in small samples, (a) the G-DINA model with BM estimation is more likely to converge successfully, (b) when prior distributions are specified reasonably, and monotonicity is not violated, the BM estimation with monotonicity tends to produce more stable item parameter estimates and more accurate person classification, and (c) the G-DINA model using the BM estimation with monotonicity is less likely to overfit the data and shows higher predictive power.
Entangled Time Hops: Doomsday Clocks, Pandemics, and Qualitative Research's Responsibility
(Sage, 2021) Shelton, Stephanie Anne; University of Alabama Tuscaloosa
This article explores the micro- and macro-level implications of the dual global pandemics of COVID-19 and racism through a narrative structure based on Barad's discussion of "timehops." Weaving personal, national, and international stories, the article explores qualitative research's responsibility and potential to offer new ways to respond to the entanglements of people, places, moments, materials, and these pandemics.
An Iterative Parametric Bootstrap Approach to Evaluating Rater Fit
(Sage, 2021) Guo, Wenjing; Wind, Stefanie A.; University of Alabama Tuscaloosa
When analysts evaluate performance assessments, they often use modern measurement theory models to identify raters who frequently give ratings that are different from what would be expected, given the quality of the performance. To detect problematic scoring patterns, two rater fit statistics, the infit and outfit mean square error (MSE) statistics are routinely used. However, the interpretation of these statistics is not straightforward. A common practice is that researchers employ established rule-of-thumb critical values to interpret infit and outfit MSE statistics. Unfortunately, prior studies have shown that these rule-of-thumb values may not be appropriate in many empirical situations. Parametric bootstrapped critical values for infit and outfit MSE statistics provide a promising alternative approach to identifying item and person misfit in item response theory (IRT) analyses. However, researchers have not examined the performance of this approach for detecting rater misfit. In this study, we illustrate a bootstrap procedure that researchers can use to identify critical values for infit and outfit MSE statistics, and we used a simulation study to assess the false-positive and true-positive rates of these two statistics. We observed that the false-positive rates were highly inflated, and the true-positive rates were relatively low. Thus, we proposed an iterative parametric bootstrap procedure to overcome these limitations. The results indicated that using the iterative procedure to establish 95% critical values of infit and outfit MSE statistics had better-controlled false-positive rates and higher true-positive rates compared to using traditional parametric bootstrap procedure and rule-of-thumb critical values.
Detecting Differential Item Functioning Using Multiple-Group Cognitive Diagnosis Models
(Sage, 2021) Ma, Wenchao; Terzi, Ragip; de la Torre, Jimmy; University of Alabama Tuscaloosa; Harran University; University of Hong Kong
This study proposes a multiple-group cognitive diagnosis model to account for the fact that students in different groups may use distinct attributes or use the same attributes but in different manners (e.g., conjunctive, disjunctive, and compensatory) to solve problems. Based on the proposed model, this study systematically investigates the performance of the likelihood ratio (LR) test and Wald test in detecting differential item functioning (DIF). A forward anchor item search procedure was also proposed to identify a set of anchor items with invariant item parameters across groups. Results showed that the LR and Wald tests with the forward anchor item search algorithm produced better calibrated Type I error rates than the ordinary LR and Wald tests, especially when items were of low quality. A set of real data were also analyzed to illustrate the use of these DIF detection procedures.
The Impact of the 2016 US Presidential Elections on Transgender and Gender Diverse People
(Springer, 2021) Price, Sarah F.; Puckett, Jae; Mocarski, Richard; University of Alabama Tuscaloosa; Michigan State University; University Nebraska Kearney
Introduction With Trump's presidency came a rise in the oppression of transgender and gender diverse (TGD) people, as the nation witnessed a removal of protections for TGD people. Methods We examined the daily experiences of 181 TGD individuals (ages 16-40, M age = 25.6) through their reflections about daily stressors over the course of 8 weeks (data collected fall 2015-summer 2017), some of which reflected shifts during the election period. Results During the 2016 presidential election, participants reported a rise in marginalization stress and the subsequent impact on safety, mental health, and well-being. There were three emergent themes: External Rejection and Stigma from Dominant Culture; Supporting the TGD Community; and Fear for the Self and Development of Proximal Stressors. Conclusions In line with marginalization stress theory, participants vocalized the progression from exterior stigmatization to proximal stressors and their heightened sense of vigilance and fear of the dominant culture. Policy Implications Based on the results of this study, policy makers and TGD advocates must work to ensure that political rhetoric and action do not serve to further marginalize and erase TGD communities.
American Indian women cancer survivors' perceptions and experiences with conventional and non-conventional mental health care for depressive symptoms
(Routledge, 2021) Burnette, Catherine E.; Liddell, Jessica; Roh, Soonhee; Lee, Yeon-Shim; Lee, Hee Yun; Tulane University; University of South Dakota; San Francisco State University; University of Alabama Tuscaloosa
Background: Despite cancer and depression being disproportionately high for American Indian and Alaska Native (AI/AN) women, such cancer survivors' help-seeking practices and perceptions related to depression are absent in extant research. A broader context of historical oppression has set the stage for unequal health outcomes and access to quality services. The purpose of this article was to explore AI women cancer survivors' experiences with conventional mental health services and informal and tribally-based assistance, as well as barriers related to mental health service utilization. Methods: A qualitative descriptive study methodology, with qualitative content analysis, was used to examine the experiences of AI women cancer survivors as they related to help-seeking experiences for depressive symptoms. The sample included 43 AI women cancer survivors (n = 14 breast cancer, n = 14 cervical cancer, and n = 15 colon and other types of cancer survivors). Results: Since receiving a cancer diagnosis, 26 (62%) participants indicated they had feelings of depression. Some participants (n = 13) described mixed perceptions of the mental health service system. Generally, participants viewed families and informal support systems as primary forms of assistance, whereas conventional services were reported as a supplementary or 'as needed' forms of support, particularly when the informal support system was lacking. Participants received help in the forms of psychotropic medications and psychotherapy, as well as help from family and AI-specific healing modalities (e.g. sweat lodges and healing ceremonies). Stigma and confidentiality concerns were primary barriers to utilizing conventional services as described by 12 (29%) participants. Discussion: Participants' help primarily came from family and tribally-based entities, with conventional mental health care being more salient when informal supports were lacking. The mixed perceptions espoused by participants may be related to a broader context of historical oppression; family and social support and tribally-based services may be protective factors for cancer survivors with depression.
Analyzing factors associated with decisional stage of adopting breast cancer screening among Korean American women using precaution adoption process model
(Routledge, 2021) Jin, Seok Won; Lee, Jongwook; Lee, Hee Yun; University of Memphis; University of Minnesota Twin Cities; University of Alabama Tuscaloosa
Background: Korean American (KA) women have experienced higher prevalence and lower survival rates of breast cancer (BC) than other ethnic groups in the United States. However, BC screening rates for KA women remain significantly lower than the national target (81.1%) specified by Healthy People 2020. Few studies have explained how the decision to adopt BC screening occurs and progresses and what factors contribute to this decision among KA women. This study used Weinstein's Precaution Adoption Process Model (PAPM) as a theoretical framework to examine characteristics and factors associated with the decisional stage of mammography adoption. Methods: A cross-sectional self-report survey was administered among KA women (N = 308) ages 50-80 from the Atlanta metropolitan area. A total of 281 KA women completed the survey, answering questions about socio-demographics, health-related information, mammography history, doctor recommendation, BC screening knowledge, self-efficacy for BC screening, decisional balance scores on attitudes and beliefs pertaining to mammography, and the seven-stage PAPM. Results: KA women reported a low rate of mammography uptake with about 24% and 35% of the participants undergoing mammography within the last year and two years, respectively. KA women in stages 5 (decided yes), 6 (action), and 7 (maintenance) were likely to have increased screening-related knowledge, positive decisional balance, and regular medical check-up compared to those in stages 1 (unaware), 2 (unengaged), and 3 (deciding). Conclusion: This study highlights important factors that could potentially facilitate BC screening among KA women in Georgia. The findings also provide implications for interventions and practice for increasing mammography screening among medically underserved populations.
Review of The Enduring Kiss: Seven Short Lessons on Love
(2022) Roach, Catherine; University of Alabama Tuscaloosa
Bridging Models of Biometric and Psychometric Assessment: A Three-Way Joint Modeling Approach of Item Responses, Response Times, and Gaze Fixation Counts
(Sage, 2022) Man, Kaiwen; Harring, Jeffrey R.; Zhan, Peida; University of Alabama Tuscaloosa; University of Maryland College Park; Zhejiang Normal University
Recently, joint models of item response data and response times have been proposed to better assess and understand test takers' learning processes. This article demonstrates how biometric information such as gaze fixation counts obtained from an eye-tracking machine can be integrated into the measurement model. The proposed joint modeling framework accommodates the relations among a test taker's latent ability, working speed and test engagement level via a person-side variance-covariance structure, while simultaneously permitting the modeling of item difficulty, time-intensity, and the engagement intensity through an item-side variance-covariance structure. A Bayesian estimation scheme is used to fit the proposed model to data. Posterior predictive model checking based on three discrepancy measures corresponding to various model components are introduced to assess model-data fit. Findings from a Monte Carlo simulation and results from analyzing experimental data demonstrate the utility of the model.
List-keepers and other carrier bag stories: Academic mothers' (in)visible labor during the COVID-19 pandemic
(Pergamon, 2023) Guyotte, Kelly W.; Melchior, Shelly; Coogler, Carlson H.; Shelton, Stephanie Anne; University of Alabama Tuscaloosa
Beginning in 2020, the COVID-19 pandemic disrupted familiar rhythms of work and life when academic women from the United States sheltered-in-place in their homes. The pandemic brought forth challenges which accentuated that caregiving with little or no support disproportionately affected mothers' abilities to navigate their new lives inside the home, where work and caregiving abruptly collided. This article takes on the (in)visible labor of academic mothers during this time-the labor mothers saw and viscerally experienced, yet that which was often unseen/unexperienced by others. Using Ursula K. Le Guin's Carrier Bag Theory as a conceptual framework, the authors engage with interviews of 54 academic mothers through a feminist-narrative lens. They craft stories of carrying (in)visible labor, isolation, simultaneity, and list-keeping as they navigate the mundaneness of everyday pandemic home/work/life. Through unrelenting responsibilities and expectations, they each find ways to carry it all, as they carry on.

UA cloudflare authentication

Browse

Browsing Research and Publications - Department of Gender and Race Studies by Issue Date

Results Per Page

Sort Options