Discussant Commentary

Symposium: Cutscore Estimation Via Two-Stage Analysis

Jihong Zhang, Ph.D.

University of Arkansas

Theoretical and Methodological Contributions

Dr. Alfonso Martinez

  • Introduced MLE estimation using a two-stage mixture model.
  • Questioned the meaning of “mastery” across populations (e.g., 7th vs. 3rd graders).
  • Proposed viewing latent ability as a continuum with cutpoints.
  • Linked to the Hybrid Model of IRT and Latent Class Models (Yamamoto, 1982).
  • Advanced understanding of continuous-discrete latent structure integration.

Dr. Jonathan Templin

  • Developed a Bayesian two-stage model.
  • Incorporated prior knowledge for enhanced inference.
  • Estimated posterior uncertainty around cutpoints.
  • Provided transparent probabilistic classification.
  • Strengthened defensible reporting of classification precision.

Practical and Substantive Applications

Mr. Sergio Haab

  • Addressed interpretation of cutpoints.
  • Advocated for behavioral and cognitive benchmarks.
  • Supported construct validity in standard setting.
  • Emphasized alignment of scores with theoretical descriptions.

Ae Kyong Jung

  • Extended two-stage methods to CAT item selection.
  • Proposed discretizing latent traits for adaptive item selection.
  • Compared Shannon entropy (DCM) vs. D-optimality (MIRT).
  • Found D-optimality yields better item selection accuracy.
  • Balanced diagnostic precision and theta estimation.

Mr. Ahmed Bediwy

  • Applied two-stage approach to standard setting.
  • Integrated with Angoff and Bookmark methods.
  • Quantified psychometric implications of panelist decisions.
  • Proposed a hybrid model balancing evidence and expert judgment.

Integrative Reflections

Emerging Themes

  1. Two-stage modeling with diverse item response types.
  2. Bayesian approaches using different priors.
  3. Emphasis on interpretability beyond statistical adequacy.
  4. Innovations in CAT scoring incorporating cutpoints.
  5. Hybrid psychometric-standard setting models as future frontier.

Conclusion

This symposium advances both theoretical landscapes and practical applications of cutscore estimation.

As psychometricians, we must estimate, explain, and justify our models clearly.

This symposium exemplifies that mission.

Open Discussion

  • How to incorporate cutscores information in CAT?
  • Addressing non-normal latent traits.
  • Multiple cutpoints per trait — how many are meaningful?
  • Real-world meaning of cutscores.
  • Uncertainty and variability around cutscores.
  • Generalizability of cutpoints across varied populations.

References

  • Yamamoto, K. (1982). Hybrid Model of IRT and Latent Class Models.
  • Hong, Y. (2014). Efficient Models for Cognitive Diagnosis with Continuous and Mixed-Type Latent Variables.