Show simple item record

dc.contributor.authorBORGSTROM, MARK CRAIG.
dc.creatorBORGSTROM, MARK CRAIG.en_US
dc.date.accessioned2011-10-31T16:59:30Zen
dc.date.available2011-10-31T16:59:30Zen
dc.date.issued1987en_US
dc.identifier.urihttp://hdl.handle.net/10150/184127en
dc.description.abstractWhen studying detection systems, parameters associated with the Receiver Operating Characteristic (ROC) curve are often estimated to assess system performance. In some applied settings it is often not possible to test the detection system with large numbers of stimuli. The resulting small sample statistics many have undesirable properties. The characteristics of these small sample ROC estimators were examined in a Monte Carlo simulation. Three popular ROC parameters were chosen for study. One of the parameters was a single parameter index of system performance, Area under the ROC curve. The other parameters, ROC intercept and slope, were considered as a pair. ROC intercept and slope were varied along with sample size and points on the certainty rating scale to form a four way factorial design. Several types of estimators were examined. For the parameter, Area under the curve, Maximum Likelihood (ML), three types of Least Squares (LS), and Distribution Free (DF) estimators were considered. Except for the DF estimator, the same estimators were considered for the parameters, intercept and slope. These estimators were compared with respect to three characteristics: bias, efficiency, and consistency. For Area under the curve, the ML estimator was the least biased. The DF estimator was the most efficient, and all the estimators except the DF estimator appeared to be consistent. For intercept and slope the LS estimator that minimized vertical error of the points from the ROC curve (line) was the least biased for both estimators. This LS estimator was also the most efficient. This estimator along with the ML estimator also appeared to be the most consistent. The other two estimators had no significant trend toward consistency. These results along with other findings, illustrate that different estimators may be "best" for different sample sizes and for different parameters. Therefore, researchers should carefully consider the characteristics of ROC estimators before using them as indices of system performance.
dc.language.isoenen_US
dc.publisherThe University of Arizona.en_US
dc.rightsCopyright © is held by the author. Digital access to this material is made possible by the University Libraries, University of Arizona. Further transmission, reproduction or presentation (such as public display or performance) of protected items is prohibited except with permission of the author.en_US
dc.subjectSignal detection (Psychology)en_US
dc.subjectParameter estimation.en_US
dc.titleESTIMATION OF RECEIVER OPERATING CHARACTERISTIC (ROC) CURVE PARAMETERS: SMALL SAMPLE PROPERTIES OF ESTIMATORS.en_US
dc.typetexten_US
dc.typeDissertation-Reproduction (electronic)en_US
dc.identifier.oclc698483837en_US
thesis.degree.grantorUniversity of Arizonaen_US
thesis.degree.leveldoctoralen_US
dc.identifier.proquest8716347en_US
thesis.degree.disciplineEducational Foundations and Administrationen_US
thesis.degree.disciplineGraduate Collegeen_US
thesis.degree.namePh.D.en_US
refterms.dateFOA2018-08-22T15:49:38Z
html.description.abstractWhen studying detection systems, parameters associated with the Receiver Operating Characteristic (ROC) curve are often estimated to assess system performance. In some applied settings it is often not possible to test the detection system with large numbers of stimuli. The resulting small sample statistics many have undesirable properties. The characteristics of these small sample ROC estimators were examined in a Monte Carlo simulation. Three popular ROC parameters were chosen for study. One of the parameters was a single parameter index of system performance, Area under the ROC curve. The other parameters, ROC intercept and slope, were considered as a pair. ROC intercept and slope were varied along with sample size and points on the certainty rating scale to form a four way factorial design. Several types of estimators were examined. For the parameter, Area under the curve, Maximum Likelihood (ML), three types of Least Squares (LS), and Distribution Free (DF) estimators were considered. Except for the DF estimator, the same estimators were considered for the parameters, intercept and slope. These estimators were compared with respect to three characteristics: bias, efficiency, and consistency. For Area under the curve, the ML estimator was the least biased. The DF estimator was the most efficient, and all the estimators except the DF estimator appeared to be consistent. For intercept and slope the LS estimator that minimized vertical error of the points from the ROC curve (line) was the least biased for both estimators. This LS estimator was also the most efficient. This estimator along with the ML estimator also appeared to be the most consistent. The other two estimators had no significant trend toward consistency. These results along with other findings, illustrate that different estimators may be "best" for different sample sizes and for different parameters. Therefore, researchers should carefully consider the characteristics of ROC estimators before using them as indices of system performance.


Files in this item

Thumbnail
Name:
azu_td_8716347_sip1_w.pdf
Size:
3.413Mb
Format:
PDF

This item appears in the following Collection(s)

Show simple item record