A simulation to evaluate the ability of nonmetric multidimensional scaling to recover the underlying structure of data under conditions of error, method of selection, and percent of missing pairs

A simulation was conducted to evaluate the ability of nonmetric MDS to recover the true structure of the data under conditions of proportion of missing pairs of dissimilarities, method of selection of missing pairs, and data with and without error. The percent of pairs missing in the matrix of observations had an effect on the ability of nonmetric ALSCAL to recover the true structure of the data. The results showed that with .10 missing pairs and with .20 missing pairs the recovery was excellent. With .30 missing pairs, recovery was good. With .40 missing pairs, and .50 missing pairs recovery was poor, and solutions had degenerate configurations with .80 missing pairs and .90 missing pairs. Method of missing and amount of error did not have an effect on either of two measures of recovery used: Correlations between recovered and true coordinates (CC) and the index of metric determinacy (M). Values of STRESS and values of RSQ obtained from the algorithm run in nonmetric ALSCAL SPSS did not represent the true recovery of the underlying structure. Ninety percent of STRESS values were good or excellent and one hundred percent of RSQ values were strong and significant even in the case of degenerate solutions. The true measures of recovery correlated poorly with the apparent measures of recovery. Therefore, it appears that values of STRESS and RSQ while informative with low levels of missing, are misleading when percent of missing pairs reach .30 or more. Conversely, scatter plots of monotonic transformation were excellent predictors of the quality of the solution at all levels of missing pairs. Researchers should view the apparent measures of fit obtained in the SPSS nonmetric MDS output with reservation and examine the plots of monotonic transformation to evaluate the quality of the nonmetric MDS solution.