Biostatistics

Biostatistics Advance Access originally published online on November 3, 2006
Biostatistics 2007 8(2):474-484; doi:10.1093/biostatistics/kxl038

This Article Abstract FREE Full Text (PDF) OA All Versions of this Article: 8/2/474    most recent kxl038v1 Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to My Personal Archive Download to citation manager Disclaimer Google Scholar Articles by Pepe, M. S. Articles by Janes, H. Search for Related Content PubMed PubMed Citation Articles by Pepe, M. S. Articles by Janes, H. Social Bookmarking          What's this?

© 2006 The Authors
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/2.0/uk/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Insights into latent class analysis of diagnostic test performance

Margaret Sullivan Pepe^*

Department of Biostatistics, University of Washington and Fred Hutchinson Cancer Research Center, 1100 Fairview Avenue North, M2-B500, Seattle, WA 98109, USA mspepe{at}u.washington.edu

Holly Janes

Department of Biostatistics, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD 21205, USA

^* To whom correspondence should be addressed.

	SUMMARY

TOP SUMMARY 1. INTRODUCTION 2. CLASSIC LATENT CLASS... 3. ANALYTIC EXPRESSIONS FOR... 4. PARAMETER INTERPRETATIONS VIA... 5. DISCUSSION APPENDIX APPENDIX B REFERENCES

 
Latent class analysis is used to assess diagnostic test accuracy when a gold standard assessment of disease is not available but results of multiple imperfect tests are. We consider the simplest setting, where 3 tests are observed and conditional independence (CI) is assumed. Closed-form expressions for maximum likelihood parameter estimates are derived. They show explicitly how observed 2- and 3-way associations between test results are used to infer disease prevalence and test true- and false-positive rates. Although interesting and reasonable under CI, the estimators clearly have no basis when it fails. Intuition for bias induced by conditional dependence follows from the analytic expressions. Further intuition derives from an Expectation Maximization (EM) approach to calculating the estimates. We discuss implications of our results and related work for settings where more than 3 tests are available. We conclude that careful justification of assumptions about the dependence between tests in diseased and nondiseased subjects is necessary in order to ensure unbiased estimates of prevalence and test operating characteristics and to provide these estimates clinical interpretations. Such justification must be based in part on a clear clinical definition of disease and biological knowledge about mechanisms giving rise to test results.

Keywords: Errors in variables; Factor analysis; Imperfect reference test; Item response theory; Latent variables; Sensitivity; Specificity

	1. INTRODUCTION

TOP SUMMARY 1. INTRODUCTION 2. CLASSIC LATENT CLASS... 3. ANALYTIC EXPRESSIONS FOR... 4. PARAMETER INTERPRETATIONS VIA... 5. DISCUSSION APPENDIX APPENDIX B REFERENCES

 
Assessments of the presence or absence of a condition cannot always be made with certainty. This is particularly true in the development of new diagnostic tests, where the very reason a new test is being developed is often because the best available test for the condition is not considered adequately accurate. A problem then arises: How can the accuracy of a new test be evaluated when there is no gold standard against which to compare it? Latent class analysis has been proposed as a statistical technique that allows such an assessment (Walter and Irwig, 1988; Dawid and Skene, 1979). Briefly, a probabilistic model is assumed for the relationship between the new diagnostic test, one or more imperfect "reference" tests, and the unobserved, or latent, disease status. The likelihood is then maximized to provide estimates of the sensitivity and specificity of the new diagnostic test. This approach is quite popular. It has been used to study markers of Behcet's disease (Ferraz and others, 1995), gastro-oesophageal reflux disease (Moayyedi and others, 2004), visceral leishmaniasis (Boelaert and others, 2004), and acute bacterial rhinosinusitis (Young and others, 2003). Moreover, it has received substantial attention from statistical methodologists to extend its applications (Qu and others, 1996; Dendukuri and Joseph, 2001; Hui and Zhou, 1998). In this paper, we discuss some inherent limitations of the latent class analysis approach.

The latent class approach has already been criticized on several grounds (Pepe and Alonzo, 2001; Pepe, 2003, pp. 203–205; Albert and Dodd, 2004). First, the analysis can and often does proceed without any formal clinical definition of disease. To the extent that disease is not defined, it follows that prevalence and test accuracy parameters are not well defined. Second, the assumed latent class model is not fully testable with the observed data, and, if the model is incorrect, it is not clear that the resulting estimates are meaningful. Third, the latent class estimates of test accuracy are obtained through a maximum likelihood procedure that does not make clear how the estimates are calculated from the raw data. This third criticism admittedly applies to many statistical techniques. In this paper, we address it by deriving analytic forms for the estimators. We determine how the raw data frequencies are used explicitly in calculating the estimates. The expressions are particularly useful for assessing the merit of the estimators when the assumed latent class structure fails.

The paper is organized as follows: We first briefly summarize the latent class analysis technique and the conditional independence (CI) assumption on which the classical latent class model is based. In Section 3, we provide analytical forms for the latent class estimators and discuss their validity both when the CI assumption holds and when it fails to hold. The EM algorithm is used in Section 4 to demonstrate relationships among the parameter estimates and to provide expressions which allow us to assess the bias in the estimates caused by conditional dependence. In Sections 3 and 4, we focus on the special case where 3 tests are available. We close with a broader discussion about the role of latent class modeling for evaluating diagnostic tests in the absence of a gold standard.

	2. CLASSIC LATENT CLASS ANALYSIS

TOP SUMMARY 1. INTRODUCTION 2. CLASSIC LATENT CLASS... 3. ANALYTIC EXPRESSIONS FOR... 4. PARAMETER INTERPRETATIONS VIA... 5. DISCUSSION APPENDIX APPENDIX B REFERENCES

 
Let the unobserved latent binary variable D indicate the presence (D = 1) or absence (D = 0) of the condition. Observed are the results of K binary test variables, {Y₁,...,Y_K}, for each of i = 1,...,n subjects. One of these variables may be the best available reference test, and the others may be new tests. A statistical model with parameter is assumed for the joint distribution of {Y₁,...,Y_K} given D, denoted by P(Y₁,...,Y_K|D). If the model has sufficient structure, and the prevalence, = P(D = 1), can be estimated by maximizing the likelihood function (,) = {P(Y_i1,...,Y_iK|D = 1) + (1 – )2P(Y_i1,...,Y_iK|D = 0)}.

The simplest and the most popular model for P(Y₁,...,Y_K|D) assumes CI: given D, the test variables {Y₁,...,Y_K} are statistically independent, and

Writing the true- and false-positive rates as _k = P(Y_ik = 1|D = 1) and _k = P(Y_ik = 1|D = 0), respectively, the parameters are and = {(_k,_k),k = 1,...,K}. With a minimum of K = 3 observed tests, the CI likelihood can be maximized because the number of parameters, (2K + 1), is equal to the available degrees of freedom, (2^K – 1). The CI assumption is the keystone of the classical latent class approach. The assumption states that, conditional on disease status, the results of the K tests are independent and knowledge of one test result gives no information about other test results.

To illustrate classical latent class analysis (LCA), consider the data shown in Table 1 for 3 tests of hearing impairment measured on n = 666 subjects, reproduced from Pepe (2003, p. 201). The maximum likelihood estimates of the 7 parameters, = prevalence and (_k,_k) for each of the 3 tests, are

View this table: [in this window] [in a new window]   Table 1. Results of the 3 tests for hearing impairment performed on n = 666 subjects

 

	3. ANALYTIC EXPRESSIONS FOR ESTIMATES

TOP SUMMARY 1. INTRODUCTION 2. CLASSIC LATENT CLASS... 3. ANALYTIC EXPRESSIONS FOR... 4. PARAMETER INTERPRETATIONS VIA... 5. DISCUSSION APPENDIX APPENDIX B REFERENCES

 
Suppose there are K = 3 observed tests, and write the probabilities of observable data as p_k = P(Y_k = 1),k = 1,2,3;p_kj = P(Y_k = 1,Y_j = 1),j > k;andp₁₂₃ = P(Y₁ = 1,Y₂ = 1,Y₃ = 1). The same notation with a "hat" denotes the observed frequency. In Appendix A, we derive the following analytic expressions for the LCA parameter estimates under the CI assumption:

(3.1)

(3.2)

where

and

(3.3)

where

For the audiology data, the frequencies in Table 1 yield . Using these estimates, we arrive at exactly the same values of , and as those calculated earlier by maximizing the likelihood. (Note here that there are 2 solutions for , one larger than and the other smaller. We choose the one that maximizes the likelihood L = (1 – ₁)^{1 – Y₁}(1 – ₂)^{1 – Y₂}(1 – ₃)^{1 – Y₃} + (1 – )(1 – ₁)^{1 – Y₁}(1 – ₂)^{1 – Y₂}(1 – ₃)^{1 – Y₃}.)

One advantage of the analytic expressions is that estimates can be calculated directly (even with a hand calculator) without requiring a numerical optimization routine to maximize the likelihood. More importantly, they describe how relationships observed in the raw data are used to infer the prevalence of the latent condition and properties of the 3 tests. In particular, the analytic expression (3.3) for the estimated prevalence reveals that the starting point for the estimation is , with V determining deviations of from 0.5; larger values of V result in lower estimates of . The factor V compares the 3-way association among tests in its numerator with the pairwise associations in its denominator. These authors do not yet have an intuitive explanation as to why prevalence is simply a function of the 3- versus 2-way association parameters under the CI LCA model. It is particularly intriguing that the marginal frequencies of positive tests, p_k, do not directly affect the prevalence estimate. These affect only the true- and false-positive rate estimates (see below). In other words, the prevalence estimate is invariant to changes in values of (p₁,p₂,p₃) as long as the 3- versus 2-way association parameter, V, remains the same.

Somewhat more intuition can be provided for the test accuracy estimates, and , given . Note that for a completely uninformative test that has no association with disease status, P(Y_k = 1|D = 1) = P(Y_k = 1) = p_k. Thus, the starting point for in (3.1) is , the true-positive rate estimate for an uninformative test. The factor , determined by the marginal positive associations between pairs of tests, increases above . This is logical, since the CI model asserts that any correlation between test results is due to their common association with the latent variable D. Therefore, if 2 tests are strongly associated, it must be because they are both accurately reflecting D. The factor C_k is curious in that its numerator reflects associations between Y_k and the other 2 tests, while its denominator reflects the association between the other 2 tests. This implies that associations between the kth test and the other tests are calibrated by the observed association between those other tests.

Analogous considerations hold for . The starting point for estimating _k is , the false-positive rate of the uninformative test. Positive associations between tests in the observed data reduce estimates of _k from this starting point.

The estimates of _k and _k are very closely linked since they are determined by exactly the same entities, , and . Observe in (3.1) and (3.2) that if is large, the kth test will be estimated to have a high true-positive "and" a low false-positive rate relative to the uninformative test. In fact, there is a direct linear relationship between and :

Higher estimates of sensitivity for the kth test also give rise to higher estimates of specificity, given and .

Under the CI LCA model, are maximum likelihood estimators and hence are consistent and efficient. Moreover, they seem to represent meaningful quantities. Consider the estimate of _k. Suppose that, in truth, 2 of the tests, Y₁ and Y₂, have high true-positive rates, and Y₃ does not. In the observed data, we would expect only weak associations between Y₁ and Y₃ and between Y₂ and Y₃ but a strong association between Y₁ and Y₂. Correspondingly, the C_k factor will be low for k = 3 because the numerator is small and the denominator is large. On the other hand, for k = 1 (or 2), the denominator and 1 component of the numerator will be small, canceling each other out to some extent, and C_k will be large due to the strong association between Y₁ and Y₂ in the numerator. Thus, (and ) will be large and will be small, as they should be. A similar exercise can be undertaken for the case where 2 tests, Y₁ and Y₂, have low true-positive rates, but Y₃ has a high true-positive rate. Compared to associations between Y₁ and Y₃ and between Y₂ and Y₃, the association between Y₁ and Y₂ will be very weak because Y₁ and Y₂ are both only weakly related to D and they are conditionally independent. This yields a high value of C₃ and hence increases well above the starting point . On the other hand, C₁ and C₂ will be dominated by the weak association between Y₁ and Y₂, assuming that the associations between Y₁ and Y₃ and between Y₂ and Y₃ are of comparable size. Hence, and will be low. We see once again that the LCA estimates make intuitive sense.

In contrast, the value of the estimators, , when the CI LCA model does not hold is questionable. Although the analytic expressions above now afford them interpretations in terms of the observed data, these do not seem to be meaningful entities or to relate to an underlying latent variable in a meaningful way. Moreover, they suggest that even when a binary latent variable exists, CI is crucial for valid estimation. Suppose, for example, that there is a latent class, D, but that 2 tests, say Y₁ and Y₂, are conditionally positively "dependent." The expressions for and suggest that the estimates will be biased toward optimistic values. Observed correlation between Y₁ and Y₂ will be stronger than is due simply to D, suggesting that will be biased large and will be biased small. Indeed, this corroborates the simulation results of Torrance-Rynard and Walter (1997).

	4. PARAMETER INTERPRETATIONS VIA THE EM ALGORITHM

TOP SUMMARY 1. INTRODUCTION 2. CLASSIC LATENT CLASS... 3. ANALYTIC EXPRESSIONS FOR... 4. PARAMETER INTERPRETATIONS VIA... 5. DISCUSSION APPENDIX APPENDIX B REFERENCES

 
In this section, we consider the EM algorithm for obtaining maximum likelihood estimates to provide alternative expressions for the parameter estimates that lead to interesting insights into their properties. These expressions are derived in Appendix B. Qu and others (1996) and Bartholomew and Knott (1999, pp. 137–139) briefly note these expressions but do not discuss them.

(4.1)

(4.2)

(4.3)

where is given by (B.6) in Appendix B.

We note that these expressions do not provide explicit formulas for calculating , , and . Rather they describe some "relationships" among the estimators. Each expression on the right-hand side is a function of all 3 parameters through the terms . Second, the expressions are intuitive, in the sense that, if is an unbiased estimate of P(D_i = 1|Y_i), then , , and are unbiased. Even if the CI assumption does not hold, the estimators of ,_k, and _k seem sensible as long as is unbiased or consistent. We can think of these as the naive estimators when D_i is observed, and when D_i is not observed, D_i is replaced with .

One avenue, therefore, for exploring bias in the estimators , and when CI fails is to consider how violations of the CI assumption affect . For example, in the case of extreme positive dependence between the 3 tests, that is, Y_i1 = Y_i2 = Y_i3 almost surely, we would anticipate that will be biased large if (Y_i1,Y_i2,Y_i3) = (1,1,1) and biased small if (Y_i1,Y_i2,Y_i3) = (0,0,0). Expressions (4.2) and (4.3) then imply overoptimistic values for even if , the average probability , is unbiased.

In the audiology data, we do in fact have a gold standard measure of disease status. Hence, we can actually compare the observed (true) and the latent class estimates of , _k, and _k. The data including D are shown in Table 2. With D observed, prevalence is calculated as 42%, whereas the LCA estimate that ignores D is 54%. The observed true- and false-positive rates are , indicating that the tests are substantially worse than the latent class analysis suggests.

View this table: [in this window] [in a new window]   Table 2. Estimated probabilities of disease by categories of test results, using LCA (that ignores D) and using the empirical proportions

 
In Table 2, we also show the subject-specific estimates of P(D_i = 1|Y_i1,Y_i2,Y_i3) from the latent class analysis of these data. One can verify that expressions (4.1), (4.2), and (4.3) do indeed yield the LCA maximum likelihood estimates of ,_k, and _k provided earlier.

Without data on D available, CI must be assumed for identifiability of LCA because only K = 3 tests are observed per subject. However, with data on D available, we can directly test if the CI assumption holds. An independence log-linear model of the form logP(Y₁ = y₁,Y₂ = y₂,Y₃ = y₃|D = 1) = ₀ + ₁y₁ + ₂y₂ + ₃y₃,{y_k = 0,1;k = 1,2,3}, yields a likelihood ratio test statistic 120.9 with 3 degrees of freedom in cases. Thus, CI does not hold. A similar exercise in nondiseased subjects yields a value of 48.5 for the likelihood ratio test. There is in fact positive dependence among tests in cases and controls. As mentioned above, this inflates LCA estimates of P(D_i = 1|Y_i) for subjects with positive tests and deflates LCA estimates of P(D_i = 1|Y_i) for subjects with negative tests. The last 2 columns of Table 2 bear this out. Correspondingly, the LCA estimates of (_k,_k) are seen to be overoptimistic relative to their true values calculated using D.

Another use of latent class analysis is to derive an operational definition of disease based on observable test results. In these data, we note that the estimates of P(D_i = 1|Y_i1,Y_i2,Y_i3) are high for certain combinations of test results and low for others. In particular, if 2 or more test results are positive, . On the other hand, if 2 or more are negative, . This result suggests the classification rule that the condition is considered present (absent) if 2 or more of the tests are positive (negative). However, comparison with the observed D indicates that this LCA-based classifier is very poor, with a false-positive rate of 42% and a false-negative rate of 30%. Again, violation of the untestable CI assumption leads to misleading inference for latent class analysis of data that do not contain the gold standard.

	5. DISCUSSION

TOP SUMMARY 1. INTRODUCTION 2. CLASSIC LATENT CLASS... 3. ANALYTIC EXPRESSIONS FOR... 4. PARAMETER INTERPRETATIONS VIA... 5. DISCUSSION APPENDIX APPENDIX B REFERENCES

 
In this paper, we derived analytic expressions for maximum likelihood parameter estimates in a latent class model with 3 observed test results. We found that the estimators are only meaningful when the tests are independent conditional on the underlying latent disease variable. Through an example where the disease status variable is known, we demonstrated that considerable bias can occur when the CI assumption is violated. We also developed intuition for the direction of the bias from the analytic forms of the estimators and from expressions based on the EM algorithm.

We contend, however, that the CI assumption often fails in practice. Most diseases are not dichotomous but occur in varying degrees of severity. This can induce correlation between tests. All tests may be positive in subjects with severe disease, while false negatives may be more likely in subjects with mild disease. In addition, mechanisms giving rise to test errors may be common to multiple tests. For example, if the tests are designed to detect a particular substance in a biological sample, the amount of substance present in the sample will affect all tests. A specimen for which the substance is erroneously absent gives rise to false-negative errors for all tests. A specimen contaminated with high levels may give rise to false-positive errors for all tests.

What course should be taken when CI is in doubt? In order to progress, a conditional dependence structure must be specified. We note that, in order to determine the accuracy of the kth test, knowledge of marginal parameters, disease prevalence, and accuracies of other tests is not very informative in and of itself. Regardless of prevalence and other tests, the observed data for test k are consistent with it being completely uninformative, that is _k = _k = p_k; it being completely sensitive if p_k, that is _k = 1 and _k = (p_k – )/(1 – ); or it being completely specific if p_k, that is _k = 0 and _k = p_k/. When K = 3, only 7 degrees of freedom are available and identifiability of prevalence, test operating characteristics, and dependence parameters requires that some of the marginal and/or dependence parameters be prespecified. Dendukuri and Joseph (2001) specified Bayesian prior distributions for the marginal parameters and, not surprisingly, found that posterior distributions were strongly dependent on prior information even with large sample sizes. They concluded that the degree of correlation between the tests must be known a priori with high precision in order for the data to be informative about the marginal parameters of interest (,_k,_k, k = 1,...,K).

Our work has focussed on the simplest setting where only 3 tests are available. However, the issues raised are relevant when more than 3 tests are available too. Knowledge of the conditional dependence structure is crucial to latent class analysis. When K 4, it is possible to simultaneously estimate parameters in a conditional dependence model and the marginal parameters. Albert and Dodd (2004) fit 2 different models for conditional dependence to real and simulated data (Albert and others, 2001; Qu and others, 1996). They found that estimates of sensitivity, specificity, and prevalence were substantially different under the 2 types of models. They also found that with K 10, it is typically very difficult to discern statistically between the 2 forms of conditional dependence. We note that with statistical analysis alone, one can never be certain about the validity of a dependence model because there are not sufficient degrees of freedom in the data to evaluate a saturated model. In other words, one cannot know from the observed data, how the kth test relates to others conditional on disease status. Yet, assumptions about this drive the test's performance estimates output from the latent class analysis.

We conclude that a latent class analysis requires careful justification of assumptions made about the conditional dependence structure. A deep understanding of the biological and technical mechanisms giving rise to test results seems necessary. This was not forthcoming in our collaborations on audiology test assessments. If we had not a gold standard reference, we would not have been able to make inference about test performance. The need to understand the mechanisms of the tests, and in particular their mutual dependence in diseased and nondiseased subjects, also highlights the need to provide a clear clinical definition for disease. Such is generally required as a first step in rigorous medical research studies. Although latent class analysis can proceed technically without ever defining what disease is, interpretations for prevalence and test accuracy estimates are dubious without it. Moreover, it seems like a prerequisite to deciding upon a conditional dependence model. If latent class analysis is to have a role in rigorous scientific evaluations, we believe that it should be held to rigorous scientific standards. In the absence of (i) a definition for the disease entity and (ii) biological rationale for models of association between test results in diseased and nondiseased subjects, the results of a latent class analysis must, in our opinion, be viewed with skepticism. Alternative approaches that employ clinical outcomes or composite references standards (Alonzo and Pepe, 1999) may be useful in some settings.

	APPENDIX

TOP SUMMARY 1. INTRODUCTION 2. CLASSIC LATENT CLASS... 3. ANALYTIC EXPRESSIONS FOR... 4. PARAMETER INTERPRETATIONS VIA... 5. DISCUSSION APPENDIX APPENDIX B REFERENCES

 

A.1 Derivation of maximum likelihood estimators

We show that under the CI model, there is a one-to-one mapping of the 7 parameters (,) = (,{(_k,_k),k = 1,2,3}) to the 7 probabilities P = (p₁,p₂,p₃,p₁₂,p₁₃,p₂₃,p₁₂₃) that characterize the probability distribution of the observable data, that is, the 2x2x2 frequency table (e.g. Table 1). Writing this mapping as g:g(P) = (,) and noting that the maximum likelihood estimates of the observable probabilities are the corresponding data frequencies , it follows that the maximum likelihood estimates of (,) are .

The following 7 equations follow from elementary probability theory and the CI assumption:

(A.1)

(A.2)

(A.3)

These define g ^{– 1}. Algebraic manipulations yield the expressions for (,{(_k,_k),k = 1,2,3}) in terms of P, that is, the function g. First, we write _k in terms of (P,,_k) using (A.1),

(A.4)

and substitute into (A.2) to yield

Thus, we can write ₂ and ₃ in terms of (P,₁,):

and substituting into the above expression for (p₂ – ₂)(p₃ – ₃), we have

where C₁ was defined in Section 3. There are 2 solutions then for ₁:. We choose which follows from the reasonable assumption that the true-positive rate is at least as large as the false-positive rate, _{1 1}. Similar steps yield and . Substituting _k into (A.4) above yields

Substituting expressions for _k and _k into (A.3) and gathering terms yield

Equivalently,

which is easily shown to equal V as defined in Section 3. Hence,

and thus

	APPENDIX B

TOP SUMMARY 1. INTRODUCTION 2. CLASSIC LATENT CLASS... 3. ANALYTIC EXPRESSIONS FOR... 4. PARAMETER INTERPRETATIONS VIA... 5. DISCUSSION APPENDIX APPENDIX B REFERENCES

 

B.1 Parameter expressions via the EM algorithm

If the latent variable D were observed, the log-likelihood for the data from the ith subject, Y_i = {Y_i1,...,Y_iK}, could be written as

Given values for = ^* and = ^*, the expected log-likelihood is

(B.1)

where

(B.2)

The EM algorithm proceeds by iteratively maximizing E_^*,^*(,) with respect to and and substituting these values for ^* and ^* in the next iteration. The algorithm is completed when (^*,^*) have converged. The value of that maximizes (B.1) is

Therefore, at convergence of the algorithm,

(B.3)

where is given by (B.2).

The development thus far is general in regards to the LCA model, P(Y_i|D_i). Adding the CI assumption and the notation = {₁,...,_K} and = {₁,...,_K} yields the following expression for the expected log-likelihood:

This expression is maximized at

and

Therefore, at convergence, the maximum likelihood estimates can be written as

(B.4)

(B.5)

Under the CI assumption, P(Y_i|D_i) = P(Y_ik|D_i), and the form for in terms of , and follows from (B.2):

(B.6)

	ACKNOWLEDGMENTS

 
Conflict of Interest: None declared.

	REFERENCES

TOP SUMMARY 1. INTRODUCTION 2. CLASSIC LATENT CLASS... 3. ANALYTIC EXPRESSIONS FOR... 4. PARAMETER INTERPRETATIONS VIA... 5. DISCUSSION APPENDIX APPENDIX B REFERENCES

 

Albert PS and Dodd LE. (2004) A cautionary note on robustness of latent class models for estimating diagnostic error without a gold standard. Biometrics 60:427–435.[CrossRef][Web of Science][Medline]

Albert PS, McShane LM, Shih JH. The U.S. National Cancer Institute Bladder Tumor Marker Network. (2001) Latent class modeling approaches for assessing diagnostic error without a gold standard: with applications to p53 immunohistochemical assays in bladder tumors. Biometrics 57:610–619.[CrossRef][Web of Science][Medline]

Alonzo T and Pepe MS. (1999) Assessing the accuracy of a new diagnostic test when a gold standard does not exist. Statistics in Medicine 18:2987–3003.[CrossRef][Web of Science][Medline]

Bartholomew DJ and Knott M. (1999) Latent Variable Models and Factor Analysis. Kendall's Library of Statistics 7 2nd edition (Arnold Publishers, London).

Boelaert M, Rijal S, Regmi S, Singh R, Karki B, Jacquet D, Chappuis F, Campino L, Desjeux P, Le Ray D. (2004) A comparative study of the effectiveness of diagnostic tests for visceral leishmaniasis. American Journal Tropical Medicine Hygiene 70:72–77.

Dawid AP and Skene AM. (1979) Maximum likelihood estimation of observer error-rates using the EM algorithm. Applied Statistics 28:20–28.[CrossRef]

Dendukuri N and Joseph L. (2001) Bayesian approaches to modeling the conditional dependence between multiple diagnostic tests. Biometrics 57:158–167.[CrossRef][Web of Science][Medline]

Ferraz MB, Walter SD, Heymann R, Atra E. (1995) Sensitivity and specificity of different diagnostic criteria for Bechet's disease according to the latent class approach. British Journal of Rheumatology 34:932–935.[Abstract/Free Full Text]

Hui SL and Zhou XH. (1998) Evaluation of diagnostic tests without gold standards. Statistical Methods in Medical Research 7:354–370.[Abstract/Free Full Text]

Moayyedi P, Duffy J, Delaney B. (2004) New approaches to enhance the accuracy of the diagnosis of reflux disease. Gut 53:iv55–iv57.[Abstract/Free Full Text]

Pepe MS. (2003) The Statistical Evaluation of Medical Tests for Classification and Prediction(Oxford University Press, New York).

Pepe MS and Alonzo TA. (2001) [Reply to letter to editor.]. Statistics in Medicine 20:656–660.[CrossRef][Web of Science][Medline]

Qu Y, Tan M, Kutner MH. (1996) Random effects models in latent class analysis for evaluating accuracy of diagnostic test. Biometrics 52:797–810.[CrossRef][Web of Science][Medline]

Torrance-Rynard VI and Walter SD. (1997) Effects of dependent errors in the assessment of diagnostic test performance. Statistics in Medicine 16:2157–2175.[CrossRef][Web of Science][Medline]

Walter SD and Irwig LM. (1988) Estimation of test error rates, disease prevalence, and relative risk from misclassified data: a review. Journal of Clinical Epidemiology 41:923–937.[CrossRef][Web of Science][Medline]

Young J, Bucher H, Tschudi P, Periat P, Hugenschmidt C, Welge-Lussen A. (2003) The clinical diagnosis of acute bacterial rhinosinusitis in general practice and its therapeutic consequences. Journal Clinical Epidemiology 56:377–384.[CrossRef]

Received June 29, 2006; revised October 18, 2006; accepted for publication October 27, 2006.


CiteULike    Connotea    Del.icio.us    What's this?




This article has been cited by other articles:

				H. Chu, S. R. Cole, Y. Wei, and J. G. Ibrahim Estimation and inference for case-control studies with multiple non-gold standard exposure assessments: with an occupational health application Biostat., October 1, 2009; 10(4): 591 - 602. [Abstract] [Full Text] [PDF]

				K. Soreide Receiver-operating characteristic curve analysis in diagnostic, prognostic and predictive biomarker research J. Clin. Pathol., January 1, 2009; 62(1): 1 - 5. [Full Text] [PDF]

				A. K. Formann and D. Bohning Letter to the editor Biostat., October 1, 2008; 9(4): 777 - 778. [Full Text] [PDF]




This Article Abstract FREE Full Text (PDF) OA All Versions of this Article: 8/2/474    most recent kxl038v1 Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to My Personal Archive Download to citation manager Disclaimer Google Scholar Articles by Pepe, M. S. Articles by Janes, H. Search for Related Content PubMed PubMed Citation Articles by Pepe, M. S. Articles by Janes, H. Social Bookmarking          What's this?

Online ISSN 1468-4357 - Print ISSN 1465-4644

Copyright © 2009 Oxford University Press

Oxford Journals Oxford University Press

• Site Map
• Privacy Policy
• Frequently Asked Questions

Other Oxford University Press sites: Oxford University Press Oxford Journals China Oxford Journals Japan American National Biography Booksellers' Information Service Children's Fiction and Poetry Children's Reference Corporate & Special Sales Dictionaries Dictionary of National Biography Digital Reference English Language Teaching Higher Education Textbooks Humanities International Education Unit Law Medicine Music Online Products Oxford English Dictionary Reference Rights and Permissions Science School Books Social Sciences Very Short Introductions World's Classics