A common misconception is that any marker that shows an association with any aspects of a disease, i.e., a risk factor, can be used as a biomarker. In practice, however, it is not that straightforward [15, 16, 26, 27]. Even a risk factor whose association with a trait yields a relative risk or odds ratio (OR) of 5.0, which is a high value for any epidemiologic association, is unlikely to be an informative biomarker at the individual level. For a risk factor to have a detection rate of 80 % for a false-positive rate of 5 %, the OR should reach more than 2,000, which corresponds to almost exclusive presence in the group of interest. This is exactly the problem with most risk factors, in particular, for genetic markers in that a lot of healthy people will also carry the marker without any sign of the disease. By using the copresence of multiple markers, a threshold may be obtained that the combination of markers would only be present in patients and absent in healthy subjects. Although achievable, this situation will only apply to a minority of patients and a minority of controls resulting in low specificity and sensitivity for the marker as a biomarker.

It may appear paradoxical that a strong risk marker yielding a very high relative risk with a very strong statistical result is not as useful for biomarker development as expected. First of all, a statistical result only shows that the marker is a classifier – in the sense that it classifies a subject as a case or control – and this is less likely to be due to chance. As will be discussed, a likelihood ratio (which incorporates specificity and sensitivity), area under the receiver operating characteristic (ROC) curve analysis, and reclassification statistics are better indicators of the value of a test as a biomarker. The ultimate aim is to have a marker able to classify a subject as diseased or healthy, rather than estimating the risk of disease, i.e., how many folds increased risk a subject has. An ideal biomarker is present in all diseased subjects (100 % sensitivity), and it is totally absent in non-diseased (healthy or those patients with other diseases) subjects (100 % specificity) yielding a likelihood ratio of infinity. As an example, the FDA-approved 70-gene signature in breast cancer for overall survival [28] corresponds to P < 0.001 and a hazard ratio of >2.0, but a sensitivity of 90 % and a specificity of 40 %, with the resulting AUC value of around 0.65 for survival. These figures correspond to a poor-to-modest classifier: to identify correctly 90 of the 100 patients with poor prognosis (and missing 10 of them), 60 of 100 patients with good prognosis are also identified as candidates for poor prognosis [27].

The initial discovery phase is just a start in a rather involved process of biomarker development. This process involves (1) discovery and replication, (2) analytic validation, (3) clinical validation, (4) determination of clinical utility, and (5) clinical use following regulatory approvals (Fig. 2). Taking a simple association study to a clinically useful biomarker is a process as detailed and elaborate as drug development [29, 30]. Such a successful translation requires work in basic, translational, and regulatory science and a comprehensive collaboration among laboratory scientists, technology developers, clinicians, statisticians, and bioinformaticians. Several guidelines have been published for different aspects of biomarker development studies: molecular epidemiology (STROBE-ME) [11], early clinical trials of novel agents [31], tumor marker prognostic studies (REMARK) [32], genomic applications (EGAPP) [33], genetic risk prediction (GRIPS) [34], and biospecimen-based studies (BRISQ) [35]. These and other [36–38] guidelines aim to prevent the use of biomarkers in the absence of high levels of evidence supporting their clinical utility and help the investigators to design biomarkers that can lead to true personalized care with high confidence.

It is important to appreciate that the discovery phase is just the beginning, but still strict guidelines of study design should be followed [30, 39, 40].

The discovery phase studies tend to be retrospective and small in size and suffer from well-recognized problems such as cross-validation for replication and extensive subgroup analysis. Instead, emphasis should be given to statistically adequately powered, prospective studies. This can be achieved using stored biospecimens of ongoing or completed cohort studies or randomized clinical trials as long as samples from placebo arm or standard treatment are used [30], bearing in mind the caveats of inclusion criteria for clinical trials. Since the rest of the development program will be based on these initial results, extreme care at this stage will pay off later.

Once the initial results are obtained in the discovery phase, the validation phase begins with examination of analytical validity as the next step. This is a technical checklist of the analysis methods used in the measurement of the biomarker candidate. The specific assay test for the biomarker is examined for its accuracy and precision. For a genetic variant, this step makes sure that the genotyping assays perform well with acceptable concordance rates by different operators, in different laboratories using different platforms. Analytical validity is measured in the laboratory by calculating the test sensitivity, which provides the probability that a positive test is truly positive (e.g., will yield is a positive results when the marker is in fact present), and test specificity, which provides the probability that the test will not detect the marker when it is not present. This calculation is performed by testing the analysis method against a gold standard or by using samples known to be positive and negative for the presence of the marker. Ideally, both parameters should be 100 %.

In the clinical validity step, the power of the biomarker candidate to show statistical correlation with the phenotype of interest or intended clinical endpoint is reexamined and confirmed. It is in this step that potential confounders (age, sex, race/ethnicity, die, medications) are also examined. Measures of sensitivity and specificity are evaluated in a representative sample of the population for whom the test is intended using appropriate epidemiologic study designs ideally including an independent replication study. These studies also establish the positive predictive value, which reflects the probability that a person with a positive test result has or will have the phenotype for which the marker is believed to be a predictor. Confirmed clinical validity does not necessarily mean that a biomarker can immediately be used in patient care. This is achieved in the next clinical utility step. Only biomarker candidates with confirmed analytical and clinical validity are submitted to the formal assessment of clinical utility. In this step, large, independent, and well-designed studies evaluate the biomarker’s expected utility using formal statistical tools. It is this stage that is usually underappreciated in discussions of the value of a marker as a biomarker. Only after passing this step, a marker attains a biomarker status with clinical utility meaning that it provides information that is useful in the clinic in making decisions about disease susceptibility, disease classification, prognostic stratification, or response to a given treatment. Once evidence of utility in real clinical settings is generated, the regulatory process begins for introduction of the biomarker for use in routine clinical care (Fig. 2).

A biomarker is a classifier that classifies a group of people into separate groups like susceptible and non-susceptible individuals, one subgroup of a disease and another, or a subgroup with good prognosis and another without. The clinical utility of a biomarker is assessed in a different manner from the way a case–control study evaluates an association. Here the focus is on whether by using a biomarker subjects can be reclassified in a class different from where they would be classified using existing evidence. An ideal biomarker, say, for disease susceptibility, with its high specificity and sensitivity (positive in every predisposed subject and negative in every subject that will remain disease-free, thus, yielding a high likelihood ratio), is well calibrated (the predicted risk for developing the disease corresponds to real risk and not over- or underestimated) and therefore would classify every subject as predisposed or not. Once this ideal stage is reached, the positive and negative predictive values for the marker will also be ideal demonstrating the benefits and risks from both positive and negative results. Such an imaginary biomarker would have a value of 100 % for each of sensitivity, specificity, and area under “ROC” curve (AUC). This scenario is only true for some monogenic Mendelian disorders where the genetic effect is not modified by the environment.

In the ultimate stage of biomarker development preceding the regulatory approval, statistical evaluation is most rigorous. Logistic regression and its output of effect size, the OR, are not good indicators of the clinical utility of the biomarker candidate. The OR is not an estimate of individual-level risk nor a diagnostic test or classifier; and the OR should not be used in a predictive study as the effect size [41]. A high OR (or relative risk) associated with a marker even after adjusting for established risk factors does not necessarily translate into better risk prediction. The statistical considerations differ between etiologic risk studies and studies on biomarkers that are going to be used to classify subjects. Studies that use the OR as the effect size use multiple markers with small effect sizes simultaneously to increase the effect size. This approach certainly increases the effect size, but the multimarker set is still not a good classifier. In a simulation study of 40 independent genetic risk markers each yielding an odds ratio of 1.2–2.0 and a sample size of one million, the best results in discriminative accuracy quantified as the AUC were obtained when the risk genotypes were all common (≥30 %) and odds ratios were closer to 2.0. Even with this unrealistic scenario, the AUC was 0.93 [42]. This example shows that even simultaneous use of most common and strong risk markers may still not have a large discriminatory value between disease and healthy states.

The ROC curve analysis is a commonly used measure of performance of a predictive test. An ideal marker has an AUC of 1 representing perfect discrimination between the diseased and nondiseased subjects; in other words, all subjects are correctly classified by the test. The baseline value for an AUC is 0.5 which represents no discrimination at all; in other words subjects are classified no more correctly than can be attributed to by chance. AUC plots the sensitivity of the marker against (1 – specificity) for all possible cutoff values. In the case of a binary marker, this is just a single point. The c-index is numerically equivalent to the AUC. A serious issue with the AUC is that it does not measure the ability of a new marker to add value to a preexisting prediction model. Thus, a new marker may have a good performance, but whether adding the new marker to existing markers will improve the performance needs to be known. This is usually done by generating two ROC curves, one with and one without the new marker using information on existing markers, and observing whether there is a difference between them. The difference can be formally assessed by the c-statistics if the change in AUC appears to be substantial enough.

However, useful ROC curves may be for classification; evaluation of predictive models cannot rely solely on the ROC curve, but should assess discrimination and calibration using new metrics (Table 1). Jakobsdottir et al. looked at this formally [16]. They compared the utility of genetic markers of a number of complex disorders identified by GWAS using ROC curve analysis against conventional risk markers already known and concluded that small P values, high ORs, and AUCs do not guarantee good prediction of actual risk. The AUC should be considered as a first step in evaluating a model or in comparing two models against each other. The AUC value is, however, insufficient on its own to show that a model would improve decision-making. One criticism aimed at the AUC analysis in clinical utility determination is that it gives equal weight to specificity and sensitivity, hence to false positives and false negatives, which may not be the case in a real clinical situation [43]. Published estimates of disease prevalence and heritability or sibling recurrence risk for 17 complex genetic diseases have been used to calculate the proportion of genetic variance that a test must explain to achieve an AUC = 0.75, which is a modest value. For 17 diseases, the proportion of genetic variance that have to be explained by genetic markers for the predictive model to attain an AUC value of 0.75 varied from 0.10 to 0.74. In other words, depending on disease prevalence and heritability, genetic markers can explain as little as 0.10 or as high as 0.74 of the heritability to yield an AUC value of 0.75, the threshold regarded as making a diagnostic classifier clinically useful when applied to a sample considered to be at increased risk [42]. On the other hand, a threshold AUC value is 0.99 for a predictive test to be a classifier when applied to the general population. Given the prevalence and heritability of RA, the maximum value for genetic markers in RA for prediction of disease susceptibility can get close (0.98) but not quite reach this threshold [48].

For clinical utility assessment, additional statistical methods including discrimination metrics (c-statistics), measures of calibration (addressing how close the predicted risks are to the actual observed risks), and reclassification (addressing whether the model including the novel biomarker changes a person’s risk sufficiently to move them to a different risk category) are needed (Table 1) [44–46, 49, 50]. Calibration is essential for good decision-making. A model is well calibrated when the predicted risk is equal to the observed risk. Calibration takes into account the average risk in a population. Although essential in biomarker development, calibration is not sufficient for clinical utility. What is most crucial for determination of clinical utility in biomarker development is reclassification, which aims to do at the individual level what AUC analysis does at the group level. In reclassification analysis, each individual’s data is considered to see whether the new marker changes their risk classification. This is achieved by reclassification tables that show changes in individual risk classifications with the addition of the new marker to the prediction model. It has been pointed out that a marker which has only modest or no effect by AUC analysis may still improve risk classification at the individual level [44, 45].

It is important to learn from past mistakes for current efforts in biomarker development to be more productive. It is well known that most claims of classification performance are overly optimistic, lacking verification by replication with questionable generalizability. Ioannidis reviewed the most common causes of biomarker failures [51]. He has pointed out that despite the introduction of so many biomarkers into clinical practice, the health impacts have not been favorable in general. Four types of failures have been recognized as summarized in Table 2. The most dramatic of these failures concerns the PSA. Initially introduced in the 1980s for monitoring treatment response, PSA testing has found a place as a biomarker for screening and early diagnosis of prostate cancer [52]. The AUC for PSA in ROC curve analysis is 0.68 for cancer versus no cancer. Given that a test that performs no better than chance yields an AUC of 0.5, this value does not suggest a high clinical utility in classifying subjects as having prostate cancer or not. PSA is believed to have done more harm and good, by drastically increasing overdiagnosis and overtreatment of prostate cancer [53]. The problem with PSA was that it was not subject to a robust assessment before being used for screening and actual clinical use disappointed. A search for biomarkers is still ongoing. The PSA example illustrates the importance of following the steps shown in Fig. 2 before introducing a biomarker to clinical use.

An ideal biomarker for clinical use should have three major characteristics: (1) It should be safe and easy to measure preferably noninvasively and with good reproducibility; (2) it should have a high sensitivity, high specificity, and high positive and negative predictive values (PPV and NPV, respectively) for its intended outcome; and (3) it should improve decision-making abilities in line with clinicopathological parameters. Any attempt to introduce a single marker to routine care as a biomarker will probably suffer one of the types of failure listed in Table 2. If a single biomarker cannot fulfill all the expectations, which is usually the case, a panel of multiple biomarkers if performing better than any single biomarker may be used. Indeed, most common biomarkers used in RA to assess disease activity (multi-biomarker disease activity or MBDA) are of this type.

In RA, associations of markers, especially those that are genetic, have been widely reported as valid for a number of outcomes, but the development of biomarkers based on those findings has been either unsuccessful or slow. As discussed above, a high OR is no indication of success for a biomarker, and the strongest HLA region association in RA has an OR of 5–10. Even the strongest HLA association in any disease with an OR of around 100 as in ankylosing spondylitis may not turn out to be a good biomarker. This highlights the difficulty with converting even the strongest risk markers to biomarkers.

As discussed below, there are a number of biomarkers used for RA diagnosis, disease activity, assessment, or prognosis. A survey analyzed the validity of biomarkers as reported in 170 articles [40]. Most common biomarkers were gene expression profiles. Flaws were identified in most reports. Less than half of the studies incorporated study-design features important for valid clinical associations: age and sex-matched groups and controlling for medications used. These issues concerned mainly the discovery stage studies. Even at that stage, which forms the foundation of a long process, no more than half of the studies were satisfactory by simple epidemiologic criteria. This is not a promising start to the process of biomarker development if biomarkers with genuine clinical utility are the aim.

To avoid future mistakes in biomarker development and to aid with valid ones, an independent initiative called OMERACT (Outcome Measures in Rheumatology) consisting of international health professionals interested in outcome measures in rheumatology was formed in 1992. OMERACT has played a critical role in the development and validation of clinical and radiographic outcome measures in RA and other rheumatic diseases. A special interest group developed validation criteria for soluble biomarkers of structural joint damage [36]. These criteria have been further developed and put to test for existing biomarkers [54–57]. Neither a baseline C-reactive protein (CRP) test nor later tests on five more soluble biomarkers receptor activator of nuclear factor kappa-B ligand (RANKL; TNFSF11), osteoprotegerin (OPG), matrix metalloprotease (MMP-3), and urine C-telopeptide of types I and II collagen (U-CTX-I and U CTX-II) produced strong evidence that these biomarkers could substitute for radiographic endpoints in RA. The OMERACT validation criteria are based on three domains: truth (is the measure truthful, does it measure what it intends to measure? Is the result unbiased and relevant?), discrimination (does the measure discriminate between situations that are of interest?), and feasibility (can the measure be applied easily, given constraints of time, money, and interpretability?) This initiative together with other published guidelines for various aspects of biomarker development studies are expected to result in the development of reliable biomarkers with enhanced validity and clinical utility.