Test Item Analysis for Credible Selection Assessment

Linking Reliability, Item Difficulty, and Item Discrimination

Selection examinations for admission or employment require high-quality assessment instruments. Item analysis is a critical process for evaluating test quality through key psychometric indices, including reliability, item difficulty (p), and item discrimination (r). These indices are interrelated and directly influence the accuracy and credibility of assessment results.

1. Reliability

Reliability refers to the consistency and stability of test scores across repeated administrations under similar conditions. Common estimation methods include Cronbach’s alpha, KR-20, test-retest, and split-half reliability. In practice, a coefficient of α or KR-20 ≥ 0.70 is considered acceptable, while ≥ 0.80 indicates good reliability.

Empirical evidence shows that items with moderate difficulty (p = 0.20–0.80) and higher discrimination (r) more effectively differentiate examinees of varying ability, thereby increasing overall test reliability (Cohen, Swerdlik, & Sturman, 2021).

Table 1 Interpretation of Cronbach’s Alpha

2. Item Difficulty (p)

Item difficulty (p-value) refers to the proportion of examinees who answer an item correctly. It indicates how easy or difficult a test item is. A p value close to 1 suggests that the item is too easy, whereas a value close to 0 indicates that the item is overly difficult (Nitko & Brookhart, 2013).

In practice, item difficulty is calculated as the ratio of correct responses to the total number of examinees. Extremely easy or extremely difficult items provide limited information and fail to effectively differentiate examinees with different ability levels.

Selecting items with p values between 0.20 and 0.80 enhances the ability of a test to distinguish between high- and low-performing examinees, which in turn increases overall test reliability (Embretson & Reise, 2013; Kline, 2016). Items that are too easy or too difficult typically show weak discrimination (Crocker & Algina, 2006).

When considered together with item discrimination (r) and reliability, appropriately difficult items contribute to higher discrimination indices and more accurate, trustworthy assessment outcomes (Embretson & Reise, 2013; Kline, 2016).

Example: If 45 out of 50 examinees answer an item correctly,
45 / 50 → p = 0.90

Table 2 Interpretation of Item Difficulty (p)

Source: Crocker & Algina (2006)

Table 3. Item Discrimination (r)

Item discrimination (r) refers to an index that indicates how well an item distinguishes between high-performing and low-performing examinees. It is commonly calculated as the correlation between an individual item score and the total test score.

A high r value indicates that the item effectively differentiates examinees according to their ability levels, meaning that high-ability examinees tend to answer the item correctly, whereas low-ability examinees tend to answer incorrectly.

When considered in relation to reliability and item difficulty (p), higher discrimination values enhance the clarity and precision of score interpretation and contribute to increased overall test reliability (Thompson, 2017).

Interpretation of Item Discrimination (r)

Source: Crocker & Algina (2006); Nitko & Brookhart (2013); Kline (2016)

Effective Test Item Analysis

Considering reliability, item difficulty (p), and item discrimination (r) together enhances the accuracy and trustworthiness of test item analysis. The key points can be summarized as follows.

1. Reliability assesses the stability and consistency of test scores. Examining reliability ensures that the test yields consistent results when administered repeatedly.
2. Item Difficulty (p) indicates whether test items are appropriately challenging— neither too easy nor too difficult— allowing for effective differentiation among examinees.
3. Item Discrimination (r) reflects how well an item distinguishes between high-performing and low-performing examinees.

Using these three indices together supports precise test design and ensures that assessment results accurately reflect the true abilities of examinees.

Relationship among p, r, and Reliability

1. Items with p values between 0.20 and 0.80 and r values greater than 0.20 are considered high-quality items, capable of differentiating examinees and enhancing overall test reliability.
2. The KR-20 formula incorporates p and q (1–p) in estimating the reliability of multiple-choice tests.
3. An applied summary is presented in Table 4.

Table 4 Applied Summary of Evaluation Criteria

Recommendations for Test Development

1. Develop and analyze p and r for each item
2. Examine α or KR-20 to evaluate overall test reliability
3. Revise items with low p or r values and refine item wording to improve test quality