Assessment regeneration
The results of these specification reviews determine the plan for regeneration of a test or test series. ETS researchers, psychometricians, and test developers analyze the reviewers' answers and comments and then use expert judgment to determine a path forward for each test or test series.
The group would typically choose one of three options, depending on the answers to the questions provided by the Standing Committee:
- Internal and external experts validate the test specifications, and no National Advisory Committee (NAC) meeting is necessary; test specifications remain unchanged.
- When updates are straightforward, we use in-house content experts to draft updated test specifications to improve alignment; external experts then validate the specifications via a virtual meeting. We make these updates when the test specifications don’t match current professional standards in the field.
- When updates are not straightforward, a NAC convenes to update the test specifications to improve alignment, and ETS performs a job analysis confirmation survey. These updates are necessary when test specifications don’t match current professional standards if they exist, or if internal and external experts determine the specifications are outdated.
When a NAC is convened, the Praxis client states help ETS recruit 12–15 licensed practicing professionals who are close to the beginning of their teaching practice, as well as college faculty who prepare teachers for that specific licensure area.
The NAC works closely with ETS assessment specialists to define the content domain and later to develop test-blueprint and test-content specifications. NAC meetings typically take place on the ETS campus in Princeton, New Jersey.
After the NAC meets to define the test domains, a job analysis survey is conducted to confirm these domains. A job survey is conducted for new tests and for existing tests that are being significantly revised. The job analysis survey enables relatively large numbers of practitioners from across the country to judge the importance of the knowledge and/or skills defined by the NAC for beginning-level practice.
The practitioners' judgments serve as an independent source of validation evidence. Once the results of the survey have been analyzed, the NAC is reconvened to develop the test blueprint and test specifications.
After the NAC develops the test blueprint and test specifications, ETS convenes panels of licensed practicing educators and college faculty from user states to conduct standard-setting studies.
Each standard-setting panel confirms that the knowledge and/or skills represented in the test content specifications are important for entry-level practice and recommends a passing score for each state to consider when setting its own passing score.
