What can we help you with?
Categories
< All Topics

Examining the Statistical Characteristics of Your Test

After assembling a test or in conjunction with the assembly process, you can examine the statistical characteristics of the test.

  • FastTest will provide you with distributional statistics on the items in a test as you assemble it, and additionally displays the test-level IRT functions if you have IRT parameters.
    • You can monitor the statistical characteristics of your test items as you add and delete items from your test, until you are satisfied that it has the statistical characteristics that you desire.
    • All statistics and displays are automatically updated as you add or delete items from your test.

Figure 4.17: The Test Statistics Dialog

Test Statistics Dialog
  • To view the summary statistics, either click the Statistics → View Test Statistics button, or right-click on a test and select View Test Statistics. This brings up the dialog in Figure 4.17.
    • The mean and standard deviation of your test statistics are presented on the left, with the IRT functions on the right.
    • To obtain actual values of the IRT functions, click the View Data button in the upper right.

Two additional piece of information are also on this tab.

  • First, the mean Angoff represents the cutscore for the test, as recommended by your Angoff panel. For a more detailed analysis of Angoff results, see the ASC Angoff Analysis Tool webpage.
  • Second, the estimated mean, standard deviation, and reliability of the number-correct scores are presented in the lower left.

Assembling Tests Using IRT

If your items have IRT parameter estimates, you can use the search capabilities of FastTest to assemble a test with a desired test information function (TIF) or conditional standard error of measurement (SEM) function.

  • The TIF shows the precision/information for a test as a function of the IRT θ (trait) variable.
  • Test information is the sum, conditional on θ, of the item information functions for all items in the test; the SEM function is the reciprocal of the square root of the TIF.

Suppose you wanted to create a test that was to be used to make a dichotomous classification of examinees. Such a test might be used to measure mastery in an educational environment or to select applicants in a personnel selection environment.

  • In IRT terms, such a test should have its highest information at the mastery/selection cutscore point on the θ continuum, while little information is necessary for scores far above or below the cutscore.

The item search capabilities of FastTest allow you to easily identify those items. Using the advanced search, specify a narrow range of θ around your θ cutoff score.

  • For example, suppose you wanted to specify θ = 1.5 as your cutscore. You might specify your first search range as a b parameter of 1.1 to 1.9. The search function will display all items that meet  these requirements, which you can then browse to select items.
  • Then return to the Test Statistics dialog to view the resulting TIF. If modification is necessary, return to your item search results to add/delete items.

Importing and Exporting Item Statistics for Items in Your Tests

To utilize the item and test statistic functionalities described above, it is of course necessary to have item statistics stored in your banks. You can also export a CSV file of the items statistics for further evaluation. FastTest provides functionalities for both importing and exporting item statistics.

  • Item statistics can be imported in two ways.
    • First, they can be imported as part of the general import process, when importing previously used banks.
    • Second, the item statistics for a specific test can be uploaded in the Test Assembler tab.
      • For example, you might write a new test, deliver it to your examinees, and then analyze in a specially designed program like Iteman or Xcalibre. You can then upload the statistics from that test.
  • To upload item statistics from a test, you must first export the current statistics of the test, which would either be blank or contain previous values.
    • To export, click the → Statistics → Export Item Metadata button, or right-click on a test and select Export → Export Item Metadata.
    • Save that file to your computer and fill in the new statistics where needed.
    • The file must be saved as a CSV file.
Name Export File

Note:  Navigate to the exported files by going to:

  • Reports Manager
  • Recent Reports (right hand side of screen)
FastTest Report Manager
  • Then, return to the Test Assembler tab and click the Statistics → Import Item Metadata button, or right-click on a test and select Import → Item Metadata.
  • Iteman Output Import
    • Locate the table of item statistics in columns. Copy the values for P and Rpbis
    • Paste those values into the exported template (Diff and Disc) and save on your computer
    • Import the saved file into FastTest by Import -> Item Metadata
Table of Contents