eatATA: a Minimal ExampleeatATA efficiently translates test design requirements for Automated Test Assembly (ATA) into constraints for a Mixed Integer Linear Programming Model (MILP). A number of efficient and user-friendly functions are available that translate conceptual test assembly constraints to constraint objects for MILP solvers, like the GLPK solver. In the remainder of this vignette we will illustrate the use of eatATA using a minimal example. A general overview over eatATA can be found in the vignette Overview of eatATA Functionality.
The eatATA package can be installed from CRAN.
install.packages("eatATA")First, eatATA is loaded into your R session. In this vignette we use a small simulated item pool, items_mini. The goal will be to assemble a single test form consisting of ten items, an average test time of eight minutes and maximum TIF at medium ability. We therefore calculate the IIF at medium ability and append it to the item pool using the calculateIFF() function.
# loading eatATA
library(eatATA)
# item pool structure
str(items_mini)
#> 'data.frame': 30 obs. of 4 variables:
#> $ item : int 1 2 3 4 5 6 7 8 9 10 ...
#> $ format : chr "mc" "mc" "mc" "mc" ...
#> $ time : num 27.8 15.5 31 29.9 23.1 ...
#> $ difficulty: num -1.881 0.843 1.119 0.729 -0.489 ...
# calculate and append IIF
items_mini[, "IIF_0"] <- calculateIIF(B = items_mini$difficulty, theta = 0)In Table 1 you can see the first five items of the item pool.
| item | format | time | difficulty | IIF_0 |
|---|---|---|---|---|
| 1 | mc | 27.786 | -1.881 | 0.1090032 |
| 2 | mc | 15.453 | 0.843 | 0.4494582 |
| 3 | mc | 31.016 | 1.119 | 0.3266106 |
| 4 | mc | 29.874 | 0.729 | 0.5033924 |
| 5 | mc | 23.134 | -0.489 | 0.6108816 |
Next, the objective function is defined: The TIF should be maximized at medium ability. For this, we use the maxObjective() function.
testInfo <- maxObjective(nForms = 1, itemValues = items_mini$IIF,
itemIDs = items_mini$item)Our further, fixed constraints are defined as additional constraint objects.
itemNumber <- itemsPerFormConstraint(nForms = 1, operator = "=",
targetValue = 10,
itemIDs = items_mini$item)
itemUsage <- itemUsageConstraint(nForms = 1, operator = "<=",
targetValue = 1,
itemIDs = items_mini$item)
testTime <- itemValuesDeviationConstraint(nForms = 1,
itemValues = items_mini$time,
targetValue = 8 * 60,
allowedDeviation = 5,
relative = FALSE,
itemIDs = items_mini$item)Alternatively, we could determine the appropriate test time based on the item pool using the autoItemValuesMinMax() function.
testTime2 <- autoItemValuesMinMaxConstraint(nForms = 1,
itemValues = items_mini$time,
testLength = 10,
allowedDeviation = 5,
relative = FALSE,
itemIDs = items_mini$item)
#> The minimum and maximum values per test form are: min = 418.09 - max = 428.09To automatically assemble the test form based on our constraints, we call the useSolver() function. In this function we define which solver should be used as back end. As a default solver, we recommend GLPK, which is automatically installed alongside this package.
solver_out <- useSolver(list(itemNumber, itemUsage, testTime, testInfo),
solver = "GLPK")
#> GLPK Simplex Optimizer, v4.47
#> 34 rows, 31 columns, 151 non-zeros
#> 0: obj = 0.000000000e+000 infeas = 4.850e+002 (1)
#> * 14: obj = 0.000000000e+000 infeas = 2.220e-016 (0)
#> * 34: obj = 6.734471402e+000 infeas = 0.000e+000 (0)
#> OPTIMAL SOLUTION FOUND
#> GLPK Integer Optimizer, v4.47
#> 34 rows, 31 columns, 151 non-zeros
#> 30 integer variables, all of which are binary
#> Integer optimization begins...
#> + 34: mip = not found yet <= +inf (1; 0)
#> + 60: >>>>> 5.989006125e+000 <= 6.732773863e+000 12.4% (17; 0)
#> + 66: >>>>> 6.729573876e+000 <= 6.729573876e+000 0.0% (16; 11)
#> + 66: mip = 6.729573876e+000 <= tree is empty 0.0% (0; 43)
#> INTEGER OPTIMAL SOLUTION FOUNDThe solution can be inspected directly via inspectSolution() or appended to the item pool via appendSolution(). Using the inspectSolution() function an additional row is created that calculates the column sums for all numeric variables.
inspectSolution(solver_out, items = items_mini, idCol = "item")
#> $form_1
#> item format time difficulty theta=0
#> 8 8 mc 30.21856 -0.36707654 0.6564876
#> 14 14 open 62.99738 0.58136415 0.5712686
#> 15 15 open 56.59458 -0.12012428 0.7150196
#> 20 20 open 87.05063 0.10201223 0.7170949
#> 22 22 order 39.92415 0.15006395 0.7108712
#> 24 24 order 40.52289 -0.53606969 0.5910511
#> 25 25 order 52.15832 0.14083641 0.7122442
#> 26 26 order 38.29060 0.02381911 0.7222039
#> 28 28 order 43.77592 0.41298287 0.6403034
#> 29 29 order 25.55363 0.24091747 0.6930294
#> Sum 211 <NA> 477.08666 0.62872568 6.7295739appendSolution(solver_out, items = items_mini, idCol = "item")
#> item format time difficulty theta=0 form_1
#> 1 1 mc 27.78586 -1.88090278 0.10900318 0
#> 2 2 mc 15.45258 0.84295865 0.44945822 0
#> 3 3 mc 31.01590 1.11881538 0.32661056 0
#> 4 4 mc 29.87421 0.72867743 0.50339241 0
#> 5 5 mc 23.13401 -0.48870993 0.61088162 0
#> 6 6 mc 25.19305 0.47273874 0.61733915 0
#> 7 7 mc 25.66340 -1.18054268 0.30183441 0
#> 8 8 mc 30.21856 -0.36707654 0.65648760 1
#> 9 9 mc 26.61642 -0.56879434 0.57682871 0
#> 10 10 mc 15.35510 1.35397237 0.23900562 0
#> 11 11 open 65.85163 -0.75879786 0.48917461 0
#> 12 12 open 35.94400 2.49927381 0.04012039 0
#> 13 13 open 78.85030 1.33165799 0.24650909 0
#> 14 14 open 62.99738 0.58136415 0.57126860 1
#> 15 15 open 56.59458 -0.12012428 0.71501958 1
#> 16 16 open 45.12778 -1.28629686 0.26229560 0
#> 17 17 open 48.11908 -0.86124314 0.44088544 0
#> 18 18 open 76.32293 0.76977036 0.48398822 0
#> 19 19 open 76.20244 -1.39388826 0.22601541 0
#> 20 20 open 87.05063 0.10201223 0.71709486 1
#> 21 21 order 22.47400 -0.43147145 0.63341304 0
#> 22 22 order 39.92415 0.15006395 0.71087118 1
#> 23 23 order 57.71593 -0.82071059 0.45992776 0
#> 24 24 order 40.52289 -0.53606969 0.59105111 1
#> 25 25 order 52.15832 0.14083641 0.71224418 1
#> 26 26 order 38.29060 0.02381911 0.72220392 1
#> 27 27 order 45.97548 2.79595336 0.02450104 0
#> 28 28 order 43.77592 0.41298287 0.64030341 1
#> 29 29 order 25.55363 0.24091747 0.69302944 1
#> 30 30 order 19.50162 -0.51434114 0.60026891 0