Item Pool Visualization

Nils Petras

2021-12-13

About this package

The IPV package is a tool to create IPV charts. The original work on IPV, including the chart concepts, can be found in: Dantlgraber, M., Stieger, S., & Reips, U. D. (2019). Introducing Item Pool Visualization: A method for investigation of concepts in self-reports and psychometric tests. Methodological Innovations, 12(3), 2059799119884283.. Please cite this paper, as well as this package (Petras, N., & Dantlgraber, M. (2020). IPV: Item Pool visualization. R Package Version 0.1.2) when using the package. The package is built to enable chart creation – quick & dirty or highly customized. Please raise an issue on github in case of bugs or to make suggestions.

Example data

The package contains two sets of example data:

  1. The model estimates from the original IPV paper ?self_confidence ?DSSEI ?SMTQ (also available as excel spreadsheets, see Appendix)
  2. Raw data from the Open-Source Psychometrics Project ?HEXACO

Terminology within the package

IPV uses the concept of item pools, that can be divided into smaller item (sub-)pools. The hierarchy of terms for item pools in this package is “construct” > “test” > “facet”, from large to small. The total item pool of a (psychological) construct may comprise multiple tests, which in turn may comprise multiple facets. This terminology is based on the common data structure in psychological assessment. But IPV can technically be used for any subdivision of an item pool, if multiple levels of specificity exist (see Dantlgraber et al., 2019). In other words, what is called “test” here does not need to be an actual test, etc.

Object structure within the package

This is an example for the data format, that is required by the chart functions and produced by ipv_est():

str(self_confidence, 2)
#> List of 2
#>  $ global:List of 2
#>   ..$ cds :'data.frame': 44 obs. of  6 variables:
#>   ..$ cors: num [1:3, 1:3] 1 0.62 0.73 0.62 1 0.75 0.73 0.75 1
#>   .. ..- attr(*, "dimnames")=List of 2
#>  $ tests :List of 3
#>   ..$ DSSEI:List of 2
#>   ..$ SMTQ :List of 2
#>   ..$ RSES :List of 2
self_confidence$tests$RSES
#> $cds
#>    factor subfactor item         cd   mean_cd aggregate_cd
#> 1    RSES        Ns    2 0.24718807 0.1719081    0.1706512
#> 2    RSES        Ns    5 0.03552244 0.1719081    0.1706512
#> 3    RSES        Ns    6 0.27899664 0.1719081    0.1706512
#> 4    RSES        Ns    8 0.18665548 0.1719081    0.1706512
#> 5    RSES        Ns    9 0.11117763 0.1719081    0.1706512
#> 6    RSES        Ps    1 0.14370647 0.2874100    0.2521100
#> 7    RSES        Ps    3 0.39970201 0.2874100    0.2521100
#> 8    RSES        Ps    4 0.43400090 0.2874100    0.2521100
#> 9    RSES        Ps    7 0.34454922 0.2874100    0.2521100
#> 10   RSES        Ps   10 0.11509134 0.2874100    0.2521100
#> 
#> $cors
#>      Ns   Ps
#> Ns 1.00 0.69
#> Ps 0.69 1.00

Note, that factor refers to an item pool, that was divided into subpools (subfactors). In this case, factor refers to the Rosenberg Self-Esteem Scale, a test that can be divided in two facets: Positive Self-Esteem (Ps), and Negative Self-Esteem (Ns), which was reversed here. As seen below, the same data structure applies on the global level, with factor referring to the overall self-confidence item pool, comprising the three tests (subfactors) RSES, SMTQ, and DSSEI.

self_confidence$global
#> $cds
#>             factor subfactor     item         cd   mean_cd aggregate_cd
#> 1  Self-Confidence     DSSEI  DSSEI.1 0.20411394 0.1327392    0.1135489
#> 2  Self-Confidence     DSSEI  DSSEI.5 0.25899148 0.1327392    0.1135489
#> 3  Self-Confidence     DSSEI  DSSEI.9 0.14573122 0.1327392    0.1135489
#> 4  Self-Confidence     DSSEI DSSEI.13 0.03366102 0.1327392    0.1135489
#> 5  Self-Confidence     DSSEI DSSEI.17 0.06495709 0.1327392    0.1135489
#> 6  Self-Confidence     DSSEI  DSSEI.3 0.13947316 0.1327392    0.1135489
#> 7  Self-Confidence     DSSEI  DSSEI.7 0.00000000 0.1327392    0.1135489
#> 8  Self-Confidence     DSSEI DSSEI.11 0.15085441 0.1327392    0.1135489
#> 9  Self-Confidence     DSSEI DSSEI.15 0.03973729 0.1327392    0.1135489
#> 10 Self-Confidence     DSSEI DSSEI.19 0.00000000 0.1327392    0.1135489
#> 11 Self-Confidence     DSSEI DSSEI.16 0.25272537 0.1327392    0.1135489
#> 12 Self-Confidence     DSSEI  DSSEI.4 0.32061476 0.1327392    0.1135489
#> 13 Self-Confidence     DSSEI DSSEI.12 0.33417252 0.1327392    0.1135489
#> 14 Self-Confidence     DSSEI  DSSEI.8 0.40921877 0.1327392    0.1135489
#> 15 Self-Confidence     DSSEI DSSEI.20 0.14234316 0.1327392    0.1135489
#> 16 Self-Confidence     DSSEI  DSSEI.2 0.00000000 0.1327392    0.1135489
#> 17 Self-Confidence     DSSEI  DSSEI.6 0.04286690 0.1327392    0.1135489
#> 18 Self-Confidence     DSSEI DSSEI.10 0.11532209 0.1327392    0.1135489
#> 19 Self-Confidence     DSSEI DSSEI.14 0.00000000 0.1327392    0.1135489
#> 20 Self-Confidence     DSSEI DSSEI.18 0.00000000 0.1327392    0.1135489
#> 21 Self-Confidence      SMTQ  SMTQ.11 0.15066918 0.3341203    0.3027064
#> 22 Self-Confidence      SMTQ  SMTQ.13 0.71794203 0.3341203    0.3027064
#> 23 Self-Confidence      SMTQ  SMTQ.14 0.33935109 0.3341203    0.3027064
#> 24 Self-Confidence      SMTQ   SMTQ.1 0.31871468 0.3341203    0.3027064
#> 25 Self-Confidence      SMTQ   SMTQ.5 0.33695594 0.3341203    0.3027064
#> 26 Self-Confidence      SMTQ   SMTQ.6 0.38444390 0.3341203    0.3027064
#> 27 Self-Confidence      SMTQ   SMTQ.3 0.21262447 0.3341203    0.3027064
#> 28 Self-Confidence      SMTQ  SMTQ.10 0.28184040 0.3341203    0.3027064
#> 29 Self-Confidence      SMTQ  SMTQ.12 0.01890422 0.3341203    0.3027064
#> 30 Self-Confidence      SMTQ   SMTQ.8 0.29763690 0.3341203    0.3027064
#> 31 Self-Confidence      SMTQ   SMTQ.7 0.56009245 0.3341203    0.3027064
#> 32 Self-Confidence      SMTQ   SMTQ.2 0.44213412 0.3341203    0.3027064
#> 33 Self-Confidence      SMTQ   SMTQ.4 0.22606678 0.3341203    0.3027064
#> 34 Self-Confidence      SMTQ   SMTQ.9 0.39030751 0.3341203    0.3027064
#> 35 Self-Confidence      RSES   RSES.1 0.19990803 0.4685707    0.4677902
#> 36 Self-Confidence      RSES   RSES.3 0.13617999 0.4685707    0.4677902
#> 37 Self-Confidence      RSES   RSES.4 0.14390308 0.4685707    0.4677902
#> 38 Self-Confidence      RSES   RSES.7 0.44631515 0.4685707    0.4677902
#> 39 Self-Confidence      RSES  RSES.10 0.46183057 0.4685707    0.4677902
#> 40 Self-Confidence      RSES   RSES.2 0.62972058 0.4685707    0.4677902
#> 41 Self-Confidence      RSES   RSES.5 0.53162016 0.4685707    0.4677902
#> 42 Self-Confidence      RSES   RSES.6 0.80121585 0.4685707    0.4677902
#> 43 Self-Confidence      RSES   RSES.8 0.60451698 0.4685707    0.4677902
#> 44 Self-Confidence      RSES   RSES.9 0.73049645 0.4685707    0.4677902
#> 
#> $cors
#>       DSSEI RSES SMTQ
#> DSSEI  1.00 0.62 0.73
#> RSES   0.62 1.00 0.75
#> SMTQ   0.73 0.75 1.00

cd is short for center distance
\[\begin{equation} cd_i = \frac{\lambda^2_{is}}{\lambda^2_{ig}} -1 \end{equation}\]
while mean_cd is the mean center distance of the items of a facet or test. aggregate_cd is the center distance of a facet or test as a whole (see cd_method). Furthermore, the matrix of latent correlations between subfactors is given as a second item of the list.

There is a raw variant ($est_raw in the output of ipv_est) of this, providing factor loadings instead of center distances.

Appendix

2. Read existing estimates (alternative Workflow)

Using ipv_est() will take less effort than estimating the models yourself and reading in the estimates. For this reason, it is recommended. On the other hand, reading in existing estimates provides you with additional options during model estimation, or the ability to use published estimates without access to the raw data.

Regardless of the input mode, you will need to provide:

To spare you the task of formatting estimates by hand, there are two automated input pathways. You can either use excel files or the manual input function. In both cases, center distances are calculated automatically and the estimates are automatically checked for (obvious) errors. Negative center distances are always set to zero before mean center distances are calculated.

… using estimates from within R

These functions allow you to reduce the manual work to a minimum. They are especially useful, when your SEM estimates are already in your R environment (e.g. because you read them from a .csv file). The functions input_manual_nested() and input_manual_simple() allow you to feed in vectors of factor loadings, item names, etc. The correct format for plotting is then generated automatically. Run input_manual_process() on the result, to automatically calculate center distances.
This is an example, where all estimates are put in completely manually for demonstration purposes:

mydata <- input_manual_simple(
test_name = "RSES",
facet_names = c("Ns", "Ps"),
items_per_facet = 5,
item_names = c(
  2, 5, 6, 8, 9,
  1, 3, 4, 7, 10),
test_loadings = c(
  .5806, .5907, .6179, .5899, .6559,
  .6005, .4932, .4476, .5033, .6431),
facet_loadings = c(
  .6484, .6011, .6988, .6426, .6914,
  .6422, .5835, .536, .5836, .6791),
correlation_matrix = matrix(
  data = c(1, .69,
           .69, 1),
  nrow = 2,
  ncol = 2))
mydata
#> $fls
#>    factor subfactor item factor_loading subfactor_loading
#> 1    RSES        Ns    2         0.5806            0.6484
#> 2    RSES        Ns    5         0.5907            0.6011
#> 3    RSES        Ns    6         0.6179            0.6988
#> 4    RSES        Ns    8         0.5899            0.6426
#> 5    RSES        Ns    9         0.6559            0.6914
#> 6    RSES        Ps    1         0.6005            0.6422
#> 7    RSES        Ps    3         0.4932            0.5835
#> 8    RSES        Ps    4         0.4476            0.5360
#> 9    RSES        Ps    7         0.5033            0.5836
#> 10   RSES        Ps   10         0.6431            0.6791
#> 
#> $cors
#>      Ns   Ps
#> Ns 1.00 0.69
#> Ps 0.69 1.00
input_manual_process(mydata)
#> $cds
#>    factor subfactor item         cd   mean_cd aggregate_cd
#> 1    RSES        Ns    2 0.24718807 0.1719081    0.1706512
#> 2    RSES        Ns    5 0.03552244 0.1719081    0.1706512
#> 3    RSES        Ns    6 0.27899664 0.1719081    0.1706512
#> 4    RSES        Ns    8 0.18665548 0.1719081    0.1706512
#> 5    RSES        Ns    9 0.11117763 0.1719081    0.1706512
#> 6    RSES        Ps    1 0.14370647 0.2874100    0.2521100
#> 7    RSES        Ps    3 0.39970201 0.2874100    0.2521100
#> 8    RSES        Ps    4 0.43400090 0.2874100    0.2521100
#> 9    RSES        Ps    7 0.34454922 0.2874100    0.2521100
#> 10   RSES        Ps   10 0.11509134 0.2874100    0.2521100
#> 
#> $cors
#>      Ns   Ps
#> Ns 1.00 0.69
#> Ps 0.69 1.00

For nested cases, use the function input_manual_nested(), and add the individual tests using input_manual_simple(). Then you can run input_manual_processs() as in the simple case. You can find a (lengthy) example below.

… using Excel files

Excel files have the advantage that you can simply copy and paste your SEM estimates into spreadsheets and the input function of the IPV package (input_excel()) does the rest. The files need to be structured as in the example, that you can find here:

system.file("extdata", "IPV_global.xlsx", package = "IPV", mustWork = TRUE)
system.file("extdata", "IPV_DSSEI.xlsx", package = "IPV", mustWork = TRUE)
system.file("extdata", "IPV_SMTQ.xlsx", package = "IPV", mustWork = TRUE)
system.file("extdata", "IPV_RSES.xlsx", package = "IPV", mustWork = TRUE)

As you can see, there is a file for each test, and a global file. You might want to use a copy as your template, so you can just fill in your values. Open a file to see how it works.

On sheet 1 you need to provide the factor loadings from your SEM estimation results, on sheet 2 you need to provide the named and complete latent correlation matrix. On sheet 1, “factor” contains a single factor name and “factor_loading” the factor loadings of items on that factor (not squared). “subfactor” contains the names of grouped factors and “subfactor_loading” the factor loadings of items on these factors (not squared). “item” contains the item names. Therefore, each row contains the full information on the respective item.
Read these excel sheets using input_excel. In the example:

global <- system.file("extdata", "IPV_global.xlsx", package = "IPV", mustWork = TRUE)
tests <- c(system.file("extdata", "IPV_DSSEI.xlsx", package = "IPV", mustWork = TRUE),
           system.file("extdata", "IPV_SMTQ.xlsx", package = "IPV", mustWork = TRUE),
           system.file("extdata", "IPV_RSES.xlsx", package = "IPV", mustWork = TRUE))
mydata <- input_excel(global = global, tests = tests)
#> New names:
#> * `` -> ...1
#> Negative center distance adjusted to 0
#> New names:
#> * `` -> ...1
#> Negative center distance adjusted to 0
#> New names:
#> * `` -> ...1
#> Negative center distance adjusted to 0
#> New names:
#> * `` -> ...1

The data will be prepared automatically, including the calculation of center distances. If any factor loading is below .1 or any center distance below 0, it is set to that value and a warning or message is displayed. IPV does not allow negative factor loadings, which is indicated by an error. If possible, recode your data appropriately.

Mix tests with and without facets

In nested charts, tests do not need to have facets. If you use input by excel, use NA instead of providing a file name.

global <- system.file("extdata", "IPV_global.xlsx", package = "IPV", mustWork = TRUE)
tests <- c(system.file("extdata", "IPV_DSSEI.xlsx", package = "IPV", mustWork = TRUE),
           system.file("extdata", "IPV_SMTQ.xlsx", package = "IPV", mustWork = TRUE),
           NA)
mydata <- input_excel(global = global, tests = tests)

If you use manual input, do not provide data on facetless tests with input_manual_simple(). Any further treatment of facetless tests is handled automatically.

Manual input in nested cases - example

Note that all values that are put in manually for presentation purposes here could be read from an arbitrarily formatted R object. Copying values manually from another source is error-prone and therefore not recommended.

# first the global level
mydata <- input_manual_nested(
  construct_name = "Self-Confidence",
  test_names = c("DSSEI", "SMTQ", "RSES"),
  items_per_test = c(20, 14, 10),
  item_names = c(
     1,  5,  9, 13, 17, # DSSEI
     3,  7, 11, 15, 19, # DSSEI
    16,  4, 12,  8, 20, # DSSEI
     2,  6, 10, 14, 18, # DSSEI
    11, 13, 14,  1,  5,  6, # SMTQ
     3, 10, 12,  8, # SMTQ
     7,  2,  4,  9, # SMTQ
     1,  3,  4,  7, 10, # RSES
     2,  5,  6,  8,  9), # RSES
  construct_loadings = c(
    .5189, .6055, .618 , .4074, .4442,
    .5203, .2479, .529 , .554 , .5144,
    .3958, .5671, .5559, .4591, .4927,
    .3713, .5941, .4903, .5998, .6616,
    .4182, .2504, .4094, .3977, .5177, .4603,
    .3271, .261 , .3614, .4226,
    .2076, .3375, .5509, .3495,
    .5482, .4627, .4185, .4185, .5319,
    .4548, .4773, .4604, .4657, .4986),
  test_loadings = c(
    .5694, .6794, .6615, .4142, .4584, # DSSEI
    .5554, .2165, .5675, .5649, .4752, # DSSEI
    .443 , .6517, .6421, .545 , .5266, # DSSEI
    .302 , .6067, .5178, .5878, .6572, # DSSEI
    .4486, .3282, .4738, .4567, .5986, .5416, # SMTQ
    .3602, .2955, .3648, .4814, # SMTQ
    .2593, .4053, .61  , .4121, # SMTQ
    .6005, .4932, .4476, .5033, .6431, # RSES
    .5806, .5907, .6179, .5899, .6559), # RSES
  correlation_matrix = matrix(
    data = c(
      1 , .73, .62,
      .73, 1, .75,
      .62, .75, 1),
    nrow = 3,
    ncol = 3))

# then add tests individually
# test 1
mydata$tests$RSES <- input_manual_simple(
  test_name = "RSES",
  facet_names = c("Ns", "Ps"),
  items_per_facet = c(5, 5),
  item_names = c(
    2, 5, 6, 8,  9,
    1, 3, 4, 7, 10),
  test_loadings = c(
    .5806, .5907, .6179, .5899, .6559,
    .6005, .4932, .4476, .5033, .6431),
  facet_loadings = c(
    .6484, .6011, .6988, .6426, .6914,
    .6422, .5835, .536, .5836, .6791),
  correlation_matrix = matrix(
    data = c(
      1, .69,
      .69, 1),
    nrow = 2,
    ncol = 2))
# test 2
mydata$tests$DSSEI <- input_manual_simple(
  test_name = "DSSEI",
  facet_names = c("Ab", "Pb", "Ph", "So"),
  items_per_facet = 5,
  item_names = c(
    2, 6, 10, 14, 18,
    16, 4, 12, 8, 20,
    3, 7, 11, 15, 19,
    1, 5, 9, 13, 17),
  test_loadings = c(
    .302 , .6067, .5178, .5878, .6572,
    .443 , .6517, .6421, .545 , .5266,
    .5554, .2165, .5675, .5649, .4752,
    .5694, .6794, .6615, .4142, .4584),
  facet_loadings = c(
    .3347, .6537, .6078, .684 , .735 ,
    .6861, .8746, .7982, .7521, .6794,
    .7947, .3737, .819 , .7099, .5785,
    .7293, .8284, .7892, .3101, .4384),
  correlation_matrix = matrix(
    data = c(
      1, .49, .66, .76,
      .49, 1, .37, .54,
      .66, .37, 1, .53,
      .76, .54, .53, 1),
    nrow = 4,
    ncol = 4))
# test 3
mydata$tests$SMTQ <- input_manual_simple(
  test_name = "SMTQ",
  facet_names = c("Cf", "Cs", "Ct"),
  items_per_facet = c(6, 4, 4),
  item_names = c(
    11, 13, 14, 1, 5, 6,
    3, 10, 12, 8,
    7, 2, 4, 9),
  test_loadings = c(
    .4486, .3282, .4738, .4567, .5986, .5416,
    .3602, .2955, .3648, .4814,
    .2593, .4053, .61  , .4121),
  facet_loadings = c(
    .4995, .3843, .5399, .4562, .6174, .6265,
    .4601, .3766, .4744, .5255,
    .3546, .5038, .7429, .4342),
  correlation_matrix = matrix(
    data = c(
      1, .71, .62,
      .71, 1, .59,
      .62, .59, 1),
    nrow = 3,
    ncol = 3))

# finally process (as in a simple case)
my_processed_data <- input_manual_process(mydata)
#> Negative center distance adjusted to 0
#> Negative center distance adjusted to 0
#> Negative center distance adjusted to 0
my_processed_data
#> $global
#> $global$cds
#>             factor subfactor     item         cd   mean_cd aggregate_cd
#> 1  Self-Confidence     DSSEI  DSSEI.1 0.20411394 0.1327392    0.1135489
#> 2  Self-Confidence     DSSEI  DSSEI.5 0.25899148 0.1327392    0.1135489
#> 3  Self-Confidence     DSSEI  DSSEI.9 0.14573122 0.1327392    0.1135489
#> 4  Self-Confidence     DSSEI DSSEI.13 0.03366102 0.1327392    0.1135489
#> 5  Self-Confidence     DSSEI DSSEI.17 0.06495709 0.1327392    0.1135489
#> 6  Self-Confidence     DSSEI  DSSEI.3 0.13947316 0.1327392    0.1135489
#> 7  Self-Confidence     DSSEI  DSSEI.7 0.00000000 0.1327392    0.1135489
#> 8  Self-Confidence     DSSEI DSSEI.11 0.15085441 0.1327392    0.1135489
#> 9  Self-Confidence     DSSEI DSSEI.15 0.03973729 0.1327392    0.1135489
#> 10 Self-Confidence     DSSEI DSSEI.19 0.00000000 0.1327392    0.1135489
#> 11 Self-Confidence     DSSEI DSSEI.16 0.25272537 0.1327392    0.1135489
#> 12 Self-Confidence     DSSEI  DSSEI.4 0.32061476 0.1327392    0.1135489
#> 13 Self-Confidence     DSSEI DSSEI.12 0.33417252 0.1327392    0.1135489
#> 14 Self-Confidence     DSSEI  DSSEI.8 0.40921877 0.1327392    0.1135489
#> 15 Self-Confidence     DSSEI DSSEI.20 0.14234316 0.1327392    0.1135489
#> 16 Self-Confidence     DSSEI  DSSEI.2 0.00000000 0.1327392    0.1135489
#> 17 Self-Confidence     DSSEI  DSSEI.6 0.04286690 0.1327392    0.1135489
#> 18 Self-Confidence     DSSEI DSSEI.10 0.11532209 0.1327392    0.1135489
#> 19 Self-Confidence     DSSEI DSSEI.14 0.00000000 0.1327392    0.1135489
#> 20 Self-Confidence     DSSEI DSSEI.18 0.00000000 0.1327392    0.1135489
#> 21 Self-Confidence      SMTQ  SMTQ.11 0.15066918 0.3341203    0.3027064
#> 22 Self-Confidence      SMTQ  SMTQ.13 0.71794203 0.3341203    0.3027064
#> 23 Self-Confidence      SMTQ  SMTQ.14 0.33935109 0.3341203    0.3027064
#> 24 Self-Confidence      SMTQ   SMTQ.1 0.31871468 0.3341203    0.3027064
#> 25 Self-Confidence      SMTQ   SMTQ.5 0.33695594 0.3341203    0.3027064
#> 26 Self-Confidence      SMTQ   SMTQ.6 0.38444390 0.3341203    0.3027064
#> 27 Self-Confidence      SMTQ   SMTQ.3 0.21262447 0.3341203    0.3027064
#> 28 Self-Confidence      SMTQ  SMTQ.10 0.28184040 0.3341203    0.3027064
#> 29 Self-Confidence      SMTQ  SMTQ.12 0.01890422 0.3341203    0.3027064
#> 30 Self-Confidence      SMTQ   SMTQ.8 0.29763690 0.3341203    0.3027064
#> 31 Self-Confidence      SMTQ   SMTQ.7 0.56009245 0.3341203    0.3027064
#> 32 Self-Confidence      SMTQ   SMTQ.2 0.44213412 0.3341203    0.3027064
#> 33 Self-Confidence      SMTQ   SMTQ.4 0.22606678 0.3341203    0.3027064
#> 34 Self-Confidence      SMTQ   SMTQ.9 0.39030751 0.3341203    0.3027064
#> 35 Self-Confidence      RSES   RSES.1 0.19990803 0.4685707    0.4677902
#> 36 Self-Confidence      RSES   RSES.3 0.13617999 0.4685707    0.4677902
#> 37 Self-Confidence      RSES   RSES.4 0.14390308 0.4685707    0.4677902
#> 38 Self-Confidence      RSES   RSES.7 0.44631515 0.4685707    0.4677902
#> 39 Self-Confidence      RSES  RSES.10 0.46183057 0.4685707    0.4677902
#> 40 Self-Confidence      RSES   RSES.2 0.62972058 0.4685707    0.4677902
#> 41 Self-Confidence      RSES   RSES.5 0.53162016 0.4685707    0.4677902
#> 42 Self-Confidence      RSES   RSES.6 0.80121585 0.4685707    0.4677902
#> 43 Self-Confidence      RSES   RSES.8 0.60451698 0.4685707    0.4677902
#> 44 Self-Confidence      RSES   RSES.9 0.73049645 0.4685707    0.4677902
#> 
#> $global$cors
#>       DSSEI SMTQ RSES
#> DSSEI  1.00 0.73 0.62
#> SMTQ   0.73 1.00 0.75
#> RSES   0.62 0.75 1.00
#> 
#> 
#> $tests
#> $tests$DSSEI
#> $tests$DSSEI$cds
#>    factor subfactor item        cd   mean_cd aggregate_cd
#> 1   DSSEI        Ab    2 0.2282804 0.2743874    0.2738012
#> 2   DSSEI        Ab    6 0.1609379 0.2743874    0.2738012
#> 3   DSSEI        Ab   10 0.3778353 0.2743874    0.2738012
#> 4   DSSEI        Ab   14 0.3541072 0.2743874    0.2738012
#> 5   DSSEI        Ab   18 0.2507761 0.2743874    0.2738012
#> 6   DSSEI        Pb   16 1.3986528 0.8627867    0.8039587
#> 7   DSSEI        Pb    4 0.8010406 0.8627867    0.8039587
#> 8   DSSEI        Pb   12 0.5453189 0.8627867    0.8039587
#> 9   DSSEI        Pb    8 0.9044000 0.8627867    0.8039587
#> 10  DSSEI        Pb   20 0.6645214 0.8627867    0.8039587
#> 11  DSSEI        Ph    3 1.0473622 1.0341577    0.8657735
#> 12  DSSEI        Ph    7 1.9794108 1.0341577    0.8657735
#> 13  DSSEI        Ph   11 1.0827449 1.0341577    0.8657735
#> 14  DSSEI        Ph   15 0.5792512 1.0341577    0.8657735
#> 15  DSSEI        Ph   19 0.4820193 1.0341577    0.8657735
#> 16  DSSEI        So    1 0.6405048 0.3101167    0.3266159
#> 17  DSSEI        So    5 0.4867197 0.3101167    0.3266159
#> 18  DSSEI        So    9 0.4233590 0.3101167    0.3266159
#> 19  DSSEI        So   13 0.0000000 0.3101167    0.3266159
#> 20  DSSEI        So   17 0.0000000 0.3101167    0.3266159
#> 
#> $tests$DSSEI$cors
#>      Ab   Pb   Ph   So
#> Ab 1.00 0.49 0.66 0.76
#> Pb 0.49 1.00 0.37 0.54
#> Ph 0.66 0.37 1.00 0.53
#> So 0.76 0.54 0.53 1.00
#> 
#> 
#> $tests$SMTQ
#> $tests$SMTQ$cds
#>    factor subfactor item         cd   mean_cd aggregate_cd
#> 1    SMTQ        Cf   11 0.23980233 0.2185428    0.1986201
#> 2    SMTQ        Cf   13 0.37108259 0.2185428    0.1986201
#> 3    SMTQ        Cf   14 0.29848382 0.2185428    0.1986201
#> 4    SMTQ        Cf    1 0.00000000 0.2185428    0.1986201
#> 5    SMTQ        Cf    5 0.06379961 0.2185428    0.1986201
#> 6    SMTQ        Cf    6 0.33808850 0.2185428    0.1986201
#> 7    SMTQ        Cs    3 0.63161260 0.5346459    0.4688794
#> 8    SMTQ        Cs   10 0.62422302 0.5346459    0.4688794
#> 9    SMTQ        Cs   12 0.69114054 0.5346459    0.4688794
#> 10   SMTQ        Cs    8 0.19160761 0.5346459    0.4688794
#> 11   SMTQ        Ct    7 0.87013272 0.5021480    0.4480744
#> 12   SMTQ        Ct    2 0.54512322 0.5021480    0.4480744
#> 13   SMTQ        Ct    4 0.48320454 0.5021480    0.4480744
#> 14   SMTQ        Ct    9 0.11013146 0.5021480    0.4480744
#> 
#> $tests$SMTQ$cors
#>      Cf   Cs   Ct
#> Cf 1.00 0.71 0.62
#> Cs 0.71 1.00 0.59
#> Ct 0.62 0.59 1.00
#> 
#> 
#> $tests$RSES
#> $tests$RSES$cds
#>    factor subfactor item         cd   mean_cd aggregate_cd
#> 1    RSES        Ns    2 0.24718807 0.1719081    0.1706512
#> 2    RSES        Ns    5 0.03552244 0.1719081    0.1706512
#> 3    RSES        Ns    6 0.27899664 0.1719081    0.1706512
#> 4    RSES        Ns    8 0.18665548 0.1719081    0.1706512
#> 5    RSES        Ns    9 0.11117763 0.1719081    0.1706512
#> 6    RSES        Ps    1 0.14370647 0.2874100    0.2521100
#> 7    RSES        Ps    3 0.39970201 0.2874100    0.2521100
#> 8    RSES        Ps    4 0.43400090 0.2874100    0.2521100
#> 9    RSES        Ps    7 0.34454922 0.2874100    0.2521100
#> 10   RSES        Ps   10 0.11509134 0.2874100    0.2521100
#> 
#> $tests$RSES$cors
#>      Ns   Ps
#> Ns 1.00 0.69
#> Ps 0.69 1.00