The vast majority of score differences for both domains and items included in this research are statistically significant. Though Standard Deviation is the preferred statistic when interpreting differences in scores, it is only appropriate when the distribution of scores is normal. UniLOA domain and item score distributions are all negatively skewed, rendering Standard Deviation an unreliable indicator of variance. As such, we have to rely on overall patterns and observations over time, which tell us that:
Score differences of 0-2 are considered “normal” and don’t draw attention from researchers. Those areas of difference are probably more based in normal statistical error than of something “real”
Score differences of 2-3 are considered worthy of additional investigation. They trigger enough curiosity to beg the question “what might be going on here?”
Score differences greater than 3 is always a “red flag” for researchers, and that “flag” gets more red and larger as the score increases beyond 3. These areas are markedly significant and show a clear differentiation either to the positive or the negative. Differences of +6.5, +8.2, and +11.0, as found in this report, are staggering.