Skip to contents

Changelog

Source: NEWS.md

intraclass (development version)

intraclass 0.2.0

Breaking changes

  • The result classes now start with the package name. icc() returns an intraclass_icc object instead of icc. d_study() returns intraclass_dstudy instead of icc_dstudy, and choose_icc() returns intraclass_recommendation instead of icc_recommendation. The performance package also uses the class icc. When performance was loaded after intraclass, its print method replaced the intraclass one, and a printed result showed only its header. Code that tests for an old class name, for example inherits(x, "icc"), must use the new name. The change has no deprecation period, because a class name cannot give a warning. Results saved with an earlier version keep the old class, so run icc() again to get the new one. If you attach both intraclass and performance, the package that you attach last still masks the icc() function of the other. Call intraclass::icc() to be sure which function runs.

New features

  • If the parameters package is installed, parameters::model_parameters() now works on icc() results. The method returns the tidy() table under the column names of easystats, the family of packages that parameters belongs to. The table is titled “Intraclass correlation coefficients”. Its main columns are Parameter, Coefficient, SE, CI, CI_low, CI_high and Method. The other columns are the identifiers Type, Level, Occasions and SF_Index. A ci value other than the confidence level of the fit is an error, because the table cannot compute the interval again. parameters is an optional dependency.

  • Four simulated teaching datasets now ship with the package. school holds pupils nested in classrooms, and school_incomplete is the same design with a fifth of the ratings dropped. ratings_replicates has three ratings per subject-by-rater cell, and ratings_twoway has 20 subjects by 4 raters. The articles and the README load them instead of simulating data inline, so the examples run from a fresh session with no setup. Each has a reference page that names the generating script and seed. The values the articles report do not change. The README’s multilevel example now shows the school results.

  • The package now ships a citation entry, which citation("intraclass") prints. The README carries a Citation section, which links to the companion software paper’s repository, https://github.com/jmgirard/intraclass-paper. A separate README section links to a new contributing guide. The guide says how to report a problem, ask for help and propose a change.

Bug fixes

  • If the scores spread far more than 1 unit, the default engine = "glmmTMB" sometimes stopped before the best solution, with the subject variance at 0. Heartbeat intervals in milliseconds are an example. icc() then reported an ICC of 0 with an interval of [0, 0], or failed to fit. Every glmmTMB fit now starts from values scaled to the spread of the scores. Multiplying every score by a constant now leaves the ICC unchanged.

  • If a default engine = "glmmTMB" fit stops with a variance at or near 0, the fit now runs once more from a second starting point. The engine is the software that does the fitting. If only the second fit has a finite result, it replaces the first. If the second fit is better by a set margin, it also replaces the first. But if the first fit’s Hessian is positive definite and the second’s is not, the first fit is kept. In all other cases the first fit is kept. Only the kept fit’s warnings and messages are shown. On test data where the default starting values of glmmTMB stopped at a false 0, the fit now reaches the ICC of the lme4 engine.

  • icc() now refuses data in which at least one subject has two or more scores and no subject has two different scores. It raises an intraclass_singular_fit error before it fits a model, on every engine. A call that an earlier argument or design check refuses keeps that check’s error. Before, such data ended in an unclassed glmmTMB or lme4 error on some paths. On such data whose scores differed between subjects, a one-way lme4 fit returned an ICC of 1 with the interval [1, 1].

  • On the glmmTMB and lme4 engines, an engine error that does not come from rlang now reaches the caller as an intraclass_singular_fit error. This covers errors while the engine fits the model or reads the fitted parameter covariance. The engine’s own error is kept as its cause. Before, such errors carried no package class, so a handler that catches by class missed them. Two-way data whose residual is exactly zero raised them, for example scores that vary only by rater. An error that R converts from a warning under options(warn = 2), and an error of class stackOverflowError, pass unchanged. If R shortened the warning’s text, the error is still re-raised.

  • A Monte-Carlo interval is built by drawing parameter values from the fitted model’s uncertainty. Under ci_method = "montecarlo" or "bootstrap", an interval whose two limits are exactly equal is no longer reported on the glmmTMB and lme4 engines. icc() stops with an intraclass_zero_width_interval error instead. Before, a glmmTMB fit with a variance stuck at 0 sometimes printed an ICC of 0 with the interval [0, 0]. A narrow interval whose limits differ is still reported.

  • Under the same two methods on the same two engines, icc() now warns about some intervals near 1. For a single-rating row, an interval whose lower limit lies within 1e-12 of 1 gets an intraclass_unresolved_interval warning. For a row that averages ratings over raters, occasions or both, the bound is 1e-12 divided by the number of ratings averaged. The warning names its rows. Data with almost no residual variation can give such an interval, and its width can then reflect rounding rather than the data. Correct intervals from scores with very little noise can also get the warning. The interval itself is unchanged.

  • On glmmTMB and lme4 fits, d_study() now checks its interval band in the same two ways. It reads each curve’s single-rating coefficient, at one rater and one occasion. If that coefficient’s two limits are exactly equal, d_study() refuses the band with an intraclass_zero_width_interval error. If the limits differ and the lower limit lies within 1e-12 of 1, it warns with intraclass_unresolved_interval. At a very large m, a projected row can print as [1, 1] through rounding without raising either. The projected numbers are unchanged.

  • When a variance in a Monte-Carlo draw overflows, the error no longer tells a glmmTMB user to refit with engine = "glmmTMB".

Documentation

  • The package’s messages, the reference manual, the README and the articles are rewritten in plainer English. The messages cover errors, warnings, notes and install prompts. They also cover the questions and advice that choose_icc() prints, and the notes that print() and summary() add to an icc() fit. Each glossary term is defined in plain words, or linked to the Glossary article, where a page first uses it. Which condition or note fires, and when, is unchanged. Nothing computed changes.

  • Five definitions in the Glossary article were imprecise or said less than the package does, and they are corrected. Every page that quoted them now carries the new text. Each glossary entry now opens with a short definition, or points to the entry that holds one.

  • The icc() help page is now titled “Intraclass correlation coefficients for interrater reliability”, and the d_study() page “Project reliability to other numbers of raters”. In the Confidence-interval methods article, each ci_method section now opens with a paragraph on what the method is for and when to choose it. Four error messages from icc() and d_study() no longer cite internal record numbers.

  • Four articles now show their comparison tables as gt tables with a title, plain column headings and rounded numbers. The four are Comparison with other packages, Estimation engines, Confidence-interval methods and Choosing an ICC. gt joins Suggests:, and without it those tables are skipped. The sentences that disagreed with their tables are corrected. The packages agree to within 0.00001, not exactly. In the Choosing an ICC incomplete-data example, the random-rater interval is wider only for the agreement coefficients, which ask whether raters give the same score.

intraclass 0.1.0

CRAN release: 2026-09-10

First public release.

intraclass estimates interrater-reliability intraclass correlation coefficients (ICCs) within the generalizability-theory framework. Variance components come from a linear mixed model rather than from classical ANOVA mean squares. A point estimate is never reported without an interval, and ci_method selects which interval that is. The package requires R 4.5.0 or newer.

What ships

  • icc() fits the model and reports the ICC family the design defines. The type argument picks absolute agreement or consistency, where consistency means raters agree apart from a constant offset per rater. The unit argument picks single or average. The raters argument picks random or fixed raters, where fixed means the observed raters are the whole population of interest. The model argument picks one-way or two-way, where two-way means each rater is tracked across the subjects they score. See Getting started and ?icc.
  • Data need not form a complete, balanced grid. There is support for imbalanced, incomplete, and multilevel (nested) designs. That covers unequal ratings per subject, missing ratings, and subjects nested in a higher-level unit such as a classroom or clinic. A cluster column on a two-way design switches on the multilevel ICC. In a multilevel design the subject level is reliability within a cluster, and the cluster level is reliability of cluster means. When the same raters span every cluster, the column also adds the cluster level. See Multilevel designs: subject and cluster level and ?icc for the layouts and where it refuses.
  • Rating a subject-by-rater cell more than once gives a within-cell replicate design. On a two-way random design icc() splits the single-rating residual into a subject-by-rater interaction and pure error, and occasions reports the reliability of one rating or, on balanced replicates, of the mean. See D-studies and within-cell replicates and ?icc for the designs replicates support and where they refuse.
  • d_study() projects a fitted reliability to other numbers of raters (m) or occasions (n_o), with a plot() and ggplot2::autoplot() curve. D-studies and within-cell replicates gives the designs each projection supports and where it refuses.
  • choose_icc() recommends which coefficient or coefficients to report, explains the reasoning, and prints the icc() call to run. Its type, unit and level questions each take "both", which asks for the pair rather than making you choose one. It gives advice only. See Choosing an ICC.
  • tidy() and glance() return tidy summaries of a fit or a projection. print(), format(), plot() and ggplot2::autoplot() methods are provided for both classes, and summary() for an icc() fit. ggplot2 is a Suggests dependency.
  • Datasets ratings and ratings_incomplete, used throughout the documentation.
  • Eight articles: Getting started, Choosing an ICC, Multilevel designs: subject and cluster level, Estimation engines, Confidence-interval methods, D-studies and within-cell replicates, Glossary, and Comparison with other packages.

Engines

  • The default engine is glmmTMB, the package’s only hard engine dependency. engine = "lme4", engine = "lavaan" and engine = "brms" are selectable. Which designs each engine covers, and where it refuses, is documented in Estimation engines and ?icc.
  • The brms engine uses a prior, the distribution placed on a parameter before seeing the data. It fits both random- and fixed-rater models under a sourced half-t(4, 0, 1) prior on every random-effect standard deviation. Its point estimate is the posterior mode, the peak of the posterior distribution. Its interval is a percentile credible interval, which holds the share of the posterior probability that the confidence level sets. Because the interval comes from the posterior draws, ci_method = "posterior" is forced. Supplying a custom prior is a deliberate deviation: icc() warns, and the coverage results this package reports no longer apply.
  • The lme4 package itself is already on your library path after a plain install, because glmmTMB lists it in its own Imports, but the lme4 engine also needs merDeriv, which does not arrive. merDeriv, lavaan and brms sit in this package’s Suggests, and a plain install fetches none of the three.

Confidence intervals

  • ci_method selects the interval: "montecarlo" (the default), "bootstrap", "npbootstrap", "searle", "burch", "mpl", and "posterior" under brms. The default is the Monte-Carlo interval, built by drawing parameter values from the fitted model’s uncertainty. Where a method does not apply to the design, the call aborts with a classed error. Confidence-interval methods compares them; ?icc gives the per-method conditions.
  • Interval coverage was studied by simulation before release, and the limitations those studies found are documented rather than smoothed over. The Monte-Carlo default under-covers when the subject effects are strongly skewed or heavy-tailed, worst 0.6725 at chi-square(1) subject effects with a true ICC of 0.6, 50 subjects and 5 raters. "burch" is no remedy there (worst 0.6655).
  • "searle" and "burch" are the two classical closed forms, and the three grids below measure their widths: the smaller grid’s 16 cells and the larger grid’s 64 cells draw only the subject effects from the non-normal family. The third draws the residual from the same family as the subject effect.
  • Which of the two closed forms gives the narrower interval is conditional. On both grids that vary only the subject effect, "burch" is the narrower of the two in 16 of 16 cells of the smaller grid and 59 of 64 cells of the larger grid.
  • That width margin holds much the same up to a true ICC of 0.3 rather than shrinking as the true ICC rises (on the larger grid; the smaller grid’s margin does shrink across its levels). The "burch" width advantage then collapses to near parity at a true ICC of 0.6, on the one grid reaching that value. It also shrinks steadily as the subject count grows, measured at 5 raters.
  • What "burch" does against "searle" also depends on the residual, and the three grids measure that: the two grids that vary only the subject effect put it narrower nearly everywhere, while the third, which draws the residual from the same family as the subject effect, puts it wider at every symmetric heavy-tailed family measured (a median width ratio of 1.2963 at t(5) with 100 subjects) and narrower at every lighter-tailed one, the normal included.
  • The ci_method = "mpl" documentation states the interpolation evidence behind off-node subject counts. The correction constant is calibrated at subject-count nodes and linearly interpolated between them. The interpolated path is coverage-validated at each supported confidence level, at the default 0.95 by three off-node cells, each clearing its pre-registered coverage floor. The documentation also says what that validation does not establish (interpolated values are validated at a handful of geometries, not calibrated, and the interval’s asymmetry direction is not uniform across rater counts). Every universal or negative claim this documentation makes about the validated cells is settled mechanically, in CI, against the committed coverage fixture (data-raw/check-mpl-doc-claims.py).

When a call fails

  • Errors are classed and name the quantities that failed the check, rather than asserting a cause. Where an interval method aborts on degenerate data, the error names another ci_method only after running it on your data and confirming it returns a usable interval.