Nested cross-validated comparator court for GazeWeave

Every method receives the same outer folds, held-out candidate sets, candidate priors, and cross-fitted warp. Registered ridge composites select their penalty using inner folds only. Frozen scores remain explicitly uncalibrated compatibility scores.

Usage

gaze_baseline_cv(
  ref_tab,
  source_tab,
  match_on,
  contrast_on = NULL,
  refvar = "fixgroup",
  sourcevar = "fixgroup",
  spec,
  split_on = match_on,
  n_folds = NULL,
  seed = 1,
  fit_source_filter = NULL,
  eval_source_filter = NULL,
  inner_split_on = split_on
)

Arguments

ref_tab, source_tab: Reference and source tables.
match_on: One or more columns defining the true template match.
contrast_on: Optional columns restricting nonmatching candidates.
refvar, sourcevar: List columns containing fixation groups.
spec: A [gaze_baseline_spec()].
split_on: Columns defining the held-out unit. Defaults to `match_on`.
n_folds: Number of cross-fitting folds. `NULL` uses up to five; for Replay, which scores each held-out row against the candidates in its own fold, the default is lowered when needed so that every fold keeps a true candidate and a nonmatch in each contrast stratum. Calibration needs more: each training fold must hold at least `2 * calibration folds` items (4 by default) per contrast stratum, or that fold falls back to an uncalibrated score and a warning of class `gaze_calibration_fallback` is raised.
seed: Fold-assignment seed. `NULL` uses the engine-specific default. Folds depend on `seed` alone: the draw uses R's default generator and locale-independent ordering whatever `RNGkind()` and locale are in effect, and the caller's generator and random number stream are left unchanged.
fit_source_filter, eval_source_filter: Optional logical vectors or functions evaluated on `source_tab`. They permit a model to be trained on positive-control rows and evaluated on both signal and independently generated null rows without admitting the null rows to preprocessing or hyperparameter selection.
inner_split_on: Columns defining inner-fold groups. Defaults to `split_on`; specify a finer training-only unit when an outer fold holds out an entire session or replication.

Value

A `gaze_baseline_fit` with one long-format row per held-out trial and comparator.

Experimental

GazeWeave is experimental. Its interface, defaults and result columns may change in future releases; record the package version with any results.

Result columns

Every GazeWeave fit (`gaze_transport_cv()`, `gaze_replay_cv()`, `gaze_baseline_cv()`) has `results` with one row per held-out source row (per source row and method for the baseline court) holding the source identifiers and these columns:

`.cv_fold`: Outer fold that held the row out.
`gaze_info_bits`: `log2(posterior_true / prior_true)`, the primary endpoint. `NA` for uncalibrated baseline methods.
`posterior_true`, `prior_true`: Held-out probability of the true candidate, and its declared prior.
`odds_bits`, `log_loss`, `brier_score`: Other proper scores of the same posterior.
`template_rank`, `top1_credit`: Rank of the true candidate (1 is best) and credit for ranking it first, split over ties.
`candidate_count`: Number of candidates scored.
`candidates`: Per-row table with at least `candidate_key`, `log_score`, `prior`, `posterior` and `is_true`.

Each engine adds its own diagnostics (for example Transport's solver counts or Replay's background level), documented on its own page. Every fit also has `spec`, `keys`, `folds` and `provenance`; `provenance$engine` names the engine, and Transport and Replay record their estimator `provenance$revision`.

Examples

# Example data: two participants study and later recall six images; each
# recall roughly repeats the study path of the same image.
set.seed(1)
make_path <- function(anchor, jitter) {
  fixation_group(
    x = anchor[1] + c(0, 2, 4, 3) + rnorm(4, sd = jitter),
    y = anchor[2] + c(0, 1, -1, 2) + rnorm(4, sd = jitter),
    onset = c(0, 250, 500, 750), duration = c(200, 220, 180, 240)
  )
}
study <- expand.grid(participant = c("p1", "p2"), image = 1:6,
                     stringsAsFactors = FALSE)
anchors <- lapply(study$image, function(i) c(3 * i, 2 * (i %% 3)))
study$fixgroup <- lapply(anchors, make_path, jitter = 0.2)
recall <- study[c("participant", "image")]
recall$fixgroup <- lapply(anchors, make_path, jitter = 0.6)
# \donttest{
spec <- gaze_baseline_spec(screen = gaze_screen(20, 10, unit = "deg"),
                           density_sigmas = 1, methods = "density")
fit <- gaze_baseline_cv(study, recall, match_on = c("participant", "image"),
                        contrast_on = "participant", spec = spec,
                        n_folds = 2)
head(broom::tidy(fit, diagnostics = TRUE))
#> # A tibble: 6 × 21
#>   participant image method         court gaze_info_bits status reason calibrated
#>   <chr>       <int> <chr>          <chr>          <dbl> <chr>  <chr>  <lgl>     
#> 1 p1              1 density_raw    froz…             NA scored NA     FALSE     
#> 2 p1              1 density_sigma… froz…             NA scored NA     FALSE     
#> 3 p1              1 density_regis… froz…             NA scored NA     FALSE     
#> 4 p1              1 density_sigma… froz…             NA scored NA     FALSE     
#> 5 p1              1 density_sigma… supe…             NA skipp… Every… FALSE     
#> 6 p1              1 density_ridge… supe…             NA skipp… Every… FALSE     
#> # ℹ 13 more variables: odds_bits <dbl>, compatibility_bits <dbl>,
#> #   compatibility_odds_bits <dbl>, compatibility_probability_true <dbl>,
#> #   compatibility_log_loss <dbl>, posterior_true <dbl>, prior_true <dbl>,
#> #   log_loss <dbl>, brier_score <dbl>, template_rank <dbl>, top1_credit <dbl>,
#> #   candidate_count <int>, .cv_fold <int>
# }