spd_learn.functional.numerical_config#

spd_learn.functional.numerical_config = NumericalConfig(eigval_clamp_scale=10000.0, eigval_log_scale=100.0, eigval_sqrt_scale=100.0, eigval_inv_sqrt_scale=1000.0, eigval_power_scale=1000.0, loewner_equal_scale=100.0, stiefel_init_scale=1000.0, division_safe_scale=100000.0, batchnorm_var_eps=1e-05, dropout_eps=1e-05, trace_norm_eps=1e-06, warn_on_clamp=True, strict_spd_check=False)#

Global configuration for numerical stability thresholds.

This class provides centralized control over numerical stability parameters used throughout the spd_learn library. All thresholds are specified as multipliers of the machine epsilon for the given dtype.

The actual threshold for a given dtype is computed as:

threshold = scale * torch.finfo(dtype).eps

For example, with eigval_clamp_scale=1e4 and dtype=torch.float32:

threshold = 1e4 * 1.19e-7 ≈ 1.19e-3
Parameters:

  • eigval_clamp_scale (float) – Scale factor for general eigenvalue clamping (ReEig layer). Default: 1e4 (yields ~1e-3 for float32).

  • eigval_log_scale (float) – Scale factor for eigenvalue clamping before log operation. Default: 1e2 (yields ~1e-5 for float32).

  • eigval_sqrt_scale (float) – Scale factor for eigenvalue clamping before sqrt operation. Default: 1e2 (yields ~1e-5 for float32).

  • eigval_inv_sqrt_scale (float) – Scale factor for eigenvalue clamping before inverse sqrt. Default: 1e3 (yields ~1e-4 for float32).

  • eigval_power_scale (float) – Scale factor for eigenvalue clamping before power operation. Default: 1e3 (yields ~1e-4 for float32).

  • loewner_equal_scale (float) – Scale factor for detecting equal eigenvalues in Loewner matrix. Default: 1e2 (yields ~1e-5 for float32).

  • batchnorm_var_eps (float) – Absolute epsilon for batch normalization scalar dispersion. This is a scalar value (mean squared Frobenius norm in tangent space), not a variance matrix. Default: 1e-5.

  • dropout_eps (float) – Absolute epsilon for dropout diagonal entries. Default: 1e-5.

  • trace_norm_eps (float) – Absolute epsilon for trace normalization. Default: 1e-6.

  • stiefel_init_scale (float) – Scale factor for Stiefel manifold initialization. Default: 1e3 (yields ~1e-4 for float32).

  • division_safe_scale (float) – Scale factor for safe division operations. Default: 1e5 (yields ~1e-2 for float32).

  • warn_on_clamp (bool) – Whether to emit warnings when eigenvalues are clamped. Default: True.

  • strict_spd_check (bool) – Whether to perform strict SPD checks (slower but safer). Default: False.

Notes

The default scale factors are chosen to balance numerical stability with accuracy [Higham, 2002]. More conservative (larger) values provide better stability but may reduce precision. Less conservative (smaller) values preserve more information but risk numerical issues.

For mixed-precision training (fp16), consider using larger scale factors as the machine epsilon for fp16 is much larger (~9.77e-4).