spd_learn.functional.NumericalConfig#

class spd_learn.functional.NumericalConfig(eigval_clamp_scale: float = 10000.0, eigval_log_scale: float = 100.0, eigval_sqrt_scale: float = 100.0, eigval_inv_sqrt_scale: float = 1000.0, eigval_power_scale: float = 1000.0, loewner_equal_scale: float = 100.0, stiefel_init_scale: float = 1000.0, division_safe_scale: float = 100000.0, batchnorm_var_eps: float = 1e-05, dropout_eps: float = 1e-05, trace_norm_eps: float = 1e-06, warn_on_clamp: bool = True, strict_spd_check: bool = False, _threshold_cache: Dict[tuple, float]=<factory>)[source]#

Bases: object

Global configuration for numerical stability thresholds.

This class provides centralized control over numerical stability parameters used throughout the spd_learn library. All thresholds are specified as multipliers of the machine epsilon for the given dtype.

The actual threshold for a given dtype is computed as:

threshold = scale * torch.finfo(dtype).eps

For example, with eigval_clamp_scale=1e4 and dtype=torch.float32:

threshold = 1e4 * 1.19e-7 ≈ 1.19e-3
Parameters:

  • eigval_clamp_scale (float) – Scale factor for general eigenvalue clamping (ReEig layer). Default: 1e4 (yields ~1e-3 for float32).

  • eigval_log_scale (float) – Scale factor for eigenvalue clamping before log operation. Default: 1e2 (yields ~1e-5 for float32).

  • eigval_sqrt_scale (float) – Scale factor for eigenvalue clamping before sqrt operation. Default: 1e2 (yields ~1e-5 for float32).

  • eigval_inv_sqrt_scale (float) – Scale factor for eigenvalue clamping before inverse sqrt. Default: 1e3 (yields ~1e-4 for float32).

  • eigval_power_scale (float) – Scale factor for eigenvalue clamping before power operation. Default: 1e3 (yields ~1e-4 for float32).

  • loewner_equal_scale (float) – Scale factor for detecting equal eigenvalues in Loewner matrix. Default: 1e2 (yields ~1e-5 for float32).

  • batchnorm_var_eps (float) – Absolute epsilon for batch normalization scalar dispersion. This is a scalar value (mean squared Frobenius norm in tangent space), not a variance matrix. Default: 1e-5.

  • dropout_eps (float) – Absolute epsilon for dropout diagonal entries. Default: 1e-5.

  • trace_norm_eps (float) – Absolute epsilon for trace normalization. Default: 1e-6.

  • stiefel_init_scale (float) – Scale factor for Stiefel manifold initialization. Default: 1e3 (yields ~1e-4 for float32).

  • division_safe_scale (float) – Scale factor for safe division operations. Default: 1e5 (yields ~1e-2 for float32).

  • warn_on_clamp (bool) – Whether to emit warnings when eigenvalues are clamped. Default: True.

  • strict_spd_check (bool) – Whether to perform strict SPD checks (slower but safer). Default: False.

Notes

The default scale factors are chosen to balance numerical stability with accuracy [Higham, 2002]. More conservative (larger) values provide better stability but may reduce precision. Less conservative (smaller) values preserve more information but risk numerical issues.

For mixed-precision training (fp16), consider using larger scale factors as the machine epsilon for fp16 is much larger (~9.77e-4).

batchnorm_var_eps: float = 1e-05#
clear_cache() → None[source]#

Clear the threshold cache after configuration changes.

division_safe_scale: float = 100000.0#
dropout_eps: float = 1e-05#
eigval_clamp_scale: float = 10000.0#
eigval_inv_sqrt_scale: float = 1000.0#
eigval_log_scale: float = 100.0#
eigval_power_scale: float = 1000.0#
eigval_sqrt_scale: float = 100.0#
get_scale(name: Literal['eigval_clamp', 'eigval_log', 'eigval_sqrt', 'eigval_inv_sqrt', 'eigval_power', 'loewner_equal', 'batchnorm_var', 'dropout', 'trace_norm', 'stiefel_init', 'division_safe']) → float[source]#

Get the scale factor for a given threshold name.

Parameters:

name (ThresholdName) – The name of the threshold.

Returns:

The scale factor for the threshold.

Return type:

float

is_absolute(name: Literal['eigval_clamp', 'eigval_log', 'eigval_sqrt', 'eigval_inv_sqrt', 'eigval_power', 'loewner_equal', 'batchnorm_var', 'dropout', 'trace_norm', 'stiefel_init', 'division_safe']) → bool[source]#

Check if a threshold uses absolute values (not scaled by eps).

Parameters:

name (ThresholdName) – The name of the threshold.

Returns:

True if the threshold is absolute, False if scaled.

Return type:

bool

loewner_equal_scale: float = 100.0#
stiefel_init_scale: float = 1000.0#
strict_spd_check: bool = False#
summary(dtype: dtype = torch.float32) → str[source]#

Return formatted string showing all thresholds for a given dtype.

Parameters:

dtype (torch.dtype, default=torch.float32) – The dtype to compute thresholds for.

Returns:

Formatted summary of all threshold values.

Return type:

str

Examples

>>> from spd_learn.functional.numerical import numerical_config
>>> print(numerical_config.summary(torch.float32))
Numerical Configuration Summary (dtype=torch.float32)
==================================================
...
trace_norm_eps: float = 1e-06#
warn_on_clamp: bool = True#