Note
Go to the end to download the full example code.
Manifold Attention with MAtt#
This tutorial demonstrates how to use MAtt (Manifold Attention Network) for EEG classification. MAtt applies attention mechanisms on the SPD manifold to weight temporal segments by their discriminative importance.
Introduction#
MAtt [Pan et al., 2022] introduces attention mechanisms on the SPD manifold:
Patch-based Processing: Segments the signal into temporal patches
Covariance per Patch: Computes SPD matrices for each segment
Manifold Attention: Weights patches using Log-Euclidean distances
Aggregation: Combines weighted SPD matrices for classification
This allows the model to focus on the most discriminative time periods within each trial, improving classification and interpretability.
Setup and Imports#
import warnings
import matplotlib.pyplot as plt
import torch
from braindecode import EEGClassifier
from moabb.datasets import BNCI2014_001
from moabb.paradigms import MotorImagery
from sklearn.metrics import accuracy_score
from sklearn.preprocessing import LabelEncoder
from skorch.callbacks import EpochScoring, GradientNormClipping
from skorch.dataset import ValidSplit
from spd_learn.models import MAtt
warnings.filterwarnings("ignore")
Loading the Dataset#
dataset = BNCI2014_001()
paradigm = MotorImagery(n_classes=4)
print(f"Dataset: {dataset.code}")
print("Paradigm: 4-class motor imagery")
Choosing from all possible events
Dataset: BNCI2014-001
Paradigm: 4-class motor imagery
Creating the MAtt Model#
MAtt architecture:
Spatial Conv: Learns spatial filters
Temporal Conv: Extracts temporal features
Patch Embedding: Segments into n_patches temporal windows
Covariance + TraceNorm: SPD matrix per patch
AttentionManifold: Queries, keys, values on SPD manifold
ReEig + LogEig: Project to tangent space
Linear: Classification
The attention mechanism computes:
where \(d_{LE}\) is the Log-Euclidean distance.
n_chans = 22
n_outputs = 4
model = MAtt(
n_chans=n_chans,
n_outputs=n_outputs,
n_patches=6, # Number of temporal segments
temporal_out_channels=32, # Temporal feature dimension
temporal_kernel_size=25, # ~100ms at 250Hz
temporal_padding=12, # Keep time dimension
attention_in_features=32, # Input to attention (must match temporal_out_channels)
attention_out_features=24, # Output from attention
)
print("MAtt Architecture:")
print(model)
MAtt Architecture:
MAtt(
(add_extra_dim): Rearrange('batch chan time -> batch 1 chan time')
(spatial_filter): Conv2d(1, 22, kernel_size=(22, 1), stride=(1, 1))
(spatial_batch_norm): BatchNorm2d(22, eps=1e-05, momentum=0.1, affine=True, bias=True, track_running_stats=True)
(temporal_feature_extractor): Conv2d(22, 32, kernel_size=(1, 25), stride=(1, 1), padding=(0, 12))
(temporal_batch_norm): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, bias=True, track_running_stats=True)
(squeeze_channel): Rearrange('batch kernel 1 time -> batch kernel time')
(patch_cov_layer): Sequential(
(0): PatchEmbeddingLayer()
(1): CovLayer()
(2): TraceNorm()
)
(manifold_attention): AttentionManifold(
(q_trans): ParametrizedBiMap(
(parametrizations): ModuleDict(
(weight): ParametrizationList(
(0): _Orthogonal()
)
)
)
(k_trans): ParametrizedBiMap(
(parametrizations): ModuleDict(
(weight): ParametrizationList(
(0): _Orthogonal()
)
)
)
(v_trans): ParametrizedBiMap(
(parametrizations): ModuleDict(
(weight): ParametrizationList(
(0): _Orthogonal()
)
)
)
)
(re_eig): ReEig()
(tangent): LogEig()
(flatten): Flatten(start_dim=1, end_dim=-1)
(linear): Linear(in_features=1800, out_features=4, bias=True)
)
Training the Model#
subject_id = 1
batch_size = 32
max_epochs = 100
learning_rate = 1e-4 # Low learning rate for stable SPD learning
device = "cuda" if torch.cuda.is_available() else "cpu"
print(f"\nUsing device: {device}")
# Cache configuration
cache_config = dict(
save_raw=True,
save_epochs=True,
save_array=True,
use=True,
overwrite_raw=False,
overwrite_epochs=False,
overwrite_array=False,
)
# Load data
X, labels, meta = paradigm.get_data(
dataset=dataset, subjects=[subject_id], cache_config=cache_config
)
# Encode labels
le = LabelEncoder()
y = le.fit_transform(labels)
# Split by session
train_idx = meta.query("session == '0train'").index.to_numpy()
test_idx = meta.query("session == '1test'").index.to_numpy()
print(f"\nData shape: {X.shape}")
print(f"Training samples: {len(train_idx)}")
print(f"Test samples: {len(test_idx)}")
# Create classifier
# Note: SPD networks benefit from gradient clipping to prevent
# divergence during training on the Riemannian manifold.
clf = EEGClassifier(
model,
criterion=torch.nn.CrossEntropyLoss,
optimizer=torch.optim.Adam,
optimizer__lr=learning_rate,
train_split=ValidSplit(0.1, stratified=True, random_state=42),
batch_size=batch_size,
max_epochs=max_epochs,
callbacks=[
(
"train_acc",
EpochScoring(
"accuracy", lower_is_better=False, on_train=True, name="train_acc"
),
),
("gradient_clip", GradientNormClipping(gradient_clip_value=1.0)),
],
device=device,
verbose=1,
)
# Train
clf.fit(X[train_idx], y[train_idx])
# Evaluate
y_pred_train = clf.predict(X[train_idx])
y_pred_test = clf.predict(X[test_idx])
train_acc = accuracy_score(y[train_idx], y_pred_train)
test_acc = accuracy_score(y[test_idx], y_pred_test)
print(f"\n{'=' * 50}")
print(f"Results for Subject {subject_id}")
print(f"{'=' * 50}")
print(f"Train Accuracy: {train_acc * 100:.2f}%")
print(f"Test Accuracy: {test_acc * 100:.2f}%")
Using device: cpu
Data shape: (528, 22, 1001)
Training samples: 240
Test samples: 288
epoch train_acc train_loss valid_acc valid_loss dur
------- ----------- ------------ ----------- ------------ ------
1 0.1823 1.4130 0.3333 1.3765 1.8364
2 0.3646 1.3572 0.2917 1.3431 1.7292
3 0.3438 1.3349 0.4583 1.3215 1.7119
4 0.4792 1.2978 0.3750 1.3080 1.7123
5 0.4792 1.2778 0.4167 1.2904 1.7114
6 0.6198 1.2457 0.5833 1.2687 1.7124
7 0.6615 1.2247 0.6250 1.2532 1.7094
8 0.6771 1.2025 0.5417 1.2324 1.7124
9 0.6354 1.1692 0.5000 1.2162 1.7236
10 0.7708 1.1387 0.5417 1.1971 1.7252
11 0.6458 1.1264 0.5417 1.1815 1.7147
12 0.6875 1.1035 0.5833 1.1569 1.7199
13 0.7135 1.0997 0.5417 1.1545 1.7167
14 0.6302 1.0643 0.5000 1.1254 1.7264
15 0.7604 1.0256 0.6250 1.1141 1.7363
16 0.7812 1.0179 0.5833 1.0962 1.3429
17 0.7812 1.0006 0.5000 1.0752 1.6212
18 0.7656 0.9795 0.6667 1.0580 1.7695
19 0.8281 0.9521 0.7500 1.0440 1.6395
20 0.8073 0.9387 0.5000 1.0308 1.3474
21 0.7708 0.9288 0.6667 1.0141 1.2814
22 0.7917 0.9018 0.5833 1.0058 1.5314
23 0.8021 0.8833 0.6667 0.9954 1.3073
24 0.8438 0.8722 0.7083 0.9774 1.3294
25 0.8281 0.8655 0.7500 0.9703 1.2836
26 0.8438 0.8409 0.7500 0.9581 1.3081
27 0.8385 0.8221 0.6250 0.9420 1.3199
28 0.8281 0.8091 0.6667 0.9272 1.3150
29 0.8542 0.7969 0.7083 0.9195 1.3050
30 0.8854 0.7630 0.7083 0.9053 1.3410
31 0.8125 0.7802 0.6667 0.8976 1.3214
32 0.8750 0.7517 0.7917 0.8861 1.3083
33 0.8073 0.7444 0.6250 0.8921 1.3008
34 0.8333 0.7283 0.7500 0.8691 1.3436
35 0.9062 0.7115 0.7500 0.8652 1.2766
36 0.8594 0.7110 0.7083 0.8576 1.3200
37 0.8802 0.6956 0.7917 0.8457 1.3086
38 0.8750 0.6931 0.7917 0.8424 1.2797
39 0.9010 0.6685 0.7917 0.8317 1.3222
40 0.8750 0.6563 0.7500 0.8220 1.3302
41 0.8542 0.6427 0.7083 0.8108 1.2964
42 0.8646 0.6510 0.7083 0.8195 1.3113
43 0.8698 0.6314 0.7083 0.8157 1.3383
44 0.8542 0.6321 0.6667 0.8033 1.4101
45 0.8750 0.6148 0.7917 0.7846 1.7817
46 0.8802 0.6024 0.7500 0.7868 1.7502
47 0.8802 0.6089 0.7500 0.7801 1.7561
48 0.8958 0.5988 0.6667 0.7879 1.7551
49 0.8958 0.5831 0.7917 0.7657 1.6958
50 0.8854 0.5884 0.7083 0.7717 1.3227
51 0.9167 0.5683 0.7500 0.7699 1.3318
52 0.9115 0.5518 0.7083 0.7537 1.4628
53 0.9010 0.5493 0.7917 0.7474 1.7344
54 0.9010 0.5391 0.7083 0.7425 1.7076
55 0.9115 0.5540 0.7500 0.7491 1.7364
56 0.8906 0.5351 0.7917 0.7412 1.7437
57 0.8958 0.5361 0.7083 0.7357 1.7854
58 0.9323 0.5170 0.7500 0.7304 1.4609
59 0.8906 0.5087 0.7917 0.7313 1.3340
60 0.8958 0.5038 0.7917 0.7237 1.3459
61 0.9219 0.4977 0.7500 0.7214 1.4262
62 0.9062 0.5143 0.7083 0.7225 1.2904
63 0.9219 0.4886 0.7917 0.7072 1.3171
64 0.9271 0.4750 0.7917 0.7139 1.2977
65 0.9219 0.4824 0.7083 0.7249 1.3074
66 0.9115 0.4759 0.7917 0.6987 1.3662
67 0.9167 0.4805 0.7917 0.7027 1.3270
68 0.9323 0.4587 0.7083 0.7043 1.4115
69 0.9323 0.4506 0.7917 0.7032 1.7320
70 0.9375 0.4480 0.7083 0.6950 1.7812
71 0.9271 0.4481 0.7500 0.6901 1.9428
72 0.9375 0.4389 0.7500 0.6826 1.9415
73 0.9375 0.4182 0.7917 0.6758 1.9045
74 0.9531 0.4368 0.7500 0.6803 1.9185
75 0.9427 0.4200 0.7917 0.6741 1.8609
76 0.9062 0.4367 0.7917 0.6778 1.7720
77 0.9271 0.4246 0.7500 0.6749 1.7737
78 0.9479 0.4119 0.7083 0.6816 1.7928
79 0.9323 0.4072 0.7917 0.6685 1.7795
80 0.9479 0.3899 0.7500 0.6707 1.7779
81 0.9479 0.3926 0.7500 0.6732 1.7769
82 0.9375 0.3874 0.7917 0.6646 1.7759
83 0.9427 0.3937 0.7083 0.6638 1.7738
84 0.9479 0.3923 0.7500 0.6614 1.7811
85 0.9479 0.3769 0.7500 0.6599 1.7779
86 0.9427 0.3716 0.7500 0.6578 1.7855
87 0.9583 0.3730 0.7500 0.6498 1.8521
88 0.9583 0.3689 0.7917 0.6382 1.7913
89 0.9688 0.3616 0.7500 0.6583 1.7634
90 0.9427 0.3671 0.7500 0.6550 1.7853
91 0.9427 0.3626 0.7083 0.6467 1.8268
92 0.9479 0.3508 0.7083 0.6624 1.5766
93 0.9583 0.3502 0.7500 0.6418 1.4841
94 0.9688 0.3454 0.7917 0.6398 1.8455
95 0.9583 0.3370 0.7500 0.6444 1.7683
96 0.9583 0.3316 0.7500 0.6428 1.8484
97 0.9688 0.3255 0.7500 0.6325 2.0322
98 0.9688 0.3272 0.7917 0.6288 1.5719
99 0.9583 0.3286 0.7083 0.6479 1.6443
100 0.9792 0.3158 0.7500 0.6247 1.2109
==================================================
Results for Subject 1
==================================================
Train Accuracy: 95.83%
Test Accuracy: 76.39%
Visualizing Results#
fig, axes = plt.subplots(1, 2, figsize=(12, 4))
# Training history
history = clf.history
epochs = range(1, len(history) + 1)
ax1 = axes[0]
ax1.plot(epochs, history[:, "train_loss"], "b-", label="Train Loss", linewidth=2)
ax1.plot(epochs, history[:, "valid_loss"], "r--", label="Valid Loss", linewidth=2)
ax1.set_xlabel("Epoch", fontsize=12)
ax1.set_ylabel("Loss", fontsize=12)
ax1.set_title("Training and Validation Loss", fontsize=14)
ax1.legend(fontsize=10)
ax1.grid(True, alpha=0.3)
ax2 = axes[1]
ax2.plot(epochs, history[:, "train_acc"], "b-", label="Train Acc", linewidth=2)
ax2.plot(epochs, history[:, "valid_acc"], "r--", label="Valid Acc", linewidth=2)
ax2.set_xlabel("Epoch", fontsize=12)
ax2.set_ylabel("Accuracy", fontsize=12)
ax2.set_title("Training and Validation Accuracy", fontsize=14)
ax2.legend(fontsize=10)
ax2.grid(True, alpha=0.3)
ax2.set_ylim([0, 1])
plt.tight_layout()
plt.show()

Understanding Manifold Attention#
The attention mechanism in MAtt operates differently from standard attention:
Standard Attention (Euclidean):
Manifold Attention (Log-Euclidean):
This respects the Riemannian geometry of SPD matrices, computing meaningful distances on the manifold rather than in Euclidean space.
Summary#
In this tutorial, we demonstrated:
Creating a MAtt model with patch-based temporal segmentation
Training for motor imagery classification
Understanding the manifold attention mechanism
MAtt is particularly useful when:
Different time segments have varying discriminative power
You want interpretable attention weights
The data has complex temporal dynamics
Total running time of the script: (2 minutes 45.694 seconds)