clearn — Continual Learning for PyTorch

clearn

Wrap once. Train forever.

Continual learning for PyTorch models. Prevent catastrophic forgetting with one line of code.

$ pip install clearn-ai

The Problem

Catastrophic forgetting kills your models

When you fine-tune a neural network on new data, it forgets everything it learned before. This is catastrophic forgetting. clearn fixes it.

Without clearn

Task 1 accuracy after training on 20 new tasks

95% →

↓ 87% forgotten

Standard SGD fine-tuning overwrites previously learned weights. By task 20, your model remembers nothing from task 1.

With clearn

Task 1 accuracy after training on 20 new tasks

95% →

94%

↓ 1% forgotten

clearn protects important weights while learning new tasks. One line of code. No architecture changes. No retraining from scratch.

Quickstart

Five lines to continual learning

Wrap your existing PyTorch model. Train on sequential tasks. Inspect what was retained. That's it.

quickstart.py

import clearn

# Wrap any PyTorch model with a continual learning strategy
model = clearn.wrap(your_model, strategy="ewc")

# Train on sequential tasks — forgetting is handled automatically
model.fit(task1_loader, optimizer, task_id="q1_fraud")
model.fit(task2_loader, optimizer, task_id="q2_fraud")

# Inspect what was retained
print(model.diff())

model.diff()

Git diff for model knowledge

The retention report shows exactly what your model remembers after each task. No other continual learning library ships this.

python — model.diff()

RetentionReport
├── q1_fraud:    94.2% retained  (-5.8%)
├── q2_fraud:    100.0% (current task)
├── plasticity:  0.87
├── stability:   0.94
└── recommendation: "stable — no action needed"

Strategies

Choose your forgetting prevention method

Every strategy is a one-line swap. Start with EWC, scale to DER++, or combine with LoRA for LLMs.

EWC

Regularization-based

Protects important weights via Fisher Information Matrix. Pure regularization — no data replay needed. Best for when you can't store past data.

clearn.wrap(model, strategy="ewc")

Online importance

Tracks each parameter's contribution to loss during training. Computes importance online without extra passes. Efficient alternative to EWC.

clearn.wrap(model, strategy="si")

DER++

Replay-based

Stores past examples in a memory buffer and matches their original logits during new training. Best general-purpose strategy for maximum retention.

clearn.wrap(model, strategy="der", buffer_size=500)

GEM

Constraint-based

Projects gradients to ensure new learning never increases loss on previous tasks. Hard constraint rather than soft penalty. Theoretically strongest guarantees.

clearn.wrap(model, strategy="gem")

LoRA-EWC

Parameter-efficient

Combines Low-Rank Adapters with EWC regularization. Designed for LLMs where full fine-tuning is impractical. Minimal memory overhead.

clearn.wrap(model, strategy="lora-ewc")

Benchmark

Measured on Split CIFAR-100

ResNet-18 trained on 20 sequential tasks (5 classes each). Task 1 accuracy measured after completing all 20 tasks.

Method	Task 1 Accuracy	Retention
Baseline SGD	~8%
clearn EWC	~82%
clearn DER++	~88%

Split CIFAR-100 · 20 sequential tasks · ResNet-18 · SGD lr=0.01

Integrations

Works with HuggingFace

Load pretrained models directly from the Hub. Train with continual learning. Push back when you're done.

huggingface_integration.py

import clearn

# Load a pretrained model with continual learning built in
model = clearn.from_pretrained("bert-base-uncased", strategy="lora-ewc")

# Train on your first task
model.fit(loader, optimizer, task_id="sentiment_v1")

# Push to Hub — checkpoint includes full task history
model.push_to_hub("your-name/my-model")