术语

• NCM：nearest class mean，NCM represents each class as a prototype vector that is the average feature vector of all examples observed for the class so far.[iCaRL] 类中心

Deep Class-Incremental Learning: A Survey (opens new window)

Data-Centric

• Data Replay
  • Direct Replay
  • Generative Replay
• Data Regularization

Direct Replay

2022

2021

• Rainbow Memory: Continual Learning with a Memory of Diverse Samples

• Continual prototype evolution: Learning online from non-stationary data streams (opens new window)

• Memory-efficient incremental learning through feature adaptation (opens new window)

• Memory efficient class-incremental learning for image classification (opens new window)

2020

• Using hindsight to anchor past knowledge in continual learning (opens new window)

2019

• Gradient based sample selection for online continual learning

2018

• Selective experience replay for lifelong learning (opens new window)

• Riemannian walk for incremental learning: Understanding forgetting and intransigence (opens new window)

• Experience replay for continual learning (opens new window)

2017

• icarl: Incremental classifier and representation learning (opens new window) CVPR
• Prototypical networks for few-shot learning (opens new window)

Generative Replay

[Continual learning with deep generative replay]

[Exemplarsupported generative reproduction for class incremental learning]

[Overcoming catastrophic forgetting for continual learning via model adaptation]

[Learning to remember: A synaptic plasticity driven framework for continual learning]

Incremental learning using conditional adversarial networks

Data Regularization

GEM-[Gradient episodic memory for continual learning] AGEM-[Efficient lifelong learning with a-gem]

Layerwise optimization by gradient decomposition for continual learning CVPR,2021.

Training networks in null space of feature covariance for continual learning CVPR,2021.

Model-Centric

• Dynamic Networks
  • Neuron Expansion
  • Backbone Expansion
  • Prompt Expansion
• Parameter Regularization

Neuron Expansion

reinforcement learning[Reinforced continual learning]

Backbone Expansion

• Der: Dynamically expandable representation for class incremental learning (opens new window) CVPR,2021

• Foster:Feature boosting and compression for class-incremental learning (opens new window)

• A model or 603 exemplars: Towards memory-efficient class-incremental learning

Prompt Expansion

Vision Transformer (ViT) Dytox: Transformers for continual learning with dynamic token expansion CVPR,2022

Visual Prompt Learning (VPT)

Parameter Regularization

Algorithm-Centric

• Knowledge Distillation
  • Logit Distillation
  • Feature Distillation
  • Relational Distillation
• Model Rectify
  • Feature Rectify
  • Logit Rectify
  • Weight Rectify

Logit Distillation

2016

• Learning without forgetting (opens new window),ECCV

Feature Distillation

Relational Distillation

Feature Rectify

Logit Rectify

Weight Rectify

Unclassified

2020

Semantic Drift Compensation for Class-Incremental Learning CVPR