Alibi Detect is an open source Python library focused on outlier, adversarial and drift detection. The package aims to cover both online and offline detectors for tabular data, text, images and time series. Both TensorFlow and PyTorch backends are supported for drift detection.
For more background on the importance of monitoring outliers and distributions in a production setting, check out this talk from the Challenges in Deploying and Monitoring Machine Learning Systems ICML 2020 workshop, based on the paper Monitoring and explainability of models in production and referencing Alibi Detect.
For a thorough introduction to drift detection, check out Drift Detection: An Introduction with Seldon. The talk covers what drift is and why it pays to detect it, the different types of drift, how it can be detected in a principled manner and also describes the anatomy of a drift detector.
- Outlier, adversarial and drift detection on CIFAR10
- Mahalanobis outlier detection on KDD Cup ‘99 dataset
- Isolation Forest outlier detection on KDD Cup ‘99 dataset
- VAE outlier detection on KDD Cup ‘99 dataset
- VAE outlier detection on CIFAR10
- AE outlier detection on CIFAR10
- AEGMM and VAEGMM outlier detection on KDD Cup ‘99 dataset
- Likelihood Ratio Outlier Detection with PixelCNN++
- Likelihood Ratio Outlier Detection on Genomic Sequences
- Time-series outlier detection using Prophet on weather data
- Time series outlier detection with Spectral Residuals on synthetic data
- Time series outlier detection with Seq2Seq models on synthetic data
- Seq2Seq time series outlier detection on ECG data
- Adversarial AE detection and correction on CIFAR-10
- Model Distillation drift detector on CIFAR-10
- Categorical and mixed type data drift detection on income prediction
- Kolmogorov-Smirnov data drift detector on CIFAR-10
- Maximum Mean Discrepancy drift detector on CIFAR-10
- Drift detection on Amazon reviews
- Text drift detection on IMDB movie reviews
- Classifier drift detector on CIFAR-10
- Model uncertainty based drift detection on CIFAR-10 and Wine-Quality datasets
- Online Drift Detection on the Wine Quality Dataset
- Drift detection on molecular graphs