Building an AI Powered Monitoring System

‍

Overview

We built an AI powered monitor for a stamping press at large automotive supplier.  We provided anomaly detection models for process control, downtime prediction models for predictive maintenance, and RCA (root cause analysis) chatbots to help make real time decisions and changes. When using the Praxis platform in the pilot, we found that the time to value from ingestion to building an AI powered monitoring system was less than 1 week.

Key Results

I. Control Limit Violations

^{Built anomaly detection models for all 13 sensors that have control limits with an average training time < 5 minutes.}

II. Downtime Prediction

^{Over a month, we sent a total of 8 alerts with 6 downtimes prevented for a total reduction of 28.5%.}

III. Max.E

^{Deployed a root cause analysis bot, Max.E, with the ability to retrieve and analyze data, contextualize insights from models and configure real time alerts for users.}

Control Limit Violations

Anomaly detection models are generated for each sensor we monitor on the machine.

There are two proprietary architectures used to train models for this objective.

I. Praxis AnomalyAR: An autoregressive encoder decoder architecture which detects anomalies based on a predicted probabilistic distribution

_{The AnomalyAR model detects anomalies by finding values that lie outside the predicted probabilistic distribution.}

‍II. Praxis AnomalyForest: Isolation forests adapted for time series data

_{The Praxis AnomalyForest model detects anomalies by finding outliers based on values, variance, and moving averages. The model is supported by a data preprocessing strategy which restricts batches at training and inference to data which all belongs to the same centerline value. Below is an example of the outputs of a centerline change detection model.}

RLHF for Anomaly Detection

In the Praxis platform, users can accept and reject anomalies to further fine tune the model. Our reinforcement learning loop is based on calculating the similarity of detected anomalies to the similarity of anomalies flagged as false positives by the user.

Downtime Prediction

Time to failure models are powerful tools to understand the probability of an event occurring as well as the largest drivers of this event occurring. The goal of a time to failure model is to understand past trends in data which cause an event. In this use case, our models are trained to predict the probability of a downtime. The inputs of the model are the sensors and detected anomaly events. The output is the predicted probability of a downtime occurring within one hour. This model requires examples of the output on which to train. In this case, our output variable is downtimes on the stamping press machine.

Users have the option of setting context window and prediction window parameters in this model. The context window defines the amount of data used to understand critical conditions. The prediction window defines the length of time we can predict a downtime within. A prediction window of one hour means we will predict the probability of a downtime occurring within one hour.

Synthetic Downtime Generation

If there are not many occurrences of downtimes, this can cause worse performance.

Users can set the parameter Synthetic Generation Factor to decide how many synthetically generated downtimes will be injected in the training dataset. A Synthetic Generation Factor of 2 will multiply the number of downtimes by 2.

Training

We train models with various architectures and choose the best model based on the highest accuracy and f1 score.

We have four options for model architectures

1. Praxis PredictiveTransformer: A transformer based model based on PatchTST architecture which breaks input time series data into patches
2. Praxis PredictiveNet: A LSTM based model
3. Praxis PredictiveGPT: Various foundational models such as MOMENT, TimeGPT, TimeFM and Praxis’ proprietary foundational model which is actively in development
4. Praxis PredictiveForest: Random forest based models adapted for time series data

Users can set a threshold parameter which identifies the probability at which we detect a downtime. This parameter defaults to 0.5.

Analysis

The Praxis platform provides a comprehensive feature analysis report for each predictive model. This helps us understand which variables were the most important when predicting a downtime. We pair model assets with advanced statistical analysis to calculate statistically significant warning ranges for important sensors.

Max.E

Max.E is Praxis’s proprietary manufacturing agent which can aid in all RCA related tasks and help users make conclusions much quicker than they would without a copilot. We can understand powerful relationships through the feature importance outputs from models. However, to complete a root cause analysis of a situation, we need to understand all the critical information about it including data in documents such as manuals and work orders. This is where LLMs can help.

We use an architecture for root cause analysis that is based on a combination of LLMs and SLMs (small language models). Small language models such as Llama 8B are less computationally expensive and can be fine-tuned to address specific tasks.

There are five main tasks for which we train SLMs.

I. Metadata: Sensor metadata such as measuring points and tag groups

II. Values: Measurements read by the sensors and basic trend analysis

III. Models: Retrieving and understanding information provided by anomaly detection, predictive and optimization models

IV. Statistical Calculations: Metrics such as standard deviation, averages and other key values as well as results of statistical tests

V. Documents: Retrieving and contextualizing information found in documents such as manuals

Each SLM is built to speak with a main agent which serves as the main node to executes queries from the user. Information for each SLM is stored in key value pairs allowing for retrieval of important information based on the context or time period referenced in the analysis. The main agent adapts chain of thought principles used in recent large language models to collect information from relevant SLMs to provide the user with accurate guidance and judgement. Further fine tuning of these models with instructions as well as examples improves the language model’s capability to understand time-based data.

‍