The LLM Applications on the sample workspace is an example of how the RagaAI Testing Platform can help with Model Quality Checks to identify performance gaps and perform regression analysis.
The RagaAI Testing Platform is designed to add science to the art of detection AI issues, performing root cause analysis and providing actionable recommendations. This is done as an automated suite of tests on the platform.
A summary view of some sample tests for LLM Summarisation on the RagaAI Testing Platform
An overview of all tests for the sample project is available here -
**1. Failure Mode Analysis**
A summary view of the test runs for LLM Summarisation on RagaAI Testing Platform
**Goal** - Identify scenarios where the LLM model performs poorly (below a set threshold) on the test dataset.
**Methodology** - RagaAI consistently identifies groups where the model might be consistently failing.
**Insight** - In this case, we see that the model performs poorly on texts belonging to the cluster 1 where the average cosine similarity score of 0.29, significantly lower than the threshold of 0.5. Analysing the texts and summaries within Cluster 1 can help reveal specific features or content types that cause the model to struggle.
**Impact -** By understanding the specific scenarios where the model fails using this test, users can develop strategies to mitigate the risks of inaccurate summaries. This can involve data augmentation, model retraining with targeted data, or investigating potential biases in the training data.
For more details, please refer to the detailed [failure mode analysis documentation.](/ragaai-prism/test-inventory/large-language-model-llm/failure-mode-analysis.md)
---
# Agent Instructions: Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter:
```
GET https://docs.raga.ai/ragaai-prism/sandbox-guide/llm-summarization.md?ask=
```
The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.