Dynamic Tracing

Dynamic Tracing: Automating Dataset Creation

Pain Point Addressed

Creating datasets manually for each run or configuration in an Agentic Application can be time-consuming and error-prone. Users often need to evaluate different model configurations, scenarios, or parameters, which requires creating distinct datasets for each variation. Traditionally, this process involves repeated manual intervention, making it inefficient and prone to inconsistencies.

Dynamic Tracing eliminates these inefficiencies by enabling users to automate dataset creation seamlessly. By dynamically setting dataset names within their code execution flow, users can automate the process, ensuring consistent and structured dataset organization.

---

Benefits of Dynamic Tracing

1. Automation: No need to manually create datasets for each model configuration or input variation.

2. Consistency: Ensures a standardized naming convention and dataset structure.

3. Scalability: Easily scales to multiple configurations or iterations.

4. Efficiency: Saves time by automating repetitive tasks.

---

How to Implement Dynamic Tracing

1. Initialize the Tracer

Start by initializing the Tracer object with the necessary configuration:

from catalyst_tracing import Tracer

# Initialize the tracer
tracer = Tracer(
    project_name="your_project_name",  # Replace with your project name
    dataset_name="your_dataset_name",  # Initial dataset name
    tracer_type="Agentic",             # Specify the tracer type
)

2. Use Dynamic Tracing to Automate Dataset Creation

Within your application's main loop or execution flow, dynamically set the dataset name for each run using the set_dataset_name method. This ensures that each run is associated with a unique dataset.

if __name__ == "__main__":
    # Automate dataset creation for multiple runs
    for i in range(3):  # Replace 3 with the number of desired iterations
        tracer.set_dataset_name(f"Dataset_{i}")  # Dynamically set dataset name
        tracer.start()
        main()          # Run your Agentic application logic
        tracer.stop()   

---

Example Use Case

Suppose you are testing an Agentic Application with three different model configurations. Instead of manually creating datasets for each configuration:

1. Define the Tracer at the global level.

2. Use a loop to iterate over the configurations.

3. Dynamically set the dataset name to reflect the configuration details.

4. Automatically generate and trace datasets for each iteration.

---