JSON Verify

Tests the validity of the JSON file format in the model response

Required arguments: prompt, response

Interpretation of results: Score 1 represents valid JSON and 0 represents invalid

Code Example:

prompt = '''Write a sample JSON for a dataset in a retail shop'''
response= '''{
  ""order_id"": ""12345"",
  ""customer_name"": ""John Doe"",
  ""items"": [
    {
      ""item_id"": 101,
      ""name"": ""Product A"",
      ""quantity"": 2,
      ""price"": 10.99
    }
    {
      ""item_id"": 102,
      ""name"": ""Product B"",
      ""quantity"": 1,
      ""price"": 24.99
    },
    {
      ""item_id"": 103,
      ""name"": ""Product C"",
      ""quantity"": 3,
      ""price"": 8.50
    }
  ],
  ""total_price"": 75.46
}'''

evaluator.add_test(
    test_names=["json_verify_guardrail"],
    data={'prompt':prompt,
          'response': response,
    },

).run()

evaluator.print_results()