max_batch_size Int (None) The absolute maximum amount of prompts to process at one time. This value is automatically adjusted based on cache size. prompt_template String (None) Name of a jinja2 chat ...