In this tutorial, we will introduce onnxruntime.SessionOptions() intra_op_num_threads.
import onnxruntime as rt sess_options = rt.SessionOptions() sess_options.intra_op_num_threads = 0 sess_options.execution_mode = rt.ExecutionMode.ORT_SEQUENTIAL sess_options.graph_optimization_level = rt.GraphOptimizationLevel.ORT_ENABLE_ALL sess_options.add_session_config_entry("session.intra_op.allow_spinning", "1")
intra_op_num_threads: it controls the total number of INTRA threads to use to run the model.
Default: (not specified or 0). sess_options.intra_op_num_threads = 0
INTRA Threads Total = Number of physical CPU Cores. Leaving at default also enables some affinitization.
E.g. 6-core machine (with 12 HT logical processors) = 6 total INTRA threads
You can learn more by this tutorial:
When sess_options.intra_op_num_threads = 1
If you plan to test the performance of onnx, you can set sess_options.intra_op_num_threads = 1. For example, if you plan to compute rtf metric.