In this tutorial, we will introduce onnxruntime.SessionOptions() intra_op_num_threads.
For example:
import onnxruntime as rt
sess_options = rt.SessionOptions()
sess_options.intra_op_num_threads = 0
sess_options.execution_mode = rt.ExecutionMode.ORT_SEQUENTIAL
sess_options.graph_optimization_level = rt.GraphOptimizationLevel.ORT_ENABLE_ALL
sess_options.add_session_config_entry("session.intra_op.allow_spinning", "1")intra_op_num_threads: it controls the total number of INTRA threads to use to run the model.
Default: (not specified or 0). sess_options.intra_op_num_threads = 0
INTRA Threads Total = Number of physical CPU Cores. Leaving at default also enables some affinitization.
E.g. 6-core machine (with 12 HT logical processors) = 6 total INTRA threads
You can learn more by this tutorial:
https://onnxruntime.ai/docs/performance/tune-performance/threading.html#set-number-of-intra-op-threads
When sess_options.intra_op_num_threads = 1
If you plan to test the performance of onnx, you can set sess_options.intra_op_num_threads = 1. For example, if you plan to compute rtf metric.