When you are fine-tuning qwen llm model, you may get this error: RuntimeError: “triu_tril_cuda_template” not implemented for ‘BFloat16.
In this tutorial, we will introduce how to fix this error.
This error looks like:
Why does this error occur?
Because the version of torch is lower?
As to us, the torch version is 2.0.1
How to fix this error?
Update your torch version.
We will update to 2.3.0 using pip
pip install torch==2.3.0 --upgrade
Then, we find this error is fixed.