When we are using Bert model by transformers, we may get this error: TypeError: ‘BertTokenizer’ object is not callable. In this tutorial, we will introduce you how to fix it.
For example:
from transformers import AutoTokenizer,BertModel
model = BertModel.from_pretrained("./pretrained_models/bert-base-cased")
tokenizer = AutoTokenizer.from_pretrained("./pretrained_models/bert-base-cased")
print(tokenizer) #<transformers.tokenization_bert.BertTokenizer object at 0x000001C5CCCF68D0>
text = "Replace me by any text you'd like."
encoded_input = tokenizer(text, return_tensors='pt')
print(encoded_input)
output = model(**encoded_input)Run this code, we will see:
How to fix this TypeError?
We should update transformers version to 3.0+
Our version is __version__ = “2.4.1”
If you have updated, this error will be fixed.
For example:
pip uninstall transformers pip install -i https://mirrors.aliyun.com/pypi/simple/ transformers==3.3 --trusted-host mirrors.aliyun.com