Fix TypeError: ‘BertTokenizer’ object is not callable – Bert Tutorial

By | January 19, 2023

When we are using Bert model by transformers, we may get this error: TypeError: ‘BertTokenizer’ object is not callable. In this tutorial, we will introduce you how to fix it.

For example:

from transformers import AutoTokenizer,BertModel
model = BertModel.from_pretrained("./pretrained_models/bert-base-cased")
tokenizer = AutoTokenizer.from_pretrained("./pretrained_models/bert-base-cased")
print(tokenizer) #<transformers.tokenization_bert.BertTokenizer object at 0x000001C5CCCF68D0>

text = "Replace me by any text you'd like."
encoded_input = tokenizer(text, return_tensors='pt')
output = model(**encoded_input)

Run this code, we will see:

Fix TypeError: 'BertTokenizer' object is not callable - Bert Tutorial

How to fix this TypeError?

We should update transformers version to 3.0+

Our version is __version__ = “2.4.1”

If you have updated, this error will be fixed.

For example:

pip uninstall transformers
pip install -i transformers==3.3 --trusted-host