如何解决为什么培训师在教程中进行培训时不报告评估指标?
我正在关注本教程以了解培训师 API。 https://huggingface.co/transformers/training.html
我复制的代码如下:
from datasets import load_dataset
import numpy as np
from datasets import load_metric
metric = load_metric("accuracy")
def compute_metrics(eval_pred):
logits,labels = eval_pred
predictions = np.argmax(logits,axis=-1)
return metric.compute(predictions=predictions,references=labels)
print('Download dataset ...')
raw_datasets = load_dataset("imdb")
from transformers import AutoTokenizer
print('Tokenize text ...')
tokenizer = AutoTokenizer.from_pretrained("bert-base-cased")
def tokenize_function(examples):
return tokenizer(examples["text"],padding="max_length",truncation=True)
tokenized_datasets = raw_datasets.map(tokenize_function,batched=True)
print('Prepare data ...')
small_train_dataset = tokenized_datasets["train"].shuffle(seed=42).select(range(500))
small_eval_dataset = tokenized_datasets["test"].shuffle(seed=42).select(range(500))
full_train_dataset = tokenized_datasets["train"]
full_eval_dataset = tokenized_datasets["test"]
print('Define model ...')
from transformers import AutoModelForSequenceClassification
model = AutoModelForSequenceClassification.from_pretrained("bert-base-cased",num_labels=2)
print('Define trainer ...')
from transformers import TrainingArguments,Trainer
training_args = TrainingArguments("test_trainer",evaluation_strategy="epoch")
trainer = Trainer(
model=model,args=training_args,train_dataset=small_train_dataset,eval_dataset=small_eval_dataset,compute_metrics=compute_metrics,)
print('Fine-tune train ...')
trainer.evaluate()
但是,它没有报告任何关于训练指标的信息,而是以下消息:
Download dataset ...
Reusing dataset imdb (/Users/congminmin/.cache/huggingface/datasets/imdb/plain_text/1.0.0/4ea52f2e58a08dbc12c2bd52d0d92b30b88c00230b4522801b3636782f625c5b)
Tokenize text ...
100%|██████████| 25/25 [00:06<00:00,4.01ba/s]
100%|██████████| 25/25 [00:06<00:00,3.99ba/s]
100%|██████████| 50/50 [00:13<00:00,3.73ba/s]
Prepare data ...
Define model ...
Some weights of the model checkpoint at bert-base-cased were not used when initializing BertForSequenceClassification: ['cls.seq_relationship.weight','cls.predictions.transform.LayerNorm.weight','cls.seq_relationship.bias','cls.predictions.transform.dense.bias','cls.predictions.bias','cls.predictions.decoder.weight','cls.predictions.transform.LayerNorm.bias','cls.predictions.transform.dense.weight']
- This IS expected if you are initializing BertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing BertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of BertForSequenceClassification were not initialized from the model checkpoint at bert-base-cased and are newly initialized: ['classifier.weight','classifier.bias']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Define trainer ...
Fine-tune train ...
100%|██████████| 63/63 [08:35<00:00,8.19s/it]
Process finished with exit code 0
教程不是更新了吗?我应该更改一些配置以报告指标吗?
解决方法
评估函数返回指标,它不打印它们。是否
metrics=trainer.evaluate()
print(metrics)
工作?此外,该消息说您正在使用基本 bert 模型,该模型未针对句子分类进行预训练,而是针对基本语言模型进行预训练。因此,它没有任务的初始化权重,应该进行训练
,你为什么要做 trainer.evaluate()
?这只是在验证集上运行验证。如果你想微调或训练,你需要做:
trainer.train()
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。