I fine tuned bert with MRPC task.experiment is present below.
tensorflow 1.11.0 # use this for training on CPU
tensorflow-gpu 1.11.0 # use this for traning on GPU
all these parameter is presented at
win10 + tensorflow 1.11.0 + CPU. eval_accuracy = 0.68
win10 + tensorflow-gpu 1.11.0 + GPU. eval_accuracy = 0.68
ubuntu + tensorflow 1.11.0 + CPU. eval_accuracy =0.84
ubuntu + tensorflow-gpu 1.11.0 + GPU. eval_accuracy = 0.68
obviously, only on ubuntu with CPU, the result is correct. so, we could only fine tuned bert on CPU in ubuntu, but it cost 2-3 hours per training. it only cost a quarter per training on GPU, but the result is wrong.
that’s is all my experiment.