Add support for MacOS Apple Metal “mps” backend

>>> classifier = pipeline("sentiment-analysis")
>>> classifier.device = torch.device("mps")
>>> classifier.model.to("mps")
>>> classifier("We are very sad to mps backend is not supporter in Transformers.")
[{'label': 'NEGATIVE', 'score': 0.5000945329666138}]

Unfortunately, as evidenced in the output, the PyTorch MPS backend is still very much broken. Even a lot of basic operations do not work correctly. For example:

>>> torch.arange(10, device="mps")
tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0], device="mps:0")
>>> torch.ones(10, device="mps").type(torch.int32)
tensor([1065353216, 1065353216, 1065353216, 1065353216, 1065353216, 1065353216,
        1065353216, 1065353216, 1065353216, 1065353216], device="mps:0",
       dtype=torch.int32)

So, it’s unlikely that you can use it yet, until the Torch maintainers shake out some bugs.

Read more here: Source link