Five basic functions dealing with tensors in pytorch

Every beginner of deep learning should know these five basic functions of pytoch.

It is one of the most sought after skills for recruiters to build neural networks in an accurate and effective way. Pytorch is a python library mainly used for deep learning. One of the most basic and important parts of pytorch is to create tensors, which are numbers, vectors, matrices, or any n-dimensional array. In order to reduce the computational speed, explicit loops must be avoided when constructing neural networks. We can use vectorization to avoid such loops. When constructing neural networks, the ability to calculate matrix operations quickly enough is very important.

“Why not use the numpy library?”

For deep learning, we need to calculate the derivatives of model parameters. Pytorch provides the ability to track derivatives during back propagation, but numpy does not. This is called “auto grad” in pytorch. Pytorch provides built-in support for fast execution using GPU. This is critical in the training model. Since numpy lacks the ability to transfer its calculations to the GPU, the training time of the model will eventually become very large.

All deep learning projects using pytorch start with the creation of tensors. Let’s look at some of the functions that must be known, which are the backbone of any deep learning project involving building neural networks.

  • torch.tensor()
  • torch.sum()
  • torch.index_select()
  • torch.stack()

After installing pytoch, you can directly import the following from the code:

# Import torch and other required modules
import torch


First, we define an auxiliary function, describe (x), which will summarize various properties of tensor x, such as tensor type, tensor dimension and tensor content.

# Helper function
def describe(x):
  print("Type: {}".format(x.type()))
  print("Shape/size: {}".format(x.shape))
  print("Values: \n{}".format(x)

Using torch Tensor creates tensors in pytorch

Pytorch allows us to create tensors in many different ways using torch packages. One way to create a tensor is to initialize a random tensor by specifying its dimension

describe(torch.Tensor(2, 3))

Creating tensors declaratively using Python lists

We can also create tensors using Python lists. We just need to pass the list as a parameter to the function, and we have its tensor form.

x = torch.Tensor([[1, 2, 3],[4, 5, 6]])

Creating tensors using numpy arrays

We can also create pytorch tensors from the numpy array. The type of tensor is double tensor instead of the default float tensor. The data type corresponding to numpy is float64, as shown below.

import numpy as np
npy = np.random.rand(2, 3)

What can’t we do with tensors? Tensors must be real or complex and should not be strings or characters.

torch.tensor([[1, 2], [3, 4, 5]])

ValueError                                Traceback (most recent call last)
<ipython-input-5-28787d136593> in <module>
      1 # Example 3 - breaking (to illustrate when it breaks)
----> 2 torch.tensor([[1, 2], [3, 4, 5]])

ValueError: expected sequence of length 2 at dim 1 (got 3)

torch. Tensor () forms the core of any pytorch project, literally because it is a tensor.


This function returns the sum of all elements in the input tensor.

describe(torch.sum(x, dim=0,keepdims=True))

If you know numpy, you may have noticed that for 2D tensors, we represent rows as dimension 0 and columns as dimension 1. torch. The sum () function allows us to calculate the sum of rows and columns.

We also pass true for keepdims to preserve the dimensions in the results. By defining dim = 1, we tell the function to collapse the array by column.


TypeError                                 Traceback (most recent call last)
<ipython-input-17-1617bf9e8a37> in <module>()
      1 # Example 3 - breaking (to illustrate when it breaks)
----> 2 torch.sum(npy,dim=1,keepdims=True)

TypeError: sum() received an invalid combination of arguments - got (numpy.ndarray, keepdims=bool, dim=int), but expected one of:
 * (Tensor input, *, torch.dtype dtype)
      didn't match because some of the keywords were incorrect: keepdims, dim
 * (Tensor input, tuple of ints dim, bool keepdim, *, torch.dtype dtype, Tensor out)
 * (Tensor input, tuple of names dim, bool keepdim, *, torch.dtype dtype, Tensor out)

This function is very useful in calculating metrics and loss functions.


This function returns a new tensor that indexes the input tensor along the dimension dim using the entry in the index (longtensor).

indices = torch.LongTensor([0, 2])
describe(torch.index_select(x, dim=1, index=indices))

We can pass the index as a tensor and define the axis as 1, which returns a new tensor size
rows_of_original_tensor x length_of_indices_tensor。

indices = torch.LongTensor([0, 0])
describe(torch.index_select(x, dim=0, index=indices))

We can pass the index as a tensor and define the axis as 0. The return size of this function is columns_ of_ original_ tensor x length_ of_ indices_ New tensor of tensor.

indices = torch.FloatTensor([0, 2])
describe(torch.index_select(x, dim=1, index=indices))

This function is useful in complex indexes such as discontinuous indexes of tensors.


This connects a series of tensors along the new dimension.

describe(torch.stack([x, x, x],dim = 0))

We can pass the tensor we want to connect as a tensor list with dim 0 to stack it along the line.

describe(torch.stack([x, x, x],dim = 1))

We can pass the tensor we want to join as a list of tensors, dim = 1, to stack it along the column.

y = torch.tensor([3,3])
describe(torch.stack([x, y, x],dim = 1))

RuntimeError                              Traceback (most recent call last)
<ipython-input-37-c97227f5da5c> in <module>()
      1 # Example 3 - breaking (to illustrate when it breaks)
      2 y = torch.tensor([3,3])
----> 3 describe(torch.stack([x, y, x],dim = 1))

RuntimeError: stack expects each tensor to be equal size, but got [2, 3] at entry 0 and [2] at entry 1

This function is the same as torch index_ Select () is very useful in combination to flatten a matrix.

This function performs matrix multiplication of a matrix.

mat1 =torch.randn(3,2)
describe(, mat1))

By simply passing the matrix as a parameter, we can easily perform matrix multiplication, which will produce a new tensor as the product of the two matrices.

mat1 = np.random.randn(3,2)
mat1 = torch.from_numpy(mat1).to(torch.float32)
describe(, mat1))

In the above example, we define a numpy array and convert it to a tensor of type float32. Now we can successfully perform matrix multiplication on tensors. The data types of the two tensors must match in order to operate successfully.

mat1 =torch.randn(2,3)
describe(, mat1))

RuntimeError                              Traceback (most recent call last)
<ipython-input-62-18e7760efd23> in <module>()
      1 # Example 3 - breaking (to illustrate when it breaks)
      2 mat1 =torch.randn(2,3)
----> 3 describe(, mat1))

RuntimeError: mat1 and mat2 shapes cannot be multiplied (2x3 and 2x3)

In order to perform a successful matrix multiplication, the columns of matrix 1 and the rows of matrix 2 must match. torch. The mm() function follows the basic rules of matrix multiplication. Even if the matrix is in the same order, it will not be automatically multiplied by the transpose of another matrix, and you must define it manually.

In order to calculate the derivative in back propagation, it is necessary to be able to perform matrix multiplication effectively, which is torch Where mm () appears.


This concludes our study of the five basic pytorch functions. From basic tensor creation to advanced and little-known functions with specific use cases, such as torch index_ Select (), pytorch provides many such functions to make the work of data science enthusiasts easier.

Author: inshal Khan

Finally, if you are interested in participating in the kaggle competition, please write to me personally and invite you to join the kaggle competition exchange group

Read more here: Source link