How to manipulate sparse tensors?

How to create Sparse Tensors in TensorFlow?

Handling Sparse Tensors: Distinguishing Zero vs Missing Values

You can manipulate sparse tensors in multiple ways using different methods. It is completely at you discretion what operation you want to do with your tensor. We will see different ways we can manipulate the sparse tensors in TensorFlow.

There is a package called tf.sparse package which consist of all the tools and methods that allow us to perform arithmetic manipulations, matrix multiplications, concatenation, and more. Let us discuss few of them.

Arithmetic Manipulations

Traditional arithmetic manipulations are not directly applicable on sparse tensors. Therefore, to add sparse tensors of same shape we use tf.sparse.add function. You can study the code give below. It shows two sparse tensors which are added and the result is stored in the result_sparse_tensor variable.

Python3

import tensorflow as tf
 
# Create two sparse tensors
sparse_tensor1 = tf.sparse.SparseTensor(indices=[[0, 3], [2, 4]],
                      values=[10, 20],
                      dense_shape=[5, 5])
sparse_tensor2 = tf.sparse.SparseTensor(indices=[[1, 2], [4, 2]],
                      values=[30, 40],
                      dense_shape=[5, 5])
 
# Add sparse tensors
result_sparse_tensor = tf.sparse.add(sparse_tensor1, sparse_tensor2)
 
# Print the result
print(result_sparse_tensor)

Output:

SparseTensor(indices=tf.Tensor(
[[0 3]
 [1 2]
 [2 4]
 [4 2]], shape=(4, 2), dtype=int64), values=tf.Tensor([10 30 20 40], shape=(4,), dtype=int32), dense_shape=tf.Tensor([5 5], shape=(2,), dtype=int64))

Sparse-Dense Matrix Multiplication

If you want to multiply a sparse tensor and a dense tensor you can use tf.sparse.sparse_dense_matmul method as given below.

Python3

import tensorflow as tf
 
# Define a sparse tensor
sparse_indices = tf.constant([[0, 2], [1, 0], [1, 2], [2, 1]], dtype=tf.int64)
sparse_values = tf.constant([1.0, 2.0, 3.0, 4.0], dtype=tf.float32)
sparse_shape = tf.constant([3, 3], dtype=tf.int64)
 
sparse_tensor = tf.sparse.SparseTensor(indices=sparse_indices, values=sparse_values, dense_shape=sparse_shape)
 
# Define a dense matrix
dense_matrix = tf.constant([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0], [7.0, 8.0, 9.0]], dtype=tf.float32)
 
# Perform sparse-dense matrix multiplication
result_dense_matrix = tf.sparse.sparse_dense_matmul(sparse_tensor, dense_matrix)
 
# Print the result
print("Sparse Tensor:")
print(sparse_tensor)
 
print("\nDense Matrix:")
print(dense_matrix)
 
print("\nResult of Sparse-Dense Matrix Multiplication:")
print(result_dense_matrix)

Output:

Sparse Tensor:
SparseTensor(indices=tf.Tensor(
[[0 2]
 [1 0]
 [1 2]
 [2 1]], shape=(4, 2), dtype=int64), values=tf.Tensor([1. 2. 3. 4.], shape=(4,), dtype=float32), dense_shape=tf.Tensor([3 3], shape=(2,), dtype=int64))
Dense Matrix:
tf.Tensor(
[[1. 2. 3.]
 [4. 5. 6.]
 [7. 8. 9.]], shape=(3, 3), dtype=float32)
Result of Sparse-Dense Matrix Multiplication:
tf.Tensor(
[[ 7.  8.  9.]
 [23. 28. 33.]
 [16. 20. 24.]], shape=(3, 3), dtype=float32)

Concatenation

In order to concatenate two sparse tensors, you can use tf.sparse.concat method. Check the example given below that concatenates two sparse tensors.

Python3

import tensorflow as tf
 
# Create two sparse tensors
sparse_indices1 = tf.constant([[0, 1], [1, 0], [2, 2]], dtype=tf.int64)
sparse_values1 = tf.constant([9, 2, 5], dtype=tf.int32)
sparse_shape1 = tf.constant([3, 3], dtype=tf.int64)
 
sparse_tensor1 = tf.sparse.SparseTensor(indices=sparse_indices1, values=sparse_values1, dense_shape=sparse_shape1)
 
sparse_indices2 = tf.constant([[3, 1], [4, 0]], dtype=tf.int64)
sparse_values2 = tf.constant([6, 8], dtype=tf.int32)
sparse_shape2 = tf.constant([5, 3], dtype=tf.int64)
 
sparse_tensor2 = tf.sparse.SparseTensor(indices=sparse_indices2, values=sparse_values2, dense_shape=sparse_shape2)
 
# Concatenate sparse tensors vertically
result_sparse_tensor = tf.sparse.concat(axis=0, sp_inputs=[sparse_tensor1, sparse_tensor2])
 
# Print the result
print("Sparse Tensor 1:")
print(sparse_tensor1)
 
print("\nSparse Tensor 2:")
print(sparse_tensor2)
 
print("\nResult of Concatenation:")
print(result_sparse_tensor)

Output:

Sparse Tensor 1:
SparseTensor(indices=tf.Tensor(
[[0 1]
 [1 0]
 [2 2]], shape=(3, 2), dtype=int64), values=tf.Tensor([9 2 5], shape=(3,), dtype=int32), dense_shape=tf.Tensor([3 3], shape=(2,), dtype=int64))
Sparse Tensor 2:
SparseTensor(indices=tf.Tensor(
[[3 1]
 [4 0]], shape=(2, 2), dtype=int64), values=tf.Tensor([6 8], shape=(2,), dtype=int32), dense_shape=tf.Tensor([5 3], shape=(2,), dtype=int64))
Result of Concatenation:
SparseTensor(indices=tf.Tensor(
[[0 1]
 [1 0]
 [2 2]
 [6 1]
 [7 0]], shape=(5, 2), dtype=int64), values=tf.Tensor([9 2 5 6 8], shape=(5,), dtype=int32), dense_shape=tf.Tensor([8 3], shape=(2,), dtype=int64))

Slicing

It is very easy to slice a sparse tensor. You simply need to use tf.sparse.slice function like give below in the code.

Python3

import tensorflow as tf
 
# Create a sparse tensor
sparse_indices = tf.constant([[0, 1], [1, 0], [2, 2], [2, 3]], dtype=tf.int64)
sparse_values = tf.constant([7, 14, 12, 11], dtype=tf.int32)
sparse_shape = tf.constant([3, 4], dtype=tf.int64)
 
sparse_tensor = tf.sparse.SparseTensor(indices=sparse_indices, values=sparse_values, dense_shape=sparse_shape)
 
# Print the original sparse tensor
print("Original Sparse Tensor:")
print(sparse_tensor)
 
# Slice the sparse tensor
sliced_sparse_tensor = tf.sparse.slice(sparse_tensor, start=[1, 0], size=[2, 3])
 
# Print the sliced sparse tensor
print("\nSliced Sparse Tensor:")
print(sliced_sparse_tensor)

Output:

Original Sparse Tensor:
SparseTensor(indices=tf.Tensor(
[[0 1]
 [1 0]
 [2 2]
 [2 3]], shape=(4, 2), dtype=int64), values=tf.Tensor([ 7 14 12 11], shape=(4,), dtype=int32), dense_shape=tf.Tensor([3 4], shape=(2,), dtype=int64))
Sliced Sparse Tensor:
SparseTensor(indices=tf.Tensor(
[[0 0]
 [1 2]], shape=(2, 2), dtype=int64), values=tf.Tensor([14 12], shape=(2,), dtype=int32), dense_shape=tf.Tensor([2 3], shape=(2,), dtype=int64))

Elementwise Operations

If you want to perform elementwise operations on nonzero values in sparse tensors (TensorFlow 2.4 and higher), you can use tf.sparse.map_values to do the same.

Python3

import tensorflow as tf
 
# Create a sparse tensor
sparse_indices = tf.constant([[0, 1], [1, 0], [2, 2], [2, 3]], dtype=tf.int64)
sparse_values = tf.constant([21,11,13, 8], dtype=tf.int32)
sparse_shape = tf.constant([3, 4], dtype=tf.int64)
 
sparse_tensor = tf.sparse.SparseTensor(indices=sparse_indices, values=sparse_values, dense_shape=sparse_shape)
 
# Print the original sparse tensor
print("Original Sparse Tensor:")
print(sparse_tensor)
 
# Define an elementwise operation function
def elementwise_operation(value):
    return value * 2
 
# Apply elementwise operation to nonzero values
result_sparse_tensor = tf.sparse.map_values(elementwise_operation, sparse_tensor)
 
# Print the result
print("\nResult of Elementwise Operation:")
print(result_sparse_tensor)

Output:

Original Sparse Tensor:
SparseTensor(indices=tf.Tensor(
[[0 1]
 [1 0]
 [2 2]
 [2 3]], shape=(4, 2), dtype=int64), values=tf.Tensor([21 11 13  8], shape=(4,), dtype=int32), dense_shape=tf.Tensor([3 4], shape=(2,), dtype=int64))
Result of Elementwise Operation:
SparseTensor(indices=tf.Tensor(
[[0 1]
 [1 0]
 [2 2]
 [2 3]], shape=(4, 2), dtype=int64), values=tf.Tensor([42 22 26 16], shape=(4,), dtype=int32), dense_shape=tf.Tensor([3 4], shape=(2,), dtype=int64))

Sparse tensors in Tensorflow

Imagine you are working with a massive dataset which is represented by multi-dimensional arrays called tensors. In simple terms, tensors are the building blocks of mathematical operations on the data. However, sometimes, tensors can have majority of values as zero. Such a tensor with a lot of zero values is called as sparse tensor.

Sparse tensors are mostly encountered in the fields of computer vision and natural language processing. These can be pretty overwhelming at times. Therefore, in this article we will be discussing various aspect related to sparse tensors. You will have the following concepts cleared when you read this article:

Table of Content

What are Sparse Tensors?
How to create Sparse Tensors in TensorFlow?
How to manipulate sparse tensors?
Handling Sparse Tensors: Distinguishing Zero vs Missing Values

Tags:

#AI-ML-DS With Python #Dev Scripter 2024 #Tensorflow #AI-ML-DS #Deep Learning #Dev Scripter

How to create Sparse Tensors in TensorFlow?

Handling Sparse Tensors: Distinguishing Zero vs Missing Values

How to manipulate sparse tensors?

Arithmetic Manipulations

Python3

Sparse-Dense Matrix Multiplication

Python3

Concatenation

Python3

Slicing

Python3

Elementwise Operations

Python3

Sparse tensors in Tensorflow

Similar Reads