Profiling Jupyter Notebook

Now let’s try to understand what are the steps needed to profile in Jupyter Notebook.

1. Installing the required libraries:

First we need to ensure that we have the necessary libraries installed in out system. As we need ‘line_profiler‘ for ‘%lprun’ and ‘cProfile‘ for ‘%prun‘. If it’s not installed already in our notebook, we can install them by using the following commad:

!pip install line_profiler

2. Load the Magic Commands

In our Jupyter Notebook, we then need to load the ‘%lprun’ and ‘%prun’ magic commands. We’ll now use the following commands to load them:

3. Profiling the Code

Now we’ll use ‘%lprun’ to profile specific lines of code and ‘%prun’ to profile entire functions as we have dicussed about this in the introduction. Let’s understand this with a small example:

Output:

Timer unit: 1e-09 s
Total time: 0.00622773 s
File: /tmp/ipykernel_601603/4220250964.py
Function: fib at line 3
Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
     3                                           def fib(n):
     4     21891    3171786.0    144.9     50.9      if n < 2:
     5     10946    1101316.0    100.6     17.7          return n
     6                                               else:
     7     10945    1954627.0    178.6     31.4          return fib(n-1) + fib(n-2)

When you run the above code, your output will be telling you about; Line, Hits, Time, Per Hit, % Time, and Line Contents. It’ll be somewhat looking like this:

Jupyter Notebook is a versatile tool or IDE widely used mostly by data scientists, researchers, and programmers for interactive computing and data analysis, dashboards, and visualizations. It offers a unique and rich set of features. Some topics are there it can perform such as: running the code in different languages, timing the code execution, debugging the code, and profiling. We’ll discuss the commands that work like magic for profiling in Jupyter Notebook. Magic Commands are the special commands in Jupyter Notebook that either start with the ‘% ‘ or ‘%% ‘ sign and perform different operations and tasks. Profiling helps in identifying bottlenecks, optimizing code, and improving overall performance. So, let’s explore the world of magic commands in Jupyter Notebook, specifically designed for profiling our Python code. Before we jump into it, we first need to understand some of the key concepts that we should know about profiling in Jupyter Notebook. So, what actually profiling is in Jupyter Notebook?