Jupyter Notebook Tools for Sphinx¶

Sphinx Setup¶

In the directory with your notebook files, run this command (assuming you have Sphinx installed already):

sphinx-quickstart

Answer the questions that appear on the screen. In case of doubt, just press the <Return> key to take the default values.

After that, there will be a few brand-new files in the current directory. You’ll have to make a few changes to the file named conf.py. You should at least check if those two variables contain the right things:

extensions = [
    'nbsphinx',
    'sphinx.ext.mathjax',
]
exclude_patterns = ['_build', '**.ipynb_checkpoints']

Once your conf.py is in place, edit the file named index.rst and add the file names of your notebooks (with or without the .ipynb extension) to the toctree directive.

Running Sphinx¶

To create the HTML pages, use this command:

sphinx-build <source-dir> <build-dir>

If you have many notebooks, you can do a parallel build by using the -j option:

sphinx-build <source-dir> <build-dir> -j<number-of-processes>

For example, if your source files are in the current directory and you have 4 CPU cores, you can run this:

sphinx-build . _build -j4

Afterwards, you can find the main HTML file in _build/index.html.

Subsequent builds will be faster, because only those source files which have changed will be re-built. To force re-building all source files, use the -E option.

To create LaTeX output, use:

sphinx-build <source-dir> <build-dir> -b latex

If you don’t know how to create a PDF file from the LaTeX output, you should have a look at Latexmk (see also this tutorial).

Sphinx can automatically check if the links you are using are still valid. Just invoke it like this:

sphinx-build <source-dir> <build-dir> -b linkcheck

Watching for Changes with sphinx-autobuild¶

If you think it’s tedious to run the command sphinx-build again and again when you make changes to your notebooks, you’ll be happy to hear that there is a way to avoid that: sphinx-autobuild!

It can be installed with

pip install sphinx-autobuild --user

You can start auto-building your files with

sphinx-autobuild <source-dir> <build-dir>

This will start a local webserver which will serve the generated HTML pages at http://localhost:8000/. Whenever you save changes in one of your notebooks, the appropriate HTML page(s) will be re-built and when finished, your browser view will be refreshed automatically. Neat!

You can also abuse this to auto-build the LaTeX output:

sphinx-autobuild <source-dir> <build-dir> -b latex

However, to auto-build the final PDF file, you’ll need an additional tool. Again, you can use latexmk for this (see above). Change to the build directory and run

latexmk -pdf -pvc

If your PDF viewer isn’t opened because of LaTeX build errors, you can use the option -f to force creating a PDF file.

Automatic Creation of HTML and PDF output on readthedocs.org¶

This is very easy!

1. Create an account on https://readthedocs.org/ and add your Github/Bitbucket repository (or any publicly available Git/Subversion/Mercurial/Bazaar repository).

2. Create a file named requirements.txt (or whatever name you wish) in your repository containing the required pip packages:

   nbsphinx
   ipykernel
   

3. In the “Advanced Settings” on readthedocs.org, specify your requirements.txt file (or however you called it) in the box labelled “Requirements file”. Kinda obvious, isn’t it?

4. Still in the “Advanced Settings”, make sure the right Python interpreter is chosen. This must be the same version (2.x or 3.x) as you were using in your notebooks!

5. Make sure that in the “Settings” of your Github repository, under “Webhooks & services”, “ReadTheDocs” is listed and activated in the “Services” section. If not, use “Add service”. There is probably a similar thing for Bitbucket.

6. Done!

After that, you only have to “push” to your repository and the HTML pages and the PDF file of your stuff are automagically created on readthedocs.org. Awesome!

You can even have different versions of your stuff, just use Git tags and branches and select in the readthedocs.org settings (under “Admin”, “Versions”) which of those should be created.

HTML Themes¶

The nbsphinx extension does not provide its own theme, you can use any of the available themes or create a custom one, if you feel like it.

The following links show how the nbsphinx input and output cells look like in different themes.

Equations¶

Equations can be formatted really nicely, either inline, like \(\text{e}^{i\pi} = -1\), or on a separate line, like

Note: Avoid leading and trailing spaces around math expressions, otherwise errors like the following will occur when Sphinx is running:

ERROR: Unknown interpreted text role "raw-latex".

See also the pandoc docs:

Anything between two $ characters will be treated as TeX math. The opening $ must have a non-space character immediately to its right, while the closing $ must have a non-space character immediately to its left, and must not be followed immediately by a digit.

Code¶

We can also write code with nice syntax highlighting:

print("Hello, world!")

Tables¶

Images¶

PNG file (local):

SVG file (local):

PNG file (remote):

SVG file (remote):

Links to Other Notebooks¶

Relative links to local notebooks can be used: a link to a notebook in a subdirectory, a link to an orphan notebook (latter won’t work in LaTeX output, because orphan pages are not included there).

This is how a link is created in Markdown:

[a link to a notebook in a subdirectory](subdir/another.ipynb)

Markdown also supports reference-style links: a reference-style link, another version of the same link.

These can be created with this syntax:

[a reference-style link][mylink]

[mylink]: subdir/another.ipynb

Links to sub-sections are also possible, e.g. this subsection.

This link was created with:

[this subsection](subdir/another.ipynb#A-Sub-Section)

You just have to remember to replace spaces with hyphens!

BTW, links to sections of the current notebook work, too, e.g. beginning of this section.

This can be done, as expected, like this:

[beginning of this section](#Links-to-Other-Notebooks)

Links to *.rst Files (and Other Sphinx Source Files)¶

Links to files whose extension is in the configuration value source_suffix, will be converted to links to the generated HTML/LaTeX pages. Example: A reStructuredText file.

This was created with:

[A reStructuredText file](rst.rst)

Links to sub-sections are not (yet?) possible.

Links to Local Files (HTML only)¶

Links to local files (other than Jupyter notebooks and other Sphinx source files) are also possible, e.g. requirements.txt.

This was simply created with:

[requirements.txt](requirements.txt)

The linked files are automatically copied to the HTML output directory. For LaTeX output, no link is created.

Special Display Formats¶

See IPython example notebook.

TODO: tables? e.g. Pandas DataFrame?

In [6]:

from IPython.display import display, Image, SVG, Math, YouTubeVideo

Local Image Files¶

In [7]:

i = Image(filename='images/notebook_icon.png')
i

Out[7]:

In [8]:

display(i)

For some reason this doesn’t work with Image(...):

In [9]:

SVG(filename='images/python_logo.svg')

Out[9]:

Image URLs¶

In [10]:

Image(url='https://www.python.org/static/img/python-logo-large.png')

Out[10]:

In [11]:

Image(url='https://www.python.org/static/img/python-logo-large.png', embed=True)

Out[11]:

In [12]:

Image(url='http://jupyter.org/assets/nav_logo.svg')

Out[12]:

In [13]:

Image(url='https://www.python.org/static/favicon.ico')

Out[13]:

In [14]:

Image(url='http://python.org/images/python-logo.gif')

Out[14]:

In [15]:

eq = Math(r"\int_{-\infty}^\infty f(x) \delta(x - x_0) dx = f(x_0)")
eq

Out[15]:

In [16]:

display(eq)

In [17]:

%%latex
\begin{equation}
\int_{-\infty}^\infty f(x) \delta(x - x_0) dx = f(x_0)
\end{equation}

In [18]:

YouTubeVideo('iV2ViNJFZC8')

Out[18]:

ANSI Colors¶

The standard output and standard error streams may contain ANSI escape sequences to change the text and background colors.

In [19]:

print('BEWARE: \x1b[1;33;41mugly colors\x1b[m!', file=sys.stderr, flush=True)
print('ABC\x1b[43mDEF\x1b[35mGHI\x1b[1mJKL\x1b[49mMNO\x1b[39mPQR\x1b[22mSTU')

BEWARE: ugly colors!

ABCDEFGHIJKLMNOPQRSTU

The following code showing the 8 basic ANSI colors is based on http://tldp.org/HOWTO/Bash-Prompt-HOWTO/x329.html. Each of the 8 colors has an “intense” variation, which is used for bold text.

In [20]:

text = ' XYZ '
formatstring = '\x1b[{}m' + text + '\x1b[m'

print(' ' * 6 + ' ' * len(text) +
      ''.join('{:^{}}'.format(bg, len(text)) for bg in range(40, 48)))
for fg in range(30, 38):
    for bold in False, True:
        fg_code = ('1;' if bold else '') + str(fg)
        print(' {:>4} '.format(fg_code) + formatstring.format(fg_code) +
              ''.join(formatstring.format(fg_code + ';' + str(bg))
                      for bg in range(40, 48)))

            40   41   42   43   44   45   46   47
   30  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ 
 1;30  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ 
   31  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ 
 1;31  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ 
   32  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ 
 1;32  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ 
   33  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ 
 1;33  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ 
   34  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ 
 1;34  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ 
   35  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ 
 1;35  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ 
   36  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ 
 1;36  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ 
   37  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ 
 1;37  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ  XYZ 

ANSI also supports a set of 256 indexed colors. The following code showing all of them is based on http://bitmote.com/index.php?post/2012/11/19/Using-ANSI-Color-Codes-to-Colorize-Your-Bash-Prompt-on-Linux.

In [21]:

formatstring = '\x1b[38;5;{0};48;5;{0}mX\x1b[1mX\x1b[m'

print('  + ' + ''.join('{:2}'.format(i) for i in range(36)))
print('  0 ' + ''.join(formatstring.format(i) for i in range(16)))
for i in range(7):
    i = i * 36 + 16
    print('{:3} '.format(i) + ''.join(formatstring.format(i + j)
                                      for j in range(36) if i + j < 256))

  +  0 1 2 3 4 5 6 7 8 91011121314151617181920212223242526272829303132333435
  0 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
 16 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
 52 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
 88 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
124 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
160 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
196 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
232 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

You can even use 24-bit RGB colors:

In [22]:

start = 255, 0, 0
end = 0, 0, 255
length = 79
out = []

for i in range(length):
    rgb = [start[c] + int(i * (end[c] - start[c]) / length) for c in range(3)]
    out.append('\x1b['
               '38;2;{rgb[2]};{rgb[1]};{rgb[0]};'
               '48;2;{rgb[0]};{rgb[1]};{rgb[2]}mX\x1b[m'.format(rgb=rgb))
print(''.join(out))

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

None¶

By default (if no cell format is selected), the cell content is included (without any conversion) in both the HTML and LaTeX output. This is typically not useful at all.

reST¶

Raw cells in “reST” format are interpreted as reStructuredText and parsed by Sphinx. The result is visible in both HTML and LaTeX output.

“I’m a raw cell in reST format.”

Markdown¶

Raw cells in “Markdown” format are interpreted as Markdown, and the result is included in both HTML and LaTeX output. Since the Jupyter Notebook also supports normal Markdown cells, this might not be useful at all.

“I’m a raw cell in Markdown format.”

HTML¶

Raw cells in “HTML” format are only visible in HTML output. This option might not be very useful, since raw HTML code is also allowed within normal Markdown cells.

“I’m a raw cell in HTML format.”

LaTeX¶

Raw cells in “LaTeX” format are only visible in LaTeX output.

Python¶

Raw cells in “Python” format are not visible at all (nor executed in any way).

Long-Running Cells¶

If you are doing some very time-consuming computations, you may go over the default timeout for a cell, which is 30 seconds. To include long-running cells, you have two choices:

1. Change the timeout for a cell either generally or on a per-notebook basis – see Long-Running Cells.
2. Execute the notebook beforehand and save the results, like it’s done in this example notebook:

In [1]:

import time

In [2]:

%time time.sleep(60 * 60)
6 * 7

CPU times: user 160 ms, sys: 56 ms, total: 216 ms
Wall time: 1h 1s

Out[2]:

42

Rare Libraries¶

You might have created results with a library that’s hard to install and therefore you have only managed to install it on one very old computer in the basement, so you probably cannot run this whenever you build your Sphinx docs.

In [3]:

from a_very_rare_library import calculate_the_answer

In [4]:

calculate_the_answer()

Out[4]:

42

Exceptions¶

If an exception is raised during the Sphinx build process, it is stopped (the build process, not the exception!). If you want to show to your audience how an exception looks like, you have two choices:

1. Allow errors – either generally or on a per-notebook basis – see Ignoring Errors.
2. Execute the notebook beforehand and save the results, like it’s done in this example notebook:

In [5]:

1 / 0

---------------------------------------------------------------------------
ZeroDivisionError                         Traceback (most recent call last)
<ipython-input-5-b710d87c980c> in <module>()
----> 1 1 / 0

ZeroDivisionError: division by zero

A Sub-Section¶

This is for testing inter-notebook links, see this section.

Links to Notebooks¶

Links to notebooks can be easily created: A Notebook in a Sub-Directory (the notebook title is used as link text). You can also use an alternative text.

The above links were created with:

:ref:`subdir/another.ipynb`
:ref:`an alternative text <subdir/another.ipynb>`

Links to subsections are also possible, e.g. A Sub-Section (the subsection title is used as link text) and alternative text.

These links were created with:

:ref:`subdir/another.ipynb#A-Sub-Section`
:ref:`alternative text <subdir/another.ipynb#A-Sub-Section>`

Sphinx Directives for Jupyter Notebook Cells¶

For comparison, this is a “normal” Sphinx code block using ipython3 syntax highlighting:

%file helloworld.py
#!/usr/bin/env python3
print('Hello, world!')

The nbsphinx extension provides custom directives to show notebook cells:

In [42]:

6 * 7

Out[42]:

42

This was created with

.. nbinput:: ipython3
    :execution-count: 42

    6 * 7

.. nboutput::
    :execution-count: 42

    42