Customize your IPython shell in Docker

Most of the projects I work on involve several team members and it would be annoying if I imposed MY preferences on them, so I did not even bother looking for a way to do it.

Awhile back, there was a project where I knew I was likely to be the only person working on it. It also happened to be the kind of project where I knew I would be using the IPython shell heavily for debugging as well as a fair bit of day to day data management. So I started to look into how I could accomplish it in a Docker Compose environment.

I’ve established an easy to use pattern that allows you to customize IPython as much as you could possibly want, without imposing those customizations on your coworkers!

Setup your own IPython profile

If you were working outside of docker you can create a specific profile with ipython profile create <name> and go from there, but we want our default IPython profile INSIDE Docker to have our customizations.

All you need to do is inject the environment variable IPYTHONDIR and set it to a local folder that is visible to your Docker container, but preferably NOT committed in git.

Here is how I set things up:

1. I usually add ./attic/ to my .gitignore file so I can have a folder of stuff that I want to keep with my repo, but never push up to Github. Call it whatever you want, I just happen to use attic.

2. I configure Docker Compose to load a .env file where I set IPYTHONDIR=/code/attic/ipython/

Here are the exact example configs for you:

# ... random other things 
attic/

IPYTHONDIR=/code/attic/ipython/

services:
   # ... misc other services 
  
   # Django runserver for local development 
   # NOTE: `WORKDIR /code` is defined in our Dockerfile for this project
   web:
    build: .
    env_from:
      - .env
    command: bash -c "./manage.py runserver 0.0.0.0:8000"
    # This exposes the current directory on my laptop as `/code/` in the 
    # container which is why it can find `./attic/ipython`
    volumes:
      - .:/code

   # ... more misc other services 

Initially the directory will be empty, but after you launch IPython it will create ./attic/ipython/profile_default/ which will contain a few files and folders to it uses for history, startup scripts, and the like.

What CAN you customize about IPython?

Quite a bit honestly. I’ve been a vi keybindings person since the late 90s. At this point it’s as much muscle memory for me as driving a car. I use them everywhere I can, so yes you can set that preference in IPython as well. If you’re a vi person this will be high on your list.

But we’re just to getting started. The customization documentation goes into tons of depth, but here are some of the more common things I think you will find useful.

You can customize your config by creating and editing ./attic/ipython/profile_default/ipython_config.py, which feels most appropriate for things that feel like a configuration. You can set your editing mode with:

c.editing_mode = "vi"

Import those things you use frequently

Got a few dependencies from PyPI you use often? Or a half dozen bits from your own project you constantly find yourself importing? That’s annoying so let’s just import them automatically.

You can do this either in your ipython_config.py or inside a Python script in ./attic/ipython/profile_default/startup

If you use want to use ipython_config.py you need to define that like this:

c.InteractiveShellApp.exec_lines = [
    'import numpy',
    'import datetime'
]

If you’d prefer, like I do, to use “startup” scripts to do these sorts of things there is one small thing to keep in mind. These files are loaded in sort order, so I find it useful to prefix them with numbers to control the loading order in case that matters to your code.

So do something like:

• ./attic/ipython/profile_default/startup/10-first.py
• ./attic/ipython/profile_default/startup/20-second-thing.py
• ./attic/ipython/profile_default/startup/20-some-other-thing.py

Organize your files and code as you see fit, I just wanted to point out the fact order can and often does matter.

So if I had 10-my-imports.py it can be just a bit bundle of random imports:

import numpy 
import datetime 

from datetime import timedelta 

from my_app.utils import make_life_easier

Now anytime you launch IPython these are all imported for you.

If you’re a Django person and consider installing django-extensions you can then run ./manage.py shell_plus which will use IPython if it’s installed for you and import all of your app’s models, your Django settings, and more for you as well!

Load specific data automatically

These next couple of examples are pulled from a Django based EdTech product I work on frequently for one of our clients. The nature of this app involves a lot of switching between student and faculty users and moving an assignment from state A to B to C and then maybe back to B again.

As I’ve written about before, we’re big fans of having a devdata command. In this project, I have various scenarios for each step and permutation of these assignment states so I can quickly put my local system into that state to add or fix a feature.

These devdata commands always create a fresh Assignment, but with the same name. So one of my IPython startup scripts load it and the faculty user associated to that Assignment.

from core.management.commands.devdata import get_assignment

try:
  # Yes I know this is a horrible variable name, 
  # but hey this is MY customization :P
  a = get_assignment()
  print("Loaded devdata Assignment as variable a")

  faculty = a.section.faculty.first() 
  print("Loaded assignment Faculty User as variable faculty")

except Exception as e:
  print("Could not find devdata assignment!")

Sometimes that assignment doesn’t exist, so I just wrap everything in this try/except and print out some feedback so I know IF it’s been loaded or not to avoid confusion.

Build custom tools for yourself!

Automatic imports, setting config options and loading data is a great start, but the real power comes when you (or your favorite LLM) build small tools to that make your life easier.

I should do this more than I do, but recently I built a little tool for this same EdTech project that has improved my life considerably.

This project uses a bunch of UUIDs in the URLs for Assignments, Courses, Student IDs, etc. Pretty common stuff, but I need to use these UUIDs with various ORM commands frequently.

And apparently highlighting and copying UUIDs in my browser’s location bar IS NOT one of my skills. I find myself only getting 98% of the full UUID or capturing the leading or trailing slash in the URL. Yes, it’s a small thing but it’s annoying, wastes time, and sometimes even breaks my flow… so it’s got to go.

So I built, or rather I had Claude build, a URL parser for my specific situation. Using rich to make it pretty and some regexp to give me quicker, more accurate access to all of the UUIDs in a given URL.

Here is the code for inspiration and completeness sake, but feel free to skip over it. The important part is how it’s used and screenshots of how it looks which is next.

import re

from rich import print
from rich.console import Console
from rich.table import Table
from rich.text import Text

UUID_PATTERN = re.compile(
    r"[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}"
)


class URLFinder:
    """Class to make it easier to find IDs in URLs"""

    def __init__(self):
        self.ids = []
        self.matches = []
        self.url = ""

    def __call__(self, position: int):
        """Return a string of the UUID"""
        try:
            print(self.ids[position])
            return self.ids[position]
        except IndexError:
            print("ID in position {position} not found!")

    def find_uuids_with_positions(self, url: str) -> list[dict]:
        """
        Find all UUIDs in a URL and return their positions.
        """
        self.url = url

        matches = []
        ids = []
        for match in re.finditer(UUID_PATTERN, url):
            matches.append(
                {"uuid": match.group(), "start": match.start(), "end": match.end()}
            )
            ids.append(match.group())

        self.matches = matches
        self.ids = ids

        return matches

    def create_highlighted_url(self, url, uuid_match):
        """
        Create a Rich Text object with the specified UUID highlighted.
        """
        text = Text()

        # Add text before the UUID
        text.append(url[: uuid_match["start"]], style="white")

        # Add the highlighted UUID
        text.append(uuid_match["uuid"], style="bright_blue")

        # Add text after the UUID
        text.append(url[uuid_match["end"] :], style="white")

        return text

    def uuid_table(self):
        """Prints the UUIDs in a nice table"""
        console = Console()

        # Create the table
        table = Table(show_header=True, header_style="bold magenta")
        table.add_column("Position", style="cyan", width=12)
        table.add_column("UUID", style="green", width=36)
        table.add_column("URL", style="white", min_width=50)

        if not self.matches:
            # Add a row showing no UUIDs found
            table.add_row("-", "No UUIDs found", Text(self.url, style="dim white"))
        else:
            # Add a row for each UUID found
            for i, match in enumerate(self.matches):
                position = f"{i}"
                highlighted_url = self.create_highlighted_url(self.url, match)

                table.add_row(position, match["uuid"], highlighted_url)

        console.print(table)

    def parse(self, url: str):
        self.find_uuids_with_positions(url)
        self.uuid_table()


ids = URLFinder()

So how do I use this?

Take this example URL:

http://localhost:8000/assignments/800dbda0-a10f-46cc-8461-8a9c2227fed5/reports/league/eed1abf3-7e9b-4290-9b38-fb9286334cd6

We’ve got a UUID for the Assignment and one for the League, which is another concept in the app. Sometimes I need one or the other, or both.

With my pre-loaded URLFinder and instance named ids I can run:

In[1]: ids.parse("http://localhost:8000/assignments/800dbda0-a10f-46cc-8461-8a9c2227fed5/reports/league/eed1abf3-7e9b-4290-9b38-fb9286334cd6")

And I get a pretty table that highlights each UUID in the URL so I can quickly and visually find the ID I need and use that position. This doesn’t scroll off the screen on my big ass monitor at the office, but sadly does when on a laptop screen, but hopefully you get the idea.

Until I call ids.parse() again, the class keeps the IDs in the positions listed and I can use them however I need. The code overrides __call__ above so I can call the object with ids(x) and it prints the ID and then returns it in case I want to set it to a specific variable or as I did here use it in a Django ORM call.

Conclusion Challenge

Hopefully I’ve shown you how simple it is to customize the hell out of your IPython experience.

I’m sure you can think of a few imports you should be automatically loading. Maybe even some data.

But I challenge you to take the time to build tools like my URLFinder for those smallish but annoying situations you find yousrelf in multiple times per week.

A half dozen of these tools are NOT too many and maybe not even enough. So go make your work life a little easier!

Frank Wiles

Founder of REVSYS and former President of the Django Software Foundation . Expert in building, scaling and maintaining complex web applications. Want to reach out? Contact me here or use the social links below.