Ipython

TODO:

These docs are a mess, and highlight the wrong info. To be reorganized. 

IPython is an interactive python shell, with all of the niceties that you have come to expect from an actual shell.

• colours
• tab autocomplete
• auto-reload all before each run
• all of these features within a debugger
• readline support

ipython is a python module that is a CROSS-LANGUAGE interpreter. Config once, and use for python, haskell, and many others. Complete with tab completion, syntax highlighting etc. very cool.

Documentation

As of the move to IPython 5.0, the documentation is in the progress of being hosted on Read The Docs. IPython also incorporated some other projects, and their configuration is also independent of ipython's own.

http://python-prompt-toolkit.readthedocs.io/en/stable/		IPython5.0 readline replacement (see here for vi-style keys)
http://ipython.readthedocs.io/en/stable/		IPython5.0 documentation
https://ipython.org/ipython-doc/3/		Legacy IPython documentation
https://github.com/ipython/ipython/wiki/IPython-kernels-for-other-languages		Other Lang Kernels (C#, Haskell, Julia,...)

Configuration

~/.pythonrc.py		python init file
~/.ipython/profile_default/ipython_config.py		ipython config
~/.ipython/<profile>/startup/		scripts in this folder are run on startup

import IPython	
IPython.utils.path.get_ipython_dir()				## Find loaded profile

%load_ext autoreload										## Autoreload all modules before each run
%autoreload 2

Usage

cd /path/to/files										## ipython has basic shell capabilities
run script.py											## run a script (even if not on pythonpath)

vim editing mode

ipython --TerminalInteractiveShell.editing_mode=vi  ## use editing-mode vi without an ipython config

readable colours on windows

If using cmder, use XTERM colourscheme. You can set highlighting_style to any pygments style.

python -m IPython \
    --TerminalInteractiveShell.editing_mode=vi \
    --TerminalInteractiveShell.highlighting_style=monokai

Logging

IPython initiates the logger before your programs are run, so all of the settings that you normally have set using logging.basicConfig() are ignored. You can modify them by using logging.root, which accesses the root logger's options.

logging.root.level = logging.DEBUG

CLI

I'm not sure why, but ipython does not create manpages. Nor do they host their CLI documentation on the web. (I have no idea why not...).

In order to get help from ipython:

ipython --help-all		## generic commands
ipython help kernel		## information about specific subcommands

Kernel

There are kernels for other languages available to ipython.

ruby iruby

ijava

Connecting to (remote) Kernel Session

I hate working in windows. CMD is painful to use, and unfortunately cygwin is slow and frequently requires special instructions. This simplifies my life so much.

Simple (same machine)

sudo pip2 install ipython
sudo pip2 install ipykernel
sudo pip2 install jupyter


## ipython (server)
ipython kernel

## ipython (client)
ipython console --existing kernel-9048.json												## connect to win32 ipython kernel from cygwin

Remote Machine (or VM)

Before attempting this make absolutely sure that:

• both the host/server share a python version
• both the host/server share an ipython version

Therre are also 2x methods that can possibly work here:

• forwarding localhost ports using SSH (requires sshd on kernel-host)
• binding ipython kernel to the external ip (potential security vulnerability)

You can find more information about what the ipython kernel flags are doing from the json files they create:

Kernel JSON file Locations
C:\Users\will\AppData\Local\Roaming\jupyter\runtime\kernel-####.json		Windows
/home/vagrant/.local/share/jupyter/runtime/kernel-####.json		Linux

Method 1 (fwd loclahost ssh)

This method requires an ssh server to be running on the windows machine. It forwards just the configured ports to the remote-machine using SSH.

If you you are connecting to the host from a VM, it does not matter what type of network connection you are using (provided you can make an SSH connection). The default NAT connection type is fine.

#### kernel.json
{
  "stdin_port": 54925,
  "ip": "127.0.0.1",
  "control_port": 54928,
  "hb_port": 54939,
  "signature_scheme": "hmac-sha256",
  "key": "d225ffe6-14ba-4718-8be8-15ff4a883e0c",
  "kernel_name": "",
  "shell_port": 54922,
  "transport": "tcp",
  "iopub_port": 54931
}


## Host (windows)
ipython2 kernel -f kernel.json															## reuse existing config (no need to parse)


## Guest (Linux VirtualMachine)
ssh -p 8610 -L  55534:127.0.0.1:55534 -N -f will@wintermute						## forward port 55534 on wintermute to localport 55534
																									## (do this for all ports in kernel.json)

cp kernel.json     /home/vagrant/.local/share/jupyter/runtime/kernel.json	## copy the windows kernel.json file to the linux machine
jupyter console --existing kernel.json

Method 2 (bind external ip)

Exacty the same as the above method, except that you introduce a potential security vulnerability (un-authenticated remote shell access), and do not need an SSHD server, and no ports need forwarding. Surprisingly, I was able to make this connection using the default NAT connection in vagrant.

#### kernel.json
{
  "stdin_port": 54925,
  "ip": "192.168.1.129",
  "control_port": 54928,
  "hb_port": 54939,
  "signature_scheme": "hmac-sha256",
  "key": "d225ffe6-14ba-4718-8be8-15ff4a883e0c",
  "kernel_name": "",
  "shell_port": 54922,
  "transport": "tcp",
  "iopub_port": 54931
}

## Host (windows)
ipython2 kernel -f kernel.json															## reuse existing config (no need to parse)

## Guest (linux) 
cp kernel.json     /home/vagrant/.local/share/jupyter/runtime/kernel.json	## copy the windows kernel.json file to the linux machine
jupyter console --existing kernel.json

troubleshooting

There are some tools you can use to test this, and some things you should know before testing. Getting this up and running can be a little frustrating, because ipython really wants to use the loopback addr.

Things to know

• you cannot forward ports on 127.0.0.1 using VirtualBox (https://www.virtualbox.org/ticket/13971)
• vagrant will refuse to forward ports that are already in use
• ipython kernel -f file.json can read from current dir
• (ipython|jupytr) console --existing file.json always reads from ipython's kernel location. (see above)

tools

• nc -z 127.0.0.1 54925 (test if port is listening - windows tool available)
• netstat -an | grep -V LISTEN | grep -v "*:*" watch for new port-communications to open up.

Trials

In the end, the day I spent trying to get this to work boiled down to me not copying the kernel.json file to the proper location - but for future reference here are the ways that I tried to connect a windows host with a linux VM.

For more information about my various errors while getting this to work, see ipython remote kernel trials.

Misc Troubleshooting

## installing pyzmq in cygwin			
https://github.com/zeromq/pyzmq/issues/113

http://stackoverflow.com/questions/24004074/connecting-to-a-windows-ipython-kernel-from-linux
http://stackoverflow.com/questions/9977446/connecting-to-a-remote-ipython-instance		orig
https://github.com/ipython/ipython/wiki/Cookbook%3a-Connecting-to-a-remote-kernel-via-ssh		works from windows to linux?