Emacs IPython Notebook (EIN) provides a IPython Notebook client and integrated REPL (like SLIME) in Emacs. While EIN makes notebook editing very powerful by allowing you to use any Emacs features, it also expose IPython features such as code evaluation, object inspection and code completion to the Emacs side. These features can be accessed anywhere in Emacs and improve Python code editing and reading in Emacs.
Highlighted features:
Other notebook features:
Links:
| [1] | You need to setup ein:console-args properly |
| [2] | Use the command ein:connect-to-notebook-command. |
Contents
This is a quick and clean way to try EIN separately from your Emacs setting. If you want to try EIN but think preparing all the requirements is too much, try this!:
git clone git://github.com/tkf/emacs-ipython-notebook.git
cd emacs-ipython-notebook/
lisp/zeroein.el
This will launch a new Emacs instance.
You can use environment variable EMACS to control Emacs executable to use.:
EMACS=emacs-snapshot lisp/zeroein.el
The above command requires /bin/sh. If the above command does not work (e.g., you are using MS Windows), try the following command:
emacs -Q -l lisp/zeroein.el
Also, EIN heavily relies on standard Emacs libraries including EWOC, EIEIO and json.el. EIN is currently tested against Emacs 23.3 and 24.3. It is known to work in Emacs 23.2, 24.1 and 24.2.
| [3] | See Gotchas and caveats > python-mode.el. |
Warning
As EIN relies on many packages and it will not work properly with outdated versions, installing it using el-get or MELPA is highly recommended.
If you use developmental version of el-get, you can install it easily using it. Emacs IPython Notebook is registered as package ein. See el-get website for more information.
Note
If you get error “Cannot open load file: request” that means you have older el-get version. You can fix this problem by either (1) install request.el manually, (2) use the latest recipe, or (3) update el-get to its master.
You can get the latest recipe here:
See issue 98 for more information.
You can install EIN using package.el when MELPA package repository is added to its setting. See MELPA website for more information.
Put Emacs lisp ein*.el files and Python file ein.py in a directory defined in your load-path.
You should byte compile EIN, especially when using MuMaMo, otherwise editing large notebook will be very slow. You can use the following command to compile EIN. If you don’t specify all the optional packages, there will be compiler warning but that is OK as long as you don’t use that optional package.
emacs -Q -batch -L . \ # don't forget the dot!
-L PATH/TO/websocket/ \
-L PATH/TO/nxhtml/util/ \ # optional (for MuMaMo)
-L PATH/TO/auto-complete/ \ # optional
-L PATH/TO/popup/ \ # optional (for auto-complete)
-L PATH/TO/fuzzy/ \ # optional (for auto-complete)
-L PATH/TO/smartrep/ \ # optional
-L PATH/TO/rst-mode/ \ # optional
-f batch-byte-compile *.el
You can start notebook by M-x ein:notebooklist-open and enter the port or URL of the IPython notebook server.
Open notebook list buffer.
Ask server to create a new notebook and open it in a new buffer.
Choose notebook from all opened notebook list and open it. Notebook is specified by a string NBPATH whose format is “URL-OR-PORT/NOTEBOOK-NAME”.
When used in lisp, CALLBACK and CBARGS are passed to ein:notebook-open.
Login to IPython notebook server.
Open a notebook to try random thing. Notebook name is determined based on ein:junk-notebook-name-template.
When prefix argument is given, it asks URL or port to use.
Keymap for ein:notebooklist-mode.
Reload current Notebook list.
Reload current Notebook list.
The following keybinds are available in notebook buffers.
Execute code type CELL.
Turn on auto-execution flag of the cells in region or cell at point. When the prefix argument is given, turn off the flag instead.
To use autoexec feature, you need to turn on auto-execution mode in connected buffers, using the ein:connect-toggle-autoexec command.
Toggle the visibility of the output of the cell at point. This does not alter the actual data stored in the cell.
Show all cell output. When prefix is given, hide all cell output.
Clear output from the current cell at point. Do not clear input prompt when the prefix argument is given.
Clear output from all cells. Do not clear input prompts when the prefix argument is given.
Show code cell at point in shared-output buffer. It is useful when the output of the cell at point is truncated. See also ein:cell-max-num-outputs.
Kill (“cut”) the cell at point or cells in region. Note that the kill-ring for cells is not shared with the default kill-ring of Emacs (kill-ring for texts).
Copy the cell at point. (Put the current cell into the kill-ring.)
Switch the current worksheet with the previous one.
Switch the current worksheet with the previous one.
Insert a new worksheet before this worksheet and open it. See also ein:notebook-worksheet-insert-next.
(fn NOTEBOOK WS &optional (RENDER t) (SHOW (function switch-to-buffer)))
Copy the cell at point. (Put the current cell into the kill-ring.)
Insert (“paste”) the latest killed cell. Prefixes are act same as the normal yank command.
Insert cell above. Insert markdown cell instead of code cell when the prefix argument is given. See also: ein:worksheet-insert-cell-below.
Insert cell below. Insert markdown cell instead of code cell when the prefix argument is given.
When used as a lisp function, insert a cell of TYPE-OR-CELL just after PIVOT and return the new cell.
Toggle the cell type of the cell at point. Use ein:worksheet-change-cell-type to change the cell type directly.
Change the cell type of the current cell. Prompt will appear in the minibuffer.
When used in as a Lisp function, TYPE (string) should be chose from “code”, “markdown”, “raw” and “heading”. LEVEL is an integer used only when the TYPE is “heading”.
Split cell at current position. Newlines at the splitting point will be removed. This can be omitted by giving a prefix argument (C-u).
Merge previous cell into current cell. If prefix is given, merge current cell into next cell.
Show the help for the object at point using tooltip. When the prefix argument C-u is given, open the help in the pager buffer. You can explicitly specify the object by selecting it.
Start completion for the code at point.
(fn KERNEL &rest ARGS &key CALLBACKS &allow-other-keys)
Show full traceback in traceback viewer.
Send request to the server to restart kernel.
Interrupt the kernel. This is equivalent to do C-c in the console program.
Kill kernel and then kill notebook buffer. To close notebook without killing kernel, just close the buffer as usual.
Close NOTEBOOK and kill its buffer.
Open IPython console. To use this function, ein:console-security-dir and ein:console-args must be set properly. This function works best with the new python.el which is shipped with Emacs 24.2 or later. If you don’t have it, this function opens a “plain” command line interpreter (comint) buffer where you cannot use fancy stuff such as TAB completion. It should be possible to support python-mode.el. Patches are welcome!
Jump to the source code of the object at point. When the prefix argument C-u is given, open the source code in the other window. You can explicitly specify the object by selecting it.
Go back to the point where ein:pytools-jump-to-source-command is executed last time. When the prefix argument C-u is given, open the last point in the other window.
Open “scratch sheet”. Open a new one when prefix argument is given. Scratch sheet is almost identical to worksheet. However, EIN will not save the buffer. Use this buffer like of normal IPython console. Note that you can always copy cells into the normal worksheet to save result.
Change worksheet name (not notebook name).
Open previous or last worksheet. See also ein:notebook-worksheet-open-next-or-first and ein:notebook-worksheet-open-prev.
Open next or first worksheet.
Try to open the worksheet to the worksheet WS using the function ein:notebook-worksheet-open-next, open the first worksheet if not found.
SHOW is a function to be called with the worksheet buffer if given.
Insert a new worksheet after this worksheet and open it. See also ein:notebook-worksheet-insert-prev.
(fn NOTEBOOK WS &optional (RENDER t) (SHOW (function switch-to-buffer)))
Delete the current worksheet. When used as a lisp function, delete worksheet WS from NOTEBOOk.
Open 1-th worksheet.
Open 2-th worksheet.
Open 3-th worksheet.
Open 4-th worksheet.
Open 5-th worksheet.
Open 6-th worksheet.
Open 7-th worksheet.
Open 8-th worksheet.
Open the last worksheet.
Execute cell at point if it is a code cell and move to the next cell, or insert if none.
Jump to the source code of the object at point. When the prefix argument C-u is given, open the source code in the other window. You can explicitly specify the object by selecting it.
Go back to the point where ein:pytools-jump-to-source-command is executed last time. When the prefix argument C-u is given, open the last point in the other window.
Insert the previous input in the execution history. You can go back further in the history by repeating this command. Use ein:worksheet-next-input-history to go forward in the history.
Insert next input in the execution history. You can go forward further in the history by repeating this command. Use ein:worksheet-previous-input-history to go back in the history.
Execute cell at point if it is a code cell and insert a cell bellow.
Evaluate a piece of code. Prompt will appear asking the code to run. This is handy when you want to execute something quickly without making a cell. If the code outputs something, it will go to the shared output buffer. You can open the buffer by the command ein:shared-output-pop-to-buffer.
Save the notebook.
Rename current notebook and save it immediately.
NAME is any non-empty string that does not contain ‘/’ or ‘’.
Delete a cell. (WARNING: no undo!) This command has no key binding because there is no way to undo deletion. Use kill to play on the safe side.
If you really want use this command, you can do something like this (but be careful when using it!):
(define-key ein:notebook-mode-map "\C-c\C-d"
'ein:worksheet-delete-cell)
Rename the current notebook based on ein:junk-notebook-name-template and save it immediately.
Close all opened notebooks.
Instant cell execution minor mode. Code cell at point will be automatically executed after any change in its input area.
You can connect any buffer (typically buffer opening Python file) to opened notebook and use the kernel of the notebook to execute the code, inspect objects, auto-complete code, jump to the other source, etc. Once the buffer is connected to the notebook, minor mode ein:connect-mode is enabled and the following keybinds are available.
Run buffer using the %run magic command or eval whole buffer if the prefix C-u is given. Variable ein:connect-run-command sets the command to run. You can change the command and/or set the options. See also: ein:connect-run-buffer, ein:connect-eval-buffer.
Reload buffer using the command set by ein:connect-reload-command.
Show the help for the object at point using tooltip. When the prefix argument C-u is given, open the help in the pager buffer. You can explicitly specify the object by selecting it.
Start completion for the code at point.
(fn KERNEL &rest ARGS &key CALLBACKS &allow-other-keys)
Toggle auto-execution mode of the current connected buffer.
When auto-execution mode is on, cells in connected notebook will be automatically executed whenever run, eval or reload command [4] is called in this buffer.
| [4] | Namely, one of
|
Note that you need to set cells to run in the connecting buffer or no cell will be executed. Use the ein:worksheet-turn-on-autoexec command in notebook to change the cells to run.
Open IPython console. To use this function, ein:console-security-dir and ein:console-args must be set properly. This function works best with the new python.el which is shipped with Emacs 24.2 or later. If you don’t have it, this function opens a “plain” command line interpreter (comint) buffer where you cannot use fancy stuff such as TAB completion. It should be possible to support python-mode.el. Patches are welcome!
Show full traceback in traceback viewer.
Jump to the source code of the object at point. When the prefix argument C-u is given, open the source code in the other window. You can explicitly specify the object by selecting it.
Go back to the point where ein:pytools-jump-to-source-command is executed last time. When the prefix argument C-u is given, open the last point in the other window.
Open “scratch sheet”. Open a new one when prefix argument is given. Scratch sheet is almost identical to worksheet. However, EIN will not save the buffer. Use this buffer like of normal IPython console. Note that you can always copy cells into the normal worksheet to save result.
Evaluate a piece of code. Prompt will appear asking the code to run. This is handy when you want to execute something quickly without making a cell. If the code outputs something, it will go to the shared output buffer. You can open the buffer by the command ein:shared-output-pop-to-buffer.
Jump to the source code of the object at point. When the prefix argument C-u is given, open the source code in the other window. You can explicitly specify the object by selecting it.
Go back to the point where ein:pytools-jump-to-source-command is executed last time. When the prefix argument C-u is given, open the last point in the other window.
Other useful commands:
Connect to notebook. When the prefix argument is given, you can choose any notebook on your server including the ones not yet opened. Otherwise, already chose from already opened notebooks.
Evaluate the whole buffer. Note that this will run the code inside the if __name__ == "__main__": block.
Run buffer using %run. Ask for command if the prefix C-u is given. Variable ein:connect-run-command sets the default command.
Traceback in notebook buffer is not easy to understand. You can open Traceback viewer by the command ein:notebook-view-traceback. In the Traceback viewer, following keybinds are available.
Keymap for ein:traceback-mode.
These commands can be used in the notebook buffer and the connected buffer.
Do the doctest of the object at point.
Execute %whos magic command and popup the result.
Draw inheritance graph of the class at point. hierarchymagic extension is needed to be installed. You can explicitly specify the object by selecting it.
Search kernel execution history then insert the selected one.
Choose opened notebook using helm interface.
Search kernel execution history then insert the selected one.
Choose opened notebook using anything.el interface.
You can link to IPython notebook from org-mode files.
| [5] | See 1.3 Activation in org-mode manual. |
You can customize EIN using the Emacs customization UI by typing M-x customize-group RET ein RET. All the configurable variables are listed below.
Set to t to use preset auto-complete configuration. Use ein:use-auto-complete-superpack when you need more powerful auto completion.
Set to t to use preset a little bit hacky auto-complete configuration. When this option is enabled, cached omni completion is available.
Maximum number of cache to store.
Set to t to use preset smartrep configuration.
Warning
When used with MuMaMo (see ein:notebook-modes), keyboard macro which manipulates cell (add, remove, move, etc.) may start infinite loop (you need to stop it with C-g). Please be careful using this option if you are a heavy keyboard macro user. Using keyboard macro for other commands is fine.
Load development helper.
List of default url-or-port values. This will be used for completion. So put your IPython servers. You can connect to servers not in this list (but you will need to type every time).
Default URL or port. This should be your main IPython Notebook server.
Load notebook list but do not pop-up the notebook list buffer.
For example, if you want to load notebook list when Emacs starts, add this in the Emacs initialization file:
(add-to-hook 'after-init-hook 'ein:notebooklist-load)
or even this (if you want fast Emacs start-up):
;; load notebook list if Emacs is idle for 3 sec after start-up
(run-with-idle-timer 3 nil #'ein:notebooklist-load)
You should setup ein:url-or-port or ein:default-url-or-port in order to make this code work.
See also: ein:connect-to-default-notebook, ein:connect-default-notebook.
Configure undo in notebook buffers.
You need to reopen the notebook buffer to reflect the change of this value.
Notebook modes to use (in order of preference).
When the notebook is opened, mode in this value is checked one by one and the first usable mode is used.
Available modes:
Examples:
Use MuMaMo if it is installed. Otherwise, use plain mode. This is the old default setting:
(setq ein:notebook-modes '(ein:notebook-mumamo-mode ein:notebook-plain-mode))
Avoid using MuMaMo even when it is installed:
(setq ein:notebook-modes '(ein:notebook-plain-mode))
Use simple python-mode based notebook mode when MuMaMo is not installed:
(setq ein:notebook-modes '(ein:notebook-mumamo-mode ein:notebook-python-mode))
Whether EIN should ask before killing unsaved notebook buffer.
Query timeout for opening notebook. If you cannot open large notebook because of timeout error, try to increase this value. Setting this value to nil means to use global setting. For global setting and more information, see ein:query-timeout.
Query timeout for saving notebook. Similar to ein:notebook-querty-timeout-open, but for saving notebook. For global setting and more information, see ein:query-timeout.
Number of traceback stack to show. Hidden tracebacks are not discarded. You can always view them using the command ein:notebook-view-traceback.
String shown in the cell prompt when the auto-execution flag is on. See also ein:connect-aotoexec-lighter.
Junk notebook name template. This value is used from ein:notebooklist-new-scratch-notebook and ein:notebook-rename-to-scratch-command. This must be a format string which can be passed to format-time-string.
Delay before executing cell after change in second.
Start completion when inserting a dot. Note that ein:use-auto-complete (or ein:use-auto-complete-superpack) must be t to enable this option. This variable has effect on notebook buffers and connected buffers.
Bind helm-ein-kernel-history to this key in notebook mode.
Example:
(setq ein:helm-kernel-history-search-key "\M-r")
This key will be installed in the ein:notebook-mode-map.
Bind anything-ein-kernel-history to this key in notebook mode.
Example:
(setq ein:anything-kernel-history-search-key "\M-r")
This key will be installed in the ein:notebook-mode-map.
Automatically construct search pattern when non-nil.
This variable applies to both helm-ein-kernel-history and anything-ein-kernel-history.
Output types to be used in notebook. First output-type found in this list will be used. This variable can be a list or a function returning a list given DATA plist. See also ein:output-type-prefer-pretty-text-over-html.
Example: If you prefer HTML type over text type, you can set it as:
(setq ein:output-type-preference
'(emacs-lisp svg png jpeg html text latex javascript))
Note that html comes before text.
Variables let-bound while calling shr-insert-document.
To use default shr setting:
(setq ein:shr-env nil)
Draw boundaries for table (default):
(setq ein:shr-env
'((shr-table-horizontal-line ?-)
(shr-table-vertical-line ?|)
(shr-table-corner ?+)))
Security directory setting.
Following types are valid:
IPython executable used for console.
Example: "/user/bin/ipython". Types same as ein:console-security-dir are valid.
Additional argument when using console.
Warning
Space-separated string is obsolete now. Use a list of string as value now.
Setting to use IPython profile named “YOUR-IPYTHON-PROFILE”:
(setq ein:console-args '("--profile" "YOUR-IPYTHON-PROFILE"))
Together with ein:console-security-dir, you can open IPython console connecting to a remote kernel.:
(setq ein:console-args '("--ssh" "HOSTNAME"))
(setq ein:console-security-dir "PATH/TO/SECURITY/DIR")
You can setup ein:console-args per server basis using alist form:
(setq ein:console-args
'((8888 . '("--profile" "PROFILE"))
(8889 . '("--ssh" "HOSTNAME"))
(default . '("--profile" "default"))))
If you want to use more complex setting, you can set a function to it:
(setq ein:console-args
(lambda (url-or-port) '("--ssh" "HOSTNAME")))
See also: ein:console-security-dir.
%run magic command used for ein:connect-run-buffer. Types same as ein:console-security-dir are valid.
Setting for ein:connect-reload-buffer. Same as ein:connect-run-command.
Whether the buffer should be saved before ein:connect-run-buffer.
Set to t to connect to the notebook after jumping to a buffer.
String appended to the lighter of ein:connect-mode (ein:c) when auto-execution mode is on. When nil, use the same string as ein:cell-autoexec-prompt.
Notebook to be connect when ein:connect-to-default-notebook is called.
Example setting to connect to “My_Notebook” in the server at port 8888 when opening any buffer in python-mode:
(setq ein:connect-default-notebook "8888/My_Notebook")
(add-hook 'python-mode-hook 'ein:connect-to-default-notebook)
ein:connect-default-notebook can also be a function without any argument. This function must return a string (notebook path of the form “URL-OR-PORT/NOTEBOOK-NAME”).
As ein:connect-to-default-notebook requires notebook list to be loaded, consider using ein:notebooklist-load to load notebook list if you want to connect to notebook without manually opening notebook list.
Connect to the default notebook specified by ein:connect-default-notebook. Set this to python-mode-hook to automatically connect any python-mode buffer to the notebook.
Major Mode for Code Cell.
Major Mode for Text Cell.
Major Mode for HTML Cell.
Major Mode for Markdown Cell.
Major Mode for Raw Cell.
Major Mode for Heading Cell.
Fallback Major Mode.
Turn on workaround for mumamo-indent-line-function.
In code cell, hitting TAB or C-j at the end of input area causes error from MuMaMo. When this variable is non-nil, EIN patches mumamo-indent-line-function to workaround this problem. This workaround is on by default.
Note that python-mode’s indentation function has other problems with MuMaMo. For example, hitting TAB twice, which decreases the indentation level by one in normal Python buffer, causes similar error in code cell. The current workaround does not fix this problem.
Convert file paths between Emacs and Python process.
This value can take these form:
Here, the functions TO-PYTHON and FROM-PYTHON are defined as:
Use ein:tramp-create-filename-translator to easily generate the pair of TO-PYTHON and FROM-PYTHON.
Generate a pair of TO-PYTHON and FROM-PYTHON for ein:filename-translations.
Usage:
(setq ein:filename-translations
`((8888
. ,(ein:tramp-create-filename-translator "MY-HOSTNAME"))))
;; Equivalently:
(setq ein:filename-translations
(lambda (url-or-port)
(when (equal url-or-port 8888)
(ein:tramp-create-filename-translator "MY-HOSTNAME"))))
This setting assumes that the IPython server which can be connected using the port 8888 in localhost is actually running in the host named MY-HOSTNAME.
Adapted from slime-create-filename-translator.
Default query timeout for HTTP access in millisecond.
Setting this to nil means no timeout.
If you do the same operation before the timeout, old operation will be canceled (see also ein:query-singleton-ajax).
Note
This value exists because it looks like url-retrieve occasionally fails to finish (start?) querying. Timeout is used to let user notice that their operation is not finished. It also prevent opening a lot of useless process buffers. You will see them when closing Emacs if there is no timeout.
If you know how to fix the problem with url-retrieve, please let me know or send pull request at github! (Related bug report in Emacs bug tracker: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=11469)
Although EIN mostly works fine, there are some deficits I noticed but have not fixed yet. It seems that they originate from some upstream bugs so there is little I can do in EIN (but I’m not sure – it’s possible that I am misusing the libraries!).
If you know how to fix/workaround them, patches are very welcome.
While using EIN, probably most of the error messages are about server connections. It looks like the problem is in url-retrieve. But in those cases you don’t loose any notebook data and your IPython kernel is fine. You can just type the command again and it will go fine most of the time. For saving notebook, I implemented code to retry when there is an error comes from url-retrieve to make it even safer.
When using MuMaMo based notebook mode, you will notice that highlighting outside of the cell input is turned off while you are in the input area. It seems there is a bug in MuMaMo [6].
If you are using smartrep and MuMaMo together, see also the warning in ein:use-smartrep document.
| [6] | See the relevant bug report I posted: https://bugs.launchpad.net/nxhtml/+bug/1013794 |
In my environment, using python-mode.el without byte-compiling it in MuMaMo based notebook mode produces segfault.
Also, mumamo-idle-set-major-mode messages error (wrong-type-argument listp python-saved-check-command) time to time, making minibuffer bit noisy while editing notebook. See Tips to fix this problem.
By telling IPython a little bit about Emacs Lisp, you can execute Emacs Lisp from IPython, just like you can execute Javascript in the web client. See emacslisp.py for more details.
In [1]:
%run PATH/TO/emacslisp.py
In [2]:
EmacsLisp('(+ 1 2 3)')
Out [2]:
6
Please use M-x ein:dev-bug-report-template to write a bug report. It pops up a buffer containing some system information and instruction for bug report.
Sometime more information that the *Message* buffer provides is needed to debug.
Execute (ein:log-set-level 'debug) (e.g., M-: (ein:log-set-level 'debug) RET).
Then do some operation which cause the problem.
Go to the log buffer _*ein:log-all* (it starts with a space) and paste the whole buffer to the issue tracker.
Please enclose the log with three backquotes to make the snippet as a code block, like this:
```
[verbose] Start logging. @#<buffer *ein: 8888/NAME*>
[info] Notebook NAME is ready @#<buffer *ein: 8888/NAME*>
[info] Kernel started: 5e4f74d1-ce91-4e7e-9575-9646adea5172 @#<buffer *scratch*>
```
See also: GitHub Flavored Markdown - Introduction
If it is too long, you can use paste bin service such as gist.
websocket.el has its logging buffer. Sometime its useful to see its log. This is how to do that.
If you are interested in debugging EIN, you should start it with calling the command ein:dev-start-debug. This command sets debug-on-error to t and do some patching to debugger. This patching is required because printing EWOC objects freezes Emacs otherwise. It also changes log level to log everything the log buffer. You can reset the patch and log level with ein:dev-stop-debug.