Setting up jupyterhub with anaconda + sudospawn

Jupyter is a great system of "literary programming" in which you can write code in python or several other high-level languages, directly view the output of text or graphics.



To do collaborative work, to use this tool to teach, it is useful to have a system that allows, also through a web interface, to launch the Jupiter notepad server, and that the servers run in the space of the user who does login.





Jupyterhub is a python package that allows different users to a server and launch jupyter notebook system sessions.


By default, jupyterhub is configured to run as root. However, if we want to expose this service to a public network, we should run this service as a user with limited privileges. This is achieved through the sudo-package. This part of the guide  https://github.com/jupyterhub/jupyterhub/wiki/Using-sudo-to-run-JupyterHub-without-root-privileges

Steps:

1.  Install anaconda. It can be downloaded from  https://www.continuum.io/downloads. Notice that  jupyterhub requires python 3.

2. Install jupyter (with  anaconda, it comes out-the-box)

3. Install jupyterhub

$ conda install --channel https://conda.anaconda.org/conda-forge jupyterhub

4. setup a user jupyterhub, with a home folder containing
    * certifies
    * the datebase
    * a folder to share among all the users.

$ mkdir [JUPYTER_HOMEPATH] 
$ mkdir [JUPYTER_HOMEPATH]/notebooks
$ mkdir [JUPYTER_HOMEPATH]/certs
$ useradd  -d [JUPYTER_HOMEPATH]  -s ""  jupyterhub
$ chown -R jupyterhub.jupyterhub [JUPYTER_HOMEPATH]
$ cd [JUPYTER_HOMEPATH]/certs 
$ openssl req -x509 -nodes -days 365 -newkey rsa:1024 -keyout server.key -out server.crt

5. Add the user jupyterhub to the group shadow

$ usermod -a -G shadow jupyterhub




6. Install sudospawner. We do that throught the anaconda's package manager.

    $ conda install --channel https://conda.anaconda.org/conda-forge sudospawner

    $ ln -s [ANACONDA_PATH]/bin/sudospawner /usr/local/bin/

7. Add to the  sudoers file (by means of  the visudo command)

$ visudo

adding at the end of the file


Cmnd_Alias JUPYTER_CMD = /usr/local/bin/sudospawner
jupyterhub ALL=(%jupyterhub) NOPASSWD:JUPYTER_CMD 




Notice that it is important that the commands
* sudospawner
* configurable-http-proxy
* jupyterhub-singleuser
* node

 are inside /usr/local/bin and not on the  anaconda's bin folder, in order to sudo recognizes it as a valid command. To do that, we need to create the proper links from the [anaconda]/bin files to /usr/local/bin.




8. Add to the group jupyterhub all the users that should be able to login in jupyterhub.

$ usermod -a -G jupyterhub [user1] 
$ usermod -a -G jupyterhub [user2]

9. Create   /etc/jupyterhub/, generate the configuration file

$ mkdir /etc/jupyterhub

$ cd /etc/jupyterhub

$ [ANACONDA_PATH]/bin/jupyterhub --generate-config   

$ chown -R jupyterhub.jupyterhub /etc/jupyterhub

$ nano jupyterhub_config.py

and establish the following parameters:

c.JupyterHub.admin_access =  True
c.JupyterHub.cookie_secret_file = '[JUPYTER_HOMEPATH]/jupyterhub_cookie_secret'
c.JupyterHub.ip = '[server IP]'
c.JupyterHub.port = 8000
c.JupyterHub.hub_port = 8082
c.JupyterHub.ssl_cert = '[JUPYTER_HOMEPATH]/certs/server.crt'
c.JupyterHub.ssl_key = '[JUPYTER_HOMEPATH]/certs/server.key'
c.JupyterHub.spawner_class = 'sudospawner.SudoSpawner'

c.Spawner.notebook_dir = '~/jupyter-notebooks'c.Spawner.ip = '[server IP]'
c.Authenticator.admin_users = set({"[Admin username]"})
c.PAMAuthenticator.service = 'login

10. Create the service  file /etc/init.d/jupyterhub. The following file is an example that works on Ubuntu

#! /bin/sh
### BEGIN INIT INFO
# Provides:          jupyterhub
# Required-Start:    $remote_fs $syslog
# Required-Stop:     $remote_fs $syslog
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Start jupyterhub
# Description:       This file should be used to construct scripts to be
#                    placed in /etc/init.d.
### END INIT INFO

# Author: Alisue
#
# Please remove the "Author" lines above and replace them
# with your own name if you copy and modify this script.

# Do NOT "set -e"

# Source function library.
#For centos
#. /etc/rc.d/init.d/functions


# PATH should only include /usr/* if it runs after the mountnfs.sh script
DESC="Multi-user server for Jupyter notebooks"
NAME=jupyterhub
RUNAS_USER="jupyterhub"
DAEMON=/opt/anaconda3/bin/jupyterhub
WORKDIRECTORY=[JUPYTER_HOME]
DAEMON_ARGS=" -f /etc/jupyterhub/jupyterhub_config.py --log-level=DEBUG"
PIDFILE=/var/run/$NAME.pid
SCRIPTNAME=/etc/init.d/$NAME

# Exit if the package is not installed
[ -x "$DAEMON" ] || exit 0

# Read configuration variable file if it is present
[ -r /etc/default/$NAME ] && . /etc/default/$NAME

# Load the VERBOSE setting and other rcS variables
. /lib/init/vars.sh

# Define LSB log_* functions.
# Depend on lsb-base (>= 3.2-14) to ensure that this file is present
# and status_of_proc is working.
. /lib/lsb/init-functions

#
# Function that starts the daemon/service
#
do_start()
{
    # Return
    #   0 if daemon has been started
    #   1 if daemon was already running
    #   2 if daemon could not be started
    start-stop-daemon -v --start -d $WORKDIRECTORY -c $RUNAS_USER  --pidfile $PIDFILE --exec $DAEMON --test  #|| return 1
    RETVAL="$?"
    [ "$RETVAL" = 1 ] && return 1
    start-stop-daemon -v --start -d $WORKDIRECTORY --background -c $RUNAS_USER --make-pidfile --pidfile $PIDFILE \
    --exec $DAEMON --  $DAEMON_ARGS || return 2
    # Add code here, if necessary, that waits for the process to be ready
    # to handle requests from services started subsequently which depend
    # on this one.  As a last resort, sleep for some time.
    return 0
}

#
# Function that stops the daemon/service
#
do_stop()
{
    # Return
    #   0 if daemon has been stopped
    #   1 if daemon was already stopped
    #   2 if daemon could not be stopped
    #   other if a failure occurred
    start-stop-daemon -v --stop  --retry=TERM/30/KILL/5  --user $RUNAS_USER
    RETVAL="$?"
    [ "$RETVAL" = 2 ] && return 2
    # Wait for children to finish too if this is a daemon that forks
    # and if the daemon is only ever run from this initscript.
    # If the above conditions are not satisfied then add some other code
    # that waits for the process to drop all resources that could be
    # needed by services started subsequently.  A last resort is to
    # sleep for some time.
   
    start-stop-daemon -v --stop  --oknodo --retry=0/30/KILL/5 --user $RUNAS_USER
    [ "$?" = 2 ] && return 2
    # Many daemons don't delete their pidfiles when they exit.
    rm -f $PIDFILE
    return "$RETVAL"
}

#
# Function that sends a SIGHUP to the daemon/service
#
do_reload() {
    #
    # If the daemon can reload its configuration without
    # restarting (for example, when it is sent a SIGHUP),
    # then implement that here.
    #
     start-stop-daemon --stop --signal 1 --quiet --pidfile $PIDFILE --name $NAME
    do_stop  || return "$?"
    do_start || return "$?"
    return 0
}

case "$1" in
  start)
    [ "$VERBOSE" != no ] && log_daemon_msg "Starting $DESC" "$NAME"
    do_start
    case "$?" in
        0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
        2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
    esac
    return 0
    ;;
  stop)
    [ "$VERBOSE" != no ] && log_daemon_msg "Stopping $DESC" "$NAME"
    do_stop
    RET=$?
    case "RET" in
        0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
        2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
        esac
    ;;
  status)
    status_of_proc "$DAEMON" "$NAME" && exit 0 || exit $?
    ;;
  #reload|force-reload)
    #
    # If do_reload() is not implemented then leave this commented out
    # and leave 'force-reload' as an alias for 'restart'.
    #
    #log_daemon_msg "Reloading $DESC" "$NAME"
    #do_reload
    #log_end_msg $?
    #;;
  restart|force-reload)
    #
    # If the "reload" option is implemented then remove the
    # 'force-reload' alias
    #
    log_daemon_msg "Restarting $DESC" "$NAME"
    do_stop
    case "$?" in
          0|1)
        do_start
        case "$?" in
            0) log_end_msg 0 ;;
            1) log_end_msg 1 ;; # Old process is still running
            *) log_end_msg 1 ;; # Failed to start
            esac
        ;;
          *)
        # Failed to stop
        log_end_msg 1
        ;;
        esac
    ;;
  *)
    echo "Usage: $SCRIPTNAME {start|stop|status|restart|force-reload}"
    exit 3
    ;;
esac

11. Add  jupyterhub to the list of  services.

$ for A in  `echo 2 3 4 5`; do ln -s  /etc/init.d/jupyterhub "/etc/rc$A.d/S99jupyterhub"; done

$ for A in  `echo 0 6`; do ln -s  /etc/init.d/jupyterhub "/etc/rc$A.d/K01jupyterhub"; done

Quantum Information Toolkit patch for ALPS Project - exact diagonalization routines

The ALPS Project provides an state-of-the-art library for numerical simulations on strongly correlated quantum systems. Here, I present an extension for the library and the exact diagonalization routines (fulldiag and  sparsediag) which allows us to obtain different quantum correlation measures for the simulated model. Also, it allow us to recover the density matrix of any subsystem.
Results are stored both on xml and hdf5


Here you can download a pre-alpha release of the extension.




Quick resume


To define a subsystem called A, corresponding to the first two sites in the lattice, we add the line

SUBSYSTEM[A] ="sites(0,1)"

In order to recover the density matrix of the state associated to the subsystem A,

ENTANGLEMENT_MEASURE[Local state A]="state(A)"

For the diag routines, it generates at the end of the calculus a folder
"/localsystems" on the output hdf5 file.

On the subfolder "definitions" we can access to the list of sites which constitutes each each subsystem defined.

On the subfolder "states" we can access to each density matrices stored. They are stored as a list of pairs (weight,state). The density matrix can be recovered as

Sum(weight[i]    state state^{\dagger},i) +    (1-sum(weight[i],i)) Id/(dimension)
 

Configuring CUDA in Ubuntu 13.04

Some months ago I bought a new Desktop PC with the aim to start learning about CUDA programming. The new PC had an integrated INTEL video card, and a NVIDIA GeForce GT620. The idea was use the integrated video card for the xserver, leaving the GeForce dedicated to run simulations.

When I tried to install the driver for  the NV card, things started to go wrong, because the NV driver replace the mesa OpenGL  libraries with theirs. Because the xserver actually works on the Intel video card, which is incompatible with the proprietary openGL libraries, openGL stopped working and as a result, also Unity got broken.


After a couple of evenings trying to make it work, I found that the trick to make all be working  is, after the installation of the nvidia driver, reinstall the original  mesa glx libraries. Here is the right procedure:


* Uninstall all NV components on the system

   # sudo nvidia-uninstall
   # sudo apt-get purge nvidia*

   
* Download the last official drivers  from the NV web site: http://www.nvidia.com/object/linux-display-amd64-319.60-driver.html


* Install the required packages for the installer

   # sudo apt-get install linux-headers-`uname -r`

* The next steps requires the xserver stopped. So,   Go to a terminal (Ctrl + Alt + F1) and stop the xserver. For instance,

     # sudo stop lightdm


* Install driver

# cd path/where/you/save/drivers
# sudo chmod +x  NVIDIA-Linux-x86_64-319.60.run
# sudo ./NVIDIA-Linux-x86_64-319.60.run


* At this point, the driver is OK, but our xserver would show problems with the Open GL libraries.  However, the GPU is ready:

# sudo nvidia-smi

should show the status of the GPU.

* Now, to avoid the problem with the Open GL libraries, we (re) install  both our xserver and the mesa libraries:

# sudo apt-get  install --reinstall xserver-xorg-core  xserver-xorg-  libgl1-mesa-glx


* Reboot the system

Now, you can install the CUDA Toolkit, avoiding to reinstall the NV drivers...

Parity implementation in ALPS

I would like share a patch to the ALPS code which allows it to handle problems with parity-like constraints. Some typical examples of this kind of symmetry are transversal spin Hamiltonias (XY mode, Dzyaloshinskii Moriya), bosonic and fermionic pairing couplings,Jaynes-Cummings like models, etc, for which Sz and N quantum numbers are not conserved.

This kind of constraints has the form

P=\prod_{j} \exp(i \pi Q_j )

being Q_j a local parity quantum number (for instance, n_j in a bosonic/fermionic model, or Sz_i+S in a spin model). At the end of this post you will find a couple of references with more details about this symmetry in some models in which I have worked.

The utility of take implement this kind of symmetry is twofold: on the one side, it allows to reduce the size of the Hilbert space, allowing to carry out simulations that would otherwise be imposible. On the other hand, it improves the convergence to the true ground state of these systems, because the gap between the ground state  of both different parity sectors are often very small.

The current code just working now for the full and sparse diagonalization code, and I am working on an implementation to the dmrg code.


Here is an example for the parameter file:

MODEL_LIBRARY="models.xml"
MODEL="spin-parity";
LATTICE="chain lattice";
CONSERVED_QUANTUMNUMBERS="

Parity(ParityCharge)";
Jxy=-1.;
Jz=0.;
L=20;
local_S=.5;
{h=.46;}

An example for the definition of the local basis, the global basis and the hamiltonian was included in the default model.xml file.


References
Two soluble models of an antiferromagnetic chain,
E. Lieb, T. Schultz and D. Mattis,
Ann. Phys. (NY) 16 , 407 (1961).

Entanglement of finite cyclic chains at factorizing fields,
R. Rossignoli, N. Canosa, J.M.Matera,
Physical Review A 77 (2008) 052322.
Separability and entanglement in finite dimer-type chains in general transverse fields,
N. Canosa, R. Rossignoli, J.M. Matera,
Physical Review B 81 (2010) 054415.
Factorization and entanglement in general XYZ spin arrays in non-uniform transverse fields,
R. Rossignoli, N. Canosa, J.M. Matera,
Physical Review A 80 (2009) 062325.

AucTeX

Some years ago, I found a very useful tool for anyone who write articles in LaTeX:
AucTeX.

It is an emacs complement that allow us to see, inside the text editor, the equations that we write, as well as figures and tables in a "WYSIWIG" way, without the necesity of compiling the text each time.



Up to the Ubuntu version 10.10,  we could install just it writing in a command line

$ sudo apt-get install auctex

Unfortunately,  since the version 11.04, the package stopped working well, seemly, because of a problem with the library versions.

To fix it, simply download the source code of the package from .... , compile and install it.

To make it easier, here you can download a .deb package generated from these sources.   After installing the official version of auctex, simply install the package with the corrections:

$ sudo apt-get install auctex

I hope you find it useful.

Curriculum vitae

Descargar versión en español.
Download version in english.