Working from Remote

On This Page

As an alternative to working in the UI, you can also work from remote. The features include:

  • Easy ssh / scp into the pod (instead of cumbersome kubectl exec/cp)
  • Jupyter Notebook frontend inside vs-code
  • Full Python debugger (including setting breakpoints in dependent python modules, etc.)

Enabling ssh

  • Enable ssh in the jupyter pod (either at creation or after the fact by using the edit option).
  • On the desktop add an entry for the pod to ~/.ssh/config . For example, with a pod at address and user uname:
    Host vmdev76.admin.jupyter2
    User iguazio  // notice the 'i' is not capital - unlike in the GUI produced message
    Port 30003
    PreferredAuthentications publickey
    IdentityFile /home/<uname>/.ssh/id_rsa_vmdev76_jupyter2 // the private key is copied from the GUI
    ForwardAgent yes
  • chmod the IdentityFile to 600.
  • Check that ssh into the pod works: ssh vmdev76.admin.jupyter2

Setting up the Remote Workspace

  1. Install vs-code Remote-ssh extension.
  2. From vs-code command-pallet select Remote-ssh: Connect to host and provide the pod address (for example, vmdev76.admin.jupyter2).
  3. Select open-folder from the vs-code menu, and enter /v3io/users/<user> to add the home-dir of user as a folder in the vs-code workspace.
  4. Install extensions in the remote: python, jupyter (might need to switch to a pre-release version).
  5. Select python interpreter "base" (/conda/bin/python).
  6. Open the notebook of interest from remote (open-file), or your desktop (check show-local in the open-file dialog box).

See more details in Remote Development using SSH.

Full Python Debugging

To debug a cell, first press execute-above-cells (located above and to the right of the cell) to execute all cells up to the cell of interest. Then, select between these two options for debugging the cell:

  • Run by line — Press execute-above-cells (located above and to the right of the cell) to execute all cells up to this cell.
  • Debug cell — Press Debug cell to run the individual cell with full python debugger (adding debug breakpoints as usual in vs-code).

Adding Packages of Interest (mlrun, storey, etc.) for Debug cell

To find the location of the package (for example storey package)

  1. Run from a python shell (inside the jupyter pod):
    • import storey
    • help (storey): prints the path to the package, should be at /conda/lib/python3.7/site-packages/storey
  2. From vs-code menu select:
    • add-folder-to-workspace <path-to-package>
  3. If setting breakpoints in dependent modules is required, you can set "jupyter.debugJustMyCode": false to disable the “Just My Code” debug feature, meaning it steps through library code, in addition to your code. Alternatively (if debugJustMyCode does not work), you can patch by:
    1. Find the file to patch:

      import pydevd

      (should be similar to /conda/lib/python3.7/site-packages/debugpy/_vendored/pydevd/

    2. Patch set_use_libraries_filter() to have use_libraries_filter=False as its the first line.

    3. Restart the kernel.