Working from Remote

On This Page

As an alternative to working in the UI from your IDE, you can also connect remotely from your IDE to the python kernel in the Jupyter service on the platform (requires the Jupyter Service).

The features include:

  • Easy ssh / scp into the pod (instead of cumbersome kubectl exec/cp)
  • Jupyter Notebook frontend inside vs-code
  • Full Python debugger (including setting breakpoints in dependent python modules, etc.)

Enabling ssh

  • Enable ssh in the jupyter pod (either at creation or after the fact by using the edit option).
  • On the desktop, add an entry for the pod to ~/.ssh/config . For example, with a pod at address default-tenant.app.vmdev76.lab.iguazeng.com and user uname: 
    Host vmdev76.admin.jupyter2
HostName default-tenant.app.vmdev76.lab.iguazeng.com
User iguazio  // notice the 'i' is not capital - unlike in the GUI produced message
Port 30003
PreferredAuthentications publickey
IdentityFile /home/<uname>/.ssh/id_rsa_vmdev76_jupyter2 // the private key is copied from the GUI
ForwardAgent yes
  • chmod the IdentityFile to 600.
  • Check that ssh into the pod works: ssh vmdev76.admin.jupyter2

Setting up the Remote Workspace

  1. Install vs-code Remote-ssh extension.
  2. From vs-code command-pallet select Remote-ssh: Connect to host and provide the pod address (for example, vmdev76.admin.jupyter2).
  3. Select open-folder from the vs-code menu, and enter /v3io/users/<user> to add the home-dir of user as a folder in the vs-code workspace.
  4. Install extensions in the remote: python, jupyter (might need to switch to a pre-release version).
  5. Select python interpreter "base" (/conda/bin/python).
  6. Open the notebook of interest from remote (open-file), or your desktop (check show-local in the open-file dialog box).

See more details in Remote Development using SSH.

Full Python Debugging

To debug a cell, first press execute-above-cells (located above and to the right of the cell) to execute all cells up to the cell of interest. Then, select between these two options for debugging the cell:

  • Run by line — Press execute-above-cells (located above and to the right of the cell) to execute all cells up to this cell.
  • Debug cell — Press Debug cell to run the individual cell with full python debugger (adding debug breakpoints as usual in vs-code).

Adding Packages of Interest (mlrun, storey, etc.) for Debug cell

To find the location of the package (for example storey package)

  1. Run from a python shell (inside the jupyter pod):
    • import storey
    • help (storey): prints the path to the package, should be at /conda/lib/python3.7/site-packages/storey
  2. From vs-code menu select:
    • add-folder-to-workspace <path-to-package>
  3. If setting breakpoints in dependent modules is required, you can set "jupyter.debugJustMyCode": false to disable the “Just My Code” debug feature, meaning it steps through library code, in addition to your code. Alternatively (if debugJustMyCode does not work), you can patch pydevd.py by:

    1. Find the file to patch:

      import pydevd
print(pydevd._file_)

      (should be similar to /conda/lib/python3.7/site-packages/debugpy/_vendored/pydevd/pydevd.py)

    2. Patch pydevd.py set_use_libraries_filter() to have use_libraries_filter=False as its the first line.

    3. Restart the kernel.