SURF Resarch Cloud: VirtualShip environment setup#

Note: This guide is specific to students who are enrolled at Utrecht University.

In the class, we will use VirtualShip in the cloud (in this case, SURF Research Cloud - called SURF RC from here-on). This has several advantages:

  • You aren’t limited to the power of your laptop

  • Input datasets are downloaded faster, as they’re downloaded to the cloud instance (and not to your laptop)

1. Accepting SURF RC invite#

In your student email you’ll have an invite from SURF Research Access Management (SRAM) to join a project on SURF RC. Accept this invite.

2. Open the environment#

Navigate to the SURF Research Cloud Dashboard, and click “access” on the shared workspace.

TIP: You may sometimes hit a server error when accessing the workspace. If this happens, keep on trying (refresh), as the server spin-up can be a bit overloaded at times, but should get through eventually. Unfortunately this is out of our control. Clearing your browser cache and cookies, and/or trying via an incognito/private window may also help.

3. Jupyter workspace layout and additional config#

Note: This only needs to be done once during setup.

In the Jupyter workspace, you’ll see the following (or similar) in your file explorer:

.
├── KERNEL-README.ipynb
├── data
│   └── datasets
|   └── virtualship-storage    <--- The shared persistent storage
└── scratch

Note: the persistent storage folder may be called something slightly different in your instance, for example it may have a name specific to the course you are enrolled on, such as data/storage-osl, data/storage-dyoc or data/storage-1-sept.

In the Jupyter launcher, you can open a Terminal session by clicking on “Terminal” button under the “Other” section, or by going to the “File” menu –> “New” –> “Terminal”. From here you can navigate the workspace directory structure and run commands.

INFO: VirtualShip is a command line interface (CLI) based tool. We will be working predominantly via the command line in Terminal (typing out commands instead of pointing and clicking). If you are unfamiliar with what a CLI is, see here for more information. In our case, the Terminal is just a way to access the CLI on the SURF RC virtual machine.

The data/virtualship-storage folder is your persistent storage. Here you can make a folder (e.g., by running mkdir data/virtualship-storage/{your-group-name} as a command in the Terminal, replacing {your-group-name} with your group name) to house your work for the unit. This folder will be visible to anyone using the workspace, but only you will be able to make edits to it. This is the primary place you should store your virtualship configs and content relevant to this unit.

To be able to run VirtualShip from the Terminal, we need to take some additional steps which are detailed in the KERNEL-README.ipynb in your home folder. This contains important information for configuring your environment. Specifically, for our uses, the “Initialize conda” section (also copied below). Do the following:

Initialize conda#

To make the already installed conda-tool available for yourself, you have to initialize your Terminal shell.

Back in the “Terminal” tab, type: /etc/miniconda/bin/conda init

Close the Terminal tab and start a new one. You will see that the Terminal prompt has changed to something like

(base) metheuser@mywsp:

This is conda telling you that you are currently in the “base” environment.

From here, you already have another environment set up for you. Running conda env list in the Terminal, you should see:

conda env list

# conda environments:
#
base                 * /etc/miniconda
virtualship            /etc/miniconda/envs/virtualship`

Here you can do conda activate virtualship to activate the environment called “virtualship”. This environment is a shared environment among all workspace users that can be centrally updated. If you want, you can create and manage your own environments by running the relevant conda commands.

With the virtualship environment, you now have access to the virtualship command in your Terminal, which can be confirmed by running virtualship --help.

From here you can cd (‘change directory’) into data/virtualship-storage/{your-group-name} and run virtualship commands as you would on your local machine (see the VirtualShip quickstart guide).

TIP: Return to your course-specific instructions/assignment notebook now if you were following along with this setup in class!

Working in Jupyter Notebooks#

Finally, when you’re working in Jupyter Notebooks, you are able to access the Conda environment with virtualship and related dependencies by switching the Kernel in the top right of the UI.

Course facilitator notes#

If virtualship is updated on GitHub, and you want to update the shared environment, you can do so by running the following commands in the Terminal:

conda activate virtualship
sudo /etc/miniconda/envs/virtualship/bin/pip install --upgrade git+https://github.com/OceanParcels/virtualship@main