SSH access and configuration
Aim: How to get access to systems using SSH
Target audience: Users of login, Stoomboot, or other linux facilities
The Secure Shell protocol (SSH) is a common tool used to connect to services and/or servers over an encrypted communication channel. SSH can be used from a terminal application to login to the server over a network connection (for example, from your home or from a university campus) to Nikhef. Operating systems (OSes) such as Ubuntu, macOS, or Windows have SSH client tools installed by default. SSH can use public key authentication for both servers and clients. It has several useful features that are worthwhile to learn.
The instructions in this documentation page focus on using a command-line interface with SSH. If you would like to use a GUI instead of the default command-line tools, there are many options available for every OS—such as PuTTY, which can be downloaded at https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html; or Visual Studio Code, which can be downloaded at https://code.visualstudio.com/download. Documentation for how to install and use these tools can be found at their respective documentation pages.
Prerequisites
To gain access to Nikhef resources, you will need a Nikhef account.
If you have a Nikhef account, there are two ways of gaining access to Nikhef internal services and servers:
- ssh to login.nikhef.nl (required to forward your ssh-agent) and then ssh to another server; or
- Use eduVPN with Institute Access and then ssh into your destination server.
SSH is required for access to the Stoomboot cluster.
For clarity, the machine at whose keyboard you are sitting is called the 'local machine' and the machine you would like to log in to is called the 'remote machine'.
Instructions
Getting started with ssh client tools
OSes have ssh client tools installed by default. Open a Terminal window from your laptop or desktop by using a Terminal application. You can check if you have the OpenSSH client tools with a command like
This will display the version of OpenSSH that is installed on the laptop or desktop.
Public Key Authentication
The basic way ssh authenticates you is by typing your password when you log in. Repeatedly typing your password can quickly become cumbersome, especially when using scp, and is less secure. To improve this, public key authentication can be used.
Generating a Key
Attention: Generate your keys on a trusted machine, keep them safe and use a strong password for the key!
The ssh-keygen command can be used to generate a private-public key pair. Either an ED25519 key or a 4096 bits RSA key is fine. Use a strong password! It is recommended to use RSA4096, as ED25519 is not supported by older ssh servers and 4096 bits RSA is considered safe.
> ssh-keygen -t rsa -b 4096
Generating public/private rsa key pair.
Enter file in which to save the key (/home/username/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/username/.ssh/id_rsa.
Your public key has been saved in /home/username/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:P0aZKsGVvHnn/PCk7l+FFu6ev4AgcIbwH4sJbCIYmTM username@localhost
The key's randomart image is:
+---[RSA 4096]----+
|.o . |
|E.. o .. . |
|o+ + + ++ . |
|. o ..B.oo o . o |
| oo+S.= . + .|
| ..=.+.o .|
| . . +.+.o .|
| . . . B.o |
| o+.*oo|
+----[SHA256]-----+
Or for an ED25519 key:
To make sure your .ssh directory is only readable by you, update its permissions:
Installing your public key
Once your keys have been generated, you can start using them by installing the public key on the remote machine.
To do this, your public key should be appended to the file ~/.ssh/authorized_keys on the remote machine. The most convenient way to do this is usually to copy it using your password to log in:
It can also be done manually by appending your key to the ~/.ssh/authorized_keys file on the remote machine. After copying the key to the clipboard, instead of using echo on the remote machine, an editor can also be used.
> cat ~/.ssh/id_rsa.pub
your_key
> ssh username@remote
> mkdir -p ~/.ssh
> chmod go-rwx ~/.ssh
> echo "your_key" >> ~/.ssh/authorized_keys
Next, pick a valid server to log into like login.nikhef.nl. You should now be able to authenticate using your private keys. For example,
> ssh -o PubkeyAuthentication=yes username@login.nikhef.nl
Enter passphrase for key '/home/username/.ssh/id_rsa':
By using ssh-add, you can bind your private key identity to the running ssh-agent so you do not have to type your passphrase everytime. The command ssh-add needs to be run each time you reboot your system since the ssh-agent process is restarted and the ssh key needs to be reloaded. More information about how to work with ssh-agents is explained below.
Configuration guide
On Linux/Unix, create a configuration file in .ssh/config. Below are some examples and options you can use in your configuration file. Type man ssh_config to see a full explanation of options and examples or see this ssh_config documentation. Note, you will not want to use the ForwardAgent option for some hosts. Improved Security with Agent for more details.
Your ssh configuration is stored (by default) in the file ~/.ssh/config. Using your ssh config file can save you a lot of typing.
The file is organised by remote machine, defined by Host sections; wildcards are allowed. If a host matches multiple wildcards, all of them are considered. For each parameter, the first one is taken. Parameters can also be specified globally.
Agents
An ssh agent is an application that runs on a machine to which you can add your private key to avoid typing its password repeatedly. Running an ssh-agent does entail a security risk if somebody else can communicate with the agent application. If a machine is secure and you trust the system administrators, this is usually acceptable.
Adding private key(s) to the agent is done using the ssh-add command. The command ssh-add needs to be run each time you reboot your system since the ssh-agent process is restarted and the ssh key(s) need to be reloaded.
Once an agent has been setup, run the following to add your keys to it:
The -c flag causes you to be asked for confirmation (using a popup screen) every time your key is used, which is strongly recommended for security reasons. See Improved Security with Agent for more details, or if no window pops up.
ssh-add searches for keys in the ~/.ssh directory generated with the default names by ssh-keygen. If you put your key somewhere different or named it differently, add the full path and key file as an argument to ssh-add, for example
Setting up the ssh agent to run automatically
See the following sections on how to setup an agent.
The ssh agent service for Windows is not enabled by default. See this Microsoft page for detailed instructions. These powershell instructions should enable it on startup:
macOS Keychain is automatically setup to act as an ssh agent.
The Gnome keyring should be able to store your keys. If you use Gnome, ssh-add -c from a terminal should just work.
Depending on the distribution, KDE Plasma does not always have an agent started by default. If one is not available, an agent can be started when Plasma starts by saving the following as: ~/.config/plasma-workspace/env/ssh-agent-statup.sh:
and the following as ~/.config/plasma-workspace/env/ssh-agent-shutdown.sh
And log out and back in.
Improved Security with Agent
To increase the security when running an agent, it is recommended to add the -c flag when adding a key to the agent using ssh-add:
This will result in the agent asking for confirmation every time the key is used.
For this to work, the SSH_ASKPASS environment variable must point to a program that can pop-up a display. Depending on your desktop environment x11-ssh-askpass, gnome-ssh-askpass or kde-ssh-askpass may be installed. To enable use of one of these, ensure the following is set before the agent is started:
Agent Forwarding
In order to allow the gateway machine (jump host), such as login.nikhef.nl, to use your local ssh-agent (for example on your laptop) when connecting to other machines, your ssh-agent needs to be forwarded to that gateway machine. This can be achieved by adding the following line to the configuration section for that gateway machine in your local ssh config file:
X forwarding
It is possible to forward X11 connections over ssh to allow you to open GUI applictions on a remote machine and display them on your own. This requires a quite responsive network-connection to be workable; your kilometrage may vary.
To enable X11 forwarding, add the following to your ssh config file for the hosts where you want it enabled:
Note that if you use a gateway machine (see Accessing Machines through a Gateway), you need to enable X forwarding both on the local machine for the gateway machine, and on the gateway machine for the destination machine (such as stbc-i*.nikhef.nl).
Test It
This should show a clock window:
Accessing Machines through a Gateway
Sometimes a machine you’d like to access cannot be reached directly from the network you are on. The interactive Stoomboot nodes, for example, can only be reached from the Nikhef networks. If you’d like to connect to one of them from home or from a university network, you’d first have to log in to the gateway machine—login.nikhef.nl in this case—before being able to connect to the remote machine; or by starting up an eduVPN session with Institute Access.
SSH allows this to be achieved using a single connection command by setting up a ProxyJump in your ssh configuration file. To connect to stbc-i2 through login.nikhef.nl add the following to the configuration section for the stoomboot nodes (create if it does not exist):
Test It
To test the configuration, ensure you are on a network where the nodes cannot be directly contacted, for example by using the “Amsterdam Science Park” wifi network.
should now allow you to connect. If you are asked for your password multiple times, ensure your ssh agent is setup and running.Port Forwarding
SSH allows tunnelling of traffic over the encrypted SSH connection. This can come in handy when dealing with firewalls or trying to access a machine that is only reachable through a gateway or a firewall.
Another use case is to connect to a service on another machine as if you were on that machine. This is very useful when connecting to a jupyter notebook running remotely; for example on an interactive stoomboot node. To show this, log in to an interactive stoomboot node and start a notebook:
> ssh stbc-i2.nikhef.nl
> source /cvmfs/sft.cern.ch/lcg/views/setupViews.sh LCG_94
x86_64-centos7-gcc7-dbg
> jupyter notebook --no-browser
Copy/paste this URL into your browser when you connect for the first time,
to login with a token:
http://localhost:8890/?token=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXx
Note the port that was opened on the remote host, in this case 8890. The default port for a notebook server is 8888, but if a machine is shared, that port might be taken by another notebook server started by another user.
In a shell on your local machine run:
The -f flag tells ssh to run in the background; -N tells it to not execute any command on the remote host. The systax of -L is local_port:remote_server:remote_port remote; local_port is opened on your local machine; remote_server can be any machine reachable from remote - which includes localhost; and remote is the machine ssh will connect to.
To view the notebook in your own browser, point the browser to the link (including token) that jupyter printed when starting up.
If you are going to run the remote service you want to connect to from an interactive ssh session itself, and you know the port number up front, the two separate ssh commands can be combined:
Authenticating with Kerberos
Kerberos is an authentication framework based on tokens. It is for example used by CERN. To authenticate using Kerberos, there are two steps:
- Obtain a ticket
- Have ssh use the ticket to authenticate
To obtain a ticket, use (for Cern, use CERN.CH):
Then for your host add:
Example with lxplus
CERN’s Linux public login user service (lxplus) interactive cluster is used by many. There are a few things to note. As a result of the presence (not clear how much longer) of afs, public key authentication shouldn’t be used as you won’t have access to your home directory when it’s used to authenticate you. Instead, PubKeyAuthentication should be turned off and kerberos should be used.
lxplus uses a dns-based login to point you to a specifice node when you log in to the general address. For this to work, a patch must have been applied to your installation of openssh to allow this to work. In particular the option GSSAPITrustDns must be available and set. An example ssh config for lxplus is:
Host lxplus*.cern.ch lxplus*
User cern_username
ForwardX11 yes
ForwardX11Trusted yes
ForwardAgent yes
PasswordAuthentication yes
PubkeyAuthentication no
GSSAPIAuthentication yes
GSSAPIDelegateCredentials yes
GSSAPITrustDns yes
More details about this issue here.
Using SSH as a Proxy Server
An SSH connection can serve as an HTTP(S) SOCKS proxy server, which can be used to access web sites or services behind a firewall or gateway machine. To enable the proxy capabilities of ssh, add -D <port> to the command. A typical port is 8080:
This starts ssh in the background. If you also require an interactive login sessions, the -f -N can be omitted.
Once the proxy is up, the browser needs to be configured to use it, see the following section for more details.
To configure firefox’s proxy settings, open “Preferences -> Network Settings” and select “Manual Proxy Configuration”. Enter “localhost” and your chosen port. Also tick the “Proxy DNS with using SOCKS v5” box. See this guide for a more extensive explanation including pictures.
Alternatively, an addon such as FoxyProxy can be used to configure a per-site proxy server or per-site exception rules. See this guide for more information.
Chrome (and Chromium) use the desktop environment’s proxy settings. To set the proxy server that chrome should use for everything, start chrome with:
Alternatively, an addon such as Proxy-SwitchyOmega can be used to configure a per-site proxy server or per-site exception rules. See this guide for more information.
Troubleshooting
When you log into a server for the first time, you are prompted with a question like:
The authenticity of host 'bosui (2001:610:120:e030::4)' can't be established.
ED25519 key fingerprint is SHA256:XXXXXXXXXXX.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])?
This message is asking you if this is the correct host that you want to connect to. It will store the key fingerprint for each server that you establish an ssh connection with in a file called .ssh/known_hosts.
If a server that you are connecting to does not have the same key fingerprint as what is stored in the known_hosts file, you will get a message like:
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the RSA key sent by the remote host is
6e:45:f9:a8:af:38:3d:a1:a5:c7:76:1d:02:f8:77:00.
Please contact your system administrator.
Add correct host key in /home/hostname /.ssh/known_hosts to get rid of this message.
Offending RSA key in /var/lib/sss/pubconf/known_hosts:4
RSA host key for pong has changed and you have requested strict checking.
Host key verification failed.
And no longer be able to log into the server for security purposes. This could be a sign from an attacker trying to eavesdrop on the connection, but it can also be that the host has been reinstalled. In the latter case, it is OK to reset the known keys. The command
will remove all known keys for the server in your known_hosts file.
When you ssh to the server again, you will be prompted to verify the new key fingerprint for the server.
Links
- Look for different terminal applications for your OS (i.e., Guake, iterm2, … ). This can allow you to customise your settings and screens to make coding easier.
- https://www.openssh.com/index.html