How to setup and configure an ArchLinux based workstation for scientific computing in a mulit-user environment.
At our lab we have multiple workstation which provide researchers of our group with the necessary computational power to reconstruct large volumetric datasets. The programming languages used by the team range from MATLAB over Python to C++ and CUDA.
To generate a multiuser environment and avoid each person having there private workstation, we rely on cheap interfacing computers at desks and powerful, centralized, and headless workstations. Even though many persons recommended me a more stable OS for this purpose, I decided to use ArchLinux for all of them due to the flexibility in the setup process. In this blog post I want to walk you through the setup process of one of those headless workstations. A lot of the document is a duplicate of the installation process described in the Arch Linux wiki and this page only contains a few modifications.
Preparations
I will not go through the details of how to setup the bootable USB stick with ArchLinux OS on it. Please refer to this wiki for further instructions. Connect your PC to ethernet, keyboard, and a screen before booting. Once you booted into the live environment, quickly make sure that you booted into UEFI mode by running
ls /sys/firmware/efi/efivars
There should be no error when executing this command. If not, power off your computer and check if UEFI boot is enabled in BIOS.
Startup network
To list all the available network devices use
ip link
and check if your card is enlisted and online. If there is nothing showing up beside lo, it is very likely that your network card is not recognized (see troubleshooting section for more).
If your network card is detected correctly, make sure that you have a running and stable internet connection by using
ping archlinux.org
Now we update the system clock by
timedatectl set-ntp true
Harddrive partitioning
The next step is to partition the hard drives. We gonna go by a standard partitioning scheme containing a separate home, root, boot, and swap partition. To check which drives you have available for this purpose, use
lsblk
Usually it makes sense to position the swap and root partition on a fast drive like an M2 SSD. The home folder can usually be on a slower drive because users will make use of it as a data storage. Consider adding a tmp drive to store often used files by users.
To partition the hard drives themselves use
fdisk /dev/sda1
where your might need to replace sda1 with the name of the harddrive you want to format.
| Mount point | Partition type | suggested size | Type | Called here | |
|---|---|---|---|---|---|
/mnt |
Linux x86-64 root | at least 256 Gb | ext4 | /dev/<root> |
|
/mnt/boot |
EFI system partition | At least 512 Mb | EFI | /dev/<boot> |
|
/mnt/home |
Linux x86-64 | at least 100 Gb / user | ext4 | /dev/<home> |
|
swap |
Linux Swap | 8 Gb to 64 Gb | swap | dev/<swap> |
After running fdisk, we first generate an empty partitioning table with g. Then we add the listed partitions with n and change their type using t if necessary. There are handy abbreviations called linux (use for root and home), swap (use for swap), and uefi (use for boot) for your partition types.
Once you are done, write the scheme to the disc using w and validate the partitioning scheme with lsblk or fdisk -l.
Creating and mounting file systems
After partitioning our disks we need to create the file systems and mount them at their corresponding mount points:
# generating the file systems mkfs.ext4 /dev/<root> mkfs.ext4 /dev/<home> mkswap /dev/<swap> mkfs.fat -F32 /dev/<boot> # mounting the file systems mount /dev/<root> /mnt mkdir /mnt/boot mount /dev/<boot> /mnt/boot mkdir /mnt/home mount /dev/<home> /mnt/home # activate swap partition swapon /dev/<swap>
Select mirrors and install
Now we select the best and fastest mirrors around us using reflector:
reflector --verbose --latest 5 --sort rate --save /etc/pacman.d/mirrorlist
Now we install archlinux onto the freshly mounted file system. Instead of just installing the most basic tools, we directly include the things we need for networking lateron:
pacstrap /mnt base linux linux-firmware vim tmux man-db man-pages texinfo iproute2 dhcpcd
Then we generate the /etc/fstab file so that it is clear how we want to mount things:
genfstab -U /mnt >> /mnt/etc/fstab
and log into our new system
arch-chroot /mnt
Basic system configuration
ln -sf /usr/share/zoneinfo/Region/City /etc/localtime hwclock --systohc
Edit /etc/locale.gen and uncomment en_US.UTF-8 UTF-8 and other needed locales and run locale-gen.
echo "LANG=en_US.UTF-8" >> /etc/locale.conf echo "<hostname>" >> /etc/hostname
# file: /etc/hosts 127.0.0.1 localhost ::1 localhost 127.0.1.1 <hostname>.localdomain <hostname>
Set root password using passwd
Bootloader
Before you are done install a boot loader:
pacman -S grub efibootmgr grub-install --target=x86_64-efi --efi-directory=/boot --bootloader-id=GRUB grub-mkconfig -o /boot/grub/grub.cfg
Enable microcode in addition, depending on your architecture either run:
pacman -S amd-ucode pacman -S intel-ucode
Time to reboot!
exit poweroff # remove usb stick
Post installation things
After restarting we need to enable our networking service first:
systemctl start dhcpcd systemctl status dhcpcd systemctl enable dhcpcd
First we install basic development and system tools which are a must have:
pacman -S base-devel git sudo openssh
Create your user and add him to the sudoer group.
useradd -m <username> passwd <username> EDITOR=vim visudo
Enable ssh access to go headless
systemctl start sshd systemctl status sshd systemctl enable sshd
The default amount of commands saved in your history is most probably around 500. For me this is way too little. To increase the bash history size, add HISTSIZE=9999 to your .bashrc.
Stable kernel
Sometimes it occurs that the latest linux kernel is not yet compatible with for example nvidia drivers. In that case things like CUDA do not work all of a sudden. In my opinion, it is the best to switch the whole workstation to a stable kernel. If you use e.g. nvidia drivers you also want to switch to the so called lts version (long term support). It is recommended to backup your system (not necessary if this is a fresh install). To check your current kernel version before switching to lts use uname -srm.
As root run
pacman -Syu pacman -S linux-lts nvidia-lts pacman -R linux nvidia grub-mkconfig -o /boot/grub/grub.cfg
After a restart you should see a older kernel version (to check use uname -srm).
Additional software
Access to user repositories
After the installation of the base system, we want to install yay to have access to the different software bundles which are not available through pacman but the arch user repository (shortname aur). For this, run the following lines of code:
git clone https://aur.archlinux.org/yay-git.git chown -R <username>:<username> ./yay-git cd yay-git makepkg -si
If you want to install packages use
yay -S <packagename>
Do not run this as root.
If you want to search for a package, just use
yay -Ss <packagename>
Furthermore, if you install packages through yay, you later on need to update them manually since sudo pacman -Syu wont update the packages installed throughyay. You can do this by just runningyay` without any further input arguments.
Sublime Text
As a commonly used text editor your can install Sublime Text by
yay -S sublime-text-3
form AUR or add the official repos through
curl -O https://download.sublimetext.com/sublimehq-pub.gpg && sudo pacman-key --add sublimehq-pub.gpg && sudo pacman-key --lsign-key 8A8F901A && rm sublimehq-pub.gpg echo -e "\n[sublime-text]\nServer = https://download.sublimetext.com/arch/stable/x86_64" | sudo tee -a /etc/pacman.conf sudo pacman -Syu sublime-text
which can be started using subl from the terminal.
Sometimes SublimeText is complaining about missing ASCII support. In this case for example the package control won’t work which does not allow you to install any pacakges. In this case, simply make an alias for it in your .bashrc or modify your desktop launcher toLANG=en_US.UTF-8 subl.
The more elegant solution is to add the variables to your /etc/locale.conf file:
LANG=en_US.UTF8-8 LC_CTYPE=en_US.UTF-8
You need to restart your machine for the changes to take effect.
File management
To mount network drives like for example Synology drives or FreeNAS drives we might need helper programs:
pacman -S nfs-utils cifs-utils
If one of your users wants to have a graphical interface for file management:
pacman -S thunar
For convenient extracation of zip files, install unzip.
X11 forwarding
To forward graphical content like browsers or GUIs, install xorg. Afterwards each user can generate their own XAuthority file by running xauth followed by q.
pacman -S xorg
To allow users to forward graphical content over ssh, open the file /etc/ssh/sshd_config and make sure the following two lines are uncommented:
X11Forwarding yes X11UseLocalhost no
Afterwards restart the ssh deamon using systemctl restart sshd. After reconnecting through ssh with either -X or -Y option you should be capable of opening graphical user interfaces.
Other libraries needed
The libraries below are mostly required for my own projects.
pacman -S cmake sdl2 glfw-x11 glfw-doc hdf5 glew
CUDA specific things
To install the latest version of CUDA and the drivers for your NVIDIA graphics card run:
pacman -S cuda nvidia
Don’t forget that if you want to use the lts version, that you should replace nvidia here with nvidia-lts.
MATLAB specific things
All details below were tested with version R2019b and might need further validation with different versions.
Installation
Download MATLAB from the official website, copy through ssh to your workstation and unzip folder:
unzip matlab_<version>.zip -d matlab
If you do not have unzip installed quickly run pacman -S unzip.
Installation should be executed as root but requires X11 forwarding. For this I copied the .Xauthority file of my user into /root/.Xauthority and afterwards started the script through sudo bash install. Do not forget to delete /root/.Xauthority after installation (we do not want X11 forwarding for root usually.
Activation and post installation
Each user needs to enter their license information prior using MATLAB. For the, the following script needs to be executed (replace R2019b with your MATLAB version):
/usr/local/MATLAB/R2019b/bin/activate_matlab.sh
To use MATLAB properly there are a few other libraries to install. Some of them are only necessary to use the graphical user interfaces over X11 forwarding from the headless machine. At startup of the graphical user interface, MATLAB will throw an error saying that the library libselinux.so.1 cannot be found. Fix this by:
yay -S libselinux
Problem with fonts in MATLAB
If the fonts are not properly shown, install the following packages:
sudo pacman -S xorg-fonts-100dpi xorg-fonts-75dpi xorg-fonts-type1
Furthermore, there might be an error occurring during startup:
/usr/local/MATLAB/R2019b/bin/glnxa64/jcef_helper: symbol lookup error: /usr/lib/libpango-1.0.so.0: undefined symbol: g_ptr_array_copy
which is due to compatibility issues and will also prevent you from opening for example user interfaces (uifigure()). To solve the problem, run the following lines of code as root:
cd /usr/local/MATLAB/RXXXXX/cefclient/sys/os/glnxa64 mkdir 0_excluded mv libg* 0_excluded
There is another error which might occur during startup:
Gtk-Message: 08:01:57.963: Failed to load module "canberra-gtk-module"
Additional information and links:
Troubleshoot
Troubleshoot ethernet connection
For me the most common problem here is that the DHCPC server was not running for some reason. To verify this, run
systemctl status dhcpcd.service
and to start it if it is not up and running use
systemctl status dhcpd.service
If you encountered an interface issue (for example there could be no appropriate driver for your super new ethernet card), it will show an corresponding error in the dhcpcd status. Check which kernel module is responsible for your ethernet card using
lspci -v | less
After identifying the correct kernel module e.g. Kernel module: r8169 check the dmesg entries for errors:
dmesg | grep r8169
If there is something saying unknown chip, search online for the driver, download it, transfer it to your new PC and install the module using
pacman -U <filename>
If you have issues installing it there, maybe use your phone with USB network sharing for the installation process and take care of the rest later.