Virtual Machine Setup Guide

We’ll be using Arch Linux virtual machines (VMs) in CS 326. Arch Linux is a simple distribution that gives us the opportunity to configure our virtual machines exactly as we want. For your reference, the Arch Linux Wiki has a wealth of information on configuring the system, swapping out components, and installing a wide variety of software.

In this lab, you’ll learn how to:

Create a virtual machine using Linux Kernel-based Virtual Machine (KVM) virtualization
Forward ports and connect to a remote VNC console
Partition disks
Configure a base Linux system, including networking, from scratch

If you’d like to go beyond the steps outlined on this page, check out the helpful Arch Linux Installation Guide.

NOTE: you’ll work through this lab in groups. Set up your group’s GitHub repo here: https://classroom.github.com/g/j8HDMnlH

Virtual Machine Setup

One of our goals in this class is to get more familiar with Linux (and Unix) systems, particularly with the command line interface (CLI). Instead of giving each student a Linux workstation to use over the course of the semester, we’ll use virtual machines. After all, virtualized infrastructure has revolutionized the computing landscape; rather than maintaining large datacenters with thousands of power-hungry servers, companies can start and stop virtual machines as needed to scale their infrastructure in or out as required.

KVM is a virtualization module that allows Linux to act as a hypervisor, or software component that supervises virtual machines. In VM terminology, you have two types of systems:

Host - the physical machine VMs run on; it hosts the VMs.
Guest - the VMs themselves are guests running on the host.

We will use our VM host gojira to create a VM guest for each student in this lab.

Aside: Cloud Computing

While the concepts behind cloud computing are not brand new, companies like Amazon leverage virtualization technology both for their own infrastructure as well as Amazon Web Services (AWS). Historically, Amazon was faced with a hardware conundrum: they needed massive computing and storage capabilities during the holiday shopping season to keep up with demand (after all, if a user can’t make a sale now, they’ll move on to the competition). A company with lots of money can certainly acquire all the hardware necessary, but what happens once the holiday shopping spree ends? Instead of just shutting down its infrastructure for 70% of the year, Amazon decided to rent it out via virtualization… leading to the birth of modern cloud computing.

If you’ve ever used AWS, Google Cloud Platform, Azure, etc., then you’ll be somewhat familiar with the concepts that follow. However, instead of using a fancy web UI to create our VMs, we are going to do it by hand!

Logging into the VM Host

Dust off your USF CS credentials and ssh into our jump host, stargate.cs.usfca.edu. Once logged in you can proceed to ssh to our VM host, gojira:

# Get to the CS network:
$ ssh mmalensek@stargate.cs.usfca.edu

# Log in to the VM host:
$ ssh mmalensek@gojira

# Let's see who else is logged in:
$ finger
Login      Name               Tty      Idle  Login Time   
afedosov   Alex Fedosov       pts/0    4:04  Dec  9 12:20 
mmalensek  Matthew Malensek   pts/1          Dec  9 17:31 
# (This output will probably be a bit more exciting once classes start)

IMPORTANT: You will be using ssh extensively in this class, so you should set up Passwordless ssh if you haven’t already.

All of our VMs will be hosted on a single server (gojira), which has 64 hardware threads (32 cores) and 64 GB of RAM. You will be able to log into the VM host to see all the VMs running.

Creating a new Virtual Machine

Virtualization functionality is provided by the libvirt toolkit. We’ll use the virt-install command to set up our VM. Note that in the following example, the backslash (\) denotes a command continuation, a way of breaking one very long command line into multiple lines. Before you simply copy and paste this into your terminal, you will need to tweak the command a bit to customize for your own account.

$ virt-install \
    --name=XXXXXXXX-vm1 \
    --cpu host --vcpus=2 \
    --memory=1024 \
    --disk=/home2/$(whoami)/vm1.qcow2,size=16 \
    --net model=virtio,bridge=virbr0 \
    --cdrom=/home2/mmalensek/iso/archlinux-XXXX.iso \
    --graphics vnc,password=XXXXXXXX,port=10XXX \
    --video vga \
    --os-type=linux --os-variant=fedora28 \
    --noautoconsole

Be sure to update:

name – Name of the new guest virtual machine instance. This must be unique. You should fill in your user name, e.g., my VM is named mmalensek-vm1.
cdrom – this is the path to the Arch Linux install CD image. You should cd into /home2/mmalensek/iso and find newest version stored there.
graphics – change the ‘password’ and ‘port’ fields. For the port, use 10000 plus your CS 326 Classroom ID. So if your 326 ID is 282, enter 10282 here.

Your VM’s virtual hard disk will be stored in /home2/$(whoami)/vm1.qcow2, where $(whoami) will automatically be replaced with your user name.

The rest of the command line flags do the following:

cpu, vcpus – sets the model and number of CPUs available to the VM.
memory – determines how much RAM the VM gets.
net – configures the network interface. We will use bridged networking in class.
video – sets the virtual video card. We’ll use a standard VGA display adapter.
os-type, os-variant – tells the hypervisor what type of VM we are running so it can optimize appropriately
noautoconsole – has the VM install process run in the background once the command finishes executing.

If you would like to check out all the command line flags available for the virt-install utility, take a look at the man pages: man virt-install.

Logging into the VNC Console

As long as virt-install succeeded, you should be ready to set up your OS. To verify, run virsh list:

# Note that without the --all flag, only running VMs will be shown.
# If your VM gets shut down later, you'll want to remember to pass
# this flag so you can check its status.
$ virsh list --all
 Id    Name                           State
 ----------------------------------------------------
 1     mmalensek-vm1                  running

Great, it’s running! Let’s log in. We’ll be using the Virtual Network Computing (VNC) remote desktop system to do this. Recall your VNC port from above (10000 plus your 326 ID) – this tells the VNC software how to communicate with your VM console. However, we have one issue to overcome first. Recall that in order to get into the CS network, we have to connect to stargate; this protects the CS network from the cold, harsh reality of the open Internet. Unfortunately, this also means that we have no way of connecting to gojira directly.

Luckily, we can use ssh tunneling to route information from our computer through the ssh connection to gojira – ssh will run in the background on your local machine and make a port available that shuttles data back and forth to/from gojira. This is fairly easy to accomplish on macOS/Linux, but Windows users may need to jump through some additional hoops (see below). To do this on macOS/Linux, first open a new terminal window and then run:

# Replace 'mmalensek' with your CS login, and 10XXX with the VNC port you
# configured above:

$ ssh -N -J mmalensek@stargate.cs.usfca.edu mmalensek@gojira -L 10XXX:localhost:10XXX

# The flags:
# -N  don't log us in, just forward the ports
# -J  jump through this host (stargate)
# -L  forward port 10XXX from gojira to our local machine (localhost)

# NOTE: if your version of ssh doesn't support the -J flag, then
# you can use the following instead. (ONLY if the command above didn't work)
ssh -o \
    'ProxyCommand ssh -q -W %h:%p mmalensek@stargate.cs.usfca.edu' \
    mmalensek@gojira -L 10XXX:localhost:10XXX

This will do… nothing. (As long as it worked). Now we can connect to the console.

NOTE: you should leave this window and terminal session open since it is forwarding ports for VNC.

VNC Client: macOS

macOS has a built-in VNC client. Simply switch to the ‘Finder’ application, then Go -> Connect to Server (or just press ⌘ + K). Enter:

vnc://localhost:10XXX

Where 10XXX is your VNC port number. Click Connect, and then enter the password you configured above. If everything works, you should be greeted with the following:

You’re done with the Mac VNC setup. Now you can move on to Installing the OS.

VNC Client: Linux

If you’re using a recent Linux distribution, there’s a very good possibility that you already have a VNC client installed. If not, vinagre is a good option. To install:

# If you are on an apt-based system (Ubuntu, Debian...)
$ sudo apt-get install vinagre

# If you are on a yum/dnf-based system:
yum install vinagre

Assuming you followed the ssh tunnel instructions above, this is all you need. Start up vinagre, click ‘Connect’, choose VNC and enter your connection details. Here’s an example using port 10282:

…then move on to Installing the OS.

VNC Client: Windows

Note: With the latest versions of Windows, you may be able to use WSL (Windows Subsystem for Linux) with the Linux instructions above.

If you don’t have WSL or are on an older version of Windows, you should install MobaXterm home edition. It supports ssh tunneling as well as VNC all in one application.

Once MobaXterm is installed, you will need to set up three tunnels. Click the Tunneling icon and add the following:

Tunnel 1

Forward Port: 2222
SSH Server: your-username@stargate.cs.usfca.edu:22
Destination Server: gojira:22

Tunnel 2

Forward Port: 10XXX (use your VNC port)
SSH Server: your-username@localhost:2222
Destination Server: localhost:10XXX

Tunnel 3

Forward Port: 2223
SSH Server: your-username@localhost:2222
Destination Server: 192.168.122.XXX:22

your-username is your CS login, and you’ll use your CS password for the ssh connections. Here’s a screenshot of my configuration (using ID number 282):

Make sure you start the tunnels before moving on.

Next, you will need to set up the VNC connection. Click the Session button in the MobaXterm toolbar, then choose VNC. Then enter:

Hostname: localhost
Port: 10XXX

Hit OK, enter your VNC password (the one you created with virt-install), and you should be good to go – the Arch Linux boot menu will display as shown above.

Installing the OS

At this point, we’ve booted up a Live CD version of Arch Linux. The live CD contains all the standard Unix utilities we need to bootstrap and install the operating system by hand onto our VM’s hard disk.

Make sure “Boot Arch Linux” is selected at hit enter. The system will boot and you will automatically be logged in as the root user (the Unix superuser or administrator). If you run ls to list the files here, you’ll see a single install.txt file that outlines the Arch Linux install process. This is useful to have if you are setting up a machine with no network access, but our VM should already be connected to the network. We will verify this with the ping command:

root@archiso ~ # ls
install.txt
root@archiso ~ # ping google.com
PING google.com (216.58.194.206): 56 data bytes
64 bytes from 216.58.194.206: icmp_seq=0 ttl=57 time=11.457 ms
64 bytes from 216.58.194.206: icmp_seq=1 ttl=57 time=4.017 ms
^C

# (As usual, Ctrl + C quits the program).

Okay, so we’re online. We have a live CD running and want to set up a new installation of Linux on our VM’s hard disk. Let’s get started!

Partitioning the Disk

The hard disk our VM starts out with is the same as one that has just been purchased and plugged in, i.e., it has absolutely nothing on it. On Unix systems, devices are represented as special files stored in /dev/. We can see the devices available on our VM with ls:

root@archiso ~ ＃ ls /dev
autofs           initctl             null      tty3   tty24  tty45  ttyS2    vcsu
block            input               port      tty4   tty25  tty46  ttyS3    vcsu1
bsg              kmsg                ppp       tty5   tty26  tty47  udmabuf  vcsu2
btrfs-control    lightnvm            psaux     tty6   tty27  tty48  uhid     vcsu3
bus              log                 ptmx      tty7   tty28  tty49  uinput   vcsu4
cdrom            loop0               pts       tty8   tty29  tty50  urandom  vcsu5
char             loop1               random    tty9   tty30  tty51  userio   vcsu6
console          loop2               rfkill    tty10  tty31  tty52  vcs      vda
core             loop3               rtc       tty11  tty32  tty53  vcs1     vfio
cpu_dma_latency  loop4               rtc0      tty12  tty33  tty54  vcs2     vga_arbiter
cuse             loop5               shm       tty13  tty34  tty55  vcs3     vhci
disk             loop6               snapshot  tty14  tty35  tty56  vcs4     vhost-net
dri              loop7               snd       tty15  tty36  tty57  vcs5     vhost-vsock
fb0              loop-control        sr0       tty16  tty37  tty58  vcs6     virtio-ports
fd               mapper              stderr    tty17  tty38  tty59  vcsa     vport1p1
full             mem                 stdin     tty18  tty39  tty60  vcsa1    zero
fuse             memory_bandwidth    stdout    tty19  tty40  tty61  vcsa2
hidraw0          mqueue              tty       tty20  tty41  tty62  vcsa3
hpet             net                 tty0      tty21  tty42  tty63  vcsa4
hugepages        network_latency     tty1      tty22  tty43  ttyS0  vcsa5
hwrng            network_throughput  tty2      tty23  tty44  ttyS1  vcsa6

But which one of these is our hard disk? We can view all the disk partitions on our system by inspecting /proc/partitions or using the lsblk command. Another option is fdisk -l to list all the block devices available:

# The /proc/partitions file is a virtual file that gets populated with
# partition data. We'll learn more about /proc later on.

root@archiso ~ ＃ cat /proc/partitions
major minor  #blocks  name

 254        0   16777216 vda
  11        0     616448 sr0
   7        0     500948 loop0

root@archiso ~ ＃ lsblk
NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
loop0    7:0    0 489.2M  1 loop /run/archiso/sfs/airootfs
sr0     11:0    1   602M  0 rom  /run/archiso/bootmnt
vda    254:0    0    16G  0 disk

root@archiso ~ ＃ fdisk -l
Disk /dev/vda: 16 GiB, 17179869184 bytes, 33554432 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/loop0: 489.2 MiB, 512970752 bytes, 1001896 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

One easy way to identify our hard disk is its size: we configured the VM with a 16 GB disk, so the 16 GB disk at /dev/vda is the one we want.

Partitioning the disk takes a block device and splits it into multiple sub-devices. So for instance, we could take our 16 GB disk and split it into 1, 5, and 10 GB pieces. We could also create three partitions of 1 GB and leave 13 GB of unused space on the disk… but that wouldn’t be very useful! The point is, you can divide the disk however you’d like and you don’t necessarily need to use the entire thing.

In our configuration, we’ll have two partitions: 100 MB for the boot partition, and the rest of the disk for our root partition. The boot partition contains the code for the bootloader, which takes care of actually starting our OS for us. The splash screen you saw when you first connected over VNC was displaying the live CD’s bootloader.

Let’s partition this disk! Here’s the steps:

Pass the block device file to the fdisk utility
Create a new partition table (the place where partition configuration information is stored)
Create a new primary partition (enter n, then p)
Configure the partition to start at the beginning of the disk (use the default value) and then extend by 100 MB (enter +100M).
Create another primary partition (defaults are fine here)
Print the partition table with p to verify your work
Toggle the boot flag for the first partition
Write the changes to the disk

Easy, right? Okay, here we go:

root@archiso ~ ＃ fdisk /dev/vda
Welcome to fdisk (util-linux 2.33).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0xc22fbe10.

Command (m for help): o
Created a new DOS disklabel with disk identifier 0xdd504842.

Command (m for help): n
Partition type
   p   primary (0 primary, 0 extended, 4 free)
   e   extended (container for logical partitions)
Select (default p): p
Partition number (1-4, default 1):
First sector (2048-33554431, default 2048):
Last sector, +/-sectors or +/-size{K,M,G,T,P} (2048-33554431, default 33554431): +100M

Created a new partition 1 of type 'Linux' and of size 100 MiB.

Command (m for help): n
Partition type
   p   primary (1 primary, 0 extended, 3 free)
   e   extended (container for logical partitions)
Select (default p): p
Partition number (2-4, default 2):
First sector (206848-33554431, default 206848):
Last sector, +/-sectors or +/-size{K,M,G,T,P} (206848-33554431, default 33554431):

Created a new partition 2 of type 'Linux' and of size 15.9 GiB.

# Okay, great, we've made it through steps 1-5. You should use the help menu
# ('m') to figure out how to:
#
#   - Set the boot flag on partition 1
#   - Write the changes to the disk
#
# After setting the boot flag, print the partition table with 'p'. Your output
# should look like this (note the * for 'Boot'):

Command (m for help): p
Disk /dev/vda: 16 GiB, 17179869184 bytes, 33554432 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xc22fbe10

Device     Boot  Start      End  Sectors  Size Id Type
/dev/vda1  *      2048   206847   204800  100M 83 Linux
/dev/vda2       206848 33554431 33347584 15.9G 83 Linux

Command (m for help): 

# Reminder: be sure to write your changes after setting the flag.

Note: many modern systems (especially those with uEFI) use the GPT partitioning scheme instead of MBR. To do this, you can use the gdisk utility instead of fdisk – it operates similarly but uses the more modern GPT partition format.

Assuming the partitioning process went well, you should be able to see the partitions with lsblk:

root@archiso ~ ＃ lsblk
NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
loop0    7:0    0 489.2M  1 loop /run/archiso/sfs/airootfs
sr0     11:0    1   602M  0 rom  /run/archiso/bootmnt
vda    254:0    0    16G  0 disk
├─vda1 254:1    0   100M  0 part
└─vda2 254:2    0  15.9G  0 part

If something is wrong, you can repeat the process described above without starting everything over again.

In the past, Unix installations frequently had several different partitions: binary executables might be on one partition, user home directories on another, configurations on another, and so on. This has a number of advantages, including:

The /boot partition can contain a basic file system that almost any OS can read
If the /home partition is separate, the entire OS can be replaced/upgraded (or completely changed, say from Ubuntu to CentOS) without bothering users’ files
Too many files (or too large) on one partition will not impact the others. If a huge application is installed to /usr, it can only fill the partition, not the whole disk.

Many Linux distributions have recently started defaulting to single partition: /. This is simple to work with and modern hardware is not as limited as it was in the past, making a one-partition setup a reasonable choice.

Creating File Systems

Now that we’re done editing the partition table, we have two raw partitions of an empty disk. Our OS needs to know how to store and retrieve files from these partitions; to do this, we’ll create a file system on both.

In our case, we’ll have one file system on /boot and another on / (root). For /boot, we’ll use the older but ubiquitous ext2, and for / we’ll use xfs. Each file system has its own set of pros and cons; if you want to do some research and use a different one, feel free!

File systems are created with the mkfs.* family of commands. We’ll use mkfs.ext2 and mkfs.xfs:

# Create the file system for /boot:
root@archiso ~ ＃ mkfs.ext2 /dev/vda1
mke2fs 1.44.5 (15-Dec-2018)
Creating filesystem with 102400 1k blocks and 25688 inodes
Filesystem UUID: a857a454-4df2-47a3-b5bc-986a27a3003a
Superblock backups stored on blocks:
        8193, 24577, 40961, 57345, 73729

Allocating group tables: done
Writing inode tables: done
Writing superblocks and filesystem accounting information: done


# Create the file system for / (root):
root@archiso ~ ＃ mkfs.xfs /dev/vda2
meta-data=/dev/vda2              isize=512    agcount=4, agsize=1042112 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=1        finobt=1, sparse=1, rmapbt=0
         =                       reflink=0
data     =                       bsize=4096   blocks=4168448, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0, ftype=1
log      =internal log           bsize=4096   blocks=2560, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0

/dev/vda1 now has an ext2 file system, and /dev/vda2 has an XFS file system. We can verify this with blkid:

root@archiso ~ ＃ blkid /dev/vda1
/dev/vda1: UUID="a857a454-4df2-47a3-b5bc-986a27a3003a" TYPE="ext2" PARTUUID="c22fbe10-01"
root@archiso ~ ＃ blkid /dev/vda2
/dev/vda2: UUID="96eb7b7e-a25a-479d-822b-951bd8bc0c9d" TYPE="xfs" PARTUUID="c22fbe10-02"

Mounting the File Systems

Our partitions have file systems and are ready to go. Now we just need to install Linux. To modify the file system contents, we need to mount them on our live CD’s file system tree. Mounting a file system grafts it onto an existing file system tree. We’ll mount our root partition on /mnt (you can probably guess what /mnt is for…):

# Take the root FS we created and stick it on /mnt:
root@archiso ~ ＃ mount /dev/vda2 /mnt
# Create a /boot directory on the root FS:
root@archiso ~ ＃ mkdir /mnt/boot
# Stick our boot partition inside the root partition:
root@archiso ~ ＃ mount /dev/vda1 /mnt/boot

# Now if we do an ls, all we get is:
root@archiso ~ ＃ ls /mnt
boot
# Pretty sad, huh? We need to fill that root directory with files!

# Let's take a look at what's mounted with df:
root@archiso ~ ＃ df -H
Filesystem      Size  Used Avail Use% Mounted on
( ... some contents removed for brevity ... )
dev             494M     0  494M   0% /dev
airootfs        269M  7.5M  262M   3% /
tmpfs           517M     0  517M   0% /tmp
/dev/vda2        18G   51M   18G   1% /mnt
/dev/vda1       102M  1.6M   95M   2% /mnt/boot

# Great, they're both mounted!

Installing the Base System

Arch Linux has a nice tool called pacstrap that will bootstrap a system into a specified directory. We want it to bootstrap our new root partition:

root@archiso ~ ＃ pacstrap /mnt base
==> Creating install root at /mnt
==> Installing packages to /mnt
:: Synchronizing package databases...
 core                 135.0 KiB   7.75M/s 00:00 [#############] 100%
 extra                1694.1 KiB  61.3M/s 00:00 [#############] 100%
 community            4.8 MiB     76.6M/s 00:00 [#############] 100%

( ... lots of packages get installed ... )

# Afterward, we can see that /mnt is populated with system files:
/mnt
├── bin -> usr/bin
├── boot
├── dev
├── etc
├── home
├── lib -> usr/lib
├── lib64 -> usr/lib
├── mnt
├── opt
├── proc
├── root
├── run
├── sbin -> usr/bin
├── srv
├── sys
├── tmp
├── usr
└── var

The arrows (->) here denote links or files/directories that point to another file/directory. Kind of like aliases or shortcuts in macOS/Windows.

Basic Configuration

The base system is ready, but we need to configure a few things before we can start using it. First, we need to create an fstab, which describes the file systems available to the OS. The fstab is how the OS will know what to mount automatically at boot. Note that your fstab will be a little different than the one shown below, since your hard disk partitions will have different UUIDs.

root@archiso ~ ＃ genfstab -U /mnt >> /mnt/etc/fstab

# Let's check the contents of the fstab:
root@archiso ~ ＃ cat /mnt/etc/fstab
# Static information about the filesystems.
# See fstab(5) for details.

# <file system> <dir> <type> <options> <dump> <pass>
# /dev/vda2
UUID=96eb7b7e-a25a-479d-822b-951bd8bc0c9d       /               xfs             rw,relatime,attr2,inode64,noquota     0 1

# /dev/vda1
UUID=a857a454-4df2-47a3-b5bc-986a27a3003a       /boot           ext2            rw,relatime,block_validity,barrier,user_xattr,acl     0 2

Once the fstab is ready, we can chroot into our new base system. chroot stands for ‘change root’ – it starts a new shell inside a specified directory as if it was the root directory of the system. So if we chroot into /mnt, the /mnt directory will now appear to be the root of the system! This is important because we can then run all the utilities we just installed to our new disk, and the changes they make will be written to the new system rather than the live CD’s environment.

Note that chroot is also useful from a security perspective – you can jail a process inside a chroot environment, and it won’t be able to modify the file system outside its virtual root directory. A more advanced version of chroot, called jails on BSD systems, allows for some basic containerization functionality. This might add a bit of context on where the term “jailbreaking” comes from (such as with iOS devices).

Enough talk! Let’s chroot in and start configuring!

# Notice how the shell prompt changes after doing the chroot. This is because
# we're running inside a "different" system with a different shell configuration.
root@archiso ~ ＃ arch-chroot /mnt
[root@archiso /]＃

# Our first order of business is to configure the time zone. We will create a link
# to specify that our local time zone is 'Pacific':
[root@archiso /]＃ ln -sfv /usr/share/zoneinfo/US/Pacific /etc/localtime
'/etc/localtime' -> '/usr/share/zoneinfo/US/Pacific'
[root@archiso /]＃ hwclock --systohc
[root@archiso /]＃ date
Tue Jan 15 13:50:12 PST 2019

Now we need to set up the locale. If we don’t do this, characters won’t show up correctly and we can have issues with date, time, currency formatting. Open up /etc/locale.gen in your editor (vi, nano, etc) and uncomment the en_US lines (around line 176). Here’s a snippet from the file with the correct lines uncommented:

#en_PH.UTF-8 UTF-8
#en_PH ISO-8859-1
#en_SC.UTF-8 UTF-8
#en_SG.UTF-8 UTF-8
#en_SG ISO-8859-1
en_US.UTF-8 UTF-8
en_US ISO-8859-1
#en_ZA.UTF-8 UTF-8
#en_ZA ISO-8859-1
#en_ZM UTF-8
#en_ZW.UTF-8 UTF-8
#en_ZW ISO-8859-1

Next, we’ll update the locale configuration and generate the locales:

[root@archiso /]＃ echo 'LANG=en_US.UTF-8' > /etc/locale.conf
[root@archiso /]＃ locale-gen
Generating locales...
  en_US.UTF-8... done
  en_US.ISO-8859-1... done
Generation complete.

Now we’ll create a user account for ourselves, and set the root password:

# It's a bad idea to always be logged in as root, so let's create our own user
# account with the useradd command. The -m flag creates a home directory for
# the user (/home/mmalensek in this case).
[root@archiso /]＃ useradd -m mmalensek

# Set the password for our new user
[root@archiso /]＃ passwd mmalensek

# Let's test out the account. Since we're root, we can actually use the
# substitute user (su) command to switch over:
[root@archiso /]＃ su - mmalensek
[mmalensek@archiso ~]$ ls

# Great, it worked. Let's switch back to root.
[mmalensek@archiso ~]$ exit

# Finally, you should set a root password. Root logins aren't allowed over SSH
# in the default configuration, but let's do it just to be safe:
$ passwd
# (in this case, not entering a username tells passwd to change the current
# user's password, so we're changing the root password)

Finally, we need a few more utilities installed on our new system. Let’s install them with pacman:

[root@archiso /]＃ pacman -Sy \
    base-devel clang git grub openssh python strace valgrind vim wget

( ... lots more packages install ... )

Make sure you perform this step! Some of the following configuration steps require these packages.

Setting the Hostname

There are two hard problems in computer science: cache invalidation, naming things, and off-by-one errors. We have to tackle one of these here: naming your VM. The /etc/hostname file contains the hostname of the machine, so let’s create this file from our shell. I will name my VM magical-unicorn:

[root@archiso /]＃ echo "magical-unicorn" > /etc/hostname

This is a great illustration of why we need chroot: if we hadn’t changed root to our new system’s root partition, then we’d just be changing the hostname of the live CD with the command above.

Network Configuration

One of the last things we need to do is set up the network. While our VM has negotiated a dynamic IP address, we want to be able to reach it at a static IP address in the future. Let’s create the network configuration file. You should edit /etc/systemd/network/20-wired.network with your favorite text editor and add the following:

[Match]
Name=ens3

[Network]
Address=192.168.122.XXX/24
Gateway=192.168.122.1
DNS=192.168.122.1

Where XXX is your 326 ID number. This tells the systemd network manager to set a static IP address for interface ens3, which means you’ll be able to ssh username@192.168.122.XXX to log in instead of having to use VNC every time.

Now we need to enable the network management daemon as well as ssh. We’ll also set up DNS:

[root@archiso /]＃ systemctl enable systemd-networkd
Created symlink /etc/systemd/system/dbus-org.freedesktop.network1.service → /usr/lib/systemd/system/systemd-networkd.service.
Created symlink /etc/systemd/system/multi-user.target.wants/systemd-networkd.service → /usr/lib/systemd/system/systemd-networkd.service.
Created symlink /etc/systemd/system/sockets.target.wants/systemd-networkd.socket → /usr/lib/systemd/system/systemd-networkd.socket.
Created symlink /etc/systemd/system/network-online.target.wants/systemd-networkd-wait-online.service → /usr/lib/systemd/system/systemd-networkd-wait-online.service.

[root@archiso /]＃ systemctl enable sshd
Created symlink /etc/systemd/system/multi-user.target.wants/sshd.service → /usr/lib/systemd/system/sshd.service.

# We also need to set up our nameserver, which handles DNS queries.
# E.g., looking up www.google.com maps to a particular IP address:
[root@archiso /]＃ echo "nameserver 192.168.122.1" >> /etc/resolv.conf

Alright, we should be all set with the network now.

Bootloader

Traditionally, operating systems need a small utility called a bootloader to start them up. This helps enforce separation of concerns; the hardware BIOS doesn’t need to be able to understand all the various file systems that exist or know how to start things up; instead, it just launches the bootloader and lets it do the heavy lifting. On modern systems, such as those with uEFI, a bootloader isn’t strictly necessary, but it still gives us more flexibility. We will install the GRUB bootloader on our VMs, but there are many different options available.

# IMPORTANT: notice how we are installing the bootloader to the *device*
# itself (vda), NOT a partition. The boot code is independent from the
# partitions.
[root@archiso /]＃ grub-install --target=i386-pc /dev/vda
Installing for i386-pc platform.
Installation finished. No error reported.

# Next, we need to generate a config file for GRUB. The automatic
# configuration works fine in our case.
[root@archiso boot]＃ grub-mkconfig -o /boot/grub/grub.cfg
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-linux
Found initrd image: /boot/initramfs-linux.img
Found fallback initrd image(s) in /boot: initramfs-linux-fallback.img
done

Now when the VM boots up, it will launch GRUB first. GRUB displays a nice menu that lets us select which OS to run (only one in this case), and we can also launch different versions of the Linux kernel from the menu.

Finishing Up

Okay, so I guess we really appreciate the GUI installers for Linux now! But hey, at least we’re done. Go ahead and reboot your VM with:

[root@archiso /]＃ exit
[root@archiso /]＃ shutdown -r now

Powering Back On

If your VM gets shut off for some reason (say, it doesn’t come back online after the last step), you will need to start it again. Log into gojira and then start your VM with virsh:

[gojira:~]$ virsh list --all
 Id    Name                           State
 ----------------------------------------------------
  8     mmalensek-vm1                 shut off

[gojira:~]$ virsh start mmalensek-vm1
Domain mmalensek-vm1 started

If everything went well, you should be able to ssh to the VM’s static IP address from gojira. If not, use your VNC console to debug (most likely network issue…).

Accessing Your VM: macOS/Linux

For general use, it’ll be easier to just ssh into your VM rather than using the VNC console – it’s much faster that way. You can set up your ssh configuration to do this. Edit ~/.ssh/config on your local machine, not the VM (create it if it doesn’t exist already) and fill in the following:

your-username should be your CS login
Replace XXX with your 326 ID number / VM IP address
Change vm-hostname to the hostname of your VM

Host gojira
    User your-username
    ProxyJump your-username@stargate.cs.usfca.edu

Host vm-hostname
    User your-username
    hostname 192.168.122.XXX
    ProxyJump your-username@gojira

Test out the connection by running ssh vm-hostname. Type in your VM user account’s password, and you should be logged in. Note that you cannot log in as ‘root’ this way. To make our life even easier, we should copy over our ssh public key:

[matthew@silicon ~]$ ssh-copy-id magical-unicorn
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/Users/matthew/.ssh/id_rsa.pub"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
mmalensek@192.168.122.282s password:

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'magical-unicorn'"
and check to make sure that only the key(s) you wanted were added.

Now I can log in instantly without my password! This comes in handy when we want to upload/download files from the VM as well. Let’s assume you’re using Cyberduck to transfer files. You will simply forward the VM’s ssh port to your local machine like so:

[matthew@silicon ~]$ ssh -N magical-unicorn -L 2222:localhost:22

Now, when configuring Cyberduck, you’ll enter localhost as the server and 2222 as the port:

Accessing Your VM: Windows

Once again, if you’re using WSL you can follow the Linux directions. However, if you are using MobaXterm, we will use the third tunnel we configured earlier to get to our VM.

Click the Session button in the MobaXterm toolbar, then choose SSH. Then enter:

Hostname: localhost
Username: your-username
Port: 2223

We can also transfer files from MobaXterm. Click the Session button in the toolbar, then choose SFTP. The parameters will be the same as above (localhost, your-username, 2223). Connect, and you should see the files on your VM.

Finalizing Your VM Installation

The last piece of the puzzle required to finish the lab is to add the class ssh key to the root account of your VM. This will let us know that you finished the lab successfully. To do this, log into your VM:

[silicon:~]$ ssh magical-unicorn
Last login: Mon Jan 21 17:00:28 2019 from 192.168.122.1

# Let's switch to root:
[mmalensek@magical-unicorn ~]$ su -
Password:
                                              # scroll --> -->
[root@magical-unicorn ~]＃ wget 'https://www.cs.usfca.edu/~mmalensek/cs326/schedule/materials/class_id_rsa.pub'
Saving to: ‘class_id_rsa.pub’
2019-01-21 17:09:26 (21.1 MB/s) - ‘class_id_rsa.pub’ saved [402/402]

# Then, after that:
[root@magical-unicorn ~]＃ mkdir .ssh
[root@magical-unicorn ~]＃ cat class_id_rsa.pub >> .ssh/authorized_keys
[root@magical-unicorn ~]＃ rm class_id_rsa.pub

This will let our grading script run to verify you completed the lab. If you have time left, you can move on to the next sections to set up some additional OS components.

Installing Software

Over the course of the semester, you’ll probably want to install more software packages on your VM. Package management is provided by the pacman utility. NOTE: you must be root to install packages (or you can also install sudo… see below).

# First, switch over to the root user:
[mmalensek@magical-unicorn ~]$ su -

# To search for packages use -Ss:
[root@magical-unicorn ~]＃ pacman -Ss some-package-name

# To install a package, use -Sy:
[root@magical-unicorn ~]＃ pacman -Sy some-package-name

# Updating Arch linux (will install all new package updates):
[root@magical-unicorn ~]＃ pacman -Syu

Installing Sudo (Optional)

You might be surprised to find out the sudo command is a package that you have to install first on Arch Linux. To set it up, you’ll need to install the package and configure the ‘sudoers’ file:

[root@magical-unicorn ~]＃ pacman -Sy sudo

# Edit the sudoers file with the following command:
[root@magical-unicorn ~]＃ visudo

# Scroll down near the bottom of the the sudoers file (type 'G' in vim) and uncomment
# this line (just remove the # before %wheel):
## Uncomment to allow members of group wheel to execute any command
%wheel ALL=(ALL) ALL

# Finally, add your regular user account to the 'wheel' group:
[root@magical-unicorn ~]＃ gpasswd -a mmalensek wheel
Adding user mmalensek to group wheel

You will have to log out of your user account and back in before the changes take effect. Then you can use sudo as usual:

[mmalensek@magical-unicorn ~]$ sudo pacman -Sy htop

Installing a GUI (Optional)

By default, Arch Linux does not come with a GUI. Just like anything else on the system, you have to install it yourself! Luckily, this isn’t that difficult.

The standard display server on most Linux distributions is called X.org or just Xorg. We also need to install a desktop environment. In this case, we’ll go with something lightweight: Xfce. Other things we’ll need:

Fonts, otherwise everything is going to look bad
A web browser (Firefox)
An editor (VS Code)

Feel free to experiment with other Desktop Environments and software packages if you’d like!

# Let's batch all these installations together:

[root@magical-unicorn ~]＃ pacman -Sy xorg xfce4 firefox code

( ... tons of packages install ... )

Next, we need to tell Xorg what to run when we start the GUI. Be sure you are logged in as your regular user (not root) and then open ~/.xinitrc in your text editor. Add the following to the file:

exec startxfce4

Save the file, then run startx from your console. The Xfce GUI should appear!