CS 326 Operating Systems

Project 4: Network File System (v 1.0)

Starter repository on GitHub: https://classroom.github.com/g/LSkRsIFS

For our fourth and final project in CS 326, we will be developing a Linux file system. This file system will operate over the network, allowing client machines to view the files on a remote server.

For a general idea of how this file system will function, imagine working on a project hosted on your Raspberry Pi. You’d like to edit the files on the Pi remotely from your workstation, but doing so requires third-party software or editor plugins. This is where your file system comes in: it will allow you to access any directory on your Raspberry Pi over the network as if was connected like a regular disk. The flow between components looks like this:

P3 Architecture

Server

Your server requires two startup parameters: a port and directory. The server will listen on the specified port for client connections, and serve the files (and subdirectories) from the specified directory. This directory is your file system’s root. For instance, if I run:

$ ./netfs_server /home/mmalensek 5555 

Then the server will start up, listen on port 5555 for incoming socket connections, and will serve my home directory (located at /home/mmalensek) to clients. If no port is specified, then a default will be used.

Notes:

Client

The client also takes two parameters: the server host name (or IP address) and the port to connect to. The following mounts NetFS at /tmp/mount (more details on the -f flag later):

$ ./netfs_client --server=mmalensek-pi.local --port=5555 -f /tmp/mount

Ok, let’s assume we’ve run the commands above. On the server machine, my home directory looks like:

[mmalensek-pi:~]$ ls
exam_solutions  p1.c  projects  test_file.c

Then, on another machine:

[mmalensek-other-pi:~]$ ./netfs_client --server=mmalensek-pi.local --port=5555 /tmp/mount
[mmalensek-other-pi:~]$ cd /tmp/mount
[mmalensek-other-pi:/tmp/mount]$ ls
exam_solutions  p1.c  projects  test_file.c

# Cool, we see the files hosted on the server! We can also read the files:
[mmalensek-other-pi:/tmp/mount]$ cat test_file.c
int main(void) {
    printf("Hello!\n");
    return 0;
}
[mmalensek-other-pi:/tmp/mount]$ 

In other words, the contents of my first Pi’s home directory are available under /tmp/mount/ on my second Pi.

To understand the flow of communication, consider a user that runs the ls command inside our file system. The ls command will call readdir() on the directory, which is handled by the OS Kernel, which then uses our file system client to handle the request. The client sends a message over the network requesting the data from the specified directory, which is received by the server process (running in user space). The server process performs a readdir() on its local file system and send the results back. Note: FUSE allows the file system implementation to run in user space, but the flow is the same conceptually.



P3 ls Demo

Implementation

You will use TCP sockets, along with the usual read() and write() system calls to communicate between the client and server. To implement the client-side file system, we’ll be using the FUSE 3 library. To install these libraries on your Pi:

$ su -
# (enter root password)

$ pacman -Sy fuse-common fuse3

Note: if you haven’t updated your Pi in a while, you may want to run pacman -Syu for a full system update.

FUSE (Filesystem in Userspace) is a library that eases development of Linux Virtual File Systems (VFS). As an added benefit, FUSE is compatible with most of the major Unix-based operating systems, including macOS.

At a minimum, your file system will support read-only access. This means that you must support the following file system operations:

Each of these operations will have a corresponding server request message represented as a struct. For instance, if I perform a readdir on /home/mmalensek/my_stuff, you will transmit the request to the server, handle it via a sub-process, and respond via well-defined message types implemented as structs.

For up to 5 points of extra credit, you can implement write functionality as well.

readdir

On the server side, your readdir implementation will look a lot like Project 1; you will scan through a directory, but instead of simply printing the contents you will transmit them across the network to the client.

getattr

You can get file attributes with the stat function and then transmit the resulting struct directly over the network. You will also need to do some manipulation of these attributes to handle differences in user accounts. After all, there is no guarantee that the client and the server have the same user and group IDs. You will determine the UID on both the client and server side, and then map these UIDs back and forth as necessary. Any other UIDs present on the server will be remapped to ‘root’ on the client.

Since your file system is read-only, you should remove the write bits from file modes. If you decide to implement write support, you can disable this functionality (but you should provide a switch so that we can test it during grading). To remove the write bits, create a bitmask that removes the write bits when ANDed with the original bits. Something like:

stbuf->st_mode = (mode_t) (~0222 & remote_st.st_mode)

open

Implementing the open function will be the easiest: just call open() on the server side, and as long as the function succeeds you should return success on the client side as well. To do this, return a single integer (boolean) to indicate success.

read

Read will require some experimentation to get right, but the starter code provides a reasonable example. Hint: the sendfile function may be useful here; it allows you to copy from one file descriptor to another.

You’ll notice that the client-side read function receives size, offset, and buf parameters. These refer to:

Finally, the function should return the amount of bytes read from the file, which may not match the original size passed in.

On the server side, you’ll need to do a couple things to prepare for the read:

Testing Your Code

At this point, you may be thinking: “hey, I thought we were only supposed to have one Raspberry Pi!”, and you are completely correct. You have a couple options here:

To test your file system, pass in the -f command line flag. This will run the file system in the foreground and print out your log messages so you can monitor what’s going on. If you run without the -f flag, then your file system will run in the background and must be unmounted via:

$ fusermount3 -u /path/to/mount/point

When we grade your code, we’ll start your server on a remote machine and have you connect the Pi as a client. So be sure to test this out while you develop the project! As usual, grading will be interactive and you must demonstrate the functionality with your Pi running Arch Linux.

Test Cases

You can count on a few different test cases:

We’ll provide the files and have your file system serve and read them.

Hints

Here’s some hints to guide your implementation:

Grading

Submission: submit via GitHub by checking in your code before the project deadline. You must include a makefile with your project. As part of the testing process, we will check out your code and run make to build it.

Restrictions: you may use the standard C and FUSE libraries. Other external libraries are not allowed unless permission is granted in advance. Your code must compile and run on your Raspberry Pi set up with Arch Linux as described in class. Failure to follow these guidelines will will result in a grade of 0.

Changelog