Lab 5: exec() and the shell

In this lab, we’ll make the command line shell (sh) more useful by implementing support for shell scripts. A shell script is simply a text file filled with commands that will be interpreted and executed by the shell, one by one. If you’ve ever looked at an executable python script, you might see something like:

#!/usr/bin/python3 -s

(python code follows)

The #! is called a shebang and tells the program loader that the file it is trying to load is a script. In fact, it does the same thing with the standard ELF executables we run on Linux or xv6 – a valid ELF binary starts with 7f 45 4c 46. (Verify this by running hexdump -C on a program and checking the first line… you’ll see .ELF in the ASCII display on the right). Fun fact: 7f is DEL, an unprintable character, to help us distinguish ELF files from plain text. If you are using a Windows or macOS derivative, then your executables are in a completely different format (PE and Mach-o, respectively)!

Many standard Unix utilities are actually just shell scripts. Check out view, for instance:

#!/usr/bin/sh

# run vim -R if available
if test -f /usr/bin/vim
then
  exec /usr/bin/vim -R "$@"
fi

# run vi otherwise
exec /usr/libexec/vi -R "$@"

All it does is run vim or vi with the -R flag set and then passes its command line arguments through ("$@"). So basically view is a shell script that runs vi(m) in read-only mode. When /usr/bin/view is run, the program loader determines that it is a script, and in turn executes /usr/bin/sh /usr/bin/view <args>. When the interpreter runs the script, it will ignore the #! because it is a comment. Maybe this is why Python comments begin with #?

Our goal for this lab is to modify the kernel to support executing shell scripts, and make any necessary modifications to sh so that it can react appropriately to being passed a script as its command line argument.

Modifying kexec

You will find the full kernel implementation of the sys_exec system call in kernel/exec.c, named kexec. One of the first things it does is read the ELF header from the file that was passed in. To add scripting support, you will:

  1. Check for #! at the beginning of the file that was passed in to kexec.
    • Hint: You’ll need to use readi to read bytes from the file’s corresponding inode (more about that later in the semester!). Since #! is two bytes, read two bytes from the file.
  2. If you found a shebang, then you need to determine the interpreter next. If the line is #!/usr/bim/sh then /usr/bin/sh is the interpreter. On our OS, sh lives in the root directory, so a script should start with #!/sh.
  3. Use readi again to get the interpreter. Note that unlike system calls you are used to like read, the readi function does not keep track of the last read you did. So you will need to continually update its read offset (off argument).
  4. Make sure you use an appropriate value for your read size; check kernel/param.h for hints.
  5. Even though the line we are reading will end with a \n newline character, that isn’t a valid part of the interpreter name. You’ll need to remove it.
  6. Finally, now that we know that we have a script and have extracted its interpreter, you should kexec the interpreter.
    • Hint: You will need to build a new argv array to pass to kexec. Be sure to unlock and end any file system log operations (mimic what kexec does when it cleans up).

When you are done, create a new shell script to and copy it into the VM’s disk during mkfs like you did in the previous lab. Start up your OS and try to execute the script. If everything worked, then it will seem like nothing happened, that’s actually a good thing — it means that a new instance of sh was executed. To confirm, press CTRL+P to view active processes and you should get something like this:

init: starting sh
$ /script.sh
$
1 sleep  init
2 sleep  sh
3 sleep  sh       <-- a wild second shell appears!

Press CTRL+D to send an EOF to the shell and you will be dropped back into the original, outer shell. You are ready to move on and add support for scripting to the shell!

[!] If you got exec /script.sh failed, then you may need to retrace your steps and do some debugging to determine why the script was not executed.

Modifying the shell

Now that the hard part is over, we just need to tweak sh a bit so that it can react appropriately to having command line arguments passed in. We need to:

  1. Modify sh so that it accepts command line arguments
  2. If there IS a command line argument passed in, then we should use our newfound knowledge of I/O redirection to make sh read from the script file instead of STDIN. You know what to do!
  3. If you are more of an instant gratification type, running a script after the previous two steps will sort of work. You will notice two issues: the shebang line gets interpreted as a command, and the prompt ($) prints everywhere, making the output look messy.
  4. Add basic support for comments by ignoring any command that begins with # (shebangs will now be ignored)
  5. When a script is passed into the shell, set a flag to indicate scripting is turned on and disable the $ prompt from printing if that is the case.

If everything worked, then you should be able to do something like this:

init: starting sh
$ /script.sh
Executing script: /script.sh
hello world
this is my script!
.              1 1 1024
..             1 1 1024
README.md      2 2 55
cat            2 3 36016
echo           2 4 34896
forktest       2 5 16944
grep           2 6 39456
init           2 7 35368
kill           2 8 34832
ln             2 9 34656
ls             2 10 37960
mkdir          2 11 34904
rm             2 12 34880
sh             2 13 58528
stressfs       2 14 35760
usertests      2 15 184240
grind          2 16 50648
wc             2 17 36960
zombie         2 18 34256
logstress      2 19 36792
forphan        2 20 35648
dorphan        2 21 35096
script.sh      2 22 63
console        3 23 0
bye!
$

Here are the contents of script.sh:

#!/sh

echo hello world
echo this is my script!
ls /
echo bye!

Congratulations, you just added support for a totally new executable format to your kernel, AND added shell scripting support to both it and the shell. Give yourself a well-deserved high five.

Grading and Submission

Once you are finished, check your changes into your OS repo. Then have a member of the course staff take a look at your lab to check it.

To receive 80% credit:

To receive full credit for this lab:

Check off procedure: