README.md

@@ -7,12 +7,13 @@ If you do not have access to a UNIX or UNIX-like environment then you can use th
 [older instructions](https://student.cs.uwaterloo.ca/~cs350/common/Install161.html) to build, run, debug and test in University of Waterloo's linux student environment.
 
 ## Windows Option A
-This is an emulated environment and allows for bash like commands, but you are running windows underneath.
+This is an emulated environment and allows for bash like commands, but you are running windows underneath. You will need to use
+Git Bash for all commands stated in this README and to run any of the provided scripts.
 * Install [Git for Windows](https://gitforwindows.org/)
 * Install [docker](https://docs.docker.com/desktop/windows/install) for Windows.
 
 ## Windows Option B
-This setup is for if you'd like to have the full linux environment, WSL2 is just a linux VM running on your computer!
+This setup will let you have a full linux environment on your windows machine, WSL2 is just a linux VM running on your computer!
 * Install [WSL](https://docs.microsoft.com/en-us/windows/wsl/install).
 * Install [docker](https://docs.docker.com/desktop/windows/install) for WSL.
 
@@ -24,8 +25,8 @@ This setup is for if you'd like to have the full linux environment, WSL2 is just
 * This repository assumes that you have a copy of the OS/161 source code on the local machine.
 * You **should** have a remote repository to manage your OS/161 source code and move between the local and linux.student environments. 
 
-* If you do not currently have a working copy of os161, you can retrieve your own by copying the
-`os161.tar.gz` file within `os161-container/dependencies` directory in the cs350-container repository.
+* If you do not currently have a working copy of OS/161, you can retrieve your own by copying the
+`os161.tar.gz` file within `os161-container/dependencies` directory from the cs350-container repository.
 ```
 cp os161-container/dependencies/os161.tar.gz ~/
 cd ~/
@@ -40,7 +41,7 @@ sudo ./install.sh
 ```
 In a terminal from within the project directory. This script will pull the assignments reposiroty and build it into a local copy of the os161-runner image.
 
-## Running OS161 in the cs350-container
+## Running OS/161 in the cs350-container
 To start using the container you use the run.sh script, which takes one argument -- the directory of your working code repository.
 
 ```
@@ -52,6 +53,14 @@ assignment number. You can run these commands from the shell container. These fu
 # build_kernel ASST0
 # test_kernel ASST0
 ```
+
+To fully evaluate youre kernel with the automated testing scripts you may use
+```
+# ./run_assignment.sh ASST#
+```
+This will run `build_kernel` and `test_kernel` as well as evaluate the output produced by `test_kernel` (The raw output for build and test kernel can be found in /logs).
+
+
 If you would like to run the kernel normally (no scripts), once built, you can run the kernel
 ```
 # cd /os-compile
@@ -71,15 +80,14 @@ fails that is okay (means there was no build to begin with), just run `build_ker
 
 * Now in another terminal, move to the project directory and use the `./connect.sh` script to connect to the previosuly running container:
 ```
-# cd /os-compile/
-# cs350-gdb kernel
 # cd os-compile/
 # cs350-gdb kernel
 
 GNU gdb 6.6
 Copyright (C) 2006 Free Software Foundation, Inc.
 ...
-(gdb) dir /kernel/kern/compile/ASST0
+(gdb) dir /os161-1.99/kern/compile/ASST0
+(gdb) target remote unix:.sockets/gdb
 ...
 ```
 You will need to use the command `dir /kernel/kern/compile/ASST#` where `#` is the assignment number. This will make sure to load the symbols
@@ -87,17 +95,37 @@ of the kernel. Symbols are the names we have given functions and variables. This
 
 Note: If run.sh fails it means there is still a os161-runner container running. Use `docker kill os161` to kill it.
 
-## Running other assignments
-You may point the `run.sh` in any directory, the script makes sure that the directory is shared between the container and you. Allowing you to
-continue to code and work on the files within the directoy and have the container running and seeing those changes. Further instructions will
-be given to the scripts used to run and evaluate userspace code.
+## Building and installing OS/161 userspace programs
+User-level programs are built and installed from the directory /os161-1.99/. 
+To build and install all of the user-level programs, in the container
+
+You can type
+```
+cd /os161-1.99/
+bmake
+bmake install
+```
+User-level programs are installed under $OS161TOP/root/ in the bin, sbin, testbin and uw-testbin directories. This is done automatically
+by the build_kernel command.
+
+## Running Linux userspace assignments
+To run the Linux userspace programs in the container, that are part of the progrmaming assignment, you must start the container using the `run.sh` script, which takes one argument -- the directory that holds your code for that userspace programming assignment, that is, the `c` files for the programming assignments. For example,  
 
-When mounting other directories that are not os161 note that the `build_kernel` and `test_kernel` functionality will not work.
+```
+sudo ./run.sh ~/cs350-teststudent/a0
+```
+Once you are inside the container, you can run
+```
+./run_assignment.sh ASSTUSER#
+```
+ where `#` represents the assignment number. 
+
+It is important to note that: when mounting other directories that are not OS/161, the `build_kernel` and `test_kernel` functionality will not work.
 
 ## [Layout](Layout)
 There are some noted directories within the container
 * `/kernel` holds YOUR assignment code
-* `/os161-1.99` holds unchanged os161 kernel code
+* `/os161-1.99` holds unchanged OS/161 kernel code
 * `/os-compile` holds the compiled version of the code
 * `/assignments` holds testing and evaluation scripts used for each assignment
 
@@ -108,3 +136,10 @@ Within the `/assignments` directory there are subdirectories for each assignment
 
 If the assignments repo has changed, either due to assignments being changed or bugs found (found at `os161-container/assignments`) you will need to just run `sudo ./install.sh` again.
 
+
+## Common Issues
+
+**Commands do not work and also there is randomly ^M, strings within the output:**
+This is due to windows placing return characters in the strings that get
+passed to the scripts. Please use git bash or a terminal that does UNIX
+string passing.

install.sh

 #!/usr/bin/env bash
+echo "Updating our own directory"
+git pull origin master
+
 echo "Cloning/Updating assignment directory"
 git clone https://git.uwaterloo.ca/krhancoc/cs350-assignments.git os161-container/assignments > /dev/null 2> /dev/null
 cd os161-container/assignments > /dev/null
 git pull origin master > /dev/null 2> /dev/null
 cd - > /dev/null
+
 echo "Building CS350 container"
 docker build -t os161-runner ./os161-container
 

os161-container/Dockerfile

@@ -10,12 +10,16 @@ RUN mkdir -p /os161/tools/bin; \
     mkdir -p /os161/tools/share/man/man1; \
     mkdir -p /os161/tools/share/mk;
 
+# Install a newer version of config.guess with ARM support. Pin it to a specific commit for stability.
+ADD https://git.savannah.gnu.org/cgit/config.git/plain/config.guess?id=20403c5701973a4cbd7e0b4bbeb627fcd424a0f1 /tmp/config.guess
+RUN chmod +x /tmp/config.guess
 
 COPY dependencies/os161-binutils.tar.gz /binutils.tar.gz
 RUN mkdir /binutils-src; \
     tar -xf /binutils.tar.gz -C /binutils-src --strip-components 1
 WORKDIR /binutils-src
 
+RUN cp -f /tmp/config.guess .
 RUN find . -name '*.info' | xargs touch; \
     ./configure --nfp --disable-werror --target=mips-harvard-os161 --prefix=/os161/tools; \
     make -j 4 && make install; \
@@ -30,6 +34,7 @@ RUN mkdir /gcc-src; \
     tar -xf /gcc.tar.gz -C /gcc-src --strip-components 1
 WORKDIR /gcc-src
 
+RUN cp -f /tmp/config.guess .
 RUN CFLAGS="-std=gnu89" ./configure -nfp --disable-shared --disable-threads \
 	--disable-libmudflap --disable-libssp --target=mips-harvard-os161 --prefix=/os161/tools; \
     make -j 4 && make install; \
@@ -65,16 +70,20 @@ COPY dependencies/os161-gdb.tar.gz /os161-gdb.tar.gz
 RUN mkdir /os161-gdb/ && tar -xf /os161-gdb.tar.gz -C /os161-gdb --strip-components 1
 
 WORKDIR /os161-gdb
+RUN cp -f /tmp/config.guess .
 RUN CFLAGS="-std=gnu89" ./configure --target=mips-harvard-os161 --prefix=/os161/tools --disable-werror && \
     make && make install
 
 WORKDIR /
 RUN rm -rf /os161-gdb /os161-gdb.tar.gz
 
+RUN rm -f /tmp/config.guess
 RUN cd /os161/tools/bin && sh -c 'for i in mips-*; do ln -s /os161/tools/bin/$i /os161/tools/bin/cs350-`echo $i | cut -d- -f4-`; done'
 
 RUN python3 -m pip install -U prettytable
+RUN apt update && apt install sudo
 COPY evaluate.py /evalaute
+RUN useradd -ms /usr/sbin/nologin user
 COPY helpers.sh /helpers.sh
 RUN echo "source /helpers.sh" >> ~/.bashrc
 COPY run_assignment.sh /run_assignment.sh

os161-container/evaluate.py

@@ -50,8 +50,9 @@ class Report:
 class Test:
     def __init__(self, log):
         log = [l for l in log if len(l) > 0]
-        self.name = log[0].split("=")[1]
-        self._log = "\n".join(log[4:])
+        self.name = log[0].split("=")[1].strip("<").strip(">")
+        # Remove the TEST=#### line
+        self._log = "\n".join(log[1:])
 
     def results(self):
         return self._log
@@ -68,9 +69,15 @@ class Namespace:
 
 def main(args):
     tests = []
-    with open(args.log, "r") as f:
+    encoding_error = False
+    with open(args.log, "rb") as f:
         log = f.read()
-        log.strip()
+        try:
+            log.decode(encoding='utf-8', errors='strict')
+        except:
+            encoding_error = True 
+        log = log.decode(encoding='utf-8', errors='ignore')
+        log = "\n".join([ l.strip() for l in log.split("\n") ])
         # Remove one cause topline is a split so would get empty line before
         tmp = log.split(">SPLIT<")[1:]
         for t in tmp:
@@ -89,6 +96,8 @@ def main(args):
     report = Report()
     for eval_func, out_of in verify.RUBRIC.items():
         name, mark, comments = eval_func(tests, helpers)
+        if encoding_error:
+            comments = "We could not decode your log into ASCII, this could lead to undefined behaviour everywhere, you have a memory corruption most likely in your code"
         report.add(name, mark, out_of, comments)
     print(str(report))
     report.to_csv("/logs/report.csv")

os161-container/helpers.sh

@@ -24,10 +24,12 @@ build_k_helper() {
 	cd $OS161_DIR
 	mkdir /os-compile 2> /dev/null
 	cp -r /kernel/kern/* $OS161_DIR/kern/
+	rm -rf $OS161_DIR/kern/compile/* 2> /dev/null
 	set -e
 	# Configure - step 1
 	echo "[CS350] $1 Configuring Kernel..."
 	touch $LOGS/configure.log
+	truncate -s 0 $LOGS/configure.log
 	./configure --ostree=/os-compile --toolprefix=mips-harvard-os161- >> $LOGS/configure.log
 	cd kern/conf
 	./config $1 >> $LOGS/configure.log
@@ -36,12 +38,19 @@ build_k_helper() {
 	cd ../compile/$1
 	echo "[CS350] $1 Making Kernel..."
 	touch $LOGS/depend.log
-	bmake depend >> $LOGS/depend.log 2>> $LOG/depend.log
+	truncate -s 0 $LOGS/depend.log
+	bmake depend >> $LOGS/depend.log 2>> $LOGS/depend.log
 
 	# Make and install - step 3
 	touch $LOGS/make.log
-	bmake WERROR= >> $LOGS/make.log 2>> $LOG/make.log
-	bmake install >> $LOGS/make.log 2>> $LOG/make.log
+	truncate -s 0 $LOGS/make.log
+	bmake WERROR= >> $LOGS/make.log 2>> $LOGS/make.log
+	bmake install >> $LOGS/make.log 2>> $LOGS/make.log
+
+	cd $OS161_DIR
+	echo "Building Userspace programs" >> $LOGS/make.log
+	bmake WERROR= >> $LOGS/make.log 2>> $LOGS/make.log
+	bmake install >> $LOGS/make.log 2>> $LOGS/make.log
 
 	cd /os-compile/
 	cp /sys161.conf .
@@ -71,6 +80,7 @@ test_kernel() {
 		TOTAL=$(wc -l < $f)
 		DONE=0
 		touch $LOG
+		truncate -s 0 $LOG
 		for p in `cat $f` 
 		do
 			echo ">SPLIT<" >> $LOG
@@ -81,7 +91,6 @@ test_kernel() {
 		done
 	done
 	cd - > /dev/null
-	/evalaute $LOGS/test_public.log /assignments/$1
 }
 
 

os161-container/run_assignment.sh

@@ -9,4 +9,4 @@ source /helpers.sh
 
 /assignments/$1/run.sh $1
 
-/evalaute $LOGS/test_public.log /assignments/$1
+PYTHONDONTWRITEBYTECODE=TRUE /evalaute $LOGS/test_public.log /assignments/$1

run.sh

@@ -15,11 +15,11 @@ fi
 
 mkdir -p "logs"
 chmod a+rwx "logs"
-KERNFILE=$1
+KERNFILE="$1"
 docker run -it --rm \
-	-v `realpath "$KERNFILE"`:/kernel \
-	-v `realpath "os161-container/assignments"`:/assignments \
-	-v `realpath "logs"`:/logs \
+	-v "`realpath "$KERNFILE"`":/kernel \
+	-v "`realpath "os161-container/assignments"`":/assignments \
+	-v "`realpath "logs"`":/logs \
 	--entrypoint bash \
 	--name os161 \
 	os161-runner