Assignment 1: Python and Domain Warmup

Due Date: Feb 11th

This assignment is meant to get you up to speed with Python and also to provide you with experience in the three primary domains we'll be working with this semester. For all assignments, you must use the provided template code as a starting point.

(30%) Word frequencies. One of the primary domains we'll work with this semester is text. The most common approach to dealing with large bodies of text is statistical, which requires counting the number of words in a document.
Write a Python function that can take a string as input and create a dictionary that maps each word in the string to the number of times it occurs in the string. The function should provide the user with the option of converting all words to lower case and also stripping off punctuation. You should also provide a "main" method that takes command-line arguments indicating whether stripping or conversion is desired, as well as the name of a file to use as input. It should also provide the user with the ability to pickle the dictionary to a file for later use. Skeleton file to start with: wordfreq.py
(40%) Graphs. Graphs are one of the most commonly used data structures in computer science. In this assignment, we'll represent a graph as an Adjacency list. This will be implemented as a dictionary that maps a vertex to a list, which contains each of its neighbors, along with associated data such as edge weight.
- To begin, write a Python program that can read in graph data as provided in the sample inputs and construct an adjacency list. For each edge in the graph, there are two numbers: distance and travel time. Both should be stored in a class, along with the source and destination vertex. See the skeleton file for examples.
- Next, implement a method that uses Dijkstra's Algorithm (you can see a visualization here) to find a shortest cost from the intial vertex to all other vertices in the graph. The return value of this function should be a dictionary that maps destination cities to a tuple containing the length of the path, and the vertices that form the path.
- You should also provide a main that takes as input the name of the input file, whether Dijkstra's algorithm is to be computed (including the starting vertex), and whether the graph should be pickled and stored in a file, or else printed to standard output. I have provided a skeleton for you.
- Sample inputs can be found here. This is geocoded data representing actual waypoints and distances in San Francisco.
- Skeleton file to start with: buildGraph.py
(30%) ARFF data. Attribute-Relation File Format (ARFF) is a common format for storing data for use in supervised learning problems. It consists of three sections:
1. The relation being learned about
2. The attributes and their possible values
3. The data itself
For this assignment, you should do the following:
- Write a program that can read in an ARFF file and generate the following data structures:
  - A dictionary mapping the index of an attribute to a dictionary that maps attribute names to either possible values (for nominal variables) or to the strings 'numeric' or 'string' (for those datatypes).
    
    For example, if the first attribute line in the file was:
```
  @attribute outlook {sunny, overcast, rainy}
```
    Your attribute dictionary would contain an entry that looked like:
```
{1: {'outlook' : ['sunny','overcast','rainy']}}
```
  - A list of tuples containing each data instance in the file.
- Often, the last attribute is a classification. For example, in the Breast Cancer data, the last attribute class that the patients fell into (recurrence or nonrecurrence). Given a large section of data, and a number of attributes other than the final classification attribute, we want want to try to determine what the final attribute would likely be (would the patient have a recurrence, for instance). We will write our first learning algorithm that takes in a set of data and a set of attributes, and returns the likely classification. Our learning algorithm will be 0R. 0R is a very simple technique that classifies data based on the most common classification in the training set. For example, for the tennis set, if 9/14 of the instances have the value 'yes' for play. 0R would always predict 'yes' for the classification attribute 'play'. The idea is that this is a baseline to compare later algorithms to; it's the most likely predication without knowing anything else about the problem. Write a method that takes in a set of attributes (ignored!) and a set of data, and returns the most common value for the final attribute.
- Skeleton file to start with: readARFF.py