Important Note:

• This code provides an abstract Environment class, as well as a NbyMVacuumEnvironment class.
• This code provides an abstract Agent class, as well as RandomAgent and TableDrivenAgent classes, as starting points for you.

1. Start by running the existing code on the NbyMVacuumEnvironment, just to get comfortable with how it works. Here's a sample session:

   >>> import environment
   >>> import sampleAgent
   >>> v=environment.NbyMVacuumEnvironment()
   >>> ag = sampleAgent.RandomAgent(sampleAgent.agent_actions)
   >>> v.add_agent(ag)
   
   >>> v.run(10)
   Initial room config:  {(1, 2): 'Dirty', (1, 1): 'Clean', (0, 2): 'Dirty', (1, 0): 'Dirty', (0, 0): 'Dirty', (0, 1): 'Dirty', (2, 0): 'Clean', (2, 1): 'Dirty', (2, 2): 'Dirty'}
   Percept: ((1, 0), 'Dirty') Action: Left
   Location: (1, 0) Action: Left
   Percept: ((0, 0), 'Dirty') Action: Left
   Location: (0, 0) Action: Left
   Percept: ((0, 0), 'Dirty') Action: Suck
   Location: (0, 0) Action: Suck
   Percept: ((0, 0), 'Clean') Action: Suck
   Location: (0, 0) Action: Suck
   Percept: ((0, 0), 'Clean') Action: Left
   Location: (0, 0) Action: Left
   Percept: ((0, 0), 'Clean') Action: Suck
   Location: (0, 0) Action: Suck
   Percept: ((0, 0), 'Clean') Action: Suck
   Location: (0, 0) Action: Suck
   Percept: ((0, 0), 'Clean') Action: Suck
   Location: (0, 0) Action: Suck
   Percept: ((0, 0), 'Clean') Action: Suck
   Location: (0, 0) Action: Suck
   Percept: ((0, 0), 'Clean') Action: Left
   Location: (0, 0) Action: Left
   Final room config:  {(1, 2): 'Dirty', (1, 1): 'Clean', (0, 2): 'Dirty', (1, 0): 'Dirty', (0, 0): 'Clean', (0, 1): 'Dirty', (2, 0): 'Clean', (2, 1): 'Dirty', (2, 2): 'Dirty'}
   >>> 
   

   As you can see, the random agent isn't very bright, but it should give you a starting point.

2. Next, run the TableDrivenAgent in the 3x3 environment. Unfortunately, the TableDrivenAgent is not able to guarantee that all of the rooms will be cleaned. (1) Concisely explain why it is that the TableDrivenAgent is unable to accomplish this task.
3. Next, create a subclass of Agent called ModelBasedAgent. The ModelBasedAgent is able to keep a 'model' (or state, or memory) of the environment. (you may choose to implement this however you wish). You will need to implement two methods (you may find that you want helper methods also):
   • __init__()
   • program()
   
   
   (2) Is your ModelBasedAgent guaranteed to clean all rooms in a 3x3 grid environment? If so, show why. If not, show why not.
   
   (3) Is your ModelBasedAgent guaranteed to be optimal? In other words, does it clean all rooms in the minimum possible number of moves? If so, prove that this is the case. If not, explain what limitations of your agent prevent it from solving the problem optimally.
4. Now let's make the problem more challenging: build a ModelBasedAgent that can solve the NxMVacuumEnvironment for other values of N and M. Notice that the values of N and M are NOT available to the agent; the only way it can figure out how large the grid is is through exploration.
   
   Hint: how can an agent figure out whether there is another room to the south/east, given that its percepts tell it only the coordinates of the current room and whether it is clean or dirty?
5. Finally, let's make the problem stochastic.
   
   If we have a particularly dirty environment, it may take several 'Sucks' to clean a room. Simulate this by making a class called ExtraDirtyVacuumEnvironment.
   • This should be a subclass of NByMVacuumEnvironment.
   • You should only need to modify one method: execute_action.
   • Change execute_action so that, half the time when the agent executes 'Suck' in a dirty room, the room's status becomes 'Clean', and half the time it remains 'Dirty'.
   • Lastly, modify your ModelBasedAgent as needed to allow it to be guaranteed to clean all of the rooms in this environment.
     
     
   (4) What changes to your agent are needed in general to deal with stochastic environments?