Homework 2: Recreating a memory effect

So I am finally writing up homework 2. After weeks of procrastination, fifteen hours of little progress and subsequent panic, and finally three hours of working results, I finished. I used JOONE to recreate the primacy and recency effect, though only succeeding in creating the primacy effect. I used a new kind of layer: the delay layer. This layer allowed me to give an input node a "delay" on being processed through the network. This worked by having undelayed nodes being processed more than the delayed nodes. A parameter, called taps, allowed me to set the number of cycles by which the process would be delayed.

My main difficulty was figuring out how to measure the results. The main idea is "how did it learn a set of data?". I gave it a set of values on the input node, and had a teacher signal give it a result, then tested it with various levels of delay. I had a large set of data so as to keep the learning fairly randomized, so I had two hidden layers as well. I then ran it through just 100 epochs for each training set. This seems somewhat consistent with a basic learning paradigm, where we would be exposed to words or some other stimulus and our neurons would synapse many times even in the few seconds that one might be exposed to a stimulus in a primacy/recency task. I was able to get a consistent result where the undelayed (first stimulus in a series) item had the lowest RMSE, which I used as my baseline for measurement. My problem came when I tried to determine how to get a recency effect. Primacy and recency are probably the result of two different processes, so a different approach is needed to get a recency result. Particularly, one needs to be able to recreate a long term and short term memory system and have some way to distinguish the recall between these two. Perhaps two input "networks" that have already been trained could be transferred to an output node via a learning "switch" that has the short term more salient and accessible, but also has things learned in there not quite so permanent. There would also need to be transfer between STM and LTM. A complicated proposal for sure!