We have been studying heuristics: "rules of thumb" that our memory system uses to make likelihood judgments. They're not perfect, but they give us a good idea about what's likely. Sure, we could be more accurate if we didn't use them, but we'd also be less efficient. The availability heuristic is particularly interesting: we make a judgment or assumption based on whatever is there. We could be given a totally unrelated figure and "latch onto it", using it in some unrelated guess.
It's as I study the issues like these of the memory system that I realize part of why we have the scientific method and, in math, the axiomatic method: we need to prevent the errors in human judgment affecting our ability to learn more about the world. If science was based on introspection and just figuring things out based on the way things appeared to be, we'd have so many false assumptions about the world. Our memory system is incredible, but it can also lead us to very powerful mistakes.
Friday, April 11, 2008
Illusory Correlations
We're studying a bit about illusory correlations and what especially interested me was the note that the author began with, that while many doctors and patients believe arthritis pain is correlated with weather changes, the scientific data is not present to support this. This interests me, as my fiancée suffers from fibromyalgia, a disease similar to arthritis that causes chronic pain and both of us seem to notice the pain is worse in winter. Yet, I looked at studies and confirmed that there is also no link between weather and fibromyalgia pain. Are we really that bad at seeing correlations? The ebb and flow of pain is constant, but we find a link because everyone else seems to find a link.
It appears that our system is quite susceptible to suggestions from others and "common sense." Mathematics, which is, of course, done by humans, had the quite serious problem that mathematicians were mostly using common sense to make claims about things. This worked when we try to prove 1+1=2, but when we try to prove things that get more complicated, it falls apart. It was as mathematics got more complicated and advanced mathematics became used more and more in practical applications that the axiomatic proof method became widespread. Mathematicians accepted nothing without a proof to go with it. I recall having to prove that any number times 0 is 0. Unfortunately, our memory system is not robust enough to be able to axiomatically and logically prove everything. We must make assumptions and inferences or the world would overwhelm us. Problems like illusory correlations become outweighed by the benefit of having the ability to make inferences. But these problems are why, as scientists, we must not make assumptions about the world without data to back it up.
It appears that our system is quite susceptible to suggestions from others and "common sense." Mathematics, which is, of course, done by humans, had the quite serious problem that mathematicians were mostly using common sense to make claims about things. This worked when we try to prove 1+1=2, but when we try to prove things that get more complicated, it falls apart. It was as mathematics got more complicated and advanced mathematics became used more and more in practical applications that the axiomatic proof method became widespread. Mathematicians accepted nothing without a proof to go with it. I recall having to prove that any number times 0 is 0. Unfortunately, our memory system is not robust enough to be able to axiomatically and logically prove everything. We must make assumptions and inferences or the world would overwhelm us. Problems like illusory correlations become outweighed by the benefit of having the ability to make inferences. But these problems are why, as scientists, we must not make assumptions about the world without data to back it up.
JOONE
So we began working with JOONE: The Java Object Oriented Neural Environment. Unfortunately, I had an emergency appendectomy during the week that the class worked on it, so I didn't have an opportunity to explore it with my peers. But I have had a chance to look through it and I've noticed a few things:
1) It violates Apple's Human Interface Guidelines! Of course, it's meant to be a cross-platform program, but having gotten used to a certain way of programs working, it's a bit hard to get used to. There are, of course, worse things that a program could do.
2) It is open source and the GUI editor we're using is just a front end to a very powerful Java programming environment. Unfortunately, most of my programming experience is in Objective-C, so it will be a bit of a learning curve for me to try to create a network just from the APIs that JOONE has, but I hope to try to give my Java chops a spin again.
3) The GUI is a bit hard to get used to and the complete guide is mostly based on the actual programming environment rather than the GUI front end, but the guide was quite useful in understanding the underlying frameworks of the network environment. It should be helpful and especially interesting were the different layers: a variety of different functions and synapses are available for different kinds of networks. I really hope I can spend a bit of time playing with some of the other kinds of layers and synapses to see what kinds of results it yields.
JOONE can run off of any UNIX environment, so I've installed it on my Linux server and I've been playing around with the source code on there to try different tricks in the builds, though most of my attempts at tricks just made the program not compile and even once impressively caused a kernel panic in my machine. I'm keeping the stable, working version running on my Mac and using my Linux server as my development test bed.
1) It violates Apple's Human Interface Guidelines! Of course, it's meant to be a cross-platform program, but having gotten used to a certain way of programs working, it's a bit hard to get used to. There are, of course, worse things that a program could do.
2) It is open source and the GUI editor we're using is just a front end to a very powerful Java programming environment. Unfortunately, most of my programming experience is in Objective-C, so it will be a bit of a learning curve for me to try to create a network just from the APIs that JOONE has, but I hope to try to give my Java chops a spin again.
3) The GUI is a bit hard to get used to and the complete guide is mostly based on the actual programming environment rather than the GUI front end, but the guide was quite useful in understanding the underlying frameworks of the network environment. It should be helpful and especially interesting were the different layers: a variety of different functions and synapses are available for different kinds of networks. I really hope I can spend a bit of time playing with some of the other kinds of layers and synapses to see what kinds of results it yields.
JOONE can run off of any UNIX environment, so I've installed it on my Linux server and I've been playing around with the source code on there to try different tricks in the builds, though most of my attempts at tricks just made the program not compile and even once impressively caused a kernel panic in my machine. I'm keeping the stable, working version running on my Mac and using my Linux server as my development test bed.
Thursday, March 20, 2008
Sigmoid Function
So with my math background, I'm always interested in functions and why they do the things they do. Since we're using a sigmoid function for our feedforward neural networks, I thought I'd investigate this function a bit.
One of the most useful things about this function is its differentiability. That is to say, the first derivative of the function is an easily expressible; you can easily figure out how this function changes as its input changes.
The derivative of the function
Where f(x) would be the function of the net input in our neural network
So, we would differentiation the function of the negative net input and multiply that by (1-(-net input)) in order to get the derivative of our sigmoid function. This result tells us how the function changes as the input changes. The derivative tells us the rate of change. This becomes useful because the easy calculation of f(x)(1-f(x)) can tell us exactly how our network is changing for any given f(x).
The sigmoid function is also useful because it is bounded between 0 and 1. If an equation doesn't have an upper bound, real world numbers can push the function into some strange locations that can cause some damage to the usefulness of a network.
A study of sigmoid functions will sometimes lead you to look at some of the other types of sigmoid functions. I'd be interested to see how useful a double sigmoid function is. It still squashes, but it can be bounded between -1 and 1, maintaining our original values. A double sigmoid function essentially bonds two sigmoid functions together. It has the useful property of providing normalization to a function. It has some problems because it has four inflection points rather than one, so the curve will change signs several times. However, this only related to the second derivative; the first derivative would not change signs. The only issue would be that the values might not be perfect at some of these inflection points.
Below is the formula for a specific double sigmoid function and its result in Mac OS X's grapher application:
Not the flat point at about (1,0). This is the "bonding" location between the two sigmoid functions.
One of the most useful things about this function is its differentiability. That is to say, the first derivative of the function is an easily expressible; you can easily figure out how this function changes as its input changes.
The derivative of the function
Where f(x) would be the function of the net input in our neural network
So, we would differentiation the function of the negative net input and multiply that by (1-(-net input)) in order to get the derivative of our sigmoid function. This result tells us how the function changes as the input changes. The derivative tells us the rate of change. This becomes useful because the easy calculation of f(x)(1-f(x)) can tell us exactly how our network is changing for any given f(x).
The sigmoid function is also useful because it is bounded between 0 and 1. If an equation doesn't have an upper bound, real world numbers can push the function into some strange locations that can cause some damage to the usefulness of a network.
A study of sigmoid functions will sometimes lead you to look at some of the other types of sigmoid functions. I'd be interested to see how useful a double sigmoid function is. It still squashes, but it can be bounded between -1 and 1, maintaining our original values. A double sigmoid function essentially bonds two sigmoid functions together. It has the useful property of providing normalization to a function. It has some problems because it has four inflection points rather than one, so the curve will change signs several times. However, this only related to the second derivative; the first derivative would not change signs. The only issue would be that the values might not be perfect at some of these inflection points.
Below is the formula for a specific double sigmoid function and its result in Mac OS X's grapher application:
Not the flat point at about (1,0). This is the "bonding" location between the two sigmoid functions.
Tuesday, March 18, 2008
Counterfactual Reasoning and rationality
So during class the other day we got into a brief discussion about whether humans are truly "rational" beings. This question came out of the discussion of counterfactual reasoning, the idea that we can rationalize events mainly in two categories, "it could have been worse" or "if only...". It seems that we often reason against facts in order to fit the way we want the world to be. Is this necessarily "irrational", though? Although our memory system is not perfect, it wouldn't make evolutionary sense for it to be. A perfect memory system would be quite resource intensive, and we'd also likely run in to problems with our retrieval system having difficulty finding relevant information. Our system does an excellent job of recalling important information. We can recall information important to survival and information that is important from an evolutionary standpoint.
I wonder, however, if memory for social activities is selected for as strongly as memory for things necessary for survival. Although there is social interaction among our evolutionary ancestors, it seems as though it becomes much more important for humans because of language development. We can interact in social situations in ways that other species cannot do. But since language is a comparatively new development in the evolutionary sense, is our memory as well suited for language memory as it is for other things? Our memories can sometimes cause problems in social interactions. We sometimes place a high standard on others to remember things about us and about our lives. But our memory system isn't perfect. We might not be able to remember things about someone's children or something else in his or her life. Although we do fairly well with social interaction, our imperfect memory system can cause problems.
I wonder, however, if memory for social activities is selected for as strongly as memory for things necessary for survival. Although there is social interaction among our evolutionary ancestors, it seems as though it becomes much more important for humans because of language development. We can interact in social situations in ways that other species cannot do. But since language is a comparatively new development in the evolutionary sense, is our memory as well suited for language memory as it is for other things? Our memories can sometimes cause problems in social interactions. We sometimes place a high standard on others to remember things about us and about our lives. But our memory system isn't perfect. We might not be able to remember things about someone's children or something else in his or her life. Although we do fairly well with social interaction, our imperfect memory system can cause problems.
Saturday, February 16, 2008
Concept Modeling
So we began working on building our own cognitive neural network models using a simple spreadsheet. We're modeling the ways different patterns and features are mapped and how a neural network model might adapt to learning situations as it gains more data and begins to understand the weights of the feature nodes.
While trying to better understand this, I stumbled upon the concept of Semantic relatedness, algorithms and other means for determining the relative meaning of other words, especially through the distance two words are from each other in meaning. One of the more powerful ways of determining semantic relatedness is through Google distance, how related two words are in terms of google searches. Specifically, one can enumerate this idea by understanding the number of hits for two search terms and the overlap of two terms.
Thus, an equation has been developed for this:
Where M is the number of google pages searched for and f(x) and f(y) the number of hits for the corresponding search terms.
As I look at this kind of model, I wonder if our mind works similarly, in that we do a proverbial Google search and see how related two words or concepts are by the number of "results" (categories) they fall under and make a judgment based on the kind of overlap there is. I think I'll want to explore semantic similarity and related concepts more.
While trying to better understand this, I stumbled upon the concept of Semantic relatedness, algorithms and other means for determining the relative meaning of other words, especially through the distance two words are from each other in meaning. One of the more powerful ways of determining semantic relatedness is through Google distance, how related two words are in terms of google searches. Specifically, one can enumerate this idea by understanding the number of hits for two search terms and the overlap of two terms.
Thus, an equation has been developed for this:
Where M is the number of google pages searched for and f(x) and f(y) the number of hits for the corresponding search terms.
As I look at this kind of model, I wonder if our mind works similarly, in that we do a proverbial Google search and see how related two words or concepts are by the number of "results" (categories) they fall under and make a judgment based on the kind of overlap there is. I think I'll want to explore semantic similarity and related concepts more.
Concepts: Representing knowledge
The next section reviews the idea of concepts and representing knowledge. Although the book focuses on social knowledge, it's evident that other types of knowledge can be represented. In fact, it's easier for us as psychologists to conceptualize the idea of knowledge about things than the idea of knowledges about people. It's easier to think of the idea and category of "chair" than it is for something like "extrovert."
Concepts are the basis for cognition. Our cognitive processes would be highly inefficient if we had no way to make inferences about an object based on its categorization. When I see a pen, I don't have to spend time trying to ascertain what it does and how to use it. Even if I've never seen this particular pen before, my experiences with other pens tell me that it's likely to be useful for writing and I would be familiar with how to hold it.
This makes life a great deal simpler than if we didn't have a way to categorize and infer things about objects. It gets messier with people, however. Stereotypes are an excellent example of this. Even though most of us don't like to think of ourselves as constantly stereotyping people and making inferences about people based on initial appearances, I imagine that most of us do. Again, life would be harder if we didn't. We may talk for someone for a few minutes and make some generalizations about how they behave and change the way we will act around them accordingly. If I talk to someone for a while and they misuse grammar and seem to have a low level vocabulary, I'm not likely to strike up a conversation about math. However, a lower level of English ability certainly doesn't imply a low level math ability. Yet, I'm still likely to put such a person in a category of "unable to understand higher math" just because of a subject-verb disagreement!
We're not on the chapter about stereotypes yet, but I'm very interested to learn more about and how we use them in both positive and negative ways.
Concepts are the basis for cognition. Our cognitive processes would be highly inefficient if we had no way to make inferences about an object based on its categorization. When I see a pen, I don't have to spend time trying to ascertain what it does and how to use it. Even if I've never seen this particular pen before, my experiences with other pens tell me that it's likely to be useful for writing and I would be familiar with how to hold it.
This makes life a great deal simpler than if we didn't have a way to categorize and infer things about objects. It gets messier with people, however. Stereotypes are an excellent example of this. Even though most of us don't like to think of ourselves as constantly stereotyping people and making inferences about people based on initial appearances, I imagine that most of us do. Again, life would be harder if we didn't. We may talk for someone for a few minutes and make some generalizations about how they behave and change the way we will act around them accordingly. If I talk to someone for a while and they misuse grammar and seem to have a low level vocabulary, I'm not likely to strike up a conversation about math. However, a lower level of English ability certainly doesn't imply a low level math ability. Yet, I'm still likely to put such a person in a category of "unable to understand higher math" just because of a subject-verb disagreement!
We're not on the chapter about stereotypes yet, but I'm very interested to learn more about and how we use them in both positive and negative ways.
Subscribe to:
Posts (Atom)