the evolving mind

I. SCIENCE IN A NUTSHELL

What is Science? My own evolving understanding of it has led me to formulate this definition:

Science is (1) the collection and analysis of data*
(2) in order to test hypotheses** and replicate previous findings
(3) and to evaluate old theories*** as well as form new ones.

*information that is in the least potentially measurable
**a hypothesis is a proposed relationship between variables
(objects/events)
***a theory is a set of statements (conceptual framework) that

(a) explains the processes/mechanisms behind the workings of some aspect of the universe and
(b) allows for testable predictions

“Doing science” can mean any or all of #1-3. A, quote, “real scientist” may specialize in one of the three elements, but will certainly acknowledge and respect the other two.  Also, the “real” or formal scientist subjects her work to peer review and practices clarity and transparency in their work.

II. THE TWO ARMS OF SCIENCE

Many thinkers have argued that science developed out of philosophy, particularly natural philosophy (the study of nature). While this may be true, any unstated assumption that science has completely moved beyond philosophy is false. Facts in a vacuum mean nothing. Besides data-gathering, science is a human activity that involves explanation, discussion and debate.  You might say that Science today relies on both of these arms:

1. Collecting Data — the “empirical arm” — observation/measurement/tests (numbers)

2. Making Arguments — the “philosophical arm” — hypotheses/theories/explanation (language/logic)

For something to be a science it absolutely must involve the first arm.  Without a firm basis in observation (data) any statements/arguments are speculative at best.  Also, because raw numbers must be put in a context of language for us to communicate and perhaps fully understand them, good science depends upon a diligent and deliberate use of language and logic.

Science today continues to rely on philosophical principles and reasoning.  To evaluate the scientific merit of an idea/hypothesis/claim/theory, we need to consider both the empirical AND the philosophical elements. Too often we neglect one and focus exclusively on the other.

III. FOUR CHARACTERISTICS OF STRONG SCIENCE

As mentioned in “The Two Arms of Science,” the work of science depends upon both data (numbers) and verbal propositions and arguments (language). As a heuristic and an explicit means of evaluating science, I have identified four characteristics by which to appraise the scientific strength of a proposition, article, claim etc. Two of the characteristics pertain to data, two to language.

1. QUANTITY OF DATA

Simply put, the more data, the better.

An interview of two dozen high school students won’t give you as an accurate a picture of the sexual behavior of teenagers than a survey of two thousand teens.

Similarly, a medical experiment with many subjects will give you a more accurate result than an experiment with fewer.

For example: A study of 900 women reported a link between the depression medication Paxil and an increased risk of breast cancer. (Of the 900 women, only 10 were on the medication.)

A follow-up study of nearly 10,000 women taking anti-depressant medications found zero increased risk. (In this study, hundreds of women had used Paxil.)

It is also important to consider that the weaker the quality of the data, the more you need. The corollary is also true. The stronger the data, the less you need. For example: if you want to know whether one vacuum cleaner is better built than another, and your data is an Internet-based consumer satisfaction survey, a few votes aren’t enough to draw a reliable conclusion from. Hundreds would be better. If, however, your data is the results of an independent laboratory test conducted by vacuum cleaner experts, then the data resulting from the testing of a dozen of each type of vacuum cleaner is probably enough to base a reliable conclusion upon.

Because many things in nature are probabilistic (rather than outright causing something, a variable increases or decreases the chance of something), more data means clearer results. You wouldn’t want to attempt to determine if a basketball player were an accurate 3-point shooter just by observing one attempt.

Consider another sports example. Did taking a golf lesson improve your game? One round won’t tell you. You’ve got to compare your average score from a number of rounds before the lesson to your average score from a number of rounds after. More data means more accurate results and thus conclusions you can have more confidence in.

2. QUALITY OF DATA (10 Types – not a complete list)

The higher the quality of data, the better.

type #1: common sense

quality = poor

If considered to be inborn knowledge or reasoning ability, there is probably no such thing as common sense. What we call common sense usually refers to basic ideas and knowledge acquired through exposure to a culture. Common sense is thus relative to a culture. (For example, most people in our country would consider it common sense for a woman to avoid walking city streets alone at night. For a woman living in Tokyo, a very safe city, this might not be considered common sense.) Frequently, when a case is based on an appeal to common sense, it reflects common assumptions.

type #2: intuition

quality =  poor

Intuition has a place and that place is the generation of hunches and ideas. Yes, there is a great deal to our minds beyond conscious thought. However, because intuition is less understood than conscious thought, this does not make it a magical revelation of truth. Importantly, just as intuitive information can be “spot on,” it can as readily be dead wrong. (A physicist working on an esoteric problem, for example, may experience many intuitive solutions to the problem that, when tested on the blackboard of real numbers and facts, turn out to be worthless.) Intuitions appear to have great value only when hits are acknowledged while misses go uncounted.

type #3: anecdote/testimonial

quality = poor

With isolated anecdotes it is impossible to determine when they reflect the rule and when they reflect the exception to the rule. Also, the veracity of personal reports is suspect due to their proneness to several types of bias. So if someone tells you he has a surefire way to make a million dollars, and then shows you a video clip of three success testimonials, be wary.

type #4 personal experience/observations/”clinical data”

quality = fair to poor

By clinical data I mean material that has been acquired informally (without controls) in a clinical setting — such as a doctor saying that the patients in his care looked or seemed better. Subjective biases of many forms are a serious problem with conclusions drawn from personal experience and informal observation.

type #5 opinion of authority

quality = variable

The value of this type of information depends upon such things as the credentials of the authority and, more importantly, the information that he/she has based their opinion upon. If the opinion reflects a consensus in the scientific community it can reasonably considered of better quality.

type #6 news item/magazine article

quality = variable

Articles prepared for a popular audience too often evidence a greater concern for entertainment value than for critical thought. Magazine and television news pieces are notorious for being slanted, superficial, and for advancing premature conclusions. Some sources, however, are certainly better than others. Articles in peer-reviewed journals, in contrast, have much higher quality compared to sources that do not screen submissions for signs of poor scholarship and shoddy research.

type #7 statistics/correlational studies

quality moderate (though highly variable)

“People who say they exercise two or more times a week have a 20% reduced risk of depression.” With any statistical correlation (vs. experiment) the following two truisms are important to keep in mind: 1)Correlation is not causation. (Is it possible that people who say they exercise get more fresh air and/or experience more daylight and/or socialize more, and these are what influence depression rate? With an uncontrolled correlational study, it is impossible to know. 2) Torture a set of numbers and they will talk. Frequently a person collecting statistical data has many numbers to choose from. If the person making the above statement also discovered that “People who say they exercise every single day had no reduced risk of depression,” if he or she didn’t also report that finding, the results would be misleading.

type #8: well-established scientific principles

quality = moderate to good

A well-established scientific principle has a lot of data supporting it. Gravity comes to mind. Thanks to a ton of research, we know that an object released in a gravitational field will fall toward the source of gravity. When this scientific principle is used appropriately, it can be considered to be a high-quality form of data. The important thing to remember is that the principles must be applied within their proper domain. For example, while relativity theory can be appropriately applied to cosmology (study of the cosmos), it would be ridiculous to claim it has anything to do with cosmetology (beautifying techniques and products). Clever thinkers are capable of patching together principled arguments that not only fool others, but also fool themselves. That said, any phenomenon that violates well established scientific principles, applied in their proper domain, or any phenomenon that relies upon principles applied outside their proper domain, should be viewed with skepticism.

type #9: experiment (double blind)

quality = moderate to good

There is no better way to determine whether B has been caused by A. Unfortunately, human error keeps experiments from being foolproof.

type #10: replicated experimental results (by a number of other experiments and experimenters)

quality = excellent

If one lab reported having transformed the aluminum from recycled beer cans into gold, it would be foolish to immediately dump any gold investments you had. If three independent labs reported having transformed used beer cans into nuggets of gold, well heck, that would be an occasion to buy a round a make a toast to science.

3. PRECISE USE OF LANUAGE

When we use specific language people will have a clearer idea what we are talking about. It also affords others the opportunity to verify for themselves any claims we make.

Consider this statement: Good parenting helps children excel at school. This claim is vague. What is good parenting and how do the children excel? A better wording would be: Elementary students who were read to by their parents as toddlers are 50% less likely to be held back a grade.

Scientists will define words so that other people understand exactly what they mean. If their definitions are “operational,” they lend themselves to measurement. While good parenting is difficult to measure,reading can be measured. How could you measure it? By pages, books, hours, etc.

Now imagine this headline: Eating Meat Increases Risk of Cancer. One of the first questions you’d hope the article clarified is what was meant by meat. All meat, or just beef? You may also wonder how great of an increased risk it posed.

Additionally, good scientists give specific details about the meaning of their modifiers (”more than,” “less than,” “increases,” “decreases”). If something is likely to increase your risk of cancer, how much of an increase is it? 1% 10%? Increases and decreases should be put in a clear context.

If the increase is a change in the rate of something, what is the original rate? If eating meat daily increases your risk of getting a brain tumor by 100% (doubling your chances), what’s the original rate? If one in 100,000 men ordinarily get brain tumors, it would mean that of those who eat meat daily, 2 in 100,000 can expect to get a brain tumor. While a 100% increase sounds ominous, an increase to 2 in 100,000 sounds less so. The same increase — 100% — to a higher initial rate is a different story. If the rate goes from 10 men in 100 to 20 men in 100, that’s a much bigger deal. The initial rate of something that increases is very important, yet often gets left out.

The CBS television show 60 Minutes once reported that for every older brother a boy has, his chances of being homosexual increases 30%. Say I have 3 older brothers. Does that mean there is a 90% chance I’m a homosexual (much to my surprise)? No. Because the original chances of a male being homosexual is about 2%. With three older brothers my chances would increase to roughly 5% (2% increased by 30% three times yields roughly 4.5%)

Precision is good. It is much harder to mislead others, and yourself, when you use language and numbers clearly.

4. WELL-REASONED ARGUMENTS
There is more to the critical evaluation of reasoning than simply tearing an argument down. The person honestly committed to scientific thinking will notice and acknowledge both the weaknesses and strengths in of argument.

Ultimately, scientific thinking is not about proof, about proving something. Rather, scientists and rational thinkers provide evidence and arguments in support of a conclusion. The realistic aim of science is not ultimate truth, but facts and theories we can have great confidence in. Where there is strong evidence and tight reasoning we can be highly confident of a conclusion. We should have little to no confidence in a conclusion unsupported by evidence and/or based on reasoning riddled with errors.

When evaluating arguments, one of the two most important questions to ask is whether or not the conclusions are justified by the evidence given. With little or weak evidence, conclusions should be stated tentatively. For example, “ten people tried vitamin x and nine reported feeling better. So it is possible that the vitamin has value.” That would be a more reasonable conclusion than “ten people took the vitamin and nine thought it helped, therefore it definitely works.”

Again, the confidence which a conclusion is made should reflect the quantity and quality of the data supporting it.

A second important question is whether or not the argumentation expresses an unbiased, curious investigation and evaluation. To the curious mind unrestricted by bias, one answered question invariably leads to more questions. Alternate theories and explanations are frequently explored. Very rarely will you hear a sincere scientist express an attitude of “this is the answer, case closed.”

Finally, logic is the use of language (and symbols) in a way that limits errors. Call it formal reasoning. A third important question to ask is whether or not an argument or explanation is free of known errors in reasoning.

Here are some reasoning errors people tend to make in their arguments, broken into four categories.

I. Arguments that Over-Simplify

> Argument by Repetition.
With this type of reasoning the person simply restates the same thing, sometimes in slightly different forms, over and over again. This technique has been show to be persuasive, but its validity is another matter altogether.

Recycling is a waste of time. You’ve got all these people sorting things into different bins, wasting their time. The only people for it are those who don’t’ mind wasting their time, and so they recycle. Why people do it, I don’t know, because it is a huge waste of time.

> Argument by Worst Case Scenario (a.k.a., “slippery slope”).
A person will argue against something, stating that if it is accepted, things will slip down a slope to the worst case.

Outlawing machine guns will lead to making hunting illegal. What’s next, outlawing scissors?

If we allow doctors to prescribe marijuana for medical uses, pretty soon kids will be smoking pot on school playgrounds.

> Careless Generalizations (either premature or too sweeping).
When a person makes a generalization from either too small a sample (premature), or too broadly applies a concept (sweeping), they have ventured beyond what can reasonably be claimed.
Premature -
(Fictitious comment made in 1990s:) Rap music will never catch on. “Ten Cent” released a rap album, and look what happened . . . it bombed.

Sweeping -
The Titanic was considered the best in human invention. And look what happened . . . it sank. That’s why we should never trust human technology.

> Confusing Coincidence with Cause
This is type of argument assumes that events occurring close in time are causally related (a.k.a. “post hoc, propter ergo hoc” – after this, therefore because of this).

My car broke down when my mother-in-law came to visit. Therefore, my mother-in-law’s visit must have caused the car troubles.

My child was diagnosed with autism shortly after being vaccinated. Therefore the vaccine must have caused it.

While temporal proximity may hint at a cause, a hint is not legitimate data.

> “Begging the question” (assuming the validity of important premises that lead to your conclusion.)
This type of argument takes a huge shortcut by skipping steps.

Guns are essential to our freedom. Our country is all about freedom. Therefore gun control is bad.

Are guns truly essential to our freedom? They may be, but we mustn’t assume it simply for the sake of argument.

An argument that “begs the question” involves hidden assumptions — unstated premises that misleadingly simplify the issue.

Soft drinks should be eliminated from schools because they are nothing but sugar.

And? What is going unsaid here?

Murder is wrong, therefore removing the feeding tube from a brain-dead person is wrong.

The unstated assumption in the previous example is that removing a feeding tube is murder. Is it? This is the question that the assumption begs us to ask. If one takes it for granted (assumes) that removing a feeding tube is murder they have over-simplified the matter.
> Argument by analogy
Comparing two different types of things and assuming that what is true for one must be true for the other.

Sometimes computers crash and must be re-booted. That is why electroshock therapy can help people.

> Circular reasoning
This type of argument merely restates the conclusion it was meant to support; the logic is circular, having gone nowhere.

Susan is a bad person because she is evil.

The above has asserted the same thing twice. Susan is bad; Susan is evil.

> False either/or arguments
The assumption that only two alternatives exist in a given situation and they may be equivalent (variant: black or white thinking).

Either you support the death penalty, or you are soft on crime.

Either evolutionary theory is correct or intelligent design is the answer.

In terms of open questions, there are usually a number of possibilities. In the case of evolution, natural selection may be partly correct, or the answer could be some other yet-conceived future theory that more adequately explains the development of life on earth. Or . . .

Note: By all reasonable standards, evolutionary theory is an incredibly well-supported theory.

Very few issues are of the black or white, either/or variety. Because the media likes the ratings that heated conflict provides — even completely manufactured conflict — it will portray an issue as if the matter were black or white. If one person believes that drinking alcohol has great health benefits, they may bring on another who claims alcohol is a poison. What is frequently missing is the middle ground. (Which isn’t to say that the answer is never found at one extreme or the other.)

If a television producer or magazine editor looked hard enough she could probably find a researcher to state on camera or in print that cigarettes are harmless and non-addicting. If a program/article were to present this “counterpoint” to a doctor who expresses the well-established consensus in the field – that cigarettes are both harmful and addictive – does that mean that both positions are equally plausible? Absolutely not. If two sides are presented, it also doesn’t mean that each side is as likely to be valid.

> Attacking a “straw man”
To attack a straw man is to present a flimsy version of the opponent’s position so you can easily fight it. It entails simplifying, exaggerating or outright misrepresenting the opponent’s position so that it is easier to refute.

Tom’s position on the merits of stem-cell research shows a total disregard for human life. And thus we should reject it.

Is that what Tom’s position is all about?

A straw man argument ignores a person’s actual position and portrays it in a distorted fashion. It frequently picks out a perceived weakness and blows it up to piñata size to then whack it apart.

People who can’t see that our government was behind the 9/11 terrorist attacks blindly believe that our government can do no wrong.

Is that why people don’t embrace that particular conspiracy theory?

II. Arguments that Veer off-Track

> “Non sequiturs” (does not follow)
A non-sequitur introduces irrelevant elements or jumps to unfounded conclusions. As is the case with this class of reasoning errors, it sometimes succeeds in diverting attention from the real issue.

Joe is a tall man. Tall men are sexy. Joe shouldn’t be granted custody of his children.

Homeopathy is a valid form of treatment because it gives people choices.

The “red herring” is a subset of the non-sequitur. With it, a false “key” to the argument is presented.

Scientology preaches the truth. A number of major critics of Scientology have ties to the psychiatric industry and so the critics shouldn’t be trusted..

III. Arguments that Use Social Concerns to Sway Opinion

With this type of reasoning a person resorts to pulling our strings, so to speak — tapping into common social and emotional concerns. The person engages in something more closely related to a sales pitch, including scare tactics and enticements to convince you.

> Appeal to tradition: It has been done this way for a long time.

Chinese medicine is effective because it is 3000 years old.

> Appeal to popularity.

Michael Jackson is the best musician ever. Could millions of fans be wrong?

Homosexuality is a perversion of natural law; 60% of Americans agree.

At one time in our history the majority of Americans believed that African Americans were an inferior race. The majority was wrong.

> “Ad hominem” appeals (”to the person”)
Using personal characteristics/evaluations to make it more or less likely an argument will be accepted.

Madeline is a genius; Madeline believes in UFOs; UFOs must exist.

Senator Smith is an idiot. Smith wants congress to pass bill #332. Bill #332 must be stupid.

> Appeal to authority
This form of argument amounts to claiming that a famous person holds the same position you do, hence it is the right position.

Israel should have its own state. Even Einstein believed so.

An appeal to authority is only valid if the authority is a well-informed, acknowledged expert in the field. If this is the case, the tactic is used as a short-cut. It is not who believes what that is important. Rather, what counts is the information and reasoning an assertion is based upon.

> Appeal to emotion
Whether an idea or claim makes you feel joy or fear doesn’t have any bearing on its truth.

We shouldn’t eat meat because Bambi is so cute.

What about eating Bambi’s ugly cousin, Wally the warthog? How is cuteness a reason?

If a lawyer argues, You must believe with me that this man is guilty because a number of people were gruesomely murdered and if we let the suspect free it might happen again, we must remember that our general fear of violence and a particular individual’s guilt are two separate issues.

> Guilt by association
This bogus reasoning tactic inappropriately associates what should be viewed as a neutral idea with a bad one. Only with a mindset of neutrality can we make the most reasonable decision.

Marijuana shouldn’t be prescribed by doctors because juvenile delinquents smoke dope.

And we shouldn’t drive cars because bank robbers drive cars?

IV. Arguments from a Subjective Stance

An argument from a subjective stance is riddled with bias and/or a narrow perspective.

> Appeal to the Remarkable (and perhaps unusual)
This type of reasoning consists of using dramatic stories that you feel are compelling and thus potentially persuasive.

My great-grandmother ate a lot of turnips and lived to 110, therefore all people should
eat a lot of turnips.

Although this arguments aspires to make a rule out of an exceptional event, one case does not prove a general principle.

There was a tragic airline accident. Of 74 passengers, one passenger survived. This must have been due to miraculous intervention.

Actually, the “rule” was that 74 died. The one survivor was an exception to the rule, and those 74 should not be forgotten.

> Biased selection of information
We live in a complex world, with a nearly unlimited amount of potential connections. Are the connections meaningful? We can’t assume so.

A scientific study revealed that Virgos were more likely to vomit during pregnancy.

The above is true. But the scientists themselves were demonstrating that if you look hard enough, you can find a link between elements, no matter how far flung. To find that correlation the researchers looked at a dozen star signs and many dozens of medical conditions. Out of hundreds of possibilities, they found one coincidental connection. (This is also an example of “data mining.”) With any correlation/connection, the important question is whether or not the link holds up in when put to the test.

> Special pleading:
This very common type of reasoning error consists of introducing new elements into an argument in order to fix the flaws in your position. It violates the famous philosophical principal of Occam’s razor: the fewer elements to an argument, the better. In other words, introducing special elements does not make an argument stronger, but weaker.

The reason why the assassination of JFK was blamed on Lee Harvey Oswald and the real
truth hasn’t come out is because all the people involved are/were paid off or killed.

Adding a massive cover-up involving hush money to your reasoning about JFK’s murder complicates the matter immensely, and thus makes it a weaker argument.

> Equivocation
Changing the meaning of a word or phrase to help your own cause.

Because you can’t explain this recent UFO citing, it must be proof of an alien visitation.

“UFO” means “unidentified flying object.” No more, no less. An event involving an unidentified object supports no position.

> Argument from personal experience
Assuming that what is true in my life must be true in yours.

I taught my dog to do a backflip – therefore any dog with proper training can do
backflips.

My husband is a happy stay-at-home dad therefore all husbands could be happy, stay-at-
home dads.

> Appeal to ignorance (misplaced burden of proof)
This type of reasoning assumes an argument is valid because contrary evidence has yet to be found. This is like insisting a claim or theory must be right because no one has proven it wrong. By shifting the burden of proof/disproof to the opponents, this tactic relieves the person who makes the claim from having to support or prove the point him or herself. But the burden of proof is always on the claimant. We mustn’t assume a conclusion is true because it hasn’t been shown to be false.

Acupuncture helps relieve allergies. No study has shown it doesn’t.

But what about studies that show it does? That’s the real question.

(udated 8/2008)

Trackback URI | Comments RSS