I remember being a young professor and not understanding the nature of “scientific rigor.” That was the term my older and wiser colleagues bandied about every time someone wanted to publish something or report “something new.” And the bandi-abouters (or bandi-doubters) had good reason. Isn’t science supposed to deal in verifiable quantities, in experiments conducted under the strictest controls and scrutiny, and in precisely calibrated apparatus? And that means that science must, ultimately, rely heavily on math to eliminate hints of subjectivism. The numbers just have “to add up” if peers are asked to accept results.
But, of course, the world is unimaginably complex. Don’t believe me? Hey, I’m not alone in this. The problem is that we cannot know all at any moment. Our lack of omniscience makes us vulnerable to error, hasty conclusion, and non-sequiturs. First, let me give you an example of the complexity from a recently published book; second let me engage you in speculation about social sciences.
First: Professor Bill Sullivan suggests a possible link among genes, germs, and characteristic beliefs and behaviors. His book, Pleased to Meet Me: Genes, Germs, and the Curious Forces That Make Us Who We Are, relates findings that genes might have an influence on political leanings and gut bacteria might have some influence on behavior. Sullivan writes, for example, “All of us like to think that we march to the beat of our own drum. But science has revealed that the rhythm is played by percussionists we can’t see with the naked eye.” * Scary thought, that. Just when you finished looking in the mirror and saying, “Good morning, I look and feel great today, and I’m really pleased with my thoughts and behavior,” some guy comes along and says you might be a bit of a marionette whose strings are pulled from the inside. For Sullivan, this means that we have a “false sense of self” (14), and that “almost everything we know about ourselves is wrong” (14). Just got scarier, right? The person in the mirror isn’t the person you imagined. One more “fact”: “About 10,000 species of bacteria reside in our gut, supplying us with an extra eight million genes [cells have about 21,000 each]. Their collective weight is up to three pounds” (31). Yeah, equivalent to a brain’s weight. Sullivan calls such science about genes and germs an “ego crusher.” It suggests we are possibly just as much controlled as controlling. Yet, and here’s my transition into the second part of this discussion, we can’t know for sure about who or what controls even in the face of these biological facts that Sullivan reveals to us biological laymen. (Before I change to my speculation, I’ll note that Sullivan doesn’t dismiss environmental conditions. They, too, influence who we are)
Finding out about this complexity hidden within us helps us to understand more about who we are, but it is a stretch to say the rigor of the biological science leads to irrefutable rigorous conclusions. I think Sullivan points out that separated twins reunited in later life discover that they have much in common, including leaning toward the same political points of view—40% of the time. Think about that: 40%. Maybe the influence of genes and germs should be taken seriously just 40%. Is that rigorous enough? And since genes appear to serve multiple functions, can we ever completely know that this or that gene is the reason that you, for example, lean this way or that politically, religiously, socially? And we can ask the same question about the validity of ascribing influence to gut bacteria. Isn’t there a hint in Sullivan’s research that the genes of individual bacteria seem to work in unison to produce in you some tendency? Is a hint a matter of rigor?
Second, and back to the matter of rigor: in my relative youthful ignorance I looked at literary criticism, ethnology, and social sciences in general to see whether or not such activities lacked scientific rigor. And what of history, also, I asked myself? I had a number of colleagues who worked in anthropology, archaeology, and history, all in my youthful mind related by the category “social science.” How could peer historians, anthropologists, and archaeologists judge work submitted to journals without applying judgments based on their biases and incomplete knowledge? Where’s the rigor in that process? Maybe we just want the “generally true” and not the “absolutely true.”
You probably want an example. For a while, and maybe still so, one of the most critically examined poems in the English language was “The Windhover” by Gerard Manley Hopkins. Just about everyone agrees that it is a great poem, but people keep arguing about it, especially about the line that contains the words “here buckle.” All those “great literary critics,” and yet, there is no final agreement. Is there rigor in literary research? Maybe in something like counting the number of polysyllabic words in a work, or in counting metaphorical references to the sun. And what of archaeology? Can the archaeologists agree on whether or not a Neanderthal burial 60,000 years ago implies a religion? Or, does it imply fear of scavengers near an encampment after a death? And do 430,000-to-540,000-year-old scratch marks on a shell found in Java represent language, math, or the amusement of a Homo erectus child? Or take anthropology. Should we question the work of Margaret Mead in light of Derek Freeman’s re-evaluation of her work and report that one of the girls Mead interviewed said she and her girlfriends were just pulling Mead’s leg about Samoan sexual mores? Another: What of behavioral biology that explains the “motives” of various animals? (Could someone please poll the orcas?)
When I began to see a movement toward “qualitative” research, I was even more confused about rigor. How does one rigorously deal with any quality without injecting some personal feeling or point of view? Yet, there was a growing body of such research, all taken quite seriously by “peers.” And that type of research seems to be proliferating. To me, much of it seemed reminiscent of nineteenth-century works that aimed to explain Man and Animal. So, recently, I went back to one of those roots of social “sciences,” not all the way into the nineteenth century, but to a century ago. It was there that I found some clue about the rise in qualitative research.
Here’s a passage from the 1921 Introduction to the Science of Sociology by Park and Burgess “Social questions have been endlessly discussed, and it is important that they should be. What the student needs to learn, however, is how to get facts rather than formulate opinions. The most important facts that sociologists have to deal with are opinions (attitudes and sentiments), but until students learn to deal with opinions as the biologists deal with organisms, that is, to dissect them—reduce them to their component elements, describe them, and define the situation (environment) to which they are a response—we must not expect very great progress in sociological science.”**
I guess I haven’t progressed much since those days long ago when I was still trying to get a handle on rigorous research. The student shouldn’t “formulate opinions,” but the facts of the “science” are “opinions (attitudes and sentiments).” That’s pretty much a matter of qualitative over quantitative research and a branch of human inquiry that begs a question about rigor. Ask yourself whether or not I can trust the validity of your opinion as you express it. Come on, admit it: You have changed over the years, but some of your past lingers in your present.
And that brings me to one aspect of social science with which everyone is familiar: Polls on feelings, attitudes, and tendencies we see—daily—that serve as the dissection of opinions Park and Burgess suggest make up a science of sociology. There is an entire industry centered on such incessant polling and dissection. Among the pollsters are mathematicians, specifically statisticians. Certainly, their presence lends rigor to polling. Or does it? They can do the math only on the data that the pollsters hand them.
The question about the rigor of the process is significant because the math makes no judgments. Its weakness might be in that it doesn’t give an answer about the design of the poll itself: Who wrote the questions? Are the questions devoid of any modifiers that can be loosely interpreted by the polled? Certainly, we might ask about the rigor of any poll that asks a single question, one requiring, for example, acknowledgement of favor or disfavor. Yet, many people attempt to engage others in such simplifications. And, of course, there’s the question of how many questions make a good poll, that is, what level of specificity should a poll attain? The rigor in polling comes, I believe, from highly specific questions. That’s when the statisticians can do their best work.
But now, I have to ask whether or not in my responding to a poll, am I responding? Are my genes responding? Are my gut bacteria responding?
Genetics, gut biology, and the social sciences all seem to have something to teach us about ourselves, but all of us who read studies from the fields need to ask first about the rigor of the research and second about the possibility that something might be missing, some unknown influence on the matter at hand. The unknowns, however, can’t be rigorously examined. I know that I don’t have to agree with everything that supposedly like-minded people hold dear. Where’s my individuality if I do? Do I associate intellectually with those with similar gut bacteria and genes? But, one can belong to a particular religion or denomination and still hold exceptions. One can belong to a political party and also disagree with some tenets of that party.
I suppose that until someone does some definitive rigorous research we can all accept on what humans are and why they believe and behave the way they do, we’ll be at the mercy of those who draw conclusions tainted by unknowns and opinions. What if I say that I believe our lack of omniscience is behind our willingness to accept levels of rigor as they serve our needs in the topics du jour? What if I argue that our tendencies have their origin in both our culture and our biology? If my argument were rigorous, would I be able to assign a proportion to the influence of both? Woe is me. I don't have the wherewithal to make rigorous studies that demonstrate the validity of any such conjectures. Sullivan's book reopens the old question about Nature vs. Nurture and once again forces us to ask about our own beliefs about why we are who we are and behave the way we do.
*Washington, D.C., National Geographic Partners, LLC, 2019. p. 11.
**Park, Robert E. and Ernest W. Burgess. Introduction to the Science of Sociology. Chicago. The University of Chicago Press, 1921. p. vi.
But, of course, the world is unimaginably complex. Don’t believe me? Hey, I’m not alone in this. The problem is that we cannot know all at any moment. Our lack of omniscience makes us vulnerable to error, hasty conclusion, and non-sequiturs. First, let me give you an example of the complexity from a recently published book; second let me engage you in speculation about social sciences.
First: Professor Bill Sullivan suggests a possible link among genes, germs, and characteristic beliefs and behaviors. His book, Pleased to Meet Me: Genes, Germs, and the Curious Forces That Make Us Who We Are, relates findings that genes might have an influence on political leanings and gut bacteria might have some influence on behavior. Sullivan writes, for example, “All of us like to think that we march to the beat of our own drum. But science has revealed that the rhythm is played by percussionists we can’t see with the naked eye.” * Scary thought, that. Just when you finished looking in the mirror and saying, “Good morning, I look and feel great today, and I’m really pleased with my thoughts and behavior,” some guy comes along and says you might be a bit of a marionette whose strings are pulled from the inside. For Sullivan, this means that we have a “false sense of self” (14), and that “almost everything we know about ourselves is wrong” (14). Just got scarier, right? The person in the mirror isn’t the person you imagined. One more “fact”: “About 10,000 species of bacteria reside in our gut, supplying us with an extra eight million genes [cells have about 21,000 each]. Their collective weight is up to three pounds” (31). Yeah, equivalent to a brain’s weight. Sullivan calls such science about genes and germs an “ego crusher.” It suggests we are possibly just as much controlled as controlling. Yet, and here’s my transition into the second part of this discussion, we can’t know for sure about who or what controls even in the face of these biological facts that Sullivan reveals to us biological laymen. (Before I change to my speculation, I’ll note that Sullivan doesn’t dismiss environmental conditions. They, too, influence who we are)
Finding out about this complexity hidden within us helps us to understand more about who we are, but it is a stretch to say the rigor of the biological science leads to irrefutable rigorous conclusions. I think Sullivan points out that separated twins reunited in later life discover that they have much in common, including leaning toward the same political points of view—40% of the time. Think about that: 40%. Maybe the influence of genes and germs should be taken seriously just 40%. Is that rigorous enough? And since genes appear to serve multiple functions, can we ever completely know that this or that gene is the reason that you, for example, lean this way or that politically, religiously, socially? And we can ask the same question about the validity of ascribing influence to gut bacteria. Isn’t there a hint in Sullivan’s research that the genes of individual bacteria seem to work in unison to produce in you some tendency? Is a hint a matter of rigor?
Second, and back to the matter of rigor: in my relative youthful ignorance I looked at literary criticism, ethnology, and social sciences in general to see whether or not such activities lacked scientific rigor. And what of history, also, I asked myself? I had a number of colleagues who worked in anthropology, archaeology, and history, all in my youthful mind related by the category “social science.” How could peer historians, anthropologists, and archaeologists judge work submitted to journals without applying judgments based on their biases and incomplete knowledge? Where’s the rigor in that process? Maybe we just want the “generally true” and not the “absolutely true.”
You probably want an example. For a while, and maybe still so, one of the most critically examined poems in the English language was “The Windhover” by Gerard Manley Hopkins. Just about everyone agrees that it is a great poem, but people keep arguing about it, especially about the line that contains the words “here buckle.” All those “great literary critics,” and yet, there is no final agreement. Is there rigor in literary research? Maybe in something like counting the number of polysyllabic words in a work, or in counting metaphorical references to the sun. And what of archaeology? Can the archaeologists agree on whether or not a Neanderthal burial 60,000 years ago implies a religion? Or, does it imply fear of scavengers near an encampment after a death? And do 430,000-to-540,000-year-old scratch marks on a shell found in Java represent language, math, or the amusement of a Homo erectus child? Or take anthropology. Should we question the work of Margaret Mead in light of Derek Freeman’s re-evaluation of her work and report that one of the girls Mead interviewed said she and her girlfriends were just pulling Mead’s leg about Samoan sexual mores? Another: What of behavioral biology that explains the “motives” of various animals? (Could someone please poll the orcas?)
When I began to see a movement toward “qualitative” research, I was even more confused about rigor. How does one rigorously deal with any quality without injecting some personal feeling or point of view? Yet, there was a growing body of such research, all taken quite seriously by “peers.” And that type of research seems to be proliferating. To me, much of it seemed reminiscent of nineteenth-century works that aimed to explain Man and Animal. So, recently, I went back to one of those roots of social “sciences,” not all the way into the nineteenth century, but to a century ago. It was there that I found some clue about the rise in qualitative research.
Here’s a passage from the 1921 Introduction to the Science of Sociology by Park and Burgess “Social questions have been endlessly discussed, and it is important that they should be. What the student needs to learn, however, is how to get facts rather than formulate opinions. The most important facts that sociologists have to deal with are opinions (attitudes and sentiments), but until students learn to deal with opinions as the biologists deal with organisms, that is, to dissect them—reduce them to their component elements, describe them, and define the situation (environment) to which they are a response—we must not expect very great progress in sociological science.”**
I guess I haven’t progressed much since those days long ago when I was still trying to get a handle on rigorous research. The student shouldn’t “formulate opinions,” but the facts of the “science” are “opinions (attitudes and sentiments).” That’s pretty much a matter of qualitative over quantitative research and a branch of human inquiry that begs a question about rigor. Ask yourself whether or not I can trust the validity of your opinion as you express it. Come on, admit it: You have changed over the years, but some of your past lingers in your present.
And that brings me to one aspect of social science with which everyone is familiar: Polls on feelings, attitudes, and tendencies we see—daily—that serve as the dissection of opinions Park and Burgess suggest make up a science of sociology. There is an entire industry centered on such incessant polling and dissection. Among the pollsters are mathematicians, specifically statisticians. Certainly, their presence lends rigor to polling. Or does it? They can do the math only on the data that the pollsters hand them.
The question about the rigor of the process is significant because the math makes no judgments. Its weakness might be in that it doesn’t give an answer about the design of the poll itself: Who wrote the questions? Are the questions devoid of any modifiers that can be loosely interpreted by the polled? Certainly, we might ask about the rigor of any poll that asks a single question, one requiring, for example, acknowledgement of favor or disfavor. Yet, many people attempt to engage others in such simplifications. And, of course, there’s the question of how many questions make a good poll, that is, what level of specificity should a poll attain? The rigor in polling comes, I believe, from highly specific questions. That’s when the statisticians can do their best work.
But now, I have to ask whether or not in my responding to a poll, am I responding? Are my genes responding? Are my gut bacteria responding?
Genetics, gut biology, and the social sciences all seem to have something to teach us about ourselves, but all of us who read studies from the fields need to ask first about the rigor of the research and second about the possibility that something might be missing, some unknown influence on the matter at hand. The unknowns, however, can’t be rigorously examined. I know that I don’t have to agree with everything that supposedly like-minded people hold dear. Where’s my individuality if I do? Do I associate intellectually with those with similar gut bacteria and genes? But, one can belong to a particular religion or denomination and still hold exceptions. One can belong to a political party and also disagree with some tenets of that party.
I suppose that until someone does some definitive rigorous research we can all accept on what humans are and why they believe and behave the way they do, we’ll be at the mercy of those who draw conclusions tainted by unknowns and opinions. What if I say that I believe our lack of omniscience is behind our willingness to accept levels of rigor as they serve our needs in the topics du jour? What if I argue that our tendencies have their origin in both our culture and our biology? If my argument were rigorous, would I be able to assign a proportion to the influence of both? Woe is me. I don't have the wherewithal to make rigorous studies that demonstrate the validity of any such conjectures. Sullivan's book reopens the old question about Nature vs. Nurture and once again forces us to ask about our own beliefs about why we are who we are and behave the way we do.
*Washington, D.C., National Geographic Partners, LLC, 2019. p. 11.
**Park, Robert E. and Ernest W. Burgess. Introduction to the Science of Sociology. Chicago. The University of Chicago Press, 1921. p. vi.
RSS Feed