Thank you for voting Crowdsignal Logo

Is Mikhail Simkin’s <a href="">scientific approach</a> to assessing the quality of literature valid? (Poll Closed)

Total Votes: 399

  • Robert Dawson - 13 years ago

    ideas > words

  • Nathan Phillips - 13 years ago

    Classic literature is sometimes referred to as timeless. It is thought that this literature, unlike all of its competition speaks to something in all of us no matter what era in which we live. Is it not possible that classic literature is only timeless, because people have been exposed to it with the express intent of getting them to relate it to the era in which they live? It seems to me that the timeless quality of great art is at least in part due to its fame. I would like to see an experiment taking unknown or previously despised authors and see whether or not people could relate to the stories. This would be a different level of context. People don't just judge a work on it's own merits, but based on their own experiences, history and surroundings. It seems plausible to me that an author might meet little acceptance at the time of publication, but much more in years ahead. In fact, it is well known that many artists have made it big only after death, some scientists even *cough* Mendel *cough*. Simkin's study was an over-simplification and it didn't specify its goal well enough. However, I cannot help but remember the many times that I have been inspired to read something largely forgotten, but read by one of the more well known writers.

  • Piero Chiabra - 13 years ago

    It seems to me that Simkin's approach is a typical case of ill-applied reductionism. Denying the "greatness" or the "artistic value" of an author, whatever we may intend for it, through analysing the style of its sentences is like, in my view, saying that we are not intelligent because we are made of molecules, none of which is intelligent. I Am Italian, and although I have what I think is a good knowledge of English, I Am accustomed to reading English writers in their Italian translation. I have also read Dickens and other English authors in the English original, and I have to say that, if the translation is good, most of their "artistic value", or their "greatness" is left unchanged in the translation, even if the sentences only "somewhat resemble" the orginal ones, but are necessarily different because of the different language structures. I think this proves that the "greatness" and "artistic value" of an author do not depend, to a large extent, from the sentences he uses (otherwise: do You think that Shakespeare is obsolete because he writes "Thou art?"), The frequent use of brackets in this comment just indicates what I think is the appreciation process of a work of art: it is a fuzzy process, difficult to quantify and to reduce to a set of deterministic laws: it appeals to non rational characteristics of human beings, which exist, as we are not entirely rational, and, more than everything else, needs to be performed by considering the work of art with a holistic approach, as a whole entity. Trying to analyse the artistic value of an author by tearing it down to pieces, and performing a mechanical comparison of its sentences with the ones of other authors is in my view, as I said, ill-applied reductionism.

  • Eric Winesett - 13 years ago

    I didn't take the quiz because I don't presume to have any expertise in judging literature. I did, however, take the abstract art quiz on Simkin's website, because I am trained in visual art. I got 83% (missed 2 of 12). The art quiz is pointless for the simple reason that a photograph of a painting is not a painting, and anyone who studies painting knows that there is much more to it than a two-dimensional image (e.g. scale and texture). These non-image aspects of the work are especially important in abstract paintings, which cannot rely on a representational image to hold the viewer.

  • Jim Hull - 13 years ago

    Bulwer-Lytton got famous in recent decades because Snoopy parodied him. But B-L wrote "The Last Days of Pompeii" and coined famous phrases like "The pen is mightier than the sword"; he was extremely popular in his time. Since he's now famous merely for snippets of his work, why not ask whether people can distinguish such snippets from those of his competitor Dickens?

    Meanwhile, the poll question itself -- "Is Mikhail Simkin’s scientific approach to assessing the quality of literature valid?" -- sows some confusion of its own. "Scientific approach" begs the question and might lead polltakers to decide, "Well, if it IS scientific, then it MUST be valid!" Then it assumes the Simkin experiment tests for literary quality, when in fact it's testing peoples' abilities to differentiate between writing styles. Artistic quality is an implied issue, but it's not what's tested. I voted "yes" with mixed feelings. (I confess I'm having trouble rewording the poll question to my own satisfaction, so I can see the difficulties.)

    The experiment is worthy of the Skeptics because it points up the fallacy of assuming humans can somehow discern an arbitrary inner quality of a work of art, or that they can step away from their own prejudices in the attempt. And, judging from the many comments on this page, it IS a subject of controversy, Dr. Shermer! With a "peek under the label", most people would choose Dickens over Bulwer-Lytton -- not because he's better, but because he's more respectable. People are tickled by this topic -- well done!

  • Jonathan - 13 years ago

    It looks like we're all missing the point. It seems to be the general consensus that taking snippets of literature and comparing them to see who is "better" is a dumb idea. However, I think the original idea is a sound one.

    There's a lot to be said for reviewing an entire piece without knowing who the author is. That way you remove any personal biases, whether you know you have them or not. To me; a valid theory, just a poor way of demonstrating it.

    Of course, this is all moot if I am the one who missed the point. I usually like to think that everyone is crazy but me.

  • Paul H - 13 years ago

    Bulwer-Lytton the worst writer? Really? The selection of this writer is surely not scientific in any sense of the word. Rienzi is a good read.
    It's laughable selecting a small piece of a book and comparing it. Besides the obvious statistical problems, the selection bias issues, let alone a definition of what "good writing" is, there are deeper aesthetic questions that underly this enquiry. The whole basis of quality evaluation in literature, as in music, or painting, or the Arts generally, is that it is NOT scientific, but largely gut-reaction, "I like it" or "I hate it". It is subjective. What then follows by the expert or authority is an attempt to rationalize your gut reaction into a coherent reason that hopefully will fool/convince others that you are talking factually. But don't kid yourself. Art appreciation is largely humbug, manufactured rationalization, nothing more. Gladwell tried to get around this in his book Blink, but foundered, largely because the gut reactions of most experts turn out to be prompted by learned responses from years of training and familiarity. It still makes me smile when I see a panel of experts in any field of the arts who all pretend to be on the same page, as though we should somehow expect them all to be coming to the same conclusion. We don't all have the same tastes, likes or dislikes, so why should we expect to come to the same conclusions in areas where there are no strict scientific criteria? It makes no sense at all.
    I don't watch American Idol or like certain movies; no music or movie critic is going to change that by an act of rationalizing his gut feeling.

  • Buzz Watkins - 13 years ago

    I think the study would be valid if the question were different. Instead of the "Quality" of the writer being judged, the voters were indicating their perception of which author they were reading, having really nothing to do with the quality of the snippet.

  • Jeff Henninger - 13 years ago

    I think in general that Mr. Simkin's approach to determining the answer to the question, "Are the very famous writers differnt from the obscure ones" is not valid. In most writing, all the words used are known and with few exceptions most sentences are not in and of themselves notable. Paragraphs, made up of such sentences, are likewise most likely to be uninteresting.

    Are we to expect that a brilliant writer should use one magnificent sentence in each and every paragraph they have produced? "It was the best of times, it was the worst of times" is easily recognizable as Dickens, but I think much of the rest of the book consists of many more mundane sentences and paragraphs then memorable ones.

    When measuring the whole of human literary creation, I would guess that less than one sentence in a million truely rises to magnificence. In poetry perhaps the ratio is a bit better, though I really have no idea what the ratios might be. Randomly selecting short snippets from any author's work and expecting it to be extraordinary does not even make sense to me.

    If Mr. Simkin's were to use a computer to randomly select passages and the computer perchance selected the sentence from, "A Tale of Two Cities" as shown above, would he not have to reject the sentence since it is so well known? What does that say about his approach in general? Why is this sentence so famous that he would need to purposely choose something else? Could it be that one author is more famous than another for a reasonable reason?

  • Michael - 13 years ago

    Like many who've commented already I think that isolated passages are not a good measure of writing quality. A couple of years ago I had my first opportunity to read and grade essays from high quality undergraduates. Some of them wrote in a positively beautiful prose that was completely void of meaning. Others, with weaker, but still good, writing skills, provided meaningful content as part of their writing. Those with good content got the better grades.

    "So, success is a combination of quality and chance."

    Agreed. Once you have a pool of competent, or better, authors (or musicians, or ....) they need a bit, often a good bit, of luck to rise to prominence. A celebrity endorsement on some talk show, a movie deal, placement on the shelf by the clerk in the bookstore, and suddenly an author rises to prominence while an equally good author remains in obscurity.

  • Russell Wadbrook - 13 years ago

    Bulwer Lytton once began a sentence that began "It was a dark and stormy night" - the full sentence is, "It was a dark and stormy night; the rain fell in torrents--except at occasional intervals, when it was checked by a violent gust of wind which swept up the streets (for it is in London that our scene lies), rattling along the housetops, and fiercely agitating the scanty flame of the lamps that struggled against the darkness." The fact that it's mocked by people who don't know it is an apt reflection of the main issue most of us have with Mr. Simkin's study: pulling paragraphs (or sentences or clauses) out of context gives no meaning insight into the writers in question.

  • Lawrence S. Lerner - 13 years ago

    The basic premise - that Bulwer Lytton is a very bad writer - is seriously flawed. It is based on a modern joke having to do with the fact that BL once began a novel with the sentence "It was a dark and stormy night," and Charles Schultz's Snoopy rang changes on that for years.
    In fact, Bulwer Lytton was quite popular in his day. He isn't read much any more because the prolix style of novels of his era is no longer in vogue. But then Dickens isn't read that much any more, either, outside of school assignments.
    I won't dispute that Dickens is a better novelist that BL. But BL is a poor choice if the aim of the project is to compare a very good novelist with the worst available. For the latter, one might easily choose any of the very many writers who eked out a living writing serial novels for the plethora of magazines that circulated at the time, and who are now mainly unknown.

  • Jon Richfield - 13 years ago

    I voted "Yes", though I was not really happy with the study and did not think it was very significant. My reason for the vote was that I thought that studies of such types are potentially of interest and possibly of value, but that I had serious reservations about attributing values such as "Good Writer" or "Bad Writer" according to the outcomes. Personally I rather dislike Dickens' works and have done so from childhood; he struck me as rather old-womanish and inimical to the "suspension of disbelief". Accordingly I have read very little of his work. Conversely, I have read no B-L works at all as far as I can remember, so I could not validly participate in such a trial, though I should like to if I could, just out of curiosity. But I reject the idea that the evidence is valid for a classification of good writing. Keyhole views of anything are likely to be misleading, not just literature. It is not to say that no brilliant fragment could be recognised as brilliant, but conversely, any mediocre writer could parody a great writer's work in snippets without fooling a competent judge in a couple of pages or a couple of chapters. My problem is largely with such polls; they demand simple answers to complex questions and . I actually share the views of most of the other comments here.
    Style matters and fashion matters, and substance and structure matter (more, I hope) and greatness is more than any or all of those in combination. Social context, originality, creativity, passion, subtlety, irony - those and many more cannot be captured in arbitrary fragments.
    At the same time, it would demand special pleading to say that no assessment of merit relevant to "greatness" (whatever that might be) based on double-blind tests could ever reveal anything of value. If all that great writing means is that schismatic schools of pretentious pundits froth or fume over works that they are familiar with, then it, the schools and the pundits are dispensable, say I.
    Scientific studies are valid to the extent that they demonstrate empiric distinctions.

  • Bart Stewart - 13 years ago

    Well, Steven Jent (in a comment above) beat me to the punch. I was going to say that a brush stroke by Renbrandt is no better than a brush stroke by me. It's the aggregate of them that makes the difference. How long were these "passages" that the study was using?

    The literature professor mentioned in the e-Skeptic article said it very well. Dickens reputation rests on the overall content of his books, not on any average passage plucked from them. Which is not to say that he couldn't produce some ringing phrases, but he is known for his social commentary, which arguably changed the world. The term "Dickensian" will be forever in our language as a description of the horrors of poverty. Now, it is true that once a writer "makes it" he is hyped to the high heavens, while others of worthwhile talent are lost to obscurity. That's a tragic situation that calls for an effort to reevaluate and rediscover forgotten authors of merit, but not to imply that Charles Dickens was overrated.

    I hope this study wasn't an attempt to prove that literary fiction is a worthless pursuit. Too many young people have swallowed that notion, and now the high point of their intellectual and emotional life is - Super Mario. Also this damnable competition between science and the arts is such a colossal net loser for humanity. There's outright hostility between the fields at times. It's tragic. Literature and the Humanities are never going to develop a new vaccine or build a laser-sail spacecraft, but they have the potential to establish a humane civilization from whence people can come forward and do those things. I always say we have moved away from having a literary society - and it shows.

  • David Sueme - 13 years ago

    The "taken out of context" arguments above are quite germane. There is another issue: style versus substance. Aristotle, in translation, is notoriously inelegant. But can there be any real dispute that he is much the exemplar of "genius"? Two thousand years after his death Aristotle is still the authority on the subject of formal logic. His prose may give you a headache but I don't see that you can ignore him.

  • Christopher Cousins - 13 years ago

    Dr Simkin has reinvented, albeit crudely, the wheel introduced by I A Richards back in the 1930s. Richards was a Lecturer in English at Cambridge University who presented his students with anonymous pieces of writing by a whole range of authors from literary giants to magazine writers. The students were given the list of the authors but made as much a hash of the task of identification as Simkin's quiz takers. Richards produced a book entitled Practical Criticism in which this study forms a part.

    What I consider alarming about Simkin's study is (a) his belief that literary quality will be revealed in an arbitrarily selected tiny sample of an author's work and (b) that this belief should be taken seriously and result in publication!
    The results were

  • Fatboy - 13 years ago

    Hmm, apparently links don't work in these comments. For anyone wanting to look up the study on their own, it was titled, "Experimental Study of Inequality and Unpredictability in an Artificial Cultural Market", and was authored by Matthew J. Salganik, Peter Sheridan Dodds, and Duncan J. Watts.

    If this works, I've pasted the URL below. If not, just use google with the info above.

  • Fatboy - 13 years ago

    Peter Hoban beat me to what I was going to say in his third paragraph. Being able to write a paragraph well is hardly the same thing as writing a story that people would want to read or that has a profound impact on them.

    As to how much better or worse popular writers are than not so popular writers, I'm reminded of this study done a few years ago. To quote part of the abstract, "We investigated this paradox experimentally, by creating an artificial “music market” in which 14,341 participants downloaded previously unknown songs either with or without knowledge of previous participants' choices. Increasing the strength of social influence increased both inequality and unpredictability of success. Success was also only partly determined by quality: The best songs rarely did poorly, and the worst rarely did well, but any other result was possible."

    So, success is a combination of quality and chance.

  • Erik - 13 years ago

    Mr. Simkin's mention of his previous similar study about modern art makes me think he has set out to purposefully "prove" the lack of value of genres and individuals he personally doesn't like. The hallmark of good research is to begin with a truly unbiased mind, or else that bias is likely to creep into the study design. Perhaps Mr. Simkin should try to reproduce his results by choosing a famous author he counts among his favorites, and see if his findings still hold.

  • Steven Jent - 13 years ago

    Who is the greater artist, Claude Monet or Thomas Kinkade (the "painter of light")? Could it be that Monet's reputation is merely arbitrary?

    To decide this question, I captured random 1" square samples of several paintings by Monet and Kinkade. I then asked test subjects to compare these unidentified blotches of color and to indicate which ones were painted by a genius and which were trite hack work.

    Remarkably, the result was a virtual tie. Evidently there is no real difference in talent between these two painters. The fact that museums around the world display the works of Monet but ignore Kinkade can be explained only as a product of herd instinct among curators and elitist art lovers.

  • gerard te meerman - 13 years ago

    The experiment has demonstrated two things: people are not able to guess which fragment is from the 'better' writer and at least some people are able to sort out which fragments belong together. Concluding that success in literature is a kind of random process is not an unlikely hypothesis but the experiment is not designed to answer that question. Literary quality is not validly evaluated by inspecting fragments of text: you need a larger body of text, e.g. a short story of a few pages. The conclusion of Simkin's interesting experiment is, and this is not uncommon especially if the conclusion itself is not entirely unlikely true, not supported at all by his data.

  • Peter Hoban - 13 years ago

    Mr Simkin does not reveal how his passages are selected. It is thus possible that he selected a poorer quality of Dickens passages and the best of those by Bulwer-Lytton. The results he finds may thus reflect his bias in the selection of passages.

    If he were to somehow randomise the selection of passages he might avoid the criticism of bias, but who would expect that the art of a good novelist will show in some random selection of unrelated paragraphs deprived of their context?

    Good writing is not just good words, it is more about plot and character development and what hope does Mr Simkin have of capturing that in an extract which is short enough for his purpose (so as to be not readily recognisable)? When stripped of its principal qualities one might expect that any meaningless abstract of grammatically correct prose might be regarded as equal with any other. Not a very profound conclusion.

    The analogy with citations is erroneous. The errors in the lists of cited papers are clearly copied not cited, and the most cited will be more frequently repeated precisely because copying is virtually costless. To suggest that Dickins's popularity might have been created by simple copying of fashion is perverse. Books and magazines cost real money and always have. People who choose to buy are allocating scarce resources - this is not just mindless copying.

    Isn't it rather more likely that Dickins is more widely read precisely because reading his whole narratives is more enjoyable for most readers? If so Mr Simkin was wasting his time and should stick to physics.


Leave a Comment

0/4000 chars

Submit Comment

Create your own.

Opinions! We all have them. Find out what people really think with polls and surveys from Crowdsignal.