January 27 is becoming a black-letter day in American literature. On that day in 2009, John Updike died and, this year, the first anniversary of that loss was marked by the news that JD Salinger was dead. It's an artificial coincidence – of a sort that authors as good as Updike and Salinger would have scorned in their stories – but the deaths in close succession of members of the literary generations born in the 1910s, 20s and 30s do have a symbolic significance. If we add the deaths within four months of 2007 of Norman Mailer and Kurt Vonnegut – members with Salinger of the set of major American writers formed by service in the second world war – it's clear that an era in American literature is coming to a close.
There is an obvious temptation to believe that the authors who have recently died form – with others who fought in the war (such as Saul Bellow and Gore Vidal) or were teenagers in America during it (Philip Roth) – the greatest literary generation the country has ever seen or ever will see. This triumphalist but nostalgic position holds that these writers took advantage of their nation's geopolitical power – and a media culture and bookstore customer-base which regarded serious writers seriously – to create a superpower of the pen to match the financial and military clout of the US during what became known as the American century.
The counter-argument is that this army of old soldiers was very male and masculine and white in its concerns – tempered only by a grudging, late admission to the halls of fame of writers such as Toni Morrison and Joyce Carol Oates – and that the standard narrative of 20th-century American literature is partial and distorted. This case is made persuasively in Elaine Showalter's recent book: A Jury of Her Peers: American Women Writers from Anne Bradstreet to Annie Proulx.
These contrasting presentations of recent American letters are explored in Capturing America, an eight-part Radio 4 series on which I've been working for several years. And – even before the death of Salinger during final editing – there had been melancholy signs that this was the right time to take stock. The programmes contain the final major interviews with Mailer, Vonnegut and Updike. The latter seemed healthy and energetic in the BBC's New York studio in the autumn of 2008 as he discussed his life-time mission to write "an alphabet of novels". But The Widows of Eastwick, three short of the intended 26 full-length fictions from this man of letters, became the last when he was diagnosed, just 10 days after our conversation (according to the dated poems in Endpoint, his final volume of verse) with the pneumonia that would lead to diagnosis of lung cancer and his death on the date that lay in wait for Salinger 12 months later. When I began to think about the series, the question of who was America's greatest living novelist would spark lively debate at a book festival. On the eve of transmission, that medal automatically defaults to Philip Roth.
There were other signs that this was the right time to analyse Am lit. Updike, in that last interview, reflected on having twice been pictured on the cover of Time magazine, part of the nation's honours system, to mark the publication of Couples in 1968 and Rabbit Is Rich in 1982. Now, the novelist who takes that prize is Dan Brown. And so the changing of the guard in American fiction is arguably not just generational but cultural: the large, interested readership who lined their shelves with Updike's Rabbit Quartet, Bellow's Herzog, Mailer's The Naked and the Dead, Roth's Portnoy's Complaint and other bestsellers of serious literary merit had perhaps migrated to the quick-read thriller and the confessional memoir.
Any overview is immediately subject to accusations of oversight which are followed just as inevitably by a defence of compression; but my definition of modern American literature concentrates on authors whose first work appeared after 1945, which was, in so many ways, a break-through date.
Roth, in The Plot Against America, imagines that a protectionist government prevented the US from entering the second world war when it did. But, if this had been historical reality, The Plot Against America is not the only major American novel we might now lack. The major American novelists of the middle years of the 20th century are all, in various ways, direct beneficiaries of their country's involvement in that conflict.
Norman Mailer served in the 112th Cavalry in the Pacific theatre, where Gore Vidal, enlisted in the US Army Reserve, was master of a supply boat. Joseph Heller was a bombardier in the 12th Air Force and Kurt Vonnegut a private in the 106th Infantry Division. Jerome David Salinger, drafted into the 4th Infantry Division of the 12th Infantry Regiment, fought on D-Day. Saul Bellow, though Canadian by birth and older than the others, signed up for the Merchant Navy.
Apart from Salinger, this squadron of future novelists saw little military action – Mailer was mainly utilised as a cook and Vonnegut rapidly became a prisoner of war – but all had found material for stories. Indeed, Mailer was clear that he had joined the army with the hope of writing the novel that became The Naked and the Dead (1948). Bellow's first novel, Dangling Man, drew on the war period, while Vidal's experiences at sea gave him the title for a volume of memoirs – Point to Point Navigation – and a combatant's jaundiced perspective which informed his long sequence of historical novels about the growth of American military ambition: Chronicles of Empire.
But the 1939-45 conflict (1941-45, in American terms) was not just a compelling subject for the country's writers; it was, for some, a passport to authorship. The Servicemen's Readjustment Act of 1944 (colloquially known as the GI Bill) was almost voted down by the nation's politicians – opponents citing anti-socialist objections similar to those afflicting Obama's healthcare proposals now – but it transformed the nation's education. Before this legislation, the level of college fees largely restricted entry to the children of the wealthy but a provision in the GI Bill to fund the studies of veterans democratised teaching. By 1947, just under half of undergraduates were recipients of this generosity.
Among them were Mailer and Bellow – who wrote early novels in Paris, courtesy of servicemen readjustment grants – and Heller and Vonnegut. Towards the end of his life – when we spoke in New York– Vonnegut had not forgotten the lucky consequences of war service for himself and others of his generation: "Heller and I would have been washing machine salesmen if it wasn't for the GI Bill."
The greatest of the novels that this legislation enabled Heller and Vonnegut to write are striking examples of the centrality of war to modern US literature. Both writers took two decades to turn their experience of conflict – Heller in the belly of bomber planes, Vonnegut as a PoW during the fire-bombing of Dresden – into books which, coincidentally, turned tragic events into savage comedy and had numbers in their name: Catch-22 (1962) and Slaughterhouse-Five (1969).
Because of their lengthy gestation, these novels accidentally became handbooks of the anti-Vietnam protesters, and this is a striking example of the overlaps that tend to occur in America's literature of conflict.
The same authors inspired and educated by the second world war remained involved – on the page at least – in subsequent 20th-century battles. Mailer published the polemic Why Are We in Vietnam? and The Armies of the Night, an account of a great anti-Vietnam march on Washington which records the literary odd couple he formed in that protest with Robert Lowell, the poet who had been imprisoned for conscientious objection during the war in Europe. And, in his final years, Mailer railed – as did his contemporary, Vonnegut – against the last American military intervention of their lifetimes: the invasion of Iraq. The latter, in A Man Without a Country, as a German-American once incarcerated in Dresden, even compared the administration of George W Bush to the Nazis.
During Vietnam, a Lowell poem predicted that America would be involved in "small war on the heels of small war, until the end of time". And, though we hopefully still have some time to go, this has so far proved accurate. A nation established by victory over the British – and, within a century, almost split by civil conflict – developed, after its unarguable role as the saviour of Europe, a doctrine of allegedly defensive interventions overseas which turned its authors into war reporters.
Even those who were teenagers during the second world war have contributed to the conflict literature: Roth, in The War Against America; John Updike in Terrorist; and EL Doctorow who, during the Bush years, published The March (a civil war novel) and Homer and Langley, set in the early 40s but in which the accounts of GIs sending home recordings to their families inevitably made us think of current troops in Iraq and Afghanistan. Stories of one war that clips at the heels of another.
And, in recent US history, definitions of peacetime have been relative: violent divisions over race, place and wealth – some of them dating from the civil war – have meant that even non-war stories are often conflict literature. The critic Harold Bloom told me that Cormac McCarthy's Blood Meridian (1985) – in which the wounds of the 19th century bleed copiously – has some claim to be the greatest modern American novel because it deals with the nation's deep tendency to violence. Bloom's view has perhaps recently been vindicated by the growing sense (helped by high-profile movies of No Country for Old Men and The Road) that McCarthy is now the country's most fashionable serious writer – although the 76-year-old from Rhode Island, who latterly adopted Texas as his home and literary location – has done almost nothing to encourage that popularity.
One of the major pleasures of my long investigation of American writing was meeting writers who have been heroes since I read as a teenager the Penguins and Picadors which – now yellowed and buckled – became research material 30 years later. Time and again, the jacket photographs miraculously came to life.
Norman Mailer, standing in greeting at the top of his tall house in Brooklyn Heights, with its view to the Statue of Liberty, and growling, in a perfect parody of his reputation for obsession with masculinity: "You're a big man. Do you box? You should box." Philip Roth skittish and wickedly jokey as the technical preparations were made, sombre and professorial as soon as the interviews began. Joyce Carol Oates, one of the most vociferous writers in literary history (around 150 publications, including all pseudonyms and genres), so softly spoken in a Princeton University office that she could hardly be heard over the purr of the heating. Toni Morrison, giving a magisterial reading and analysis of America on the brink of electing Obama. John Updike, arriving at a snowy Boston hotel, wearing a black knitted cap and clutching a Dunkin Donuts cup of decaf coffee.
And just hearing these voices was a kind of literary criticism. The theatre director Sir Peter Hall once said that if you want to know how a play should sound on stage, you should listen to the playwright speaking, because the tone of authors' prose or dialogue will generally reflect their speech patterns. And I thought of that as Edward Albee – on a summer day in a Soho loft filled with an impressive art collection made possible by the royalties from Who's Afraid of Virginia Woolf?, Three Tall Women and The Goat – delivered witty, twinkly, stinging sentences about his plays and his critics.
In this odd position of having coffee with set-texts, I also often thought of the narrator of The Catcher in the Rye who comments that a good story makes you want to get the author on the telephone and talk to him. But, as Salinger possibly realised with a quiet laugh even in 1951, the writer of those lines was among the few, in an age of strenuous literary publicity, from whom we never heard.
The paradox of Am lit is that it is notable for possessing both the most publicity-conscious writers in literary history – Mailer had an eye for photo-ops generally only found in reality TV contestants – and the most publicity-shy. Salinger refused interviews and public appearances throughout his career, an example followed by Harper Lee and Thomas Pynchon.
For decades, the only pictures of Salinger and Pynchon were school yearbook photos, captured before they took their vows of invisibility. Eventually, the Salinger gallery extended to two exhibits, when a paparazzo snapped him on an errand. Don DeLillo – who featured a reclusive writer in his novel Mao II – told me that this image of a startled old man looking over his shoulder at the shutter-click he had for so long avoided is one of the most upsetting he has ever seen. But that – as the illustrations to the obituary coverage showed – did not stop a couple of other cameras subsequently snapping him.
Perhaps the reason for this Mailer/Salinger dichotomy – one happy to run for public office, the other running from the clicking shutter – is that literary fame in the US is potentially so vast that responses need to be extreme: absolute promiscuity, total celibacy. Those who have tried to take a middle path of occasional cooperation – Roth, McCarthy – have suffered intrusive coverage and unwanted attention.
The level of visibility that a major writer is offered may be one explanation for the centrality of the self in modern American literature. Mailer, in a literary equivalent of a conversational tactic pioneered by sportsmen, frequently wrote about himself in the surname third-person, a tactic which can be seen as ego but which may also have acknowledged the increasing impossibility, in a time of furious curiosity about writers, of the observing character being a neutral "I".
In a similar strategy, Roth and Updike responded to the increasingly looming presence of the alter ego who was out there selling the books – and, often, being described and reviewed as brutally as the novels – by summoning up fictional surrogates.
Roth (Nathan Zuckerman), Updike (Henry Bech) – these novelists like to write about writers. Vonnegut's characters included a science fiction author called Kilgore Trout, who feels like a self-portrait, and three of the major novels of John Irving – The World According to Garp, A Widow for One Year and Last Night in Twisted River – have protagonists who are novelists. These authorial stand-ins can be viewed as self-indulgence but a more charitable interpretation would be that they are self-protection against the energetic efforts, in American letters, to appropriate a writer's identity.
Bellow, although offering no authorly surrogate as openly declared as Zuckerman or Bech, seems to have been a routinely autobiographical writer, once describing each of his novels as "a bulletin on my own condition". Fairly typically, when Bellow left the university where he was teaching for Bucharest, to visit the mother of his then wife, the result was The Dean's December (1982), in which an American academic takes a trip to see his mother-in-law in Romania. The story also incorporates, flimsily rewritten, two actual murders that had occurred contemporaneously in his home city of Chicago.
Such direct memoir is often seen as a weakness in fiction: "All the men are Saul and the women are the wives" has been a frequent complaint against Bellow's novels; Harold Bloom made a version of it when we met. But we only know because we know; if Bellow had done a Pynchon or Salinger, we might have taken the events in Bucharest as vivid imagination. And so one of the consequences of the industrialisation of publicity in the US book business has been to expose the origins of novels in a way that can then be turned against them.
Many of the nation's poets, however, have willingly participated in this striptease, without apparent misgivings. At least Bellow's bulletins on his own condition changed the names and occasional details. The output of a group of New England poets – Lowell (1917-1977), Sylvia Plath (1932-63) and Anne Sexton (1928-74) – perfected the genre of "confessional" verse, in which the life (and, in the cases of Plath and Sexton, likely future death by suicide) frequently seems to undergo little change beyond rhythmic shaping to fit the lines.
This verse was often literally therapeutic – Lowell, Plath and Sexton were all treated at the same psychiatric clinic in Massachusetts – but began a debate about whether the genre should sometimes be subject to an equivalent of medical confidentiality. Lowell – in Notebook (1969) and The Dolphin (1973) – quoted directly from the letters of an ex-wife. Whether or not this was ethical, it was true to two increasingly important ideas in American culture during this period: the primacy of the self and a prejudice that fact had more validity than fiction.
Those perceptions also drove an influential new genre which emerged at the same time as confessional poetry: the new journalism. Tom Wolfe (born in 1931) and Hunter S Thompson (1937-2005) overturned two well-cemented tenets of American journalism – the reporter as a discreet, objective presence, and a reverence for fact over opinion – to create a new strain of factual narrative in which the reporter is a star of the story. Books such as Wolfe's The Kandy-Kolored Tangerine-Flake Streamline Baby (1965) and Thompson's Fear and Loathing on the Campaign Trail (1973) introduced the devices of fiction to journalism and would eventually encourage the same development in reverse.
Perhaps conscious that arguably the finest work of new journalism had been written by a novelist – Mailer's The Executioner's Song (1980), which recreated, in visceral physical and psychological detail, the life of the murderer Gary Gilmore – Wolfe responded, within a decade, by producing the finest novel written by a new journalist: The Bonfire of the Vanities (1987). In promoting this book, he also provoked a long-running and entertaining feud with career novelists – including the New England Johns, Irving and Updike – by suggesting that their work was insufficiently observant of the real world.
This energising slippage between fact and fiction continues in the work of two of the most exciting talents of the new generation: Jonathan Safran Foer's Everything Is Illuminated (2002), published as fiction, and A Heart-Breaking Work of Staggering Genius (2000) by Dave Eggers, released as non-fiction, are both genre-crossing family memoirs that combine agonising truth with storytelling tricks and have unreliable narrators with the author's own name. True to one of the key developments in modern American writing, Safran Foer and Eggers achieved literary celebrity through first books that acted as though they already had it.
The ambition of the nation's prose writers is a commonplace of American literary studies: the idea that its authors are competing to compose the great American novel. But this contest is probably a myth – wasn't it won, as early as 1851, by Herman Melville's Moby-Dick? A different source of extraordinary boldness and scope is American theatre.
Between the eve of the second world war and the beginning of the 1960s, a series of plays appeared which revolutionised American drama: Thornton Wilder's Our Town (1938), The Glass Menagerie (1944) and A Streetcar Named Desire (1947) by Tennessee Williams, Arthur Miller's Death of a Salesman (1949) and Who's Afraid of Virginia Woolf? (1962) by Albee. Each of these dramas is set predominantly within a domestic residence of its era and has a surface of realism; each has become a standard of the classroom and the provincial theatre repertoire, with the stamp of conservatism that such endorsements inevitably bring.
Yet all of these plays contain significant non-naturalistic or experimental elements: dream sequences or flashes forward or back. Seeing Our Town last year – in the acclaimed off-Broadway revival by David Cromer for the Barrow Street Theatre – I was startled by the darkness and strangeness, in both structure and tone, of a script which I remembered as a linear hymn to small-town life. No sooner are characters introduced than the audience is told of when and how they will die horribly; an entire act takes place in a graveyard filled with people looking back on unfulfilled lives.
British theatre did not achieve a radical change in content and form until the 50s and 60s – driven first by John Osborne's stable-cleansing Look Back in Anger and then the abolition of censorship by the Lord Chamberlain's office – but the equivalent breakthrough in the playhouses of the US occurred at least a decade and a half earlier.
It is also notable that America's dramatists, though the mecca of their profession has always been the commercial stages of Broadway, consistently questioned the optimistic rhetoric of politicians and businessmen about the supremacy of its way of living. The dominant figure of postwar American drama is the fantasist or liar with a life which is in some way unsustainable: Miller's Willy Loman, Williams's Blanche DuBois, Albee's George and Martha.
This radicalism of tone and structure continued among the younger generation of dramatists. Though the leader of the new pack is a minimalist – David Mamet, whose plays, including American Buffalo and Glengarry Glen Ross, rarely detain the audience beyond two hours – US stages still spawn plays of a scale more commonly associated with multi-episode television serials.
Tony Kushner's Angels in America (1992) runs, across its two parts, for around six hours and, as its subtitle ("A Gay Fantasia on National Themes") makes clear, follows those pioneering plays of the immediate postwar period in mixing the naturalistic with the non-realistic and even the supernatural. So too does August Wilson's The Pittsburgh Cycle (1982-2005), which has an architecture even larger than Kushner's: 10 plays, each dealing with a different decade of African-American history in the 20th century. Wilson also moved freely between fact and fantasy: his characters include a 322-year-old woman.
The most recent serious play to become a box-office hit on Broadway – August: Osage County (2007) by Tracy Letts – is another of these daring constructs: a three-act, three-hour-plus attempt to show that domestic tragedy can still be written in an ironic age. Though working within a system that worships commerce – Miller, Williams and Albee all suffered spells of neglect in which they were grateful for subsidised theatre in the UK – American playwrights have, when it comes to form and politics, consistently dared to go for broke.
Writers are frequently seen as being unworldly figures, but, as it turns out, the White House and the CIA would have been better prepared for 9/11 if they had read American novelists and dramatists rather than field reports. After the attacks, the intelligence community reportedly consulted Hollywood screenwriters about likely future threats, having spotted that movies such as Die Hard anticipated the methods and level of terrorist threat to the US, but they might just as fruitfully have called in DeLillo, Charles McCarry and Kushner.
DeLillo's most resonant books so far have examined the politics of the American past – Libra (1981), about the JFK assassination and Underworld (1997), exploring the cold war era – but his earlier fiction proves to have been percipient. Though the threat of terrorism entered general consciousness in the US only after 9/11, it figured in DeLillo's work from the 70s, an insight he attributed to having lived in Greece.
McCarry is a former servant of the secret world – working as a CIA agent under deep cover in Asia and the Middle East during the cold war – who now has some claim to be the best-kept secret on the great American writers shelf. His The Tears of Autumn (1974) is one of the three best literary explorations of the JFK assassination – the others are Libra and Mailer's Oswald's Tale (1995) – and the one which perhaps explains most plausibly what happened.
Though far less well known than John le Carré, McCarry has been just as geopolitically aware and is the writer who came closest to directly predicting the 11 September attacks. His book The Better Angels (1979) includes suicide bombers sending planes against America, directed by an Arab malcontent whom contemporary readers will inevitably visualise as Osama bin Laden. Kushner's play Homebody/ Kabul (2001), written before the attacks, includes an Afghan character warning Americans that the Taliban are "coming to New York".
In the interviews they gave at what turned out to be, in too many cases, the end of their lives, the great fictionalists of the US were almost uniformly gloomy about the future of serious writing. Mailer and Updike detected the retreat of a readership for complex stories. Among living practitioners, Albee feared that Broadway ticket prices mean that only sentimentality and spectacle can sell, complaining of the "middlebrowism that is afflicting American theatre because it is a commercial theatre".
Roth was also concerned about a coarsening of culture: "The population of intelligent, attentive readers capable of concentration and focus of the kind that is required by a serious novel . . . has decreased. Not because there aren't the same number of intelligent people around but because they have been torn away like Lady Macbeth says she tore away the child from her breast. They have been torn away from the breast of literature by the screen."
Vidal, with characteristic dyspepsia, argued that America cannot have suffered a cultural decline because "we never had a culture", but accepted that his earlier work was published at a more receptive time: "The attention of readers has shifted away . . . it feels to me very much like a dying moment for literary culture in my country."
The history of sport, though, warns us that the great players of the past are prone to believing that the finest achievements belonged to their own era and will not be bettered by the disappointing generation which follows.
A more optimistic reading is that intelligent literary culture will adapt to the new conditions of the marketplace and may be revived, as the country always has been, by immigration. The Jewish-American, Irish-American, African-American and European-American writers of the great postwar generations may be followed by authors who are, say, Indian-American (Jhumpa Lahiri, left, with Unaccustomed Earth), Dominican-American (Junot Díaz's The Brief Wondrous Life of Oscar Wao) or Korean-American (Chang-rae Lee, whose novel The Surrendered, published this spring, extends the nation's rich war literature by treating the Korean war from an Asian perspective). With these books and others, a new phase is beginning.
Capturing America begins on Radio 4 on 11 February at 11.30am. To listen to Mark Lawson's interviews with American writers visit bbc.co.uk/radio4/