Revealed, the one battle Winston Churchill most feared losing during the Second World War

Bitterly fought over three million square miles of hostile ocean, the struggle to prevent Hitler's U-boats - his "grey wolves" - starving Britain into submission began on the first day of the war and ended on the last. Casting his mind back over five bloody and uncertain years, Winston Churchill later declared the "U-boat peril" was the only thing that ever really frightened him. Battles might be won or lost but the country's very existence depended on the Atlantic. As the PM said in June 1940: "Without victory there is no survival."

Yet despite the existential threat, little was reported at the time. In the words of one Royal Navy veteran, it was an "unseen war", and the seamen who paid the ultimate price have no grave but the ocean. When war began the country looked to the Royal Navy, historically the world's most powerful, for protection. John Adams, a young lieutenant on destroyer HMS Walker, remembered thinking: "It's going to be bloody but we've been pretty good on the sea for centuries, we'll get through."

The Navy was confident it could prevent vital imports reaching Germany and, at the same time, secure Britain's lifeline. Before the war, we imported 60 million tons of food and raw materials a year and every drop of oil - mostly across the Atlantic from America - via a merchant fleet of 3,000 vessels. By concentrating ships into fleets of 30 or more, the convoy system was able to reduce the number of targets for the enemy. What's more, the Admiralty was confident its new echo-sounding sonar device would strip German subs of their "cloak of invisibility".

The U-boat had taken Britain to the brink of defeat during the First World War. But by 1939 the German navy was a shadow of its former self, with only 27 subs capable of Atlantic operations. Their commander, Karl Dönitz, was confident a larger fleet could win the war but he needed "the boldest of bold enterprises" to convince Hitler. His plan was an attack on British battleships in their historic home base, Scapa Flow in Orkney.

On the night of October 8, 1939, U-47 slipped through the Royal Navy's defences and fired four torpedoes into the battleship Royal Oak. The sea flooded through savage holes torn in her side. Most of her 1,200-strong crew were asleep below decks. "They had no chance," survivor Joe Instance told me, with tears in his eyes. "The last torpedo set off the cordite magazine. The chap alongside me went up like a match."

The Royal Oak sank in just 13 minutes, taking 833 officers and men with her. Instance was one of the lucky few to be pulled from the sea. "I was burnt to buggery," he recalled. The German U-boat crew was subsequently flown to Berlin and driven through streets lined with cheering crowds to the Reich Chancellery, where Hitler presented its commander, Gunther Prien, with the new Knight's Cross of the Iron Cross. The idea began to take shape in Hitler's mind that the unthinkable was possible: the Royal Navy could be beaten. But while the sinking of the Royal Oak was great propaganda, the U-boat fleet was still too small and too far from convoy routes to threaten Britain's lifeline.

That was to change with the defeat of France in June 1940. Dönitz wasted no time transferring his U-boats to ports on the French coast. Instead of the long passage from Germany, they would now be able to sail directly into the North Atlantic.Roaming far into the ocean, his crews scoured the horizon for plumes of smoke indicating an approaching convoy. When seen, a signal would be sent to U-boats nearby to assemble into a "wolf pack".

In October 1940, Slow Convoy 7 was on its way from the Canadian port of Sydney when it steamed into a pack of five U-boats. "We were making six knots. You could walk faster," Liverpudlian Frank Holding recalled. His merchant vessel, the Beatus, was hit and sunk but not before the crew could take to a lifeboat. Those aboard the nearby Cree were not so lucky - their ship was carrying iron ore and sank like a stone.

The Navy had placed great faith in its sonar underwater detector but the enemy was using the cover of darkness to launch surface attacks. "I remember feeling so helpless when we saw these ships being sunk that night," James Keachie, a lieutenant with convoy escort HMS Bluebell, one of the Flower Class corvettes immortalised in the 1953 film The Cruel Sea, told me. "What were we supposed to do?"

As the night dragged on, his ship abandoned her hunt for the enemy and trawled for survivors. In the end, 20 merchant ships were lost from SC7 - and with them thousands of tons of essential supplies. A veil of secrecy was drawn over the disaster, with survivors warned not to speak of their ordeal. In the six months from May to December, Dönitz's U-boats sank 300 ships. U-boat crews returned to their new French bases to a heroes' welcome. This was their "happy time" - they were the new heroes of the Reich. But within a matter of a few months many of those heroes were either dead or in captivity.

In the spring of the following year, Dönitz was to lose his three most successful U-boats. The British code-breakers at Bletchley Park had cracked the Enigma cipher used to direct Atlantic operations. With this priceless intelligence, the Navy began to reroute convoys away from wolf packs. Dönitz's U-boat fleet was growing but so was the number of sinkings. By autumn 1941, his high hopes were forgotten. To make matters worse, American ships had begun to screen British convoys on the first leg of their journey across the Atlantic.

Hitler had given orders not to attack US warships. But in the early hours of October 31, U-boat ace Erich Topp torpedoed the USS Reuben James, cutting the ship in two. The forward section exploded, sending flames, twisted metal and bodies into the night sky. Fuel from its tanks spilled into the sea, covering survivors in choking oil. From the water they heard the screams of those trapped inside the sinking ship. "I only found out about all this much later," Erich Topp told me. "I couldn't get the catastrophe out of my head."

Only 45 of the US warship's crew of 160 men were rescued. She was the first US navy ship to be sunk and her loss was a profound shock. A little more than a month later Germany and the United States were at war. Dönitz was delighted. He believed America had been waging a stealth war on his U-boats for months. He outlined his plan for an attack on shipping off the US coast. Hunting in "virgin waters" would be good. Six long-range U-boats were dispatched to sink ships off the US coast before they could assemble into convoys.

Reinhard Hardegen's U-123 was in this first wave. "Ships were brightly lit," he told me. "It was incomprehensible. They were totally unprepared for what was about to happen." He was confident enough to surface and finish ships with the deck gun. This was the start of the "second happy time".

Tempers in London and Washington frayed. The US navy was too slow to introduce a blackout and convoy the ships in its waters. But the Allies had not stood still. Not only were they able to read the enemy's signals but escort vessels were now equipped with radar and radio-direction-finding equipment to track U-boats on the surface. Their escort shield could also count on air support from long-range U-boat hunters like the B-24 Liberator.

The Battle of the Atlantic was reaching a crescendo and the stakes could not have been higher. The U-boat menace had to be defeated before the Allies could contemplate D-Day. Spring 1943 was the decisive turning point. In March, Dönitz was able to concentrate 40 U-boats in the "black pit" south of Greenland where convoys could not count on air support.

The Allies lost 120 ships, the fifth highest monthly total of the war. There was panic in government but naval intelligence took a different view. It was, it declared, "a gambler's last throw" with everything in place to deal a crushing blow. The opportunity came in April 1943 when convoy ONS 5 was intercepted by a super-pack of 53 U-boats - the largest concentration ever assembled.

ONS 5 was protected by the seven ships of the B7 escort group - experienced, well commanded and equipped with radar and wireless direction-finding sets, plus the latest depth-charge throwers. The battle raged for a week, with the escorts repelling multiple nighttime attacks.

Seven U-boats were sunk, seven more damaged and, despite overwhelming odds, all but 13 of the convoy's ships were saved. It was a triumph. After ONS 5, the convoy battles followed a new pattern: U-boats sunk for little or no loss. Days later, five submarines were lost in an attack on ONS7 with only one ship sunk. Aircraft and well-equipped escorts robbed the U-boats of their power to track convoys and attack on the surface.

The hunters had become the hunted. The Germans would fight on for another two years but the outcome was no longer in doubt. Allied aircraft began to pursue the U-boats to their bomb-proof bunkers in France while new Allied merchant ships were racing down the slipways of America.

By the end of the year, 231 frontline U-boats had been lost and morale was at rock bottom. The success of D-Day in June 1944 was the final humiliation. Dönitz flung "every available boat" at the Allied armada in a kamikaze-style attack. "Press home your attack, even at the cost of your boat," he told crews. It was to no avail. The grim grey walls of the U-boat memorial near Kiel hold the names of more than 30,000 men. Of the 859 U-boats that carried out patrols during the war, 648 were lost - one third of them on their first voyage.

The grim balance sheet records that 3,500 Allied merchant ships were sunk in the North Atlantic and British home waters. The merchant service lost more than 32,000 men, the Royal Navy 50,758. Was Allied victory in the Atlantic ever in doubt? An official historian for the Royal Navy believed so. "We knew beyond doubt that the peril in which our nation so long stood derived mainly from those utterly ruthless enemies, and that only by destroying them could we survive," he wrote.

But victory was won at a terrible price.

• Andrew Williams is author of the Battle of the Atlantic (BBC Books)