The Digital Dark Ages

Nicholas Ward
Jul 7, 2021
10 min read

The internet never forgets.

This simple axiom shapes many young peoples lives. And we carefully take to sculpting our social media profiles for respectability. Knowing with certainty that the vast amounts of information that we put on the internet is there to stay forever whether we want it to be or not.

But what if I told you not only is that not true but that digital information is so impermanent. That we may now be living in one of the greatest historical dark ages of all time?

Digital information has over the past 30 years taken over everything. And every year more organizations, switch to paperless. We are spending hundreds of millions of dollars digitizing archives, libraries, and records departments.

Because of course we are. Where microfilm could store thousands of pages in a few square inches. Digital can store hundreds of thousands. And allow it to be accessed by anyone anywhere at any time.

But what’s cost? How long does digital information last?

Forever right? I mean the answer has to be forever…

Try 5 to 10 years.

The problem with digital hard drives is that they degrade incredibly fast. You can extend the life of a hard drive by using it regularly. But beyond 5 years a hard drive is almost worthless for archiving large or important files, and beyond 10 it is a liability.

Now this problem is at the moment largely invisible because we can keep up a constant rate of using and replacing hard drives. Which is the problem. Hard drives need to be constantly used to keep them viable so how do you safely maintain that data long term? The answer is you don’t all you can all you can do is to keep copying those files into new data banks every 5 to 10 years.

Now for an individual the risk is low. But for corporations, governments, and universities, groups who maintain large and extremely important archives. There is no safe way to store your data.

Take Hollywood. How do multi billion dollar companies at the pinnacle of tech store their priceless archived films? Massive data banks? Solid state drives? No they use old school 16mm and 35mm analogue film. Even today movies shot in digital, are converted to film to archive.

Hollywood studios did briefly try archiving digital but the costs were so monumental they abandoned them.

Physical film is a stained piece of polyester, and so long as it is stored in a cool dry place can last upwards of a thousand years with zero maintenance.

And this is why all major archiving organizations like the library of congress still use film and microfilm

See digital information has the archival shelf life of warm milk. But we have traded safety for convenience and today all our information is just 5 to 10 years away from poof disappearing. Even short interruptions to power grids or global trade networks could be catastrophic for global data.

One of the only ways to store digital info long ish term is Magnetic tape. Magnetic tape can very safely and easily be archived for 30 to 40 years. It’s fairly cheap the expensive part are the special readers you need which can cost thousands of dollars. However, a couple of square inches of magnetic tape can hold hundreds of terrabytes of information. And even when corrupted info on magnetic tape can often still be saved.

So why don’t we use magnetic tape to store digital info? Well we used to. They were called VCR’s and floppy disks.

Some companies like google are still backing up much of some of their data on magnetic tape.

However in the long term the few decades that magnetic tape lasts is pretty pathetic from an archival standpoint.

And you run into the next major issue in our growing digital dark age. digital obsolescence.

Any information today still left on floppy disks. Is pretty much gone. Most people don’t have floppy disk readers anymore and all that info’s just going in the bin. CD rom drives are disappearing now as well. Even hard disk drives are moving on with interfaces shifting from USB A’s to USB C’s and the prevailing disk getting replaced with Solid State Drives. In 10 years if you don’t constantly update your storage all of your data might become completely unreadable simply from technological obsolescence.

Digital obsolescence is a huge threat to contemporary record keeping. And one of the chief concerns of academics archivists and tech professionals fretting about modern archiving and the digital dark age

It has become the pet project of Google Vice President Vint Cerf. Cerf’s wants to archive all outdated software and hardware creating x ray blueprints of outdated hardware, like floppy disk readers, then storing them in the cloud. So that future generations always have the option to recreate these machines and read outdated formats.

The internet archive is another group who are trying to tackle the problems of digital obsolescence. The Internet archive claims to have archived over 500 billion webpages. Their founder Brewster Kahle founded the non profit in 1996 with the humble goal of becoming the repository of all knowledge. And over the last two decades his nonprofit has done some truly remarkable work digitizing 28 million books, 14 million audio recordings, 6 millions video, 3 million images and 500’000 software programs.

And 100 life size ceramic figure of the employees of the archive.

At their headquarters in San Francisco his team has assembled a vast collection of old tech, microfilm, cassettes, records, and basically whatever else takes their fancy.

Like Cerf their idea of preventing the digital dark age from obsolescence is to preserve the tech and software of every generation so that obsolete files can always be read.

Which is great…

But of the ice berg that is the digital dark ages the part that is digital obsolesce is not the part sticking out of the water, but a single snow flake on the mountain of an iceberg the size of wales.

See many archivists and tech professionals are quietly sounding the alarm. That the massive and unstoppable move to digital could very easily be catastrophic. Because there are so many events which could plausibly wipe out worldwide digital records permanently.

A solar flare, or nuclear blast, power surges, deliberate attacks, or natural disasters, even just time. Could all spell complete annihilation for massive amounts of global data.

Because let’s go back to that idea by Vint Cerf, of archiving using the cloud to future proof data. The cloud is not real.

The cloud is just massive data banks stored around the world.

Virtually all of google Microsoft’s and apples cloud storage is stored in San Francisco bay. And those magnetic tape archives of google are largely stored in silicon valley.

And San Francisco happens to be one of the least geographically stable places on the planet lying dead on the San Andreas fault line.

Even just a temporary interruption to the maintenance of data banks due to anything from economic problems to natural disasters could have devastating effects on digital information.

But you know earthquakes are one thing. But what are the odds of long term power cuts otherwise?

Oh hello year 2000 Enron. What’s that you’re off to manipulate California’s power network. And your going to trigger rolling blackouts across California? Well good for you. Burn baby burn.

Currently data banks require near constant maintenance to keep them functioning. And historically 5 years is just not a long period of time.

How much of your data would you lose if Apple, Google and Microsoft lost all of their cloud storage.

Now just losing cloud storage probably wouldn’t cripple the world. But you know what could cripple the world the near total loss of all digital data.

See digital data is generally recorded using magnetic recording. Do you know what destroys magnetic recording. Powerful magnetic fields.

Which is why when my new magnetic super weapon comes online the world will tremble before my might

Bow down peasants! You’re new god is come!

Sorry those are my… other notes.

Today when organization’s want to wipe data they stick them in special containers and hit them with a powerful electromagnetic pulse EMP which completely wipes them.

If you’re wondering how common EMP’s are. EMP’s are everywhere from the spark of a car battery to lightening.

To nuclear explosions. A nuclear bomb triggers an EMP. And if detonated in the atmosphere the resulting EMP would be strong enough to knock out power grids and data across an entire continent.

Unsurprisingly militaries are working on weaponized non nuclear EMP’s

But nuclear war can seem like one of those far away future things that we don’t need to worry about right now.

But you know what’s not? Solar flares and storms. Solar flares are an explosion of energy from the sun and when this energy hits earth it can have devastating effects on electronics. Normally solar flares are too small to interfere with day to day tech. But not always

The largest solar storm on record the 1859 Carrington Event was so powerful there were auroras over Hawaii and it knocked out telegraph systems around the world setting cables on fire.

The worlds most recent major solar storms hit in 2000 and 2003 causing damage to power networks around the world. A small storm in 89 saw 6 million people lose power for nine hours.

But do you want to know what’s even more terrifying? Of course you do.

In 2012 a solar storm the same size as the one in 1859 missed earth by just nine days. Do you want to know what the effects of a storm like that would be in the 21st century and how prepared we were for it?

You don’t but I’m gonna tell you anyway.

Research by the Atmospheric and Environmental Research Inc. A consultancy firm Estimated the economic cost of the storm would be between between 600 billion and 2.6 trillion US dollars, for the US alone. And the American North East corridor would suffer long term blackouts ranging from 16 days to 2 years.

If that wasn’t terrifying enough its report opens with:

"A Carrington level, extreme geomagnetic storm is almost inevitable in the future."

Another study a year later by UC Berkley and China’s State Key Laboratory of Space Weather, estimated it would have taken the world between four and ten years to recover from the damage.

And that is only of the largest storms that we know of. Because an even larger storm is entirely possible.

Oh and did I forget to mention that in a strong enough EMP not only would power grids and your electronic devices get destroyed. But your car wouldn’t work either.

But how would the destruction of electronics disrupt a mechanical internal combustion engine.

Well it wouldn’t. What it would destroy is the computer that your car needs to run. See almost all modern cars computers are so integrated into their operating they can’t run without them. Turning your car into an expensive lawn ornament.

And here you thought the worst consequence of the digital dark age was losing your half finished novel and cat gifs. No no no.

Worst case scenario is almost complete economic and social collapse. Because imagine trying to coordinate providing food for the world without cars, phones, the internet, computers, most planes, or trains.

Unsurprisingly researching this took me deep into the realms of survivalists forums full of hilariously made up statisitics like one kept coming across “Experts estimate within 18 months of an EMP attack 70 – 90% of Americans would be dead DUN DUN DUN.

I added the dun dun dun. I was curious who these, experts were, and after a little digging. It was from a group called the united west, whose mission goals is "dedicated to defending and advancing Western Civilization against the kinetic and cultural onslaught of Shariah Islam."

And their experts? What research did they conduct? Well they didn’t say. But I’m sure that is a totally factual claim.

But let’s get back to the digital dark age. Because an Armageddon level EMP would not be necessary to trigger massive data loss.

As we’ve become more dependent on digital. We have started shielding hard drives against minor EMP’s.

But the only way to properly shield electronics from EMP’s is to use a faraday cage. Which just isn’t a large scale solution.

Leaving all of our digital date completely at the mercy of fate.

And like an earthquake a colossal solar flare could happen at literally any second plunging the entire world into indefinite darkness.

Yeah that really makes the internet archive and Vint Certs plans to back up digital data using digital data seem kinda dumb doesn’t it.

As we prioritise digital more and more, the historical implications of that become increasingly catastrophic.

And it is entirely possible, we today, right now, are living in what will be one of the greatest black spots in history.

But before you panic and start carving all your tweets into stone there is hope and a few archivists have banded together to create doomsday proof archives.

Norway’s famous doomsday seed vault has now built an archive deep in an old mine shaft. Where companies and governments can pay to archive their digital files by the millennia. How do they do it? They imprint QR codes onto analogue 35mm film

Oddly enough even if every human disappeared tomorrow and a million years passed leaving even those doomsday archives as nothing but dust there would still be one archive left on the planet. Mount Rushmore. Mt Rushmore is designed to have identifiable faces on it for a minimum of 2.4 million years.

So, in 2.4 million years when the slug people begin populating the earth we can take comfort that Mt Rushmore is going to confuse the ever living crap out of them.

But there is one more hope for archives that promises to not just imprint data for thousands or even millions of years, but billions.

Crystal storage.

If you’re a scifi fan this will probably sound familiar. Crystal storage has been a favorite of scifi for decades. But as with all good scifi it’s based on real science.

Crystal storage or 5D optical storage is a process of permanently writing digital information into fused quartz. And don’t ask how because I don’t know.

And in theory this information will last billions of years. Crystal storage was first seriously proposed in the 1990’s. And the very first information imprinted into Quartz was demonstrated in 2013 by the University of Southampton.

The university produced a series of disks about the size of a CD which can contain 360 terabytes of information. Can be stored for up to 14 billion years at temperatures up to 180 degrees Celsius. And withstand half a ton of pressure.

The future is now!

Unsurprisingly archival groups are now champing at the bit to get access to the tech for their own files. But that may be a long way off. Because this is a brand new technology.

For instance though the claim is the data will last billions of years…Microsoft own crystal storage research is aiming for the much more modest time frame of 10’000 years.

Likewise the claim that each disk will hold 360terrabytes is theoretical at the moment. Because the current demonstrated copies hold information like the Magna Carta, and the UN Declaration of human rights. Which are only a few kilobytes of data.

Also the term 5D data strorage is marketing jargon.

Unsurprisingly the hype is outstripping reality. At the moment costs from equipment, to power, to materials, is astronomically high and bulky. And write times are slow. So don’t expect crystal storage in your laptop any time soon.

In theory Crystal storage will be the archival material of the future. But at the moment it seems to be little different to the gold records sent out on the Voyager space craft.

People much smarter than I am seem very confident however that this is a technology that will only get better. So in a few decades we will hopefully have long term effective digital