The Brontobyte and Big Data: The Future of Data Storage
If you were to hypothesize on the future of big data, what comes to mind? Cloud Storage? Data centers? Software-Defined Storage? Or maybe you are asking yourself:
- What is used to store data for future use?
- Will we run out of data storage?
- Is DNA the future of data storage?
- What is the future of SSD?
- What is the Future of Data Storage?
Predicting the future of stored data is anything but exact, however, if you look at the trendlines you will see a convergence of sophisticated new emerging technologies and methodologies that provide a glimpse into the foreseeable future. This will make storage management more flexible, smarter and easier to manage.
Right now IT leaders everywhere are looking for more productive ways of dealing with the oncoming tidal wave of data that will be generated by AI and IoT. Statista’s market survey report demonstrated that the total amount of data being consumed globally was forecasted to increase to 64.2 zettabytes in 2020 and 79 zettabytes in 2021 while it is projected to grow to over 180 zettabytes up to 2025.
Furthermore, Forbes predicted that more than 150 zettabytes or 150 trillion gigabytes of real-time data will need analysis by 2025.
At the time of publication, Google was performing just under 4 billion searches per day, that’s 97,000 per second. With the data deluge that companies like Google, Amazon, Facebook, and Microsoft are subjected to, they typically measure their data in Exabytes (that’s 1,000,000 terabytes).
While today most people are familiar with the terabyte, a handful of years ago few in the public knew what a terabyte was, now it’s the norm.
What comes after the terabyte? According to the International System of Units:
Petabyte
Exabyte
Zettabyte
Yottabyte
To give you a comparison, the Yottabyte is approximately 1 septillion bytes or equivalent to a quadrillion gigabytes (GB).
So where is the Brontobyte, why doesn’t it appear on this list? It has yet to become a standard unit for data storage. This means that the Bronto is only a theoretical quantity of data at the present time. Does it mean that you should not pay attention to it, heck no.
Storage Sizes and Mediums
Humans have been recording data since around the 4th millennium BC, ever since the invention of writing. Handwriting, phonographic recording, magnetic tape, and MP3’s are all examples of storage media. Some even believe that human DNA storage is the future of data.
As far as machine-readable medium or digital storage is concerned, it all started with the ‘bit’. According to Wikipedia, ‘the bit is the most basic unit of information in computing and digital communications. The name is a contraction of binary digit’.
If the digital age is defined as the age where more devices stored more digital information than on analog, then you would get to the year 2002, according to Science (journal) You see, by 1986 the world’s data only had about 1% stored in digital format (that’s only about three compressed exabytes). By 1993 it grew to 3%, by 2000 to 25% and by 2007, a massive jump to 97%. With new storage technologies available, storage started to double about every three years and server data centers exploded.
Data Storage Units Chart
| Unit | Abbreviation | Capacity |
|---|---|---|
| Bit | b | 1 or 0 (on or off) |
| Byte | B | 8 bits |
| Kilobyte | KB | 1024 bytes |
| Megabyte | MB | 1024 kilobytes |
| Gigabyte | GB | 1024 megabytes |
| Terabyte | TB | 1024 gigabytes |
| Petabyte | PB | 1024 terabytes |
| Exabyte | EB | 1024 petabytes |
| Zettabyte | ZB | 1024 exabytes |
| Yotabyte | YB | 1024 zettabytes |
Current State of Data Storage
NVMe, SSD and Computational Storage
As the need for storage capacity grows, there have been some pretty sizable advances in data storage technologies, with flash media (non-volatile memory), cloud storage, shingled magnetic recording, and solid-state drives. IT managers are currently developing plans and strategies on how to deal with the massive wave of data set to come and led by the AI and IoT revolution. The applications that run AI and IoT require even larger volumes of high-performance storage options.
Currently, data is stored in the main memory by the host server, leaders are looking for an approach where processing can be done at the storage layer. Edge technology enabled by maximum performance SSDs move computers closer to the storage devices and is a common trend we are seeing become increasingly more mainstream. This is why many are looking at computational storage systems. This delivers in-storage compute technology, bringing high-performance computing power to storage devices.
It will process and analyze the data where it is created and stored. This empowers an organization and its users to perform a variety of actions at the device level like filtering data before sending it back to the cloud to store in arrays.
This is why more IT managers are planning to transition to NVMe and container-based infrastructures, and have already started identifying applications that could benefit from enhanced efficiencies of computational storage.
With NVMe and SSD, there are numerous performance upsides which is why they are currently dominating storage marketing. Although talk of the demise of HDD is just that, talk. For the foreseeable future, hard disks will always be around in some shape or form. There will always be technologies and case uses that will benefit from the cost-effectiveness of hard disk drives such as Cloud computing or Chia Farming.
Software-Defined Storage
According to Red Hat
“Software-defined storage (SDS) is a storage architecture that separates storage software from its hardware. Unlike traditional network-attached storage (NAS) or storage area network (SAN) systems, SDS is generally designed to perform on any industry-standard or x86 system, removing the software’s dependence on proprietary hardware.”
SDS has a number of key benefits and many companies are switching to this model. It allows administrators and IT managers to consolidate and manage all of their data centrally from one place while delivering features and functionality.
SDS is virtualization software, permitting you to control and manage physical storage in ‘virtual pools’. It also offers thin provisioning for maximum storage capacity and once the pool is generated, “virtual disks” are also created in the pool as raw LUNs and then sent to your host servers to store data. The best feature of SDS controller software is that it works without concern about the capacity or usability of the hardware. It takes what it finds and maximizes capacity.
By switching to a virtual model, you will be able to access all of the possible storage in your virtual machines. This offers a unified view, extending the range to server storage and hyperscale server data centers to edge computing.
All of these trends have come about due to the unrelenting growth of data and gargantuan cloud data centers, driven by hyperscalers and convergence. Increasingly, software is expected to be the way we control the storage of data in the future. We are entering an era where we will substantially consolidate resources, based on the protocol convergence that underlie the physical infrastructure of computing, providing a stronger foundation for operational structure.
Test your knowledge and company memory vs storage.
What is Driving the Storage Revolution?
What we do know is that the need for more data center storage is rapidly increasing and as people adopt IoT and AI, the need will continue to grow. Every day people use Alexa, add new features to their Apple Watch, install a Google Nest, Ring and on and on. The current trend in big data currently seems to have data coming from three primary sources.
Consumer data from social media giants like Meta, TikTok, YouTube, Pinterest, LinkedIn and others consume a large portion of the data storage pie. Likes, Shares, Video Uploads, Comments and Messages equate to space on a drive. Equally important are public utilities and online services such as Google’s suite of free services. Many SaaS business apps are using the freemium subscription model and will offer free storage, trials, online support with tutorials and communities, hoping their subscription fees will grow with the user’s business.
A less-talked-about sector of data storage comes from industry using sensors on equipment and devices. Machine data is expected to grow progressively with the increasing adoption of AI and IoT. From health to military applications and from space to automotive, these devices and sensors will capture data and send it to other devices across the internet which will track and store it.
The third type of data is Transactional, stemming from all of the transactions that happen each day in your life and mine. Credit cards, bank transactions, grocery purchases, online purchases, points, they all get stored, likely in an array, safe and secure.
Want to know the difference between data storage types? (link when posted)
The Future of Data Storage
While the Brontobyte may be theoretical, you can bet that it will get standardized in the not-to-distant future. The dimensions are enormous. In decimal notation, 1 brontobyte is 1,000,000,000,000,000,000,000,000,000 (1027) bytes. In binary notation, 1 “brobibyte” is 1,237,940,039,285,380,274,899,124,224 (290) byte. Currently, the largest quantity of data is the zettabyte. Just to give you some scope, in 2016, worldwide traffic exceeded one zettabyte, bringing us into the zettabyte era. Imagine, one brontobyte is equivalent to 1,000,000 zettabytes.
Can you guess who is the world’s largest data company? As you may have guessed, it’s Google. They store about 10 exabytes (that’s 10 billion GBs!). Facebook on the other hand has about 2.5 billion individual content pieces, 2.7 billion likes and 300 million photos.
You would think that the company with the most data, would have the most servers. If you guessed Google, you would be mistaken, the answer is Amazon. It is difficult to pinpoint exactly how much data they store, but in 2015 it was estimated to be around 1,000,000,000 gigabytes of data across more than 1,400,000 servers.
But what is bigger than a brontobyte?
There is now talk in Big Data circles about the next theoretical level above the brontobyte: the gegobyte or geopbyte. The equivalent of 1,000 brontobytes.
With data coming from industry, the public web, transactions, the future will look much different with newer industries like space and satellite driving huge volumes of data. Streaming technology combined with a pandemic converged to foster the current golden era of television, reaching unprecedented levels of video storage. Sensors, devices, quantum computing, smart homes, and autonomous cars will all drive us into the Brontobyte era.
From laptops to servers, the primary unit of measure for data is in the vicinity of the terabyte, which evolves with storage technology capacities. Apparently, if you had all of the hard drives in the world, you still would not have enough capacity to store one brontobyte. Like much in tech, we do not know exactly what emerging technologies will bring or how soon they will arrive
Planning For Your Needs
ServerMania has been helping businesses with their data and storage needs for more than 20 years and is committed to providing our customers with reliable, satisfying experiences. We have eight data centers across the world to serve all of your needs. Book your consultation today with one of our knowledgeable reps to learn more about Storage server hosting.
About the author
