Big data is aggressive in its production, and with the merger of Cloud computing and IoT, the huge volumes of data generated are increasingly challenging the storage capacity of data centres. This has led to a growing data-capacity gap in big data storage. Unfortunately, the limitations faced by current storage technologies have severely handicapped their potential to meet the storage demand of big data. Consequently, storage technologies with higher storage density, throughput and lifetime have been researched to overcome this gap. In this paper, we first introduce the working principles of three such emerging storage technologies, and justify their inclusion in the study based on the tremendous advances received by them in the recent past. These storage technologies include Optical data storage, DNA data storage & Holographic data storage. We then evaluate the recent advances received in storage density, throughput and lifetime of these emerging storage technologies, and compare them with the trends and advances in prevailing storage technologies. We finally discuss the implications of their adoption, evaluate their prospects, and highlight the challenges faced by them to bridge the data-capacity gap in big data storage.

Introduction

Big Data is large in volume, complex in structure, and aggressive in its production. The value promised by big data has been envisioned across all the fields. However, the challenges faced by the technology to deliver that promise are still being discussed, and the solutions are yet to be finalized [1]. At the same time, integration of Internet of Things (IoT) with Cloud computing is eminent [2]. IoT is expected to grow to 30 billion units by 2020 making them one of the main sources of big data [3]. Consequently, it will have a massive impact on volume, velocity, and variety (and other axes) of big data. Data centres that are mainly based on magnetic storage technology, with petabyte (PB) and even exabyte (EB) capacities have proved to be the core platforms for cloud computing and big data storage [4]. However, there is a growing gap between the volume of digital data being created and the extent of available storage capacities. As an example, in a report prepared by International Data Corporation (IDC) [5], it is forecast that the amount of data generated globally will reach 44 zettabytes (ZBs) in year 2020. This forecast is based on the estimation that the information generated worldwide doubles every two years. IDC’s report also noted that the rate of production of data continues to outpace the growth of storage capacity. In 2013, the available storage capacity could hold just 33% of the digital universe, and by 2020, it will be able to store less than 15% [6]. Recent estimate of IDC suggests that 13 ZBs of 44 ZBs generated in 2020 will be critical, and should be stored. However, since the storage capacity available at that time will be able to hold only 15% of 44 ZBs, a minimum data-capacity gap of over 6 ZBs (which is nearly double all the data produced in 2013) is expected in year 2020. Prevailing storage technologies are increasingly challenged by their limited storage density and throughput as well as the shortcomings associated with energy consumption, capacity footprint, lifetime, and other like features. Accordingly, storage technologies with greater storage densities, higher throughput, lower energy consumption and longer lifetimes are in high demand to support big data centres. Even though many next-generation mass storage technologies have been actively researched [7], three emerging storage technologies have received tremendous advances in the recent past, and are emerging as the next-generation storage technologies for big data storage.