A cloud storage gateway is a network appliance or server which resides at the customer premises and translates cloud storage APIs such as SOAP or REST to block-based storage protocols such as iSCSI or Fibre Channel or file-based interfaces such as NFS or SMB. According to a 2011 report by Gartner Group, cloud gateways were expected to increase the use of cloud storage by lowering monthly charges and eliminating the concern of data security.
History
The term "cloud storage gateway" is a rather new item. But the technology was built already in 1992 by a Belgian company called Filepool. The company was acquired by EMC Corporation in 2001. This system was sold under the brand name "Centera" and as the communication protocol was API based, they called it content-addressable storage. The Centera system was the first object storage array. One of the major problems of the system was the missing application support by developers. Accordingly, EMC Corporation developed an interface called CUA – Centera Universal Access: The first cloud storage gateway. The CUA was able to bridge between Cenetera API and NFS protocol. It was only possible to use the CUA in front of a Centera system - accordingly it was a proprietary device. content-addressable storage evolved later to object storage, but the problem still exists in missing applications for object storage – one of the main enablers for cloud storage gateways.
Technology
Features
Modern applications use network attached storage by means of REST and SOAP with hypertext transfer protocol on the protocol layer. The related storage is provided from arrays that offer these as object storage. Classic applications use network attached storage by means of Network File System NFS, iSCSI or Server Message Block SMB. To make use of all the advantages of object storage, existing applications need to be rewritten and new applications must be object storage aware, which is not the case by default. This problem is addressed by cloud storage gateways. They offer object storage via classic native storage protocols like Network File System NFS or Server Message Block SMB. As a rule of thumb you can now use classic applications with cloud native object storage by using cloud storage gateways.
Functionality
In enterprise infrastructures NFS is mainly used by Linux systems whereas Windows systems are using SMB. Object storage needs data in form of objects rather than files. For all cloud storage gateways it is mandatory to cache the incoming files and destage them to object storage on a later step. The time of destaging is subject to the gateway and a policy engine allows functions like
pinning = bind specific files to the cache and destage them only for mirroring purpose
content based destaging = move only files with specific characteristics to object storage e.g. all MP3 files
multi-cloud mirroring = mirror all files to two different object stores
Least Recently use = fill the local cache to maximum, move all files to object storage and delete files in cache on a LRU algorithm
encrypt prior of destage = files are encrypted on the cloud storage gateway and destaged to object storage in an encrypted form
compress and / or dedupliation prior of destage = files are deduplicated and/or compressed prior of destaging
backup data in a native backup format
Combinations of these functions are usual. Default sorting schematics spanning the retrieval interface generally rely on zero-fault content processing, which carries the obvious requirement that two or more of the above functions are synchronized.
Extensions
Nearly all object storage gateways support Amazon S3 protocol as a quasi-standard. Some offer as well Microsoft Azure Blob, Google Storage, or Openstack SWIFT. Most gateways support public cloud storage e.g. from Amazon or Microsoft as an object store and Dropbox as a file drive store, there are as well a lot of vendors that support private cloud storage as well – including off and on prem storage.
Deployment methods
There are multiple variants to deploy such gateways – and some vendors support as well different variants as of their product line:
software on top of an operating system – aka FUSE based
Software appliances as well as FUSE-based gateways can be installed on public cloud infrastructures.
Advantages
Cloud storage gateway avoid the need to change existing applications by providing a standard interface. You can make use of all advantages of object storage without rewriting your applications. As well IT users are used to existing protocols – like SMB or NFS. They can make use of cloud storage with the advantage of still using their existing infrastructures. Whereas cloud storage gateways initially covered a niche only, the got more attraction as of multi-cloud technologies. As an example: It is possible to run a cloud storage gateway in form of a software appliance on top of a public or private cloud infrastructure by offering docker volume drivers that enable containers for automatic provisioning of storage used by these containers in a consistent form. They are using the hypervisors disks as a cache only, but destabe data on least recently used algorithm to the underlying cloud storage. The de facto standard for object storage is Amazon S3 – it had the most popularity and capacity installed on object storage. But every object storage vendor can offer Amazon S3 storage – even there is no real "standard" S3 API: Every vendor is a little bit different in implementing S3 API. Since 2018 we see more and more cloud storage gateways that hide this complexity by offering S3 on northbound. As such you can make use of a richer S3 implementation on northbound than the southbound supports.
Disadvantages
By using cloud storage gateways the complexity to use object storage is hidden, but that also hides some of the advantages of object storage:
the ability of horizontal scaling
ability to add high efficient metadata to the data content
to use extended WORM and archiving capabilities of object storage
As applications change to cloud-aware applications, cloud storage gateways will change from multiprotocol gateways to multi-cloud gateways, providing access to multiple cloud providers as well as multiple southbound protocols and act as relays between different clouds.
Market
The cloud storage gateway market was estimated at $74 million in 2012, up from $11 million at the end of 2010. One analyst predicted in 2013 that the cloud storage gateway market might reach US$860 million by 2016.
