Tim Rodgers, Senior Cloud Software Development Manager with Masstech outlines how we are developing inbound support for the S3 interface, allowing a standardized access method to assets stored within our platform.
What is an S3 interface?
Amazon introduced the S3 (Simple Storage Service) in 2006, along with a simple REST API to access it, that has since become the de-facto standard object storage interface.
The S3 interface is a simple, file-based service with in-built security to prevent unauthorized access, and granularity of communications and instructions to the available resources. The interface allows a user to discover, read, write, copy, and delete files. It also allows metadata to be stored and retrieved for each file, providing a richer file system than those used by standard operating systems.
With the vast majority of vendors that work with 3rd party storage providing an integration with S3, it has become an unwritten rule that if a vendor provides an S3 interface to access their storage, they can also integrate with almost anything that may wish to access the data stored in their platform.
Kumulate has always supported outbound communications via S3 interface, enabling our support for AWS and other S3 based cloud storage. However, we are now also developing inbound support for the S3 interface, allowing a standardized access method to assets stored within our platform.
What does the S3 interface support?
The S3 interface provides a simple and standardized method to store, manage and enrich your content. By providing an S3 interface to a storage management system such as Kumulate, third party vendors have the ability to access and control all of the content that is managed by the storage platform through a single name space, whether that content is stored on tape, disk, or cloud.
With the S3 interface, it is not only possible to access existing content, but also to introduce new content into the system, and subsequently to request contednt operations, such as requesting that a redundant copy is made to a different storage tier.
Users who have implemented AI enrichment services as part of their storage workflows (e.g. to automatically perform speech-to-text extraction, and/or person and object detection in videos stored in their ecosystem) can permit this information to be retrieved through the S3 interface as metadata attached to the file.
How is Masstech using the S3 interface?
Masstech’s Kumulate manages content stored on a large number of different storage platforms, providing the capability to move, transform, and enrich that content.
As well as the ability to control on-premise storage systems such as tape libraries, Kumulate uses the outbound S3 interface to provide users with full cloud connectivity to S3 based cloud platforms. Essentially this offers unlimited storage capacity in both on-prem systems and in the cloud, through a simple and commonly supported API.
Once content is in the Masstech eco-system, the Kumulate workflow Orchestrator enables it to be automatically processed, transformed, and moved/copied to destinations such as tape or cloud. These destinations will appear as buckets within the S3 interface and can be discovered through requests to the ‘list buckets’ S3 command.
Do you want to move some H.264 content from tape to a video server that requires MPG2 for playback? Simply send a copy object command on the S3 interface specifying the clip id & video server name and Kumulate will do the rest. Do you need to generate a proxy of a high-res video, store it in the cloud, and generate some closed captioning and a set of markers whenever a car, animal or person appears in the video? That’s right, send a copy object command and sit back and let Kumulate do the rest!
What this means for Kumulate users
Kumulate users will now have an industry standard interface to programmatically access and manipulate their content in a single namespace, whether that is on disk, tape, or even cloud. Have you got an application that can work with AWS for example, but not Azure or Google cloud storage? Now with Kumulate’s S3 interface your application can work with them all.
The inbound S3 interface for Kumulate is now entering final testing stages, and we’re really excited about releasing it to our Kumulate community. We can’t wait to see the myriad ways that our users will use it to enhance their existing systems and processes, and the new workflows that this enhanced functionality will enable for them.
If you’d like to know more about our new S3 interface, please drop me a line.