Google today announced that its Cloud Dataproc service for Kubernetes has launched in alpha as the company seeks to bring greater efficiency to processing data across platforms.
Since its public release three years ago, Cloud Dataproc has helped developers seeking to manage growing volumes of data. It is based on Hadoop and Spark open source big data software.
The new version for Kubernetes is aimed at enterprises that find themselves grappling with hybrid systems of private and public clouds, plus legacy infrastructure. Open source Kubernetes, originally developed by a Google team, helps manage and deploy “microservices” or “cloud native computing” that make it easier to write applications across platforms.
In a blog post from Google Cloud project managers Christopher Crosbie and James Malone, the company says that hybrid evolution is causing inefficiencies, as some machines sit unused for long period and some software grows out of date.
“With this announcement, we are bringing enterprise-grade support, management, and security to Apache Spark jobs running on GKE clusters,” the pair write.
The release is also part of the continuing arms race between giant cloud providers such as Amazon Web Services and Microsoft Azure. Each is seeking to provide the most complete suite of services to attract the growing number of companies that are migrating their digital operations from premises-based to online.
Though still in alpha, Cloud Dataproc on Kubernetes should make it easier for users to manage Apache Spark jobs that operate on Google Kubernetes Engine. And in doing so, the company is hoping to simplify the transition to containers for enterprises.
Over the long haul, the shift to containers and microservices is expected to lead to greater stability and security for cloud-based services.
Additional technical details can be found here.