The Ascend for Qubole integration enables joint customers to self-service advanced data pipelines for modern, cloud data lake environments
Ascend.io, the data engineering company, and Qubole, the open data lake company, today announced a new, strategic partnership and platform integration, bringing together the world’s most advanced data pipeline automation technology with the most comprehensive data lake platform. The Ascend for Qubole integration enables data teams to create self-service data pipelines 7x faster with 95% less code, and reduces infrastructure costs by 50% or more with greater data processing efficiency.
The Ascend Unified Data Engineering Platform automates the implementation, maintenance, and operations of data pipelines, freeing scarce data engineering talent to meet the changing requirements of accelerated business transformations. Harnessing the power of the Qubole Open Data Lake Platform to optimize the operation of data lake infrastructure, the new partnership eliminates the need for joint customers to constantly manage infrastructure operations and optimize costs. Teams using the new integration can build, run, and manage data pipelines natively within Ascend, with full access to Qubole’s platform and suite of end-to-end data lake services.
Read More: Enterprise Cloud Usage Slated to Become Entirely Virtual by 2021
“The partnership between Ascend.io and Qubole will serve as a game changer for customers as they look to unlock their next level of data maturity. Customers will receive an end-to-end solution to simplify data engineering and reduce the time it takes to extract valuable business insights from real-time, large-scale analytics,” said Mike Leone, senior analyst at Enterprise Strategy Group. “The joint solution’s power and simplicity is impressive. Not only does it put time back into the hands of data teams who are under constant pressure to do more with data, but it presents a combination of extensive customization and configurability with easy-to-use, visual, low-code interfaces. It truly offers the best of both worlds regardless of an organization’s data maturity.”
Ascend for Qubole Democratizes Data Engineering to Deliver on Business Needs
As the business demand for information and insights continues to surge, data teams today are being asked to do more with less. According to a recent Ascend.io research study, 97% of data scientists, data engineers, and enterprise architects indicate their teams are at or over capacity – with just 3% citing they have extra capacity for new projects.
“Until now, data lake ETL required months of dedicated data engineering resources to build and deploy data pipelines. Even then, scarce data engineering talent was bogged down maintaining brittle code, managing changes in data payloads, and tuning pipelines,” said Sean Knapp, CEO and founder of Ascend.io. “Ascend automates the underlying engineering complexity of data pipelines to dramatically increase the productivity of data teams. Through our partnership with Qubole, we further our mission to democratize data engineering and enable our customers to utilize the most complete data lake solution for the entire data team.”
The Ascend for Qubole integration combines the strengths of the two platforms, helping organizations succeed in managing and governing the large data volumes that flow into and through modern cloud data lake environments. This includes standing up long-running data pipelines, migrating data workloads to the cloud, and solving more business problems faster and with greater insights. Ascend’s low-code approach eases the complexity of Python, pySpark, SQL, and Scala programming for Apache Spark ETL workloads, while Qubole delivers near-zero administration and maintenance of the data lake platform.
Read More: Top Five Ways CIOs Can Boost Productivity for the Remote Workforce
“In response to the tremendous increase in demand for machine learning, streaming analytics, data exploration, and more, data pipelines have emerged as the standard technique for data movement in and out of the data lake,” said Dave Lassiter, VP global cloud partnerships and alliances at Qubole. “The increasing complexity of these interconnected pipelines, however, has far outpaced the capabilities of traditional ETL and open-source tools. Ascend cuts the time required to design, build, and deploy data pipelines down to hours – versus weeks or even months with other solutions – with almost zero maintenance.”
The Ascend Unified Data Engineering Platform revolutionizes pipelines by freeing data teams from the monotony of traditional ETL. Through advanced AI and automation, teams can focus their efforts on data modeling and architecture, tapping into their creativity to create production-ready systems faster. The platform spans the entire lifecycle of data pipelines with automated maintenance, pipeline optimization, and integrated governance capabilities.
Key benefits of the Ascend for Qubole integration include:
- Self-service data pipelines: Ascend for Qubole enables data engineers, data scientists, and data analysts to self-service data pipelines by automating the complexity of data engineering. With low-code, declarative configurations, the integration yields significant team productivity gains.
- Reduced infrastructure costs: Ascend’s DataAware intelligence observes and maintains detailed records of data movement and processing, code changes, and user activity. As a result, Ascend continually optimizes cloud usage by only processing new data and code, and precisely reporting the running costs of individual pipelines and operations. Qubole’s Heterogeneous Cluster Lifecycle Management capabilities and Intelligent Spot Management provide the most cost-effective combination of on-demand and spot virtual machines (VMs) to deliver significantly more efficient data processing at the lowest cost available.
- Increased performance: Leveraging the power of optimized open-source frameworks on Qubole, such as Presto, Spark, Hive, among others, users can process and query data with industry-leading response times, without changing their normal workflow or manually tuning their Ascend-based pipelines.
- Native connectors: Qubole’s native connectors allow users to query unstructured or semi-structured data on any data lake regardless of the storage file format – CSV, JSON, AVRO, or Parquet. Meanwhile, Ascend’s extensive connector framework is capable of ingesting data from industry leading databases, warehouses, APIs, and more to ingest, process, and deliver the data.
- Built-in financial and lineage governance: Qubole’s native financial governance capabilities provide immediate visibility into platform usage costs with advanced tools for budget allocation, chargeback, monitoring, and control of cloud spend. Meanwhile, Ascend’s DataAware intelligence provides full data lineage governance for all pipelines across the enterprise with historical timelines spanning both data and code, and allocates costs to the pipeline and data transform level so data teams know exactly where to tune their code.
HNI Corporation, a global leader in workplace furnishings and residential building products, has selected Qubole as the open data lake platform to complement its selection of Ascend as the data engineering platform earlier this year. According to Tom Kozlowski, HNI’s VP of decision science, “With the Ascend DataAware automation powering our production data pipelines from ingestion to consumption, and Qubole providing tooling such as Spark, Hive, and Presto, our decision science team now has a full complement of capabilities to invest in innovative data processing and to grow the operational analytics that drive transformation across our business. In addition, the integration between Qubole and Ascend means we now have a single version of the data shared between the two platforms. With Ascend data pipelines running natively on Qubole, we can further consolidate our data infrastructure, simplify our architecture, and reduce operational costs with Qubole’s Heterogeneous Cluster Lifecycle Management. Crucially, the declarative environment of the Ascend platform enables our data scientists and analysts to self-serve, building and deploying robust new data products themselves, freeing our data engineers to work on new ideas and initiatives.”