Databricks Fully Open Sources Delta Lake

By

Company will contribute the storage framework to the Linux Foundation.

Databricks has announced that it will contribute the entirety of its Delta Lake storage framework to the Linux Foundation and open source all Delta Lake APIs as part of the Delta Lake 2.0 release.

The Delta Lake framework enables building a “Lakehouse architecture” on top of data lakes. It provides ACID transactions for concurrency control, scalable metadata handling, and unifies streaming and batch data processing. The new 2.0 release of Delta Lake features improved query performance as well as general improvements for writing large scale performance benchmarks.

Databricks also released MLflow 2.0, which includes a new Pipelines feature to accelerate and simplify ML model deployments. The company additionally introduced Spark Connect, which allows Apache Spark to run on any device, and Project Lightspeed, a next-generation Spark streaming engine.

07/01/2022

Related content

comments powered by Disqus
Subscribe to our ADMIN Newsletters
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs



Support Our Work

ADMIN content is made possible with support from readers like you. Please consider contributing when you've found an article to be beneficial.

Learn More”>
	</a>

<hr>		    
			</div>
		    		</div>

		<div class=