.NET for Apache Spark 2.0.0 released

.NET for Apache Spark 2.0.0 released

Microsoft just recently announced the release of .NET for Apache Spark 2.0.0.

It provides a couple of new features, like the support for the Apache Spark 3.1.0 APIs, for example.

Apache Spark 2.3.x support has now officially been dropped, however.

I have also updated my different docker images, so that you can try out all the different combinations of the supported Apache Spark versions along with the new .NET for Apache Spark 2.0.0 runtime.

As you might be aware, there are actually 3 different kinds of docker images provided.

Interactive Jupyter Notebook

This image allows you … more

Build .NET for Apache Spark with VS Code in a browser

Build .NET for Apache Spark with VS Code in a browser

My last article explained how you can use .NET for Apache Spark together with Entity Framework to stream data to an SQL Server. There is one caveat though. You have to build Microsoft.Spark.Worker yourself.
This time I’ll show you how you can actually build .NET for Apache Spark with VS Code in a browser yourself, including building and running the C# examples.

Setting up your own development environment to build and test .NET for Apache Spark can be tricky and time-consuming. However, as a regular reader, you are probably aware that I like to use docker … more

Scroll to top