Tag: Apache Spark

20. June 2021 .NET Core, Apache Spark, Big Data, C#, Docker, Jupyter, Visual Studio Code

.NET for Apache Spark 2.0.0 released

.NET for Apache Spark 2.0.0 released

Microsoft just recently announced the release of .NET for Apache Spark 2.0.0.

It provides a couple of new features, like the support for the Apache Spark 3.1.0 APIs, for example.

Apache Spark 2.3.x support has now officially been dropped, however.

I have also updated my different docker images, so that you can try out all the different combinations of the supported Apache Spark versions along with the new .NET for Apache Spark 2.0.0 runtime.

As you might be aware, there are actually 3 different kinds of docker images provided.

Interactive Jupyter Notebook

This image allows you … more

27. October 2020 .NET Core, Apache Spark, Big Data, C#, Data Science, Docker, Jupyter, Learning

.NET for Apache Spark 1.0 is available

Microsoft has released version 1.0 of .NET for Apache® Spark™. The official announcement provides a brief overview of the basic features, performance and the future of .NET for Apache Spark.

It even mentions my docker images, which are now available for version 1.0.0, too. Some of my articles are mentioned on the Open Source Blog as well. A big Thank You to everyone supporting me by trying out the images, providing feedback or contributing in any other way.

Speaking of docker images: In case you didn’t know yet, there are now three different types of … more

22. September 2020 .NET Core, Apache Spark, Big Data, C#, Database, Entity Framework, SQL Server, Visual Studio Code

.NET for Apache Spark – Stream to SQL Server

In this article I am going to describe how to use .NET for Apache Spark with EntityFrameworkCore to stream data to a Microsoft SQL Server. If you have tried this before, you probably stumbled upon the following exception: Microsoft.Data.SqlClient is not supported on this platform.

So let’s find out, how that can be fixed.

Preparation

If you want to stream to an SQL Server, you obviously need to have access to an SQL Server instance first.

Using docker, it is very easy to fire up a related container. I’ve just named it sqlserver, as … more

27. May 2020 .NET Core, Apache Spark, Big Data, C#, Docker, Unix

.NET for Apache Spark Docker Image available on GitHub

.NET for Apache Spark Docker Image available on GitHub

I am really honored by the fact, that a lot of people seem to use my .NET for Apache Spark docker image to explore how C# and Apache Spark can work together, for example.

Additionally, I am getting a lot of request lately, asking whether I would be willing to share the code for creating the images.

And finally, after tidying it up a bit (e.g. removing the experimental Windows support), it is now available on GitHub.

So thanks to everyone who made this image such a success and of course you are very welcome to … more

8. May 2020 .NET Core, Apache Spark, Big Data, C#, Docker

.NET for Apache Spark 0.11.0 docker image

My .NET for Apache Spark v0.11.0 docker image is now available

.NET for Apache Spark 0.11.0 is now available and I have also updated my related docker images for Linux and Windows on the docker hub.

If you are interested, check out the official resources, or one of the following articles.

… more
Scroll to top