My .NET for Apache Spark docker image at .NET Conf 2020

This year’s .NET Conference covered many interesting topics in various online sessions. One of them was titled “The Missing Piece – Diving into the World of Big Data with .NET for Apache Spark”.
Since I wrote about .NET for Apache Spark in the past, I obviously was very curious to find out what this session might cover.

I almost couldn’t believe my eyes when I saw my name mentioned on one of the slides. But the next surprise wasn’t long in coming. One of my .NET for Apache Spark docker images was even used during … more

.NET for Apache Spark 1.0 is available

Microsoft has released version 1.0 of .NET for Apache® Spark™. The official announcement provides a brief overview of the basic features, performance and the future of .NET for Apache Spark.

It even mentions my docker images, which are now available for version 1.0.0, too. Some of my articles are mentioned on the Open Source Blog as well. A big Thank You to everyone supporting me by trying out the images, providing feedback or contributing in any other way.

Speaking of docker images: In case you didn’t know yet, there are now three different types of … more

.NET for Apache Spark – interactive notebook Docker image

If you are reading this, you are probably aware of my .NET for Apache Spark Docker images that I’ve made available so far. Just recently I’ve added a development image that allows you to easily build .NET for Apache Spark with VS Code in a browser. Today I want to introduce you to the latest member of the family:

The .NET for Apache Spark interactive notebook Docker image.

Jupyter Notebooks

In case you are not aware of what Jupyter Notebooks are, here’s a quick summary quote from the Jupyter project site.

The Jupyter Notebook

more

Build .NET for Apache Spark with VS Code in a browser

Build .NET for Apache Spark with VS Code in a browser

My last article explained how you can use .NET for Apache Spark together with Entity Framework to stream data to an SQL Server. There is one caveat though. You have to build Microsoft.Spark.Worker yourself.
This time I’ll show you how you can actually build .NET for Apache Spark with VS Code in a browser yourself, including building and running the C# examples.

Setting up your own development environment to build and test .NET for Apache Spark can be tricky and time-consuming. However, as a regular reader, you are probably aware that I like to use docker … more

.NET for Apache Spark Docker Image available on GitHub

.NET for Apache Spark Docker Image available on GitHub

I am really honored by the fact, that a lot of people seem to use my .NET for Apache Spark docker image to explore how C# and Apache Spark can work together, for example.

Additionally, I am getting a lot of request lately, asking whether I would be willing to share the code for creating the images.

And finally, after tidying it up a bit (e.g. removing the experimental Windows support), it is now available on GitHub.

So thanks to everyone who made this image such a success and of course you are very welcome to … more

Scroll to top