.NET for Apache Spark 1.0 is available

Microsoft has released version 1.0 of .NET for Apache® Spark™. The official announcement provides a brief overview of the basic features, performance and the future of .NET for Apache Spark.

It even mentions my docker images, which are now available for version 1.0.0, too. Some of my articles are mentioned on the Open Source Blog as well. A big Thank You to everyone supporting me by trying out the images, providing feedback or contributing in any other way.

Speaking of docker images: In case you didn’t know yet, there are now three different types of … more

.NET for Apache Spark – interactive notebook Docker image

If you are reading this, you are probably aware of my .NET for Apache Spark Docker images that I’ve made available so far. Just recently I’ve added a development image that allows you to easily build .NET for Apache Spark with VS Code in a browser. Today I want to introduce you to the latest member of the family:

The .NET for Apache Spark interactive notebook Docker image.

Jupyter Notebooks

In case you are not aware of what Jupyter Notebooks are, here’s a quick summary quote from the Jupyter project site.

The Jupyter Notebook

more

Build .NET for Apache Spark with VS Code in a browser

Build .NET for Apache Spark with VS Code in a browser

My last article explained how you can use .NET for Apache Spark together with Entity Framework to stream data to an SQL Server. There is one caveat though. You have to build Microsoft.Spark.Worker yourself.
This time I’ll show you how you can actually build .NET for Apache Spark with VS Code in a browser yourself, including building and running the C# examples.

Setting up your own development environment to build and test .NET for Apache Spark can be tricky and time-consuming. However, as a regular reader, you are probably aware that I like to use docker … more

.NET for Apache Spark – Stream to SQL Server

In this article I am going to describe how to use .NET for Apache Spark with EntityFrameworkCore to stream data to a Microsoft SQL Server. If you have tried this before, you probably stumbled upon the following exception: Microsoft.Data.SqlClient is not supported on this platform.

So let’s find out, how that can be fixed.

Preparation

If you want to stream to an SQL Server, you obviously need to have access to an SQL Server instance first.

Using docker, it is very easy to fire up a related container. I’ve just named it sqlserver, as … more

NumSharp Cheat Sheet

NumPy to NumSharp Cheat Sheet

Recently, I published a small example project to utilize the htm.core AI algorithm by consuming its REST API via C#. As the API also transfers serialized multi-dimensional NumPy arrays, I was looking for an easy way to get them back into C# objects. I’ve tried out a couple of approaches and finally decided on using the NumSharp library, as I wanted a solution that works on multiple platforms.

I find DataCamps’ Data Science Cheat Sheets very useful and was hoping to find something similar for NumSharp. Well, I didn’t, but obviously that gave me a … more

Scroll to top