NumSharp Cheat Sheet

NumPy to NumSharp Cheat Sheet

Recently, I published a small example project to utilize the htm.core AI algorithm by consuming its REST API via C#. As the API also transfers serialized multi-dimensional NumPy arrays, I was looking for an easy way to get them back into C# objects. I’ve tried out a couple of approaches and finally decided on using the NumSharp library, as I wanted a solution that works on multiple platforms.

I find DataCamps’ Data Science Cheat Sheets very useful and was hoping to find something similar for NumSharp. Well, I didn’t, but obviously that gave me a … more

.NET for Apache Spark – VSCode with Docker on Linux and df.Collect()

.NET for Apache Spark docker image

Overview

My last article explained, how a .NET for Apache Spark project can be debugged in Visual Studio 2019 under Windows. I have also mentioned some limitation at the end of the article.
In this article I will extend the project a bit and demonstrate the aforementioned limitation using version 0.8.0 of my docker image for .NET for Apache Spark.
Furthermore, I will show a possible workaround that can be used, if you are running Docker and Visual Studio Code under Linux (Ubuntu 18.04).

The extended application

In order to demonstrate the issue, I have … more

Happy New Year & the answer to the Christmas puzzle

Happy New Year & the answer to the Christmas puzzle

Happy New Year to everyone everywhere!

My last post in 2019 is titled “.NET for Apache Spark – UDF, VS2019, Docker for Windows and a Christmas Puzzle“.
As the name implies, it contains a small puzzle related to Christmas. Were you able to find out, what it was about?

Well, here is the resolution.

As you might have guessed correctly, the key was the “coordinates.json” file.

Extracting the coordinates from the file and replacing semicolon with a comma, allows you to put the resulting coordinate-format directly into Google Maps for example.

37.2350540,-122.081146
37.2350540,-122.060310
37.2408218,-121.994064
more

.NET for Apache Spark – UDF, VS2019, Docker for Windows and a Christmas Puzzle

.NET for Apache Spark Container

Overview

The latest version of my docker image for .NET for Apache Spark tries to support direct debugging from Visual Studio 2019 and Visual Studio Code.
This is the first article of a small series that will show how this can be done on different environments (Windows and Linux), and what limitations might exist.

Test application & data

I have put together a very simple C# application, named “HelloUdf”, for demonstration purposes. It is supposed to read a JSON file (coordinates.json) that contains one coordinate string per line.
Besides reading the file, the application’s task is … more

Debug .NET for Apache Spark with Visual Studio and docker

Greatly simplify debugging your .NET for Apache Spark project by using docker

You do want to test and debug your .NET for Apache Spark application with Visual Studio? But you don’t want to set up Apache Spark yourself?
Then read along and find out how my docker image might be able to help.

Before we dig into the details however, I specifically want to thank Devin Martin for sharing his idea about such a docker image with me!

Background

As you might be aware, you can debug your .NET for Apache Spark application directly in Visual Studio by starting the related DotnetRunner in Debug mode.

Obviously that means … more

Scroll to top