.NET for Apache Spark – Stream to SQL Server

In this article I am going to describe how to use .NET for Apache Spark with EntityFrameworkCore to stream data to a Microsoft SQL Server. If you have tried this before, you probably stumbled upon the following exception: Microsoft.Data.SqlClient is not supported on this platform.

So let’s find out, how that can be fixed.

Preparation

If you want to stream to an SQL Server, you obviously need to have access to an SQL Server instance first.

Using docker, it is very easy to fire up a related container. I’ve just named it sqlserver, as … more

NumSharp Cheat Sheet

NumPy to NumSharp Cheat Sheet

Recently, I published a small example project to utilize the htm.core AI algorithm by consuming its REST API via C#. As the API also transfers serialized multi-dimensional NumPy arrays, I was looking for an easy way to get them back into C# objects. I’ve tried out a couple of approaches and finally decided on using the NumSharp library, as I wanted a solution that works on multiple platforms.

I find DataCamps’ Data Science Cheat Sheets very useful and was hoping to find something similar for NumSharp. Well, I didn’t, but obviously that gave me a … more

.NET for Apache Spark Docker Image available on GitHub

.NET for Apache Spark Docker Image available on GitHub

I am really honored by the fact, that a lot of people seem to use my .NET for Apache Spark docker image to explore how C# and Apache Spark can work together, for example.

Additionally, I am getting a lot of request lately, asking whether I would be willing to share the code for creating the images.

And finally, after tidying it up a bit (e.g. removing the experimental Windows support), it is now available on GitHub.

So thanks to everyone who made this image such a success and of course you are very welcome to … more

.NET for Apache Spark 0.11.0 docker image

My .NET for Apache Spark v0.11.0 docker image is now available

.NET for Apache Spark 0.11.0 is now available and I have also updated my related docker images for Linux and Windows on the docker hub.

If you are interested, check out the official resources, or one of the following articles.

more

htm.core parameters – High Order Sequence Memory

In part 2, I used htm.core as a single order sequence memory by allowing only one cell per mini-column. In this post I’ll finally have a first look at the high order sequence memory.

Before we do that, I want to show you one last single order memory example however.

Single Order Sequence Memory Recap

As you might remember from the last post, these were the settings for our htm.core temporal memory (aka sequence memory).

    columns = 8
    inputSDR = SDR( columns )
    cellsPerColumn = 1
    tm = TM(columnDimensions          = (inputSDR.size,),
            cellsPerColumn            = cellsPerColumn,     
more
Scroll to top