htm.core and C#

In case you weren’t aware, htm.core now also provides a REST interface to it’s Network API. With this in place, it is possible to utilize the built-in encoders, Spatial Pooler and Temporal Memory from almost any programming language. In this post I briefly want to touch on how to use htm.core and C# together.

Introduction

Before you can start playing around with the REST interface, you will need to have the example REST server running.

To do that, you can either

  • download the source and build it yourself. Once built, the server executable (rest_server) should
more

.NET for Apache Spark Docker Image available on GitHub

.NET for Apache Spark Docker Image available on GitHub

I am really honored by the fact, that a lot of people seem to use my .NET for Apache Spark docker image to explore how C# and Apache Spark can work together, for example.

Additionally, I am getting a lot of request lately, asking whether I would be willing to share the code for creating the images.

And finally, after tidying it up a bit (e.g. removing the experimental Windows support), it is now available on GitHub.

So thanks to everyone who made this image such a success and of course you are very welcome to … more

.NET for Apache Spark ForeachWriter & PostgreSQL

.NET for Apache Spark IForeachWriter implementation

Introduction

A couple of months ago I’ve described how to transfer data from Apache Spark to PostgreSQL by creating a Spark ForeachWriter in Scala.

This time I will show how this can be done in C#, by creating a ForeachWriter for .NET for Apache Spark.

To create a custom ForeachWriter, one needs to provide an implementation of the IForeachWriter interface, which is supported from version 0.9.0 onward. I am going to use version 0.10.0 in this article, however.

Documentation of the C# Interface is provided within the related source code:

https://github.com/dotnet/spark/blob/master/src/csharp/Microsoft.Spark/Sql/ForeachWriter.cs

The example project I am … more

.NET for Apache Spark – VSCode with Docker on Linux and df.Collect()

.NET for Apache Spark docker image

Overview

My last article explained, how a .NET for Apache Spark project can be debugged in Visual Studio 2019 under Windows. I have also mentioned some limitation at the end of the article.
In this article I will extend the project a bit and demonstrate the aforementioned limitation using version 0.8.0 of my docker image for .NET for Apache Spark.
Furthermore, I will show a possible workaround that can be used, if you are running Docker and Visual Studio Code under Linux (Ubuntu 18.04).

The extended application

In order to demonstrate the issue, I have … more

.NET for Apache Spark – UDF, VS2019, Docker for Windows and a Christmas Puzzle

.NET for Apache Spark Container

Overview

The latest version of my docker image for .NET for Apache Spark tries to support direct debugging from Visual Studio 2019 and Visual Studio Code.
This is the first article of a small series that will show how this can be done on different environments (Windows and Linux), and what limitations might exist.

Test application & data

I have put together a very simple C# application, named “HelloUdf”, for demonstration purposes. It is supposed to read a JSON file (coordinates.json) that contains one coordinate string per line.
Besides reading the file, the application’s task is … more

Scroll to top