Author: Indy

6. October 2020 .NET Core, Apache Spark, C#, Docker, Scala, Unix, Visual Studio Code

Build .NET for Apache Spark with VS Code in a browser

Build .NET for Apache Spark with VS Code in a browser

My last article explained how you can use .NET for Apache Spark together with Entity Framework to stream data to an SQL Server. There is one caveat though. You have to build Microsoft.Spark.Worker yourself.
This time I’ll show you how you can actually build .NET for Apache Spark with VS Code in a browser yourself, including building and running the C# examples.

Setting up your own development environment to build and test .NET for Apache Spark can be tricky and time-consuming. However, as a regular reader, you are probably aware that I like to use docker … more

22. September 2020 .NET Core, Apache Spark, Big Data, C#, Database, Entity Framework, SQL Server, Visual Studio Code

.NET for Apache Spark – Stream to SQL Server

In this article I am going to describe how to use .NET for Apache Spark with EntityFrameworkCore to stream data to a Microsoft SQL Server. If you have tried this before, you probably stumbled upon the following exception: Microsoft.Data.SqlClient is not supported on this platform.

So let’s find out, how that can be fixed.

Preparation

If you want to stream to an SQL Server, you obviously need to have access to an SQL Server instance first.

Using docker, it is very easy to fire up a related container. I’ve just named it sqlserver, as … more

17. August 2020 .NET Core, Big Data, C#, Data Science, Jupyter, Python

NumSharp Cheat Sheet

NumPy to NumSharp Cheat Sheet

Recently, I published a small example project to utilize the htm.core AI algorithm by consuming its REST API via C#. As the API also transfers serialized multi-dimensional NumPy arrays, I was looking for an easy way to get them back into C# objects. I’ve tried out a couple of approaches and finally decided on using the NumSharp library, as I wanted a solution that works on multiple platforms.

I find DataCamps’ Data Science Cheat Sheets very useful and was hoping to find something similar for NumSharp. Well, I didn’t, but obviously that gave me a … more

10. June 2020 .NET Core, Artificial Intelligence, C#, Hierarchical Temporal Memory, Learning

htm.core and C#

In case you weren’t aware, htm.core now also provides a REST interface to it’s Network API. With this in place, it is possible to utilize the built-in encoders, Spatial Pooler and Temporal Memory from almost any programming language. In this post I briefly want to touch on how to use htm.core and C# together.

Introduction

Before you can start playing around with the REST interface, you will need to have the example REST server running.

To do that, you can either

  • download the source and build it yourself. Once built, the server executable (rest_server) should
… more
27. May 2020 .NET Core, Apache Spark, Big Data, C#, Docker, Unix

.NET for Apache Spark Docker Image available on GitHub

.NET for Apache Spark Docker Image available on GitHub

I am really honored by the fact, that a lot of people seem to use my .NET for Apache Spark docker image to explore how C# and Apache Spark can work together, for example.

Additionally, I am getting a lot of request lately, asking whether I would be willing to share the code for creating the images.

And finally, after tidying it up a bit (e.g. removing the experimental Windows support), it is now available on GitHub.

So thanks to everyone who made this image such a success and of course you are very welcome to … more

Scroll to top