.NET for Apache Spark 1.0 is available

Microsoft has released version 1.0 of .NET for Apache® Spark™. The official announcement provides a brief overview of the basic features, performance and the future of .NET for Apache Spark.

It even mentions my docker images, which are now available for version 1.0.0, as well. A big Thank You to everyone supporting me by trying out the images, providing feedback or contributing in any other way.

Speaking of docker images: In case you didn’t know yet, there are now three different types of images available.

Interactive Jupyter Notebook

This image allows you to explore .NET more

.NET for Apache Spark – interactive notebook Docker image

If you are reading this, you are probably aware of my .NET for Apache Spark Docker images that I’ve made available so far. Just recently I’ve added a development image that allows you to easily build .NET for Apache Spark with VS Code in a browser. Today I want to introduce you to the latest member of the family:

The .NET for Apache Spark interactive notebook Docker image.

Jupyter Notebooks

In case you are not aware of what Jupyter Notebooks are, here’s a quick summary quote from the Jupyter project site.

The Jupyter Notebook

more

htm.core and C#

In case you weren’t aware, htm.core now also provides a REST interface to it’s Network API. With this in place, it is possible to utilize the built-in encoders, Spatial Pooler and Temporal Memory from almost any programming language. In this post I briefly want to touch on how to use htm.core and C# together.

Introduction

Before you can start playing around with the REST interface, you will need to have the example REST server running.

To do that, you can either

  • download the source and build it yourself. Once built, the server executable (rest_server) should
more

htm.core parameters – High Order Sequence Memory

In part 2, I used htm.core as a single order sequence memory by allowing only one cell per mini-column. In this post I’ll finally have a first look at the high order sequence memory.

Before we do that, I want to show you one last single order memory example however.

Single Order Sequence Memory Recap

As you might remember from the last post, these were the settings for our htm.core temporal memory (aka sequence memory).

    columns = 8
    inputSDR = SDR( columns )
    cellsPerColumn = 1
    tm = TM(columnDimensions          = (inputSDR.size,),
            cellsPerColumn            = cellsPerColumn,     
more

htm.core parameters – Single Order Sequence Memory

To allow the htm.core temporal memory to learn sequences effectively, it is import to understand the impact of the different parameters in more detail.

In this part I will introduce

  • columnDimensions
  • cellsPerColumn
  • maxSegmentsPerCell
  • maxSynapsesPerSegment
  • initialPermanence
  • connectedPermanence
  • permanenceIncrement
  • predictedSegmentDecrement

Temporal Memory – Previously on this blog…

Part 1 just covered enough basics of htm.core to get us started, and we actually saw how the single order memory got trained.
A cycle of encoded increasing numbers from 0 to 9 was very easy to predict, as there was always just one specific value that could follow the … more

Scroll to top