.NET for Apache Spark – MQTT Streaming

.NET for Apache Spark 0.4.0 was released recently. Therefore, it is now time to test, if it can be used for MQTT Streaming as well.

If you followed my series about a real-time data processing pipeline, you probably remember that I have used Apache Bahir to retrieve streaming data from Apache ActiveMQ via the MQTT protocol. The data itself was generated by IanniX and forwarded to ActiveMQ utilizing my osc2activemq docker image.

Preparing .NET for Apache Spark for MQTT Streaming

For the most parts, you can follow this quick intro tutorial, which walks … more

.NET for Apache Spark 0.4.0 docker image

.NET for Apache Spark

.NET for Apache Spark 0.4.0 has been released.
If you want to test it out, you might find my Docker image useful.
Details are available at https://hub.docker.com/r/3rdman/dotnet-spark


There’s a new image available. Click here for more details.

Quick reference

The image is based on Ubuntu 18.04, Apache Spark 2.4.3 with Hadoop 2.7, .NET Core 2.1.801 and .NET for Apache Spark 0.4.0. It is intended for the purpose of testing .NET for Apache Spark, without the need to install the required bits manually.
Per default, the related container will start up one … more

Real-time data processing pipeline – Part 2 – OSC to ActiveMQ

Real-time data processing pipeline – Part 2 – OSC to ActiveMQ

Welcome back to the second part of my series, showcasing a real-time data processing pipeline!
In part 1, I explored visual real-time sensor data simulation, as the entry point into our pipeline.
Now it’s time to find out, how we can get the generated data into Apache ActiveMQ, by transferring it via the OSC protocol.

Apache ActiveMQ™ is the most popular open source, multi-protocol, Java-based messaging server. It supports a variety of Cross Language Clients and Protocols, and therefore makes it an excellent choice for our pipeline.

Get ActiveMQ up and running

I won’t … more

.NET Core 3.0.0 Preview 3 has been released

.NET Core 3.0.0 Preview 3 has been released.

Follow the links to view the Release Notes and Known Issues

You may want to read the full announcement at the .NET Blog

Scroll to top