GitHub

Creating a StreamSets Spark Transformer in Java

The Spark Evaluator stage allows you to implement custom processing in an Apache Spark application. The Spark Application runs for the lifetime of the pipeline, processing batches of records as they ...
InfoWorld

Spark tutorial: Get started with Apache Spark

Apache Spark has become the de facto standard for processing data at scale, whether for querying large datasets, training machine learning models to predict future trends, or processing streaming data ...
GitHub

Creating a StreamSets Spark Transformer in Scala

This tutorial explains how to create a simple Apache Spark application, using Scala, that will compute the type of a credit card from its number, and configure the Spark Evaluator to use it. A ...
IEEE

Spark in Action, Second Edition: Covers Apache Spark 3 with Examples in Java, Python, and Scala

Book Abstract: The Spark distributed data processing platform provides an easy-to-implement tool for ingesting, streaming, and processing data from any source. In Spark in Action, Second Edition, ...
InfoWorld

Tutorial: Spark application architecture and clusters

A Spark application contains several components, all of which exist whether you’re running Spark on a single machine or across a cluster of hundreds or thousands of nodes. Each component has a ...