Getting started

This section helps you to set up a stream and start sending data to Stream Machine.

Setting up a stream

This section assumes that you have created an account on the portal, and have used the strm command line interface client to create an input and an output stream.

Upon creation of your data stream, credentials have been provided for the in and output streams, as well as your billing id.

Make sure you store these credentials safe and securely, as they’re only provided once.

Client drivers

The way to interact with Stream Machine, is through one of our API drivers. Our drivers handle authentication (including refreshing of credentials), serializing a payload with the schema of your choice, and sending the payload to our infrastructure. The same applies for receiving data.

Currently, client drivers for Stream Machine are available for Java and Python (the Python driver is currently being reviewed before it’s released). Fear not, we’re always trying to create more drivers, to support more languages. And if no driver is available for the language of your choice, you can always follow our API docs and create your own.

The rest of this guide assumes that all language dependent prerequisites are fulfilled (i.e. for Java, you have installed the JDK on your machine).

All client drivers are designed to work asynchronously.

Schema

In order to send data using the schema that matches the stream that you’ve configured, your application needs a dependency on the schema.

The client driver dependency assumes that the schema dependency is available in your application, therefore it is required in order to successfully send data to Stream Machine.

Configuring your application

The configuration of your application includes three steps:

Installing / including a dependency on the driver and on the schema that you use
Configuring a StreamMachineClient in your application
Sending or receiving data

Every stream requires its own StreamMachineClient, as the client holds credentials for a single stream (regardless of whether that stream is input or output).

Java

Dependencies

For a Java based application, first, add the following dependencies, which are available through Maven central:

Maven

<dependency>
    <groupId>io.streammachine</groupId>
    <artifactId>java-driver</artifactId>
    <version>${streammachine.java-driver.version}</version>
</dependency>
<!-- TODO ensure that you use the correct schema, this is an example -->
<dependency>
    <groupId>io.streammachine.schemas.strmcatalog</groupId>
    <artifactId>clickstream</artifactId>
    <version>${streammachine.schemas.clickstream.version}</version>
</dependency>

Gradle

compile "io.streammachine:java-driver:$streammachineJavaDriverVersion"
compile "io.streammachine.schemas.strmcatalog:clickstream:$streammachineSchemasClickstreamVersion"

Initialization

A client driver can be created using four parameters: billingId (your unique customer id), clientId (matches the stream you’re publishing to / consuming from), clientSecret (the password required to interact with this stream), and the client configuration.

StreamMachineClient client = StreamMachineClient.builder()
    .billingId(billingId)
    .clientId(clientId)
    .clientSecret(clientSecret)
    .config(Config.builder().build())
    .build();

Sending events

Sending data requires a StreamMachineEvent to be created, which can be done using the schema. In this dependency, StreamMachineEvent is implemented, which then is accepted by the driver. Next, calling the .send(streamMachineEvent, SerializationType.AVRO_BINARY) method will authenticate your request, serialize the payload with the provided SerializationType and sends it to the Gateway.

Receiving events

Receiving data is done through the .startReceivingSse() method, which requires two parameters. The first is a boolean, stating whether the received data should be JSON, and the second parameter is a Consumer, that defines the actions that should be done upon receiving data.

Python

Dependencies

For a Python based application, first, add the following dependencies, which are available through PyPi:

Installation

pip install streammachine-driver

# TODO ensure that you use the correct schema, this is an example
pip install streammachine-schemas-catalog-clickstream-avro

Initialization

A client driver can be created using four parameters: billingId (your unique customer id), clientId (matches the stream your creating / consuming), clientSecret (the password required to interact with this stream), and the 'ClientConfig'.

from streammachine.driver import StreamMachineClient, ClientConfig

client = StreamMachineClient(
    billing_id,
    client_id,
    client_secret,
    ClientConfig()
)

Sending events

Receiving events

Receiving data is done through the .start_receiving_sse() method, which requires two parameters. The first is a boolean, stating whether the received data should be JSON, and the second parameter is a Callable, that defines the actions that should be done upon receival of data.