Message streaming is an almost real-time streaming of device messages to a backend, platform, or cloud. The CTC platform receives device messages, handles exceptions, acknowledges them, transforms them (from a proprietary protocol to a human-readable format), enriches them with other metadata such as account, group, and asset, and makes them available as a stream to a data consumer.
Message Streaming Types
CTC supports two types of message streaming:
-
PULL-based streaming (CTC PULL [Data Pump]): A pull-based FIFO (first in first out) queue of device messages. The message consumer polls the queue at a predefined frequency. The TTL (time to live) for the queue is 48 hours, which means that messages older than 48 hours would be overwritten if they are not consumed.
-
PUSH-based streaming (CTC PUSH): CTC pushes messages to a custom destination that is hosted by the consumer (client). Currently, the PUSH capability is available for the Amazon Web Services (AWS) cloud platform (using either the AWS_Kinesis or AWS_SQS mechanism) and Apache Kafka (using KAFKA_SASL_SCRAM via AWS MSK or KAFKA_SASL_PLAIN via Confluent Cloud and Azure Event Hubs).
CTC is constantly adding new capabilities to deliver device messages to other popular cloud platforms.
CTC has different channels to provide clients access to their device messages:
- Data Pump: A real-time stream of messages that clients can pull from. The Data Pump streams are hosted within CTC that the clients poll to fetch messages.
- Results Services: Enables you to access past messages. You can search through the messages based on different criteria and time frames.
- Custom Destination Streaming: The PUSH-based solution in which the client hosts the destination to which CTC will push messages. The client can then pull the messages from their hosted destination.
Data Storage
Your telematics data is stored in the CTC database if your subscription package includes the Data Storage feature. If your package doesn’t include that, the data will be deleted from the messaging stream as soon as it is read.
You access your stored data via the set of APIs called Results Services, mentioned above. See the [Results Services] (https://developer.calamp.com/calamp/reference/results-services-historical-data) API documentation for more information.
The number of days your data is saved is an account-level setting, but this is commonly set for 90 days for telematics messages.