Skip to content

Shipping

Vessel data in real time with Apache Kafka instead of manual services.

For a vessel data analytics provider, we moved the existing architecture of Elasticsearch and manual services to a scalable streaming architecture with Apache Kafka.

  • Apache Kafka
  • Amazon MSK
  • Kafka Streams
  • Spring Boot
  • Java
  • Schema Registry
  • Kafka Connect
  • MirrorMaker
  • GitLab CI/CD
  • Kubernetes
  • DigitalOcean

01

The starting point

A provider of vessel data analytics processed raw vessel data with an architecture built on Elasticsearch and manual services. The goal was to move to a streaming architecture: raw data should be processed continuously and analysed for registered customers, and the system had to scale with growing data volumes.

02

Our approach

  1. 01 Selecting and provisioning a suitable Apache Kafka setup on Amazon MSK
  2. 02 Streaming architecture: raw data flows from an input topic into further partitioned topics, from which insights are extracted for registered customers
  3. 03 Implementing the microservices with Java, Spring Boot and Kafka Streams, including Schema Registry, Kafka Connect and MirrorMaker
  4. 04 Integration into the existing CI/CD environment with GitLab
  5. 05 Running the microservices on managed Kubernetes at DigitalOcean

03

The result

  • A streaming architecture instead of manual services
  • High scalability: because the vessel data is already partitioned, the services scale horizontally through replicas
  • Analyses for registered customers straight from the data streams
  • Automated delivery of the microservices via GitLab CI/CD

More case studies

IT communications

Fully automated releases for three CPU architectures and many distributions.

For an appliance and middleware solution, every release variant is now built automatically: signed, tested and delivered straight to the customer's community.

  • CI/CD
  • Jenkins
  • Ansible
  • Docker
  • Package signing
  • C
  • Java
Automotive

A modern platform for .NET applications on Amazon EKS.

A modernised application environment on Kubernetes with blue-green deployments, a CI/CD pipeline and clear compliance rules, plus consulting for development and product management.

  • AWS
  • Amazon EKS
  • .NET
  • Blue-green deployment
  • Atlassian Bamboo
  • Compliance

Sounds like your project?

Tell us where you are right now. We'll tell you honestly whether and how we can help.

Get in touch

You'll hear back from us within one working day. The first conversation is free and non-binding.