Skip to main content
Version: 13.x (Current)

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[1.5.2] - 2023-10-23

Added

  • introduce support for snappy compression in Java native image
  • introduce environment variable JAVA_OPTS for configuring native image parameters

Changed

  • updated Docker image employed for build and runtime stages

[1.5.1] - 2023-10-05

Fixed

  • An incoming Kafka message with null key is now handled correctly

[1.5.0] - 2023-08-10

Added

  • messages are cached until they reach a tunable size and then uploaded
  • cached messages are always uploaded after a tunable timeout to reach lag=0 on topics
  • introduced config key bss.cache-upload-timeout which controls lag=0 policy timeout
  • introduced config key bss.max-bytes-per-file (default 8MB) to establish the average file size to upload. Setting 0 recovers v1.4.0 behavior.
  • introduced config key bss.max-cache-size (default 10MB)

Fixed

  • kafka consumer, event producer and reingestion producer are functioning java beans
  • introduced missing shutdown logic
  • kafka deserializer is ByteArrayDeserializer (no content knowledge of messages payload or key is needed here)
  • log levels of internal libraries com.google.api.client, org.apache.kafka.common, io.netty.buffer, and io.opencensus lifted to WARN

[1.4.0] - 2023-06-28

Added

  • new configuration parameter bucket-folder for each ingestion topic, which allows to customize the folder where topic messages are stored

[1.3.2] - 2023-03-15

Fixed

  • bulding native image works again

[1.3.1] - 2023-03-15 [Removed]

Fixed

  • Post-ingestion topics are now optional when forwarding ingestion messages is disabled (enable-post-ingestion: false)

Changed

  • upgraded service dependencies

[1.3.0] - 2023-01-13

Changed

  • refactored service settings to remove references to Fast Data
  • upgraded service dependencies

[1.2.0] - 2022-12-30

Added

  • new metric to monitor the number of messages written to the bucket

Changed

  • file name now includes again the offset of the first message contained in the file
  • review README.md to clarify concepts and correct configurations
  • rename variables, error codes and metrics to reduce coupling with Fast Data concepts
  • upgrade service dependencies

[1.1.3] - 2022-11-30

Changed

  • remove default topics from app settings in application.yml config file

[1.1.2] - 2022-11-28

Changed

  • rename image name generated by the CI
  • update service dependencies

[1.1.1] - 2022-11-24

Added

  • introduce custom metric to monitor bucket requests

[1.1.0] - 2022-11-23

Added

  • introduce support for Buckets employing S3 APIs

Changed

  • improve code removing unnecessary coroutine creation
  • updated service dependencies

[1.0.4] - 2022-11-10

Changed

  • upgraded service dependencies
  • mechanism to generate output object name: now a file name contains also the topic name from which messages arrives, whereas offset of first record has been replaced with timestamp of when the object is created

Fixed

  • re-balance test

[1.0.3] - 2022-11-02

Fixed

  • re-balance listener logic has been reviewed to avoid deadlock situations where the consumer stops processing
  • removed Main entrypoint to allow the application properly shutdown gracefully

[1.0.2] - 2022-10-28

Changed

  • add newline after the last record written to a batch file to avoid potential issues with edge cases in file processors

[1.0.1] - 2022-10-27

Changed

  • review logging when offsets are committed

[1.0.0] - 2022-10-27

Initial Release

This is the initial version of the service, which subscribes to Fast Data ingestion topics, store those incoming messages into a bucket and emits a notification event onto a predisposed Kafka topic.

Optionally, it can forward messages from ingestion topics towards post-ingestion ones.