Skip to content
Failed

Changes

Summary

  1. add more debug logs to SshTunnelActor; (details)
  2. adapt log level to warn in case the SSH tunnel failed; (details)
  3. Implement config for toggling whether messages whose enrichment failed should be published anyways (details)
  4. Issue failed acks for failed enrichments (details)
  5. improve logging of ThingUpdater etc. in order to be better able to find out which changne caused a "failed patch update" (details)
  6. fix parallel execution issues in ThingUpdater/MongoSearchUpdaterFlow (details)
  7. added unit test ThingUpdaterTest.parallelUpdatesProvokingIncorrectPatchUpdate in order to test bugfix for parallelism problems in ThingUpdater (details)
  8. fixed behavior switch which did not handle too many messages in the previous "awaitingBulkWriteResult" (details)
  9. reduce logging when stashing WriteModels (details)
  10. improved logging for queries and counts in SearchActor (details)
  11. fix toString() methods of various model classes (details)
  12. filter for incorrect element types in jsonArray of feature definitions; (details)
  13. remove throttling limits currently in place by default for Ditto: (details)
  14. introduced explicit "enabled" option for ThrottlingConfig and by default disable it in Ditto (details)
  15. simplified jsonString checking logic when creating a FeatureDefinition from a `featureDefinitionEntriesAsJsonArray` (details)
  16. fixed WARNING logging in RequestTimeoutHandlingDirective to log warnings for search requests only > 5s (details)
  17. update scala version to 2.13.8 and fluency to 2.6.3 (details)
  18. made Kafka client "session.timeout.ms" and "heartbeat.interval.ms" explicit in Ditto config (details)
  19. removed file name config from logback.xml because of a bug in logback core which causes the creation of tmp files that are never deleted; (details)
  20. update docu for use of DITTO_LOGGING_TOTAL_LOG_FILE_SIZE in installation-operating.md; (details)
  21. extend documentation for use of Ditto log file feature; (details)
  22. remove uncommented line and fix fileNamePattern of concierge logback.xml; (details)
  23. don't run parallel maven build (details)
  24. fixed default docker-compose.yml (details)
Commit e66bf385a51beee069138f8f594708f7e2ab7d39 by Stefan Maute
add more debug logs to SshTunnelActor;
configure connecting-min-timeout to 50s because this timeout needs to be smaller than ditto.connectivity.connection.client-actor-ask-timeout;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>
The file was modified connectivity/service/src/main/resources/connectivity.conf (diff)
The file was modified connectivity/service/src/main/java/org/eclipse/ditto/connectivity/service/messaging/BaseClientActor.java (diff)
The file was modified connectivity/service/src/main/java/org/eclipse/ditto/connectivity/service/messaging/persistence/ConnectionPersistenceActor.java (diff)
The file was modified connectivity/service/src/main/java/org/eclipse/ditto/connectivity/service/messaging/tunnel/SshTunnelActor.java (diff)
Commit 9cba450cf45df1f37af481fa5438e0e5edab51a7 by Stefan Maute
adapt log level to warn in case the SSH tunnel failed;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>
The file was modified connectivity/service/src/main/java/org/eclipse/ditto/connectivity/service/messaging/tunnel/SshTunnelActor.java (diff)
Commit 09e297ab2d7bbd895ac49a312b2d01f0e695ef33 by David Schwilk
Implement config for toggling whether messages whose enrichment failed should be published anyways

Signed-off-by: David Schwilk <david.schwilk@bosch.io>
The file was modified connectivity/service/src/main/resources/connectivity.conf (diff)
The file was modified connectivity/service/src/main/java/org/eclipse/ditto/connectivity/service/config/mapping/DefaultMappingConfig.java (diff)
The file was modified connectivity/service/src/main/java/org/eclipse/ditto/connectivity/service/messaging/OutboundMappingProcessorActor.java (diff)
The file was modified connectivity/service/src/test/resources/mapping-test.conf (diff)
The file was modified connectivity/service/src/test/java/org/eclipse/ditto/connectivity/service/config/mapping/DefaultMappingConfigTest.java (diff)
The file was modified connectivity/service/src/main/java/org/eclipse/ditto/connectivity/service/config/mapping/MappingConfig.java (diff)
Commit 52ec18228527458bd3e2b5e13b3b15ac6d401662 by David Schwilk
Issue failed acks for failed enrichments

Signed-off-by: David Schwilk <david.schwilk@bosch.io>
The file was modified connectivity/service/src/test/resources/test.conf (diff)
The file was modified connectivity/service/src/main/java/org/eclipse/ditto/connectivity/service/messaging/OutboundMappingProcessorActor.java (diff)
The file was modified connectivity/service/src/test/java/org/eclipse/ditto/connectivity/service/messaging/OutboundMappingProcessorActorTest.java (diff)
Commit 2410aa9b0a828271a4f1be8e87689792055e4805 by Thomas Jaeckle
improve logging of ThingUpdater etc. in order to be better able to find out which changne caused a "failed patch update"
* also use "trace" logging for logging potentially huge complete "writeModels"

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>
The file was modified thingsearch/service/src/main/java/org/eclipse/ditto/thingsearch/service/updater/actors/ThingUpdater.java (diff)
The file was modified thingsearch/service/src/main/java/org/eclipse/ditto/thingsearch/service/updater/actors/ThingsUpdater.java (diff)
The file was modified thingsearch/service/src/main/java/org/eclipse/ditto/thingsearch/service/persistence/write/streaming/ChangeQueueActor.java (diff)
The file was modified thingsearch/service/src/main/java/org/eclipse/ditto/thingsearch/service/persistence/write/streaming/MongoSearchUpdaterFlow.java (diff)
The file was modified thingsearch/service/src/main/java/org/eclipse/ditto/thingsearch/service/persistence/write/streaming/BulkWriteResultAckFlow.java (diff)
Commit 76795881327a48fa2c7830d1b6b51046789aba65 by Thomas Jaeckle
fix parallel execution issues in ThingUpdater/MongoSearchUpdaterFlow
* which could lead to "failed patch updates" when by small chance a new WriteModel was received in ThingUpdater at almost the same time when the BulkWrite was processed in MongoSearchUpdaterFlow
* a BulkWriteComplete message now signals the ThingUpdater that it may continue accepting WriteModels again
* made "force-update-after-start-enabled" by default `false`
* fixed unit tests by adding the expected order of messages in there

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>
The file was modified thingsearch/service/src/main/java/org/eclipse/ditto/thingsearch/service/updater/actors/ThingUpdater.java (diff)
The file was addedthingsearch/service/src/main/java/org/eclipse/ditto/thingsearch/service/persistence/BulkWriteComplete.java
The file was modified thingsearch/service/src/main/java/org/eclipse/ditto/thingsearch/service/persistence/write/streaming/MongoSearchUpdaterFlow.java (diff)
The file was modified thingsearch/service/src/test/java/org/eclipse/ditto/thingsearch/service/updater/actors/ThingUpdaterTest.java (diff)
The file was modified thingsearch/service/src/main/java/org/eclipse/ditto/thingsearch/service/common/config/UpdaterConfig.java (diff)
The file was modified thingsearch/service/src/main/resources/things-search.conf (diff)
The file was modified thingsearch/service/src/main/java/org/eclipse/ditto/thingsearch/service/persistence/PersistenceConstants.java (diff)
Commit ce57d243c065bb4277ce58e280f3f0a86496bdc8 by Thomas Jaeckle
added unit test ThingUpdaterTest.parallelUpdatesProvokingIncorrectPatchUpdate in order to test bugfix for parallelism problems in ThingUpdater

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>
The file was modified thingsearch/service/src/test/java/org/eclipse/ditto/thingsearch/service/updater/actors/ThingUpdaterTest.java (diff)
Commit b44ebcb1c009b45cebcf63ff7e77b15cbb9e5d02 by Thomas Jaeckle
fixed behavior switch which did not handle too many messages in the previous "awaitingBulkWriteResult"
* now only 2 behaviors exist: one waiting for bulkWrite result and one not, while the one waiting for bulkWrite result only stashes writeModels which occur during waiting for the bulkWrite result

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>
The file was modified thingsearch/service/src/main/java/org/eclipse/ditto/thingsearch/service/updater/actors/ThingUpdater.java (diff)
Commit eb34037c1749f5ff60d38f1480101cfe8f92e279 by Thomas Jaeckle
reduce logging when stashing WriteModels

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>
The file was modified thingsearch/service/src/main/java/org/eclipse/ditto/thingsearch/service/updater/actors/ThingUpdater.java (diff)
Commit a081de0c2fc1ff44fcfa5874a5ffe2cb23053238 by Thomas Jaeckle
improved logging for queries and counts in SearchActor

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>
The file was modified thingsearch/service/src/main/java/org/eclipse/ditto/thingsearch/service/starter/actors/SearchActor.java (diff)
Commit a5dee831c13ef979c185672c9bdcd9703360cbff by Thomas Jaeckle
fix toString() methods of various model classes

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>
The file was modified things/model/src/main/java/org/eclipse/ditto/things/model/signals/events/ThingMerged.java (diff)
The file was modified things/model/src/main/java/org/eclipse/ditto/things/model/signals/events/FeatureDesiredPropertyModified.java (diff)
The file was modified things/model/src/main/java/org/eclipse/ditto/things/model/signals/commands/modify/MergeThingResponse.java (diff)
The file was modified things/model/src/main/java/org/eclipse/ditto/things/model/signals/events/FeaturePropertyModified.java (diff)
The file was modified things/model/src/main/java/org/eclipse/ditto/things/model/signals/events/FeatureDesiredPropertyCreated.java (diff)
The file was modified things/model/src/main/java/org/eclipse/ditto/things/model/signals/commands/modify/MergeThing.java (diff)
The file was modified things/model/src/main/java/org/eclipse/ditto/things/model/signals/events/FeaturePropertyCreated.java (diff)
Commit ebd78dcadc4f953acf6dabc38bdd497dcc5bd7b3 by Stefan Maute
filter for incorrect element types in jsonArray of feature definitions;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>
The file was modified things/model/src/main/java/org/eclipse/ditto/things/model/ImmutableFeatureDefinition.java (diff)
The file was modified things/model/src/test/java/org/eclipse/ditto/things/model/ImmutableFeatureDefinitionTest.java (diff)
The file was modified gateway/service/src/main/java/org/eclipse/ditto/gateway/service/endpoints/routes/things/FeaturesRoute.java (diff)
Commit cab08f8d6cdfa1fff0e13117b6136da34bd87f34 by Thomas Jaeckle
remove throttling limits currently in place by default for Ditto:
* Kafka consumer was limited to receive 100 msg/s
* AMQP 1.0 consumer was limited to receive 100 msg/s
* Websocket was limited to receive 100 msg/s

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>
The file was modified connectivity/service/src/main/resources/connectivity.conf (diff)
The file was modified connectivity/service/src/main/java/org/eclipse/ditto/connectivity/service/messaging/kafka/AtMostOnceConsumerStream.java (diff)
The file was modified gateway/service/src/main/resources/gateway.conf (diff)
The file was modified connectivity/service/src/main/java/org/eclipse/ditto/connectivity/service/messaging/InboundMappingSink.java (diff)
The file was modified connectivity/service/src/main/java/org/eclipse/ditto/connectivity/service/config/Amqp10ConsumerConfig.java (diff)
The file was modified connectivity/service/src/main/java/org/eclipse/ditto/connectivity/service/messaging/kafka/AtLeastOnceConsumerStream.java (diff)
The file was modified base/service/src/main/java/org/eclipse/ditto/base/service/config/ThrottlingConfig.java (diff)
Commit 0076d4bbd357a3f13f8e9c22f3ec4aa52cb44523 by Thomas Jaeckle
introduced explicit "enabled" option for ThrottlingConfig and by default disable it in Ditto

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>
The file was modified connectivity/service/src/main/java/org/eclipse/ditto/connectivity/service/config/DefaultAmqp10ConsumerConfig.java (diff)
The file was modified connectivity/service/src/main/java/org/eclipse/ditto/connectivity/service/config/Amqp10ConsumerConfig.java (diff)
The file was modified connectivity/service/src/main/java/org/eclipse/ditto/connectivity/service/config/ConnectionThrottlingConfig.java (diff)
The file was modified documentation/src/main/resources/pages/ditto/installation-operating.md (diff)
The file was modified connectivity/service/src/test/java/org/eclipse/ditto/connectivity/service/messaging/kafka/KafkaConsumerActorTest.java (diff)
The file was modified base/service/src/test/resources/throttling-test.conf (diff)
The file was modified gateway/service/src/main/java/org/eclipse/ditto/gateway/service/endpoints/routes/AbstractRoute.java (diff)
The file was modified connectivity/service/src/main/java/org/eclipse/ditto/connectivity/service/messaging/kafka/KafkaConsumerStreamFactory.java (diff)
The file was modified base/service/src/main/java/org/eclipse/ditto/base/service/config/ThrottlingConfig.java (diff)
The file was modified base/service/src/test/java/org/eclipse/ditto/base/service/config/DefaultThrottlingConfigTest.java (diff)
The file was modified connectivity/service/src/main/java/org/eclipse/ditto/connectivity/service/config/DefaultConnectionThrottlingConfig.java (diff)
The file was modified base/service/src/main/java/org/eclipse/ditto/base/service/config/DefaultThrottlingConfig.java (diff)
The file was modified connectivity/service/src/test/resources/test.conf (diff)
The file was modified gateway/service/src/main/java/org/eclipse/ditto/gateway/service/endpoints/routes/websocket/WebSocketRoute.java (diff)
The file was modified gateway/service/src/test/resources/streaming-test.conf (diff)
The file was modified connectivity/service/src/test/java/org/eclipse/ditto/connectivity/service/config/DefaultAmqp10ConfigTest.java (diff)
The file was modified connectivity/service/src/main/java/org/eclipse/ditto/connectivity/service/messaging/kafka/AtLeastOnceConsumerStream.java (diff)
The file was modified connectivity/service/src/test/resources/amqp10-test.conf (diff)
The file was modified connectivity/service/src/main/java/org/eclipse/ditto/connectivity/service/messaging/amqp/MessageRateLimiterBehavior.java (diff)
The file was modified gateway/service/src/main/resources/gateway.conf (diff)
The file was modified connectivity/service/src/main/java/org/eclipse/ditto/connectivity/service/messaging/kafka/AtMostOnceConsumerStream.java (diff)
The file was modified connectivity/service/src/main/java/org/eclipse/ditto/connectivity/service/messaging/InboundMappingSink.java (diff)
The file was modified connectivity/service/src/test/resources/kafka-test.conf (diff)
The file was modified connectivity/service/src/main/java/org/eclipse/ditto/connectivity/service/messaging/amqp/MessageRateLimiter.java (diff)
The file was modified gateway/service/src/test/resources/websocket-test.conf (diff)
The file was modified connectivity/service/src/main/resources/connectivity.conf (diff)
The file was modified connectivity/service/src/test/resources/connection-throttling-test.conf (diff)
Commit b68a88acb9bd177e58b3752316ac16aefd68853e by Thomas Jaeckle
simplified jsonString checking logic when creating a FeatureDefinition from a `featureDefinitionEntriesAsJsonArray`

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>
The file was modified things/model/src/main/java/org/eclipse/ditto/things/model/ImmutableFeatureDefinition.java (diff)
Commit 21d6a2d65ff3c696a957f2f1b2c3556cc517173a by Thomas Jaeckle
fixed WARNING logging in RequestTimeoutHandlingDirective to log warnings for search requests only > 5s

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>
The file was modified connectivity/service/src/main/java/org/eclipse/ditto/connectivity/service/messaging/httppush/DefaultHttpPushFactory.java (diff)
The file was modified gateway/service/src/main/java/org/eclipse/ditto/gateway/service/endpoints/directives/RequestTimeoutHandlingDirective.java (diff)
Commit 48a740bd8296aaad6869278ea19d32d0419b1bfc by Thomas Jaeckle
update scala version to 2.13.8 and fluency to 2.6.3

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>
The file was modified bom/pom.xml (diff)
Commit d54a98547411c247d669d5adc5984f0e7d9ca810 by Thomas Jaeckle
made Kafka client "session.timeout.ms" and "heartbeat.interval.ms" explicit in Ditto config
* and provide environment variables for overwriting those

Signed-off-by: Thomas Jaeckle <thomas.jaeckle@bosch.io>
The file was modified connectivity/service/src/main/resources/connectivity.conf (diff)
Commit 1097d9de7cfccb70654fa00ea8b3c30aaad725f1 by Stefan Maute
removed file name config from logback.xml because of a bug in logback core which causes the creation of tmp files that are never deleted;
add possibility to configure clean up of old log files and archives at start up;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>
The file was modified gateway/service/src/main/resources/logback.xml (diff)
The file was modified things/service/src/main/resources/logback.xml (diff)
The file was modified connectivity/service/src/main/resources/logback.xml (diff)
The file was modified policies/service/src/main/resources/logback.xml (diff)
The file was modified documentation/src/main/resources/pages/ditto/installation-operating.md (diff)
The file was modified concierge/service/src/main/resources/logback.xml (diff)
The file was modified thingsearch/service/src/main/resources/logback.xml (diff)
Commit 8e0aab47e148c2ea6a2882a80efe3626d1386ecf by Stefan Maute
update docu for use of DITTO_LOGGING_TOTAL_LOG_FILE_SIZE in installation-operating.md;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>
The file was modified documentation/src/main/resources/pages/ditto/installation-operating.md (diff)
Commit aa6221c4cf7eebaa4ba8b1720486ecfcffc73501 by Stefan Maute
extend documentation for use of Ditto log file feature;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>
The file was modified connectivity/service/src/main/resources/logback.xml (diff)
The file was modified thingsearch/service/src/main/resources/logback.xml (diff)
The file was modified gateway/service/src/main/resources/logback.xml (diff)
The file was modified things/service/src/main/resources/logback.xml (diff)
The file was modified policies/service/src/main/resources/logback.xml (diff)
The file was modified concierge/service/src/main/resources/logback.xml (diff)
The file was modified documentation/src/main/resources/pages/ditto/installation-operating.md (diff)
Commit b9a6a411142ba8bc96ec57715df4d704dbacab6c by Stefan Maute
remove uncommented line and fix fileNamePattern of concierge logback.xml;

Signed-off-by: Stefan Maute <stefan.maute@bosch.io>
The file was modified things/service/src/main/resources/logback.xml (diff)
The file was modified concierge/service/src/main/resources/logback.xml (diff)
Commit 972c59dd082bb44642768c0e5ef4ebb33c6a0e73 by noreply
don't run parallel maven build
The file was modified .github/workflows/maven.yml (diff)
Commit 8a3100a785339b8bf3b7bb82461f2023902eee79 by noreply
fixed default docker-compose.yml

by not using a specific command by default
The file was modified deployment/docker/docker-compose.yml (diff)