What's new on the cloud for data engineers - part 11 (06-09.2023)

It's time for another part of "What's new on the cloud for data engineers". Let's see what happened in the last 4 months.

Data Engineering Design Patterns

Looking for a book that defines and solves most common data engineering problems? I'm currently writing one on that topic and the first chapters are already available in πŸ‘‰ Early Release on the O'Reilly platform

I also help solve your data engineering problems πŸ‘‰ contact@waitingforcode.com πŸ“©

This 11th part covers all that happened between 28.05.2023 and 16.09.2023. As previously, I highlighted the most interesting news.

tl;nr. The blog post includes all major changes on the data engineering-like services. If you don't have time passing through all of them, you can find a short list of my top picks for that period:

  1. AWS: Managed Apache Flink in EMR.
  2. AWS: Timestamp-based starting position for Lambda on Apache Kafka.
  3. AWS: Support for querying Apache Iceberg tables
  4. Azure: Entra ID is the new name for Azure Active Directory.
  5. Azure: General availability of the auto-scaling for Stream Analytics jobs.
  6. GCP: Data clean rooms for a simplified data sharing on BigQuery.
  7. GCP: Preview of BigQuery Studio for enhanced data discovery experience.
  8. GCP: Pub/Sub direct synchronization with GCS.
  9. GCP: Event-driven transfers for Storage Transfer Service.

AWS

Athena

Aurora

Batch

Data Sync

Database Migration Service

DocumentDB

DynamoDB

EMR

EventBridge

Timestream

QuickSight

Azure

Backup

Cache for Redis

Cosmos DB

Data Explorer

Database Migration

Event Grid

Functions

HDInsight

Monitor

SQL Database


If you liked it, you should read:

πŸ“š Newsletter Get new posts, recommended reading and other exclusive information every week. SPAM free - no 3rd party ads, only the information about waitingforcode!