Big Data patterns implemented articles

on waitingforcode.com
Articles tagged with Big Data patterns implemented. There are 3 article(s) corresponding to the tag Big Data patterns implemented. If you don't find what you're looking for, please check related tags: Cerberus + PySpark, Change Data Capture, custom state store, custom state store, hybrid orchestration and coordination, idempotent consumer, random algorithms, Spark AI Summit Europe 2019 articles, Spark SQL aggregations, Spark SQL and Avro.

Check out my new course on Data Engineering!

Are you a data scientist who wants to extend his data engineering skills? Or a software engineer who wants to work with Big Data? If not, maybe a BI developer who wants to evolve to engineering position? My course will help you to achieve your goal! Join the class →

Big Data patterns implemented - automated processing metadata insertion

Sometimes metadata is disregarded but very often it helps to retrieve the information easier and faster. One of such use cases are the headers of Apache Parquet where the stats about the column's content are stored. The reader can, without parsing all the lines, know whether what is he looking for is in the file or not. The metadata is also a part of one of Big Data patterns called automated processing metadata insertion. Continue Reading →