CLI tips

I'm the author of Data Engineering Design Patterns (O'Reilly), a Databricks MVP, and a freelance data engineer specializing in Apache Spark and Databricks. I help teams move from working pipelines to resilient architectures.
I'm currently accepting new projects for Jun 2026. Whether you need a 2-day architectural audit, a hands-on lead for a complex data engineering problem, or a workshop let's discuss your project here.

How to gzip every file separately ?

find command and pipes ("|") in Unix-like systems are very powerful features. The former retrieves the files matching the specified pattern. The latter one helps to transform the output of the previou...

Continue Reading β†’