I'm the author of Data Engineering Design Patterns (O'Reilly),
a Databricks MVP, and
a freelance data engineer specializing in Apache Spark and Databricks.
I help teams move from working pipelines to resilient architectures.
I'm currently accepting new projects for Jun 2026. Whether you need a 2-day architectural audit, a hands-on lead for a
complex data engineering problem, or a workshop
let's discuss your project here.
Recently I had to upload multiple big datasets on Redshift just for test purposes. To do that, I used COPY command which makes data loading very fast. Everything was working great for all datasets exc...
I encountered that problem at table's creation time: [Amazon](500310) Invalid operation: ----------------------------------------------- error: Maximum column length exceeds 255 code: 8001 cont...
