How to implement a STRING_AGG function in Apache Spark SQL?

You need to migrate a SQL Server code that uses STRING_AGG function. Unfortunately, it's a SQL Server-specific feature. To migrate it, you will have to use other transformations available in Apache Spark SQL. Below is one of the examples that supports sorting on a column which is not part of the aggregation (called here sort_column):

CONCAT_WS('; ', 
  FILTER(AGGREGATE(
    SORT_ARRAY(COLLECT_LIST(CONCAT_WS(' :: ', sort_column, output_column))), 
    ARRAY(''), (string_so_far, new_entry) -> ARRAY_APPEND(string_so_far, SPLIT_PART(new_entry, ' :: ', 2))),
   entry -> entry != '')