Pyspark: Split And Select Part Of The String Column Values

How can I select the characters or file path after the Dev\” and dev\ from the column in a spark DF? Sample rows of the pyspark column: \\D\Dev\johnny\Desktop\TEST \\D\Dev\matt\

Solution 1:

The following modification [Dd] matches both upper and lower case d.

df = df.withColumn(
        "sub_path",
        F.element_at(F.split(F.col("path"), "[Dd]ev\\\\"), -1)
    )

Let me know if this works for you.