Last October, a "minor" dependency bump in a PySpark job cost us four hours of downtime and roughly $12,000 in cloud compute credits. A developer had tested a new UDF locally using an older version of delta-spark than what we ran on our EMR cluster. Locally, the serialization worked fine.
Source: [Dev.to](https://dev.to/aniketsoni/why-are-you-still-debugging-spark-on-your-host-machine-141e)