Upgrading MLflow Pipelines Across a Python Version Boundary

Picture this: you are running automated daily Machine Learning pipelines at scale, and it is time for a critical environment upgrade, such as moving to a new Python version. Consequently, you need to upgrade your Databricks Runtime from LTS 14.3 to LTS 17.3. However, the upgrade cannot proceed because the logs show a cryptic error:

TypeError: code() argument 13 must be str, not int.

This was the exact challenge our team faced recently during a Databricks upgrade that uses MLflow. At our core, we manage daily automated pipelines spanning data preparation, training, and inference across separate Dev, Staging, and Prod Databricks workspaces managed via Declarative Automation Bundles (DAB).