Python 2 lost support on January 1, 2020. It’s time to upgrade to Python 3.
However, is there a way to proceed without interrupting application development and operation?
This guide will show you how to upgrade to Python 3 quickly, easily, and cost-effectively.
Suppose you already have the latest Python 2.7, and you are targeting Python 3.6 or higher.
It is not recommended to use any version lower than Python 3.5, because it is already the oldest version still supported, and its service end-of-life is on September 13, 2020.
Before reading the immigration guide, there is one last thing: if you want to know why you only use Python 3, this article here will definitely answer any questions you may encounter. This is the “cause” of the “method” of the next guide.
I. The First steps toward Python 3 compatibility
1. Every new piece of code should be compatible with Python 3
Even if you are just beginning to consider switching to Python 3, you need to immediately develop a strategy for code development: at least, in theory, every new code segment submitted to the repository should be Python 3. This is the “best-effort” transaction here.
If your product is under active development, just following this principle will make the actual migration smoother. It is also super cheap.
No need to change your continuous integration pipeline, which may be good. Just because you currently do not have the authority to make pipeline changes and the delay rule makes no sense.
You may be busy introducing new features. By making any Python 3 code compatible with you, you can pass without spending a lot of development time.
This is indeed an ideal starting point.
2. Quality assurance tools are your friends
Good test coverage, linters, and other tools run under your continuous integration systems are lifesavers whenever you introduce far-reaching changes to your application.
If you do not use one of them, it is recommended that you consider using it.
Here are some quality assurance tools that may be very useful when moving to Python 3:
Linters are the easiest to introduce, but that doesn’t mean they have little value. They will provide a welcome boost to your migration efforts.
Tests are pretty essential and unfortunately require a certain time investment, especially at the start, but they’re well worth it. For a sizable application, even the most basic happy path tests will save you countless hours you would otherwise spend on laborious manual testing and fighting regressions.
c) Continuous integration
Continuous integration brings all your software development efforts together in an automated manner. Once again, this is a time-saving measure, especially important if more than one person works on your product.
d) Error tracking
Error tracking is yet another tool that can prove really helpful should something slip through the cracks of pre-production testing.
These quality assurance tools aren’t strictly required for migrating to Python 3. However, it will be much harder to ensure your software keeps running smoothly without them. Introducing QA tools will also improve your standard development workflow.
All in all, the faster you implement the tools, the better for you.
3. Safe compatibility fixes
To kickstart your efforts, use automatic code conversion.
Here’s what you should do, step by step:
- Add a new “six” dependency to your application dependencies.
- Run “pip install modernize.”
- Run “python-modernize -w” in your project directory.
- Review the changes. They should be reasonably safe, but sometimes visibly inefficient — adjust them as you see fit.
- Test your app on Python 2.
If all goes well, you should have relatively safe compatibility fixes already applied to your code base. You can use them as a point of reference when adding new code until you fully switch to Python 3.
4. Updating dependencies, part one: the easy wins
Your application is already on the way to reach full Python 3 compatibility, but the issue of its dependencies still remains.
It’s not uncommon for projects to accumulate dependencies that are no longer maintained and consequently lack Python 3 support.
In some cases, all you’ll need to do is update a dependency on a newer version; in others, you’ll have to make sure the update is the latest version, compatible with both Python 2 and 3. That’s because certain packages may have already dropped Python 2.
Regardless, at this point, it’s best to concentrate on the easy wins.
Most of your packages are likely already compatible or only require an update to a newer version. As a rule of thumb, we suggest updating to the latest release of each dependency to be on the safe side.
Checking each dependency one by one can be time-consuming in larger projects. You can facilitate the process by running “caniusepython3” on your “requirements.txt” (create one with “pip freeze > requirements.txt” if you don’t have one).
This tool is far from accurate, but it’s good enough to achieve our main goal here: assessing how much work remains before you can make the final switch to Python 3.
II. Running Python 3
1. Updating dependencies, part two: finishing the job
After some time has passed, you’re welcome to once more look for Python 3-compatible alternatives to your dependencies that you weren’t able to find before.
If you still come up empty, you’d be wise to consider whether you want to be stuck with an unsupported and unmaintained package. And if removing it from your codebase would take up too much of your time — fork it and apply the exact same process. In the case of most packages, a single run of “python-modernize” might very well fix the issue.
Once you’re done, either publish your fork on PyPI for others to benefit from or directly install it from the repository and use it that way. Just make sure to credit the author and remember to include the original license, since it’s required by most open-source licenses.
2. Last steps in your Python 3 migration
At this stage, you’re very close to being 100% Python 3 ready. However, there are still several steps left to help you avoid potential problems during production deployment.
a) Continuous integration
If you are not doing already start running your app under Python 3 parallel to Python 2 in your continuous integration environment. Even if the tests start failing, it will be helpful in measuring your progress and preventing regressions.
If you don’t have full-fledged continuous integration, consider using tox for local testing under multiple versions of Python.
b) Cache and shared application state
Remember to flush your cache right before deployment. With such a significant change, an application state that is anything other than blank will make debugging significantly more cumbersome.
This is especially true for objects pickled using the “pickle” module under Python 2; they won’t be compatible with Python 3, so be sure to remove or recreate them before deployment. Otherwise, your app may fail outright, even if it was working just fine during testing on a clean test environment.
c) Manual regression tests
No matter how comprehensive your tests are, you can always miss some things, like differences in the configuration or the processed data. To make sure that’s not the case, a final manual check for regressions is a must before you begin production deployment.
Beside happy paths, which are the most important, remember to test what is most often missed, but may become a real problem during production setup. Your application needs to work correctly with:
- the WSGI production server (as opposed to the built-in server in development mode);
- emails and other external means of communication that may have been mocked in the dev/test setup;
- production-like data as well as an empty database.
3. Gradual deployment or a leap of faith
Depending on a number of factors, such as the resources available or your service-level agreement, you should consider gradual deployment.
If your service is load-balanced across multiple server instances, it may be a good idea to launch Python 3 on only one of them, then start increasing the role of Python 3 as your confidence in the migration grows.
This is the safest option, but it does require additional work, especially since some artefacts, such as the aforementioned cache, have to be kept separate in Python 2 and Python 3 instances.
Or… you could just go for it and deploy the whole thing at once!
Either way, some minor bugs are bound to appear, so be prepared to respond to them quickly. An error aggregation tool or sound logging management should allow you to solve them at the first sign of trouble in no time at all.
III. Final thoughts
If you want to speed up your Python 3 migration, it helps to have someone with experience in both Python 2 and 3 codebases.
Even though a substantial part of the process is automated, it’s still far from perfect. What’s worse, steps such as updating or even swapping some of the dependencies can’t be done without high working knowledge of them.
For that reason, when it comes to larger applications, we suggest hiring outside specialists if you don’t currently have the right people for the job in-house.