You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/backend/upgrading/version-specific-migration/upgrade-to-python3.md
+82-60Lines changed: 82 additions & 60 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -40,23 +40,25 @@ In general, you should follow these steps to port add-ons:
40
40
41
41
In the GitHub repository of the add-on:
42
42
43
-
- Open a ticket with the title "Add support for Python 3".
44
-
- Create a new branch named `python3`.
43
+
- Open a ticket with the title "Add support for Python 3".
44
+
- Create a new branch named `python3`.
45
+
46
+
47
+
### Using released Plone 5.2
45
48
46
49
```{warning}
47
-
The following section is valid until the final release of Plone 5.2.
50
+
This section is valid until the final release of Plone 5.2.
48
51
Upon the final release of Plone 5.2, something else will take its place.
49
52
```
50
53
51
-
### Using Released Plone 5.2
54
+
Usually you can use the latest Plone 5.2 release.
55
+
The version pins for the latest release can be found for `pip` at https://dist.plone.org/release/5.2-latest/requirements.txt and for `buildout` at https://dist.plone.org/release/5.2-latest/versions.cfg.
56
+
Install Plone with Python 3.6, 3.7, or 3.8, and then add your add-ons as source using `mr.developer`.
52
57
53
-
Usually it is fine to use the latest Plone 5.2 release.
54
-
The version pins for the latest release can be found for pip at [https://dist.plone.org/release/5.2-latest/requirements.txt] and for buildout at [https://dist.plone.org/release/5.2-latest/versions.cfg].
55
-
Install Plone with Python 3.6 or 3.7 and then add your addons as source using mr.developer\`.
56
58
57
-
### Using Core Development Buildout
59
+
### Using core development buildout
58
60
59
-
With [buildout.coredev](https://github.com/plone/buildout.coredev) the latest development version of Plone can be used.
61
+
With [`buildout.coredev`](https://github.com/plone/buildout.coredev), the latest development version of Plone can be used.
60
62
It contains everything for porting an add-on to Python 3.
Then the source of the add-on package will be checked out into the `src` folder.
106
108
109
+
```shell
110
+
./bin/buildout -c local.cfg
111
+
```
112
+
107
113
````{note}
108
-
You can also add development tools like [Products.PDBDebugMode](https://pypi.org/project/Products.PDBDebugMode/), [plone.reload](https://pypi.org/project/plone.reload/) and [Products.PrintingMailHost](https://pypi.org/project/Products.PrintingMailHost/) to your buildout.
114
+
You can also add development tools like [`Products.PDBDebugMode`](https://pypi.org/project/Products.PDBDebugMode/), [`plone.reload`](https://pypi.org/project/plone.reload/) and [`Products.PrintingMailHost`](https://pypi.org/project/Products.PrintingMailHost/) to your `buildout`.
109
115
110
116
Especially `Products.PDBDebugMode` will help a lot with issues during porting to Python 3.
111
117
@@ -124,10 +130,6 @@ auto-checkout +=
124
130
```
125
131
````
126
132
127
-
```shell
128
-
./bin/buildout -c local.cfg
129
-
```
130
-
131
133
Now everything is prepared to work on the migration of the package.
132
134
133
135
For small packages or packages that have few dependencies, it is a good idea to try starting your instance now.
@@ -138,16 +140,17 @@ For small packages or packages that have few dependencies, it is a good idea to
138
140
139
141
If it does not start up, you should continue with the next steps instead of trying to fix each issue as it appears.
140
142
141
-
## 2. Automated Fixing With Modernize
143
+
144
+
## 2. Automated fixing with modernize
142
145
143
146
`python-modernize` is a utility that automatically prepares Python 2 code for porting to Python 3.
144
147
After running `python-modernize`, there is manual work ahead.
145
-
There are some problems that `python-modernize`can not fix on its own.
148
+
There are some problems that `python-modernize`cannot fix on its own.
146
149
It also can make changes that are not really needed.
147
150
You need to closely review all changes after you run this tool.
148
151
149
-
`python-modernize` will warn you, when it is not sure what to do with a possible problem.
150
-
Check this [Cheat Sheet](http://python-future.org/compatible_idioms.html)with idioms for writing Python 2/3 compatible code.
152
+
`python-modernize` will warn you when it is not sure what to do with a possible problem.
153
+
Check this [Cheat Sheet](http://python-future.org/compatible_idioms.html) with idioms for writing Python 2/3 compatible code.
151
154
152
155
`python-modernize` adds an import of the compatibility library `six` if needed.
153
156
The import is added as the last import, therefore it is often necessary to reorder the imports.
@@ -156,6 +159,7 @@ Check the [Python style guide for Plone](https://docs.plone.org/develop/stylegui
156
159
157
160
If `six` is used in the code, make sure that `six` is added to the `install_requires` list in the `setup.py` of the package.
158
161
162
+
159
163
### Installation
160
164
161
165
Install `modernize` into your Python 3 environment with `pip`.
@@ -170,9 +174,10 @@ Install `isort` into your Python 3 environment with `pip`.
170
174
./bin/pip install isort
171
175
```
172
176
177
+
173
178
### Usage
174
179
175
-
The following command is a dry-run. I shows all changes that `modernize` would make.
180
+
The following command is a dry-run. It shows all changes that `modernize` would make.
@@ -198,6 +203,7 @@ You can use `isort` to fix the order of imports:
198
203
199
204
After you run the commands above, you need to review all changes and fix what `modernizer` did not get right.
200
205
206
+
201
207
## 3. Use `precompiler`
202
208
203
209
You can make use of `plone.recipe.precompiler` to identify syntax errors quickly.
@@ -218,28 +224,31 @@ If you want to avoid running the complete buildout every time, you can use the `
218
224
./bin/buildout -c local.cfg install precompiler
219
225
```
220
226
221
-
## 4. Start The Instance
222
227
223
-
As a next step we recommend that you try to start the instance with your add-on.
228
+
## 4. Start the instance
229
+
230
+
As a next step, we recommend that you try to start the instance with your add-on.
224
231
This will fail on all import errors (e.g., relative imports that are not allowed in Python 3).
225
232
If it works then you can try to install the add-on.
226
233
227
234
You need to fix all issues that appear before you can do manual testing to check for big, obvious issues.
228
235
236
+
229
237
### Common Issues during startup
230
238
231
239
The following issues will abort your startup.
232
240
You need to fix them before you are able to test the functionality by hand or run tests.
233
241
234
-
#### A - Class Advice
235
242
236
-
This kind of error message:
243
+
#### Class advice
237
244
238
-
```shell
245
+
If you get an error message similar to the following.
246
+
247
+
```console
239
248
TypeError: Class advice impossible in Python3. Use the @implementer class decorator instead.
240
249
```
241
250
242
-
tells you that there is a class that is using an `implements` statement which needs to be replaced by the `@implementer` decorator.
251
+
This tells you that there is a class using an `implements` statement which needs to be replaced by the `@implementer` decorator.
243
252
244
253
For example, code that is written as follows:
245
254
@@ -259,28 +268,32 @@ from zope.interface import implementer
259
268
classGroup(form.BaseForm):
260
269
```
261
270
262
-
The same is the case for `provides(IFoo)` and some other Class advices.
263
-
These need to be replaced with their respective decorators like `@provider`.
271
+
The same is true for `provides(IFoo)` and some other class advices.
272
+
These need to be replaced with their respective decorators, such as `@provider`.
273
+
274
+
275
+
#### Relative imports
264
276
265
-
#### B - Relative Imports
277
+
Relative imports such as `import permissions` are no longer permitted.
278
+
Instead, use fully qualified import paths, such as `from collective.package import permissions`.
266
279
267
-
Relative imports like `import permissions` are no longer permitted.
268
-
Instead use fully qualified import paths such as `from collective.package import permissions`.
269
280
270
-
#### C - Syntax Error On Importing Async
281
+
#### Syntax error on importing async
282
+
283
+
Starting with Python 3.7, you can no longer have a module called `async` (see https://github.com/celery/celery/issues/4849).
284
+
You need to rename all such files, folders, or packages (such as `zc.async` and `plone.app.async`).
271
285
272
-
In Python 3.7 you can no longer have a module called `async` (see <https://github.com/celery/celery/issues/4849>).
273
-
You need to rename all such files, folders or packages (like `zc.async` and `plone.app.async`).
274
286
275
287
## 5. Test functionality manually
276
288
277
-
Now that the instance is running you should do the following and fix all errors as they appear.
289
+
Now that the instance is running, you should do the following, and fix all errors as they appear.
290
+
291
+
- Install the add-on.
292
+
- Test basic functionality, for example, adding and editing content types and views.
293
+
- Uninstall the add-on.
278
294
279
-
- Install the add-on.
280
-
- Test basic functionality (e.g., adding and editing content-types and views).
281
-
- Uninstall the add-on.
295
+
For this step, it is recommended that you have installed `Products.PDBDebugMode` to help debug and fix issues.
282
296
283
-
For this step it is recommended that you have installed `Products.PDBDebugMode` to help debug and fix issues.
284
297
285
298
## 6. Run Tests
286
299
@@ -292,38 +305,44 @@ Remember that you can run `./bin/test -s collective.package -D` to enter a `pdb`
292
305
293
306
With some luck, there will not be too many issues left with the code at this point.
294
307
295
-
It you are unlucky then you have to fix Doctests.
308
+
If you are unlucky, then you have to fix doctests.
296
309
These should be changed so that Python 3 is the default.
297
310
For example, string types (or text) should be represented as `'foo'`, not `u'foo'`, and bytes types (or data) should be represented as `b'bar'`, not `'bar'`.
298
311
Search for examples of `Py23DocChecker` in Plone's packages to find a pattern which allows updated doctests to pass in Python 2.
299
312
300
-
- To test your code against `buildout.coredev`, start by browsing to [Add-ons \[Jenkins\]](https://jenkins.plone.org/view/Add-ons/).
301
-
- Note there are jobs set up for Plone 4.3, 5.1, and 5.2 on Python 2, and two jobs that run tests for Plone 5.2 on Python 3.6 and Python 3.7.
302
-
- Click the link {guilabel}`log in` on Jenkins website (top right). For the first login, you must authorize Jenkins to have access to your GitHub account to authenticate.
303
-
- Click the link for the job you want to run, for example, {guilabel}`Test add-on against Plone 5.2 on Python3.7`.
304
-
- Choose the link {guilabel}`Build with parameters` in the menu on the left-hand side.
305
-
- Fill the fields {guilabel}`ADDON_URL` and {guilabel}`ADDON_BRANCH` with your repository's URL and the branch name ("python3" if you followed these instructions).
306
-
- Start the build with the {guilabel}`Build` button.
313
+
- To test your code against `buildout.coredev`, start by browsing to [Add-ons \[Jenkins\]](https://jenkins.plone.org/view/Add-ons/).
314
+
- Note there are jobs set up for Plone 4.3, 5.0, 5.1, and 5.2 on Python 2, and three jobs that run tests for Plone 5.2 on Python 3.6, Python 3.7, Python 3.8, and Python 3.9.
315
+
- Click the link {guilabel}`log in` on Jenkins website (top right).
316
+
For the first login, you must authorize Jenkins to have access to your GitHub account to authenticate.
317
+
- Click the link for the job you want to run.
318
+
For example, {guilabel}`Test add-on against Plone 5.2 on Python3.8`.
319
+
- Choose the link {guilabel}`Build with parameters` in the menu on the left-hand side.
320
+
- Fill the fields {guilabel}`ADDON_URL` and {guilabel}`ADDON_BRANCH` with your repository's URL and the branch name ("python3" if you followed these instructions).
321
+
- Start the build with the {guilabel}`Build` button.
322
+
307
323
308
-
## 7. Update Add On Information
324
+
## 7. Update add-on information
309
325
310
-
Add the following three entries of the classifiers list in setup.py:
326
+
Add the following four entries of the classifiers list in `setup.py`.
311
327
312
328
```python
313
329
"Framework :: Plone :: 5.2",
314
330
# ...
315
331
"Programming Language :: Python :: 3.6",
316
332
"Programming Language :: Python :: 3.7",
333
+
"Programming Language :: Python :: 3.8",
317
334
```
318
335
319
336
Make an entry in the `CHANGES.rst` file.
320
337
321
-
## 8. Create A Test Setup That Tests In Python 2 And Python 3
322
338
323
-
You need to update the buildout of the add-on you are migrating to also support Plone 5.2 and Python 3.
324
-
Since the buildout of most add-ons are different we cannot offer advice that works for all add-ons.
339
+
## 8. Create a test setup that tests in Python 2 and Python 3
325
340
326
-
But it is be a good idea to create a empty new package with {py:mod}`bobtemplates.plone` and either copy the code of the add-on in there or the new skeleton-files into the old add-on. The least you can do is look at the files created by {py:mod}`bobtemplates.plone` and copy whatever is appropriate to the add-on you are working on.
341
+
You need to update the `buildout` of the add-on you are migrating to also support Plone 5.2 and Python 3.
342
+
Since the `buildout` of most add-ons are different, we cannot offer advice that works for all add-ons.
343
+
344
+
But it is a good idea to create an empty new package with {py:mod}`bobtemplates.plone`, and either copy the code of the add-on in there or the new skeleton files into the old add-on.
345
+
The least you can do is look at the files created by {py:mod}`bobtemplates.plone`, and copy whatever is appropriate to the add-on you are working on.
Always use the newest version of {py:mod}`bobtemplates.plone`!
334
353
335
-
Add-ons created like this contain a setup that allows testing in Python 2 and Python 3 and various Plone versions locally and on travis-ci using {py:mod}`tox`. Look at the files `tox.ini` and `travis.yml`.
354
+
Add-ons created like this contain a setup that allows testing in Python 2 and Python 3, and various Plone versions locally, and on Travis-CI using {py:mod}`tox`.
355
+
Look at the files `tox.ini` and `travis.yml`.
356
+
336
357
337
358
## 9. Frequent Issues
338
359
360
+
339
361
### Text and Bytes
340
362
341
363
This is by far the biggest issue when porting to Python 3.
@@ -346,7 +368,7 @@ As a rule of thumb, you can assume that in Python 3 everything should be text.
346
368
Only in very rare cases will you need to handle bytes.
347
369
```
348
370
349
-
`python-modernize` will **not** fix all your text/bytes issues.
371
+
`python-modernize` will _not_ fix all your text/bytes issues.
350
372
It only replaces all cases of `unicode` with `six.text_type`.
351
373
You need to make sure that the code you are porting will remain unchanged in Python 2 and (at least in most cases) use text in Python 3.
352
374
@@ -361,8 +383,8 @@ do_something(value)
361
383
362
384
You can use the helper methods `safe_text` and `safe_bytes` (`safe_unicode` and `safe_encode` in Plone 5.1).
363
385
364
-
`python-modernize` also does not touch the import statement `from StringIO import StringIO` even though this works only in Python 2.
365
-
You have to check whether you are dealing with text or binary data and use the appropriate import statement from `six` (<https://six.readthedocs.io/#six.StringIO>).
386
+
`python-modernize` also does not touch the import statement `from StringIO import StringIO`, even though this works only in Python 2.
387
+
You have to check whether you are dealing with text or binary data and use the appropriate import statement from `six` (https://six.readthedocs.io/#six.StringIO).
0 commit comments