|
|
|
@@ -0,0 +1,355 @@
|
|
|
|
|
Metadata-Version: 2.1
|
|
|
|
|
Name: zodbpickle
|
|
|
|
|
Version: 1.0.2
|
|
|
|
|
Summary: Fork of Python 3 pickle module.
|
|
|
|
|
Home-page: https://github.com/zopefoundation/zodbpickle
|
|
|
|
|
Author: Python and Zope Foundation
|
|
|
|
|
Author-email: zodb-dev@zope.org
|
|
|
|
|
License: PSFL 2 and ZPL 2.1
|
|
|
|
|
Keywords: zodb pickle
|
|
|
|
|
Platform: any
|
|
|
|
|
Classifier: Development Status :: 4 - Beta
|
|
|
|
|
Classifier: License :: OSI Approved :: Zope Public License
|
|
|
|
|
Classifier: License :: OSI Approved :: Python Software Foundation License
|
|
|
|
|
Classifier: Programming Language :: Python
|
|
|
|
|
Classifier: Programming Language :: Python :: 2
|
|
|
|
|
Classifier: Programming Language :: Python :: 2.7
|
|
|
|
|
Classifier: Programming Language :: Python :: 3
|
|
|
|
|
Classifier: Programming Language :: Python :: 3.4
|
|
|
|
|
Classifier: Programming Language :: Python :: 3.5
|
|
|
|
|
Classifier: Programming Language :: Python :: 3.6
|
|
|
|
|
Classifier: Programming Language :: Python :: 3.7
|
|
|
|
|
Classifier: Programming Language :: Python :: Implementation :: CPython
|
|
|
|
|
Classifier: Programming Language :: Python :: Implementation :: PyPy
|
|
|
|
|
Classifier: Programming Language :: Python :: Implementation :: Jython
|
|
|
|
|
Classifier: Framework :: ZODB
|
|
|
|
|
Classifier: Topic :: Database
|
|
|
|
|
Classifier: Topic :: Software Development :: Libraries :: Python Modules
|
|
|
|
|
Classifier: Operating System :: Microsoft :: Windows
|
|
|
|
|
Classifier: Operating System :: Unix
|
|
|
|
|
Classifier: Operating System :: MacOS :: MacOS X
|
|
|
|
|
Provides-Extra: test
|
|
|
|
|
Provides-Extra: testing
|
|
|
|
|
Requires-Dist: setuptools
|
|
|
|
|
Provides-Extra: test
|
|
|
|
|
Provides-Extra: testing
|
|
|
|
|
Requires-Dist: nose; extra == 'testing'
|
|
|
|
|
Requires-Dist: coverage; extra == 'testing'
|
|
|
|
|
|
|
|
|
|
``zodbpickle`` README
|
|
|
|
|
=====================
|
|
|
|
|
|
|
|
|
|
.. image:: https://travis-ci.org/zopefoundation/zodbpickle.svg?branch=master
|
|
|
|
|
:target: https://travis-ci.org/zopefoundation/zodbpickle
|
|
|
|
|
|
|
|
|
|
.. image:: https://coveralls.io/repos/github/zopefoundation/zodbpickle/badge.svg
|
|
|
|
|
:target: https://coveralls.io/github/zopefoundation/zodbpickle
|
|
|
|
|
:alt: Coverage status
|
|
|
|
|
|
|
|
|
|
.. image:: https://img.shields.io/pypi/v/zodbpickle.svg
|
|
|
|
|
:target: https://pypi.python.org/pypi/zodbpickle
|
|
|
|
|
:alt: PyPI
|
|
|
|
|
|
|
|
|
|
.. image:: https://img.shields.io/pypi/pyversions/zodbpickle.svg
|
|
|
|
|
:target: https://pypi.python.org/pypi/zodbpickle
|
|
|
|
|
:alt: Python versions
|
|
|
|
|
|
|
|
|
|
This package presents a uniform pickling interface for ZODB:
|
|
|
|
|
|
|
|
|
|
- Under Python2, this package forks both Python 2.7's ``pickle`` and
|
|
|
|
|
``cPickle`` modules, adding support for the ``protocol 3`` opcodes.
|
|
|
|
|
It also provides a new subclass of ``bytes``, ``zodbpickle.binary``,
|
|
|
|
|
which Python2 applications can use to pickle binary values such that
|
|
|
|
|
they will be unpickled as ``bytes`` under Py3k.
|
|
|
|
|
|
|
|
|
|
- Under Py3k, this package forks the ``pickle`` module (and the supporting
|
|
|
|
|
C extension) from both Python 3.2 and Python 3.3. The fork add support
|
|
|
|
|
for the ``noload`` operations used by ZODB.
|
|
|
|
|
|
|
|
|
|
Caution
|
|
|
|
|
-------
|
|
|
|
|
|
|
|
|
|
``zodbpickle`` relies on Python's ``pickle`` module.
|
|
|
|
|
The ``pickle`` module is not intended to be secure against erroneous or
|
|
|
|
|
maliciously constructed data. Never unpickle data received from an
|
|
|
|
|
untrusted or unauthenticated source as arbitrary code might be executed.
|
|
|
|
|
|
|
|
|
|
Also see https://docs.python.org/3.6/library/pickle.html
|
|
|
|
|
|
|
|
|
|
General Usage
|
|
|
|
|
-------------
|
|
|
|
|
|
|
|
|
|
To get compatibility between Python 2 and 3 pickling, replace::
|
|
|
|
|
|
|
|
|
|
import pickle
|
|
|
|
|
|
|
|
|
|
by::
|
|
|
|
|
|
|
|
|
|
from zodbpickle import pickle
|
|
|
|
|
|
|
|
|
|
This provides compatibility, but has the effect that you get the fast implementation
|
|
|
|
|
in Python 3, while Python 2 uses the slow version.
|
|
|
|
|
|
|
|
|
|
To get a more deterministic choice of the implementation, use one of::
|
|
|
|
|
|
|
|
|
|
from zodbpickle import fastpickle # always C
|
|
|
|
|
from zodbpickle import slowpickle # always Python
|
|
|
|
|
|
|
|
|
|
Both modules can co-exist which is helpful for comparison.
|
|
|
|
|
|
|
|
|
|
But there is a bit more to consider, so please read on!
|
|
|
|
|
|
|
|
|
|
Loading/Storing Python 2 Strings
|
|
|
|
|
--------------------------------
|
|
|
|
|
|
|
|
|
|
In all their wisdom, the Python developers have decided that Python 2 ``str``
|
|
|
|
|
instances should be loaded as Python 3 ``str`` objects (i.e. unicode
|
|
|
|
|
strings). Patches were proposed in Python `issue 6784`__ but were never
|
|
|
|
|
applied. This code base contains those patches.
|
|
|
|
|
|
|
|
|
|
.. __: http://bugs.python.org/issue6784
|
|
|
|
|
|
|
|
|
|
Example 1: Loading Python 2 pickles on Python 3 ::
|
|
|
|
|
|
|
|
|
|
$ python2
|
|
|
|
|
>>> import pickle
|
|
|
|
|
>>> pickle.dumps('\xff', protocol=0)
|
|
|
|
|
"S'\\xff'\np0\n."
|
|
|
|
|
>>> pickle.dumps('\xff', protocol=1)
|
|
|
|
|
'U\x01\xffq\x00.'
|
|
|
|
|
>>> pickle.dumps('\xff', protocol=2)
|
|
|
|
|
'\x80\x02U\x01\xffq\x00.'
|
|
|
|
|
|
|
|
|
|
$ python3
|
|
|
|
|
>>> from zodbpickle import pickle
|
|
|
|
|
>>> pickle.loads(b"S'\\xff'\np0\n.", encoding='bytes')
|
|
|
|
|
b'\xff'
|
|
|
|
|
>>> pickle.loads(b'U\x01\xffq\x00.', encoding='bytes')
|
|
|
|
|
b'\xff'
|
|
|
|
|
>>> pickle.loads(b'\x80\x02U\x01\xffq\x00.', encoding='bytes')
|
|
|
|
|
b'\xff'
|
|
|
|
|
|
|
|
|
|
Example 2: Loading Python 3 pickles on Python 2 ::
|
|
|
|
|
|
|
|
|
|
$ python3
|
|
|
|
|
>>> from zodbpickle import pickle
|
|
|
|
|
>>> pickle.dumps(b"\xff", protocol=0)
|
|
|
|
|
b'c_codecs\nencode\np0\n(V\xff\np1\nVlatin1\np2\ntp3\nRp4\n.'
|
|
|
|
|
>>> pickle.dumps(b"\xff", protocol=1)
|
|
|
|
|
b'c_codecs\nencode\nq\x00(X\x02\x00\x00\x00\xc3\xbfq\x01X\x06\x00\x00\x00latin1q\x02tq\x03Rq\x04.'
|
|
|
|
|
>>> pickle.dumps(b"\xff", protocol=2)
|
|
|
|
|
b'\x80\x02c_codecs\nencode\nq\x00X\x02\x00\x00\x00\xc3\xbfq\x01X\x06\x00\x00\x00latin1q\x02\x86q\x03Rq\x04.'
|
|
|
|
|
|
|
|
|
|
$ python2
|
|
|
|
|
>>> import pickle
|
|
|
|
|
>>> pickle.loads('c_codecs\nencode\np0\n(V\xff\np1\nVlatin1\np2\ntp3\nRp4\n.')
|
|
|
|
|
'\xff'
|
|
|
|
|
>>> pickle.loads('c_codecs\nencode\nq\x00(X\x02\x00\x00\x00\xc3\xbfq\x01X\x06\x00\x00\x00latin1q\x02tq\x03Rq\x04.')
|
|
|
|
|
'\xff'
|
|
|
|
|
>>> pickle.loads('\x80\x02c_codecs\nencode\nq\x00X\x02\x00\x00\x00\xc3\xbfq\x01X\x06\x00\x00\x00latin1q\x02\x86q\x03Rq\x04.')
|
|
|
|
|
'\xff'
|
|
|
|
|
|
|
|
|
|
Example 3: everything breaks down ::
|
|
|
|
|
|
|
|
|
|
$ python2
|
|
|
|
|
>>> class Foo(object):
|
|
|
|
|
... def __init__(self):
|
|
|
|
|
... self.x = 'hello'
|
|
|
|
|
...
|
|
|
|
|
>>> import pickle
|
|
|
|
|
>>> pickle.dumps(Foo(), protocol=0)
|
|
|
|
|
"ccopy_reg\n_reconstructor\np0\n(c__main__\nFoo\np1\nc__builtin__\nobject\np2\nNtp3\nRp4\n(dp5\nS'x'\np6\nS'hello'\np7\nsb."
|
|
|
|
|
>>> pickle.dumps(Foo(), protocol=1)
|
|
|
|
|
'ccopy_reg\n_reconstructor\nq\x00(c__main__\nFoo\nq\x01c__builtin__\nobject\nq\x02Ntq\x03Rq\x04}q\x05U\x01xq\x06U\x05helloq\x07sb.'
|
|
|
|
|
>>> pickle.dumps(Foo(), protocol=2)
|
|
|
|
|
'\x80\x02c__main__\nFoo\nq\x00)\x81q\x01}q\x02U\x01xq\x03U\x05helloq\x04sb.'
|
|
|
|
|
|
|
|
|
|
$ python3
|
|
|
|
|
>>> from zodbpickle import pickle
|
|
|
|
|
>>> class Foo(object): pass
|
|
|
|
|
...
|
|
|
|
|
>>> foo = pickle.loads("ccopy_reg\n_reconstructor\np0\n(c__main__\nFoo\np1\nc__builtin__\nobject\np2\nNtp3\nRp4\n(dp5\nS'x'\np6\nS'hello'\np7\nsb.", encoding='bytes')
|
|
|
|
|
>>> foo.x
|
|
|
|
|
Traceback (most recent call last):
|
|
|
|
|
File "<stdin>", line 1, in <module>
|
|
|
|
|
AttributeError: 'Foo' object has no attribute 'x'
|
|
|
|
|
|
|
|
|
|
wait what? ::
|
|
|
|
|
|
|
|
|
|
>>> foo.__dict__
|
|
|
|
|
{b'x': b'hello'}
|
|
|
|
|
|
|
|
|
|
oooh. So we use ``encoding='ASCII'`` (the default) and ``errors='bytes'`` and
|
|
|
|
|
hope it works::
|
|
|
|
|
|
|
|
|
|
>>> foo = pickle.loads("ccopy_reg\n_reconstructor\np0\n(c__main__\nFoo\np1\nc__builtin__\nobject\np2\nNtp3\nRp4\n(dp5\nS'x'\np6\nS'hello'\np7\nsb.", errors='bytes')
|
|
|
|
|
>>> foo.x
|
|
|
|
|
'hello'
|
|
|
|
|
|
|
|
|
|
falling back to bytes if necessary ::
|
|
|
|
|
|
|
|
|
|
>>> pickle.loads(b'\x80\x02U\x01\xffq\x00.', errors='bytes')
|
|
|
|
|
b'\xff'
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Support for ``noload()``
|
|
|
|
|
------------------------
|
|
|
|
|
|
|
|
|
|
The ZODB uses `cPickle`'s ``noload()`` method to retrieve all persistent
|
|
|
|
|
references from a pickle without loading any objects. This feature was removed
|
|
|
|
|
from Python 3's pickle. Unfortuantely, this unnecessarily fills the pickle
|
|
|
|
|
cache.
|
|
|
|
|
|
|
|
|
|
This module provides a ``noload()`` method again.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
===========
|
|
|
|
|
Changelog
|
|
|
|
|
===========
|
|
|
|
|
|
|
|
|
|
1.0.2 (2018-08-10)
|
|
|
|
|
==================
|
|
|
|
|
|
|
|
|
|
- Add support for Python 3.7.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1.0.1 (2018-05-16)
|
|
|
|
|
==================
|
|
|
|
|
|
|
|
|
|
- Fix a memory leak in pickle protocol 3 under Python 2. See `issue 36
|
|
|
|
|
<https://github.com/zopefoundation/zodbpickle/issues/36>`_.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1.0 (2018-02-09)
|
|
|
|
|
================
|
|
|
|
|
|
|
|
|
|
- Add a warning to the readme not to use untrusted pickles.
|
|
|
|
|
|
|
|
|
|
- Drop support for Python 3.3.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
0.7.0 (2017-09-22)
|
|
|
|
|
==================
|
|
|
|
|
|
|
|
|
|
- Drop support for Python 2.6 and 3.2.
|
|
|
|
|
|
|
|
|
|
- Add support for Jython 2.7.
|
|
|
|
|
|
|
|
|
|
- Add support for Python 3.5 and 3.6.
|
|
|
|
|
|
|
|
|
|
0.6.0 (2015-04-02)
|
|
|
|
|
==================
|
|
|
|
|
|
|
|
|
|
- Restore the ``noload`` behaviour from Python 2.6 and provide the
|
|
|
|
|
``noload`` method on the non-C-accelerated unpicklers under PyPy and
|
|
|
|
|
Python 2.
|
|
|
|
|
|
|
|
|
|
- Add support for PyPy, PyPy3, and Python 3.4.
|
|
|
|
|
|
|
|
|
|
0.5.2 (2013-08-17)
|
|
|
|
|
==================
|
|
|
|
|
|
|
|
|
|
- Import accelerator from *our* extension module under Py3k.
|
|
|
|
|
See https://github.com/zopefoundation/zodbpickle/issues/6,
|
|
|
|
|
https://github.com/zopefoundation/zodbpickle/issues/7.
|
|
|
|
|
|
|
|
|
|
- Fix unpickler's ``load_short_binstring`` across supported platforms.
|
|
|
|
|
|
|
|
|
|
0.5.1 (2013-07-06)
|
|
|
|
|
==================
|
|
|
|
|
|
|
|
|
|
- Update all code and tests to Python 2.6.8, 2.7.5, 3.2.5, 3.3.2 .
|
|
|
|
|
|
|
|
|
|
- Add the modules ``zodbpickle.fastpickle`` and ``zodbpickle.slowpickle``.
|
|
|
|
|
This provides a version-independent choice of the C or Python
|
|
|
|
|
implementation.
|
|
|
|
|
|
|
|
|
|
- Fix a minor bug on OS X
|
|
|
|
|
|
|
|
|
|
0.5.0 (2013-06-14)
|
|
|
|
|
==================
|
|
|
|
|
|
|
|
|
|
- Removed support for the ``bytes_as_strings`` arguments to pickling APIs:
|
|
|
|
|
the pickles created when that argument was true might not be unpickled
|
|
|
|
|
without passing ``encoding='bytes'``, which ZODB couldn't reliably enforce.
|
|
|
|
|
On Py3k, ZODB will be using ``protocol=3`` pickles anyway.
|
|
|
|
|
|
|
|
|
|
0.4.4 (2013-06-07)
|
|
|
|
|
==================
|
|
|
|
|
|
|
|
|
|
- Add protocol 3 opcodes to the C version of the ``noload()`` dispatcher.
|
|
|
|
|
|
|
|
|
|
0.4.3 (2013-06-07)
|
|
|
|
|
==================
|
|
|
|
|
|
|
|
|
|
- Packaging error: remove spurious ``-ASIDE`` file from sdist.
|
|
|
|
|
|
|
|
|
|
0.4.2 (2013-06-07)
|
|
|
|
|
==================
|
|
|
|
|
|
|
|
|
|
- Fix NameError in pure-Python version of ``Unpickler.noload_appends``.
|
|
|
|
|
|
|
|
|
|
- Fix NameError in pure-Python version of ``Unpickler.noload_setitems``.
|
|
|
|
|
|
|
|
|
|
0.4.1 (2013-04-29)
|
|
|
|
|
==================
|
|
|
|
|
|
|
|
|
|
- Fix typo in Python2 version of ``zodbpickle.pickle`` module.
|
|
|
|
|
|
|
|
|
|
0.4 (2013-04-28)
|
|
|
|
|
================
|
|
|
|
|
|
|
|
|
|
- Support the common pickle module interface for Python 2.6, 2.7, 3.2, and 3.3.
|
|
|
|
|
|
|
|
|
|
- Split the Python implementations / tests into Python2- and Py3k-specific
|
|
|
|
|
variants.
|
|
|
|
|
|
|
|
|
|
- Added a fork of the Python 2.7 ``_pickle.c``, for use under Python2.
|
|
|
|
|
The fork adds support for the Py3k ``protocol 3`` opcodes.
|
|
|
|
|
|
|
|
|
|
- Added a custom ``binary`` type for use in Python2 apps.
|
|
|
|
|
Derived from ``bytes``, the ``binary`` type allows Python2 apps to pickle
|
|
|
|
|
binary data using opcodes which will cause it to be unpickled as ``bytes``
|
|
|
|
|
on Py3k. Under Py3k, the ``binary`` type is just an alias for ``bytes``.
|
|
|
|
|
|
|
|
|
|
0.3 (2013-03-18)
|
|
|
|
|
================
|
|
|
|
|
|
|
|
|
|
- Added ``noload`` code to Python 3.2 version of ``Unpickler``. As with
|
|
|
|
|
the Python 3.3 version, this code remains untested.
|
|
|
|
|
|
|
|
|
|
- Added ``bytes_as_strings`` option to the Python 3.2 version of
|
|
|
|
|
``Pickler``, ``dump``, and ``dumps``.
|
|
|
|
|
|
|
|
|
|
0.2 (2013-03-05)
|
|
|
|
|
================
|
|
|
|
|
|
|
|
|
|
- Added ``bytes_as_strings`` option to ``Pickler``, ``dump``, and ``dumps``.
|
|
|
|
|
|
|
|
|
|
- Incomplete support for Python 3.2:
|
|
|
|
|
|
|
|
|
|
- Move ``_pickle.c`` -> ``_pickle_33.c``.
|
|
|
|
|
|
|
|
|
|
- Clone Python 3.2.3's ``_pickle.c`` -> ``_pickle_32.c`` and apply the
|
|
|
|
|
same patch.
|
|
|
|
|
|
|
|
|
|
- Choose between them at build time based on ``sys.version_info``.
|
|
|
|
|
|
|
|
|
|
- Disable some tests of 3.3-only features.
|
|
|
|
|
|
|
|
|
|
- Missing: implementation of ``noload()`` in ``_pickle_32.c``.
|
|
|
|
|
|
|
|
|
|
- Missing: implementation of ``bytes_as_strings=True`` in ``_pickle_32.c``.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
0.1.0 (2013-02-27)
|
|
|
|
|
==================
|
|
|
|
|
|
|
|
|
|
- Initial release of Python 3.3's pickle with the patches of Python
|
|
|
|
|
`issue 6784`__ applied.
|
|
|
|
|
|
|
|
|
|
.. __: http://bugs.python.org/issue6784#msg156166
|
|
|
|
|
|
|
|
|
|
- Added support for ``errors="bytes"``.
|
|
|
|
|
|
|
|
|
|
|
Esther Kleinhenz