Data transformation

You can have more control over how is JSON is generated from your data by supplying a transform function. A transform function accepts two positional arguments:

class_name

The full dotted name of a persistent object’s class.

state

The object’s state as a bytes string in JSON format.

The transform function must return a new state string or None. If None is returned, then the original state is used.

A transform function may return an empty string to indicate that a record should be skipped and not written to the newt table.

For example, persistent.mapping.PersistentMapping objects store their data in a data attribute, so their JSON representation is more complex than we might want. Here’s a transform that replaces the JSON representation of a PersistentMapping with its data:

import json

def flatten_persistent_mapping(class_name, state):
    if class_name == 'persistent.mapping.PersistentMapping':
       state = json.loads(state)
       state = state['data']
       return json.dumps(state)

We can supply a transform function to the Python constructor using the transform keyword argument:

import newt.db

conn = newt.db.connection('', transform=flatten_persistent_mapping)

To specify a transform in text configuration, use a transform option to supply the dotted name of your transform function in the newt configuration element:

%import newt.db

<newtdb>
  <zodb>
    <relstorage>
      keep-history false
      <newt>
        transform myproject.flatten_persistent_mapping
        <postgresql>
          dsn dbname=''
        </postgresql>
      </newt>
    </relstorage>
  </zodb>
</newtdb>