Serialization & Deserialization

Burr provides a set of tools to make loading and saving state easy. These are functions that will be used by lifecycle hooks to save and load state.

If you want to implement your own serialization and deserialization, you should implement the serialize & deserialize functions, which act as an interface. serialize uses singledispatch and operates on a Type level. While deserialize registers a string value to a function.

class burr.core.serde.StringDispatch

Class to capture how to deserialize something.

We register a key with a deserializer function. It’s like single dispatch but based on a string key value.

Example usage:

from burr.core import serde

@serde.deserializer.register("pickle")
def deserialize_pickle(value: dict, pickle_kwargs: dict = None, **kwargs) -> cls:
    if pickle_kwargs is None:
        pickle_kwargs = {}
    return pickle.loads(value["value"], **pickle_kwargs)

What this does is register the function deserialize_pickle with the key “pickle”. This should mirror the appropriate serialization function - which is what sets the key value to match the deserializer function against.

Notice that this namespaces its kwargs. This is important because we don’t want to have a collision with other kwargs that might be passed in.

burr.core.serde.deserialize(value: Any, **kwargs) → Any

Main function to deserialize a value.

Looks for a key in the value if it’s a dictionary and calls the appropriate deserializer function.

burr.core.serde.serialize(value, **kwargs) → Any
burr.core.serde.serialize(value: bool, **kwargs) → str | int | float | bool
burr.core.serde.serialize(value: float, **kwargs) → str | int | float | bool
burr.core.serde.serialize(value: int, **kwargs) → str | int | float | bool
burr.core.serde.serialize(value: str, **kwargs) → str | int | float | bool
burr.core.serde.serialize(value: dict, **kwargs) → dict[str, Any]
burr.core.serde.serialize(value: list, **kwargs) → list[Any]
burr.core.serde.serialize(value: Document, **kwargs) → dict
burr.core.serde.serialize(value: BaseMessage, **kwargs) → dict
burr.core.serde.serialize(value: BaseModel, **kwargs) → dict
burr.core.serde.serialize(value: DataFrame, pandas_kwargs: dict, **kwargs) → dict

This is the default implementation for serializing a value.

All other implementations should be registered with the @serialize.register decorator.

Each function should output a dictionary, and include the KEY & value to use for deserialization.

Parameters:

  • value – The value to serialize

  • kwargs – Any additional keyword arguments. Each implementation should namespace their kwargs.

Returns:

A dictionary representation of the value

Each serialize function needs to a mirror deserialize function. To know which deserialize function to use, the serialize function needs to return a dictionary and have burr.core.serde.KEY as one of the keys mapping to the appropriate value. This is used to identify the deserialization function to use.

Out of the box Implementations

The following implementations are available assuming you have the right dependencies in your environment:

LangChain Objects

burr.integrations.serde.langchain.deserialize_lc_document(value: dict, **kwargs) → Document

Deserializes langchain documents.

burr.integrations.serde.langchain.deserialize_lc_document_hack(value: dict, **kwargs) → Document

Deserializes langchain documents that we didn’t know about into a document.

burr.integrations.serde.langchain.deserialize_lc_document_with_state(value: dict, **kwargs) → Document

Deserializes langchain documents with state.

burr.integrations.serde.langchain.deserialize_lc_message(value: dict, **kwargs) → BaseMessage

Deserializes langchain messages.

burr.integrations.serde.langchain.deserialize_lc_message_hack(value: dict, **kwargs) → BaseMessage

Deserializes langchain messages that we didn’t know how to serialize.

burr.integrations.serde.langchain.serialize_lc_docs(value: Document, **kwargs) → dict

Serializes langchain documents.

burr.integrations.serde.langchain.serialize_lc_messages(value: BaseMessage, **kwargs) → dict

Serializes langchain messages.

Pandas Objects

burr.integrations.serde.pandas.deserialize_pandas_df(value: dict, pandas_kwargs: dict, **kwargs) → DataFrame

Custom deserializer for pandas dataframes.

Parameters:

  • value – the dictionary to pull the path from to load the parquet file.

  • pandas_kwargs – other args to pass to the pandas read_parquet function.

  • kwargs

Returns:

pandas dataframe

burr.integrations.serde.pandas.serialize_pandas_df(value: DataFrame, pandas_kwargs: dict, **kwargs) → dict

Custom serde for pandas dataframes.

Saves the dataframe to a parquet file and returns the path to the file. Requires a path key in the pandas_kwargs dictionary.

Parameters:

  • value – the pandas dataframe to serialize.

  • pandas_kwargs – path key is required – this is the base path to save the parquet file. As well as any other kwargs to pass to the pandas to_parquet function.

  • kwargs

Returns:

Pickle-able Objects

burr.integrations.serde.pickle.register_type_to_pickle(cls)

Register a class to be serialized/deserialized using pickle.

Note: pickle_kwargs are passed to the pickle.dumps and pickle.loads functions.

This will register the passed in class to be serialized/deserialized using pickle.

class User:
    def __init__(self, name, email):
        self.name = name
        self.email = email

from burr.integrations.serde import pickle
pickle.register_type_to_pickle(User) # this will register the User class to be serialized/deserialized using pickle.
Parameters:

cls – The class to register

Pydantic Objects

burr.integrations.serde.pydantic.deserialize_pydantic(value: dict, **kwargs) → BaseModel

Deserializes a pydantic object from a dictionary. This will pop the __pydantic_class and then import the class.

burr.integrations.serde.pydantic.serialize_pydantic(value: BaseModel, **kwargs) → dict

Uses pydantic to dump the model to a dictionary and then adds the __pydantic_class to the dictionary.