Skip to content

prefect.utilities.hashing

file_hash(path, hash_algo=_md5)

Given a path to a file, produces a stable hash of the file contents.

Parameters:

Name Type Description Default
path str

the path to a file

required
hash_algo

Hash algorithm from hashlib to use.

_md5

Returns:

Name Type Description
str str

a hash of the file contents

Source code in src/prefect/utilities/hashing.py
37
38
39
40
41
42
43
44
45
46
47
48
def file_hash(path: str, hash_algo=_md5) -> str:
    """Given a path to a file, produces a stable hash of the file contents.

    Args:
        path (str): the path to a file
        hash_algo: Hash algorithm from hashlib to use.

    Returns:
        str: a hash of the file contents
    """
    contents = Path(path).read_bytes()
    return stable_hash(contents, hash_algo=hash_algo)

hash_objects(*args, hash_algo=_md5, raise_on_failure=False, **kwargs)

Attempt to hash objects by dumping to JSON or serializing with cloudpickle. On failure of both, None will be returned; to raise on failure, set raise_on_failure=True.

Source code in src/prefect/utilities/hashing.py
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
def hash_objects(
    *args, hash_algo=_md5, raise_on_failure: bool = False, **kwargs
) -> Optional[str]:
    """
    Attempt to hash objects by dumping to JSON or serializing with cloudpickle.
    On failure of both, `None` will be returned; to raise on failure, set
    `raise_on_failure=True`.
    """
    try:
        serializer = JSONSerializer(dumps_kwargs={"sort_keys": True})
        return stable_hash(serializer.dumps((args, kwargs)), hash_algo=hash_algo)
    except Exception:
        pass

    try:
        return stable_hash(cloudpickle.dumps((args, kwargs)), hash_algo=hash_algo)
    except Exception:
        if raise_on_failure:
            raise

    return None

stable_hash(*args, hash_algo=_md5)

Given some arguments, produces a stable 64-bit hash of their contents.

Supports bytes and strings. Strings will be UTF-8 encoded.

Parameters:

Name Type Description Default
*args Union[str, bytes]

Items to include in the hash.

()
hash_algo

Hash algorithm from hashlib to use.

_md5

Returns:

Type Description
str

A hex hash.

Source code in src/prefect/utilities/hashing.py
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
def stable_hash(*args: Union[str, bytes], hash_algo=_md5) -> str:
    """Given some arguments, produces a stable 64-bit hash of their contents.

    Supports bytes and strings. Strings will be UTF-8 encoded.

    Args:
        *args: Items to include in the hash.
        hash_algo: Hash algorithm from hashlib to use.

    Returns:
        A hex hash.
    """
    h = hash_algo()
    for a in args:
        if isinstance(a, str):
            a = a.encode()
        h.update(a)
    return h.hexdigest()