Skip to content

prefect.blocks.abstract

CredentialsBlock

Bases: Block, ABC

Stores credentials for an external system and exposes a client for interacting with that system. Can also hold config that is tightly coupled to credentials (domain, endpoint, account ID, etc.) Will often be composed with other blocks. Parent block should rely on the client provided by a credentials block for interacting with the corresponding external system.

Source code in src/prefect/blocks/abstract.py
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
class CredentialsBlock(Block, ABC):
    """
    Stores credentials for an external system and exposes a client for interacting
    with that system. Can also hold config that is tightly coupled to credentials
    (domain, endpoint, account ID, etc.) Will often be composed with other blocks.
    Parent block should rely on the client provided by a credentials block for
    interacting with the corresponding external system.
    """

    @property
    def logger(self) -> Logger:
        """
        Returns a logger based on whether the CredentialsBlock
        is called from within a flow or task run context.
        If a run context is present, the logger property returns a run logger.
        Else, it returns a default logger labeled with the class's name.

        Returns:
            The run logger or a default logger with the class's name.
        """
        try:
            return get_run_logger()
        except MissingContextError:
            return get_logger(self.__class__.__name__)

    @abstractmethod
    def get_client(self, *args, **kwargs):
        """
        Returns a client for interacting with the external system.

        If a service offers various clients, this method can accept
        a `client_type` keyword argument to get the desired client
        within the service.
        """

logger: Logger property

Returns a logger based on whether the CredentialsBlock is called from within a flow or task run context. If a run context is present, the logger property returns a run logger. Else, it returns a default logger labeled with the class's name.

Returns:

Type Description
Logger

The run logger or a default logger with the class's name.

get_client(*args, **kwargs) abstractmethod

Returns a client for interacting with the external system.

If a service offers various clients, this method can accept a client_type keyword argument to get the desired client within the service.

Source code in src/prefect/blocks/abstract.py
52
53
54
55
56
57
58
59
60
@abstractmethod
def get_client(self, *args, **kwargs):
    """
    Returns a client for interacting with the external system.

    If a service offers various clients, this method can accept
    a `client_type` keyword argument to get the desired client
    within the service.
    """

DatabaseBlock

Bases: Block, ABC

An abstract block type that represents a database and provides an interface for interacting with it.

Blocks that implement this interface have the option to accept credentials directly via attributes or via a nested CredentialsBlock.

Use of a nested credentials block is recommended unless credentials are tightly coupled to database connection configuration.

Implementing either sync or async context management on DatabaseBlock implementations is recommended.

Source code in src/prefect/blocks/abstract.py
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
class DatabaseBlock(Block, ABC):
    """
    An abstract block type that represents a database and
    provides an interface for interacting with it.

    Blocks that implement this interface have the option to accept
    credentials directly via attributes or via a nested `CredentialsBlock`.

    Use of a nested credentials block is recommended unless credentials
    are tightly coupled to database connection configuration.

    Implementing either sync or async context management on `DatabaseBlock`
    implementations is recommended.
    """

    @property
    def logger(self) -> Logger:
        """
        Returns a logger based on whether the DatabaseBlock
        is called from within a flow or task run context.
        If a run context is present, the logger property returns a run logger.
        Else, it returns a default logger labeled with the class's name.

        Returns:
            The run logger or a default logger with the class's name.
        """
        try:
            return get_run_logger()
        except MissingContextError:
            return get_logger(self.__class__.__name__)

    @abstractmethod
    async def fetch_one(
        self, operation, parameters=None, **execution_kwargs
    ) -> Tuple[Any]:
        """
        Fetch a single result from the database.

        Args:
            operation: The SQL query or other operation to be executed.
            parameters: The parameters for the operation.
            **execution_kwargs: Additional keyword arguments to pass to execute.

        Returns:
            A list of tuples containing the data returned by the database,
                where each row is a tuple and each column is a value in the tuple.
        """

    @abstractmethod
    async def fetch_many(
        self, operation, parameters=None, size=None, **execution_kwargs
    ) -> List[Tuple[Any]]:
        """
        Fetch a limited number of results from the database.

        Args:
            operation: The SQL query or other operation to be executed.
            parameters: The parameters for the operation.
            size: The number of results to return.
            **execution_kwargs: Additional keyword arguments to pass to execute.

        Returns:
            A list of tuples containing the data returned by the database,
                where each row is a tuple and each column is a value in the tuple.
        """

    @abstractmethod
    async def fetch_all(
        self, operation, parameters=None, **execution_kwargs
    ) -> List[Tuple[Any]]:
        """
        Fetch all results from the database.

        Args:
            operation: The SQL query or other operation to be executed.
            parameters: The parameters for the operation.
            **execution_kwargs: Additional keyword arguments to pass to execute.

        Returns:
            A list of tuples containing the data returned by the database,
                where each row is a tuple and each column is a value in the tuple.
        """

    @abstractmethod
    async def execute(self, operation, parameters=None, **execution_kwargs) -> None:
        """
        Executes an operation on the database. This method is intended to be used
        for operations that do not return data, such as INSERT, UPDATE, or DELETE.

        Args:
            operation: The SQL query or other operation to be executed.
            parameters: The parameters for the operation.
            **execution_kwargs: Additional keyword arguments to pass to execute.
        """

    @abstractmethod
    async def execute_many(
        self, operation, seq_of_parameters, **execution_kwargs
    ) -> None:
        """
        Executes multiple operations on the database. This method is intended to be used
        for operations that do not return data, such as INSERT, UPDATE, or DELETE.

        Args:
            operation: The SQL query or other operation to be executed.
            seq_of_parameters: The sequence of parameters for the operation.
            **execution_kwargs: Additional keyword arguments to pass to execute.
        """

    # context management methods are not abstract methods because
    # they are not supported by all database drivers
    async def __aenter__(self) -> Self:
        """
        Context management method for async databases.
        """
        raise NotImplementedError(
            f"{self.__class__.__name__} does not support async context management."
        )

    async def __aexit__(self, *args) -> None:
        """
        Context management method for async databases.
        """
        raise NotImplementedError(
            f"{self.__class__.__name__} does not support async context management."
        )

    def __enter__(self) -> Self:
        """
        Context management method for databases.
        """
        raise NotImplementedError(
            f"{self.__class__.__name__} does not support context management."
        )

    def __exit__(self, *args) -> None:
        """
        Context management method for databases.
        """
        raise NotImplementedError(
            f"{self.__class__.__name__} does not support context management."
        )

logger: Logger property

Returns a logger based on whether the DatabaseBlock is called from within a flow or task run context. If a run context is present, the logger property returns a run logger. Else, it returns a default logger labeled with the class's name.

Returns:

Type Description
Logger

The run logger or a default logger with the class's name.

__aenter__() async

Context management method for async databases.

Source code in src/prefect/blocks/abstract.py
300
301
302
303
304
305
306
async def __aenter__(self) -> Self:
    """
    Context management method for async databases.
    """
    raise NotImplementedError(
        f"{self.__class__.__name__} does not support async context management."
    )

__aexit__(*args) async

Context management method for async databases.

Source code in src/prefect/blocks/abstract.py
308
309
310
311
312
313
314
async def __aexit__(self, *args) -> None:
    """
    Context management method for async databases.
    """
    raise NotImplementedError(
        f"{self.__class__.__name__} does not support async context management."
    )

__enter__()

Context management method for databases.

Source code in src/prefect/blocks/abstract.py
316
317
318
319
320
321
322
def __enter__(self) -> Self:
    """
    Context management method for databases.
    """
    raise NotImplementedError(
        f"{self.__class__.__name__} does not support context management."
    )

__exit__(*args)

Context management method for databases.

Source code in src/prefect/blocks/abstract.py
324
325
326
327
328
329
330
def __exit__(self, *args) -> None:
    """
    Context management method for databases.
    """
    raise NotImplementedError(
        f"{self.__class__.__name__} does not support context management."
    )

execute(operation, parameters=None, **execution_kwargs) abstractmethod async

Executes an operation on the database. This method is intended to be used for operations that do not return data, such as INSERT, UPDATE, or DELETE.

Parameters:

Name Type Description Default
operation

The SQL query or other operation to be executed.

required
parameters

The parameters for the operation.

None
**execution_kwargs

Additional keyword arguments to pass to execute.

{}
Source code in src/prefect/blocks/abstract.py
272
273
274
275
276
277
278
279
280
281
282
@abstractmethod
async def execute(self, operation, parameters=None, **execution_kwargs) -> None:
    """
    Executes an operation on the database. This method is intended to be used
    for operations that do not return data, such as INSERT, UPDATE, or DELETE.

    Args:
        operation: The SQL query or other operation to be executed.
        parameters: The parameters for the operation.
        **execution_kwargs: Additional keyword arguments to pass to execute.
    """

execute_many(operation, seq_of_parameters, **execution_kwargs) abstractmethod async

Executes multiple operations on the database. This method is intended to be used for operations that do not return data, such as INSERT, UPDATE, or DELETE.

Parameters:

Name Type Description Default
operation

The SQL query or other operation to be executed.

required
seq_of_parameters

The sequence of parameters for the operation.

required
**execution_kwargs

Additional keyword arguments to pass to execute.

{}
Source code in src/prefect/blocks/abstract.py
284
285
286
287
288
289
290
291
292
293
294
295
296
@abstractmethod
async def execute_many(
    self, operation, seq_of_parameters, **execution_kwargs
) -> None:
    """
    Executes multiple operations on the database. This method is intended to be used
    for operations that do not return data, such as INSERT, UPDATE, or DELETE.

    Args:
        operation: The SQL query or other operation to be executed.
        seq_of_parameters: The sequence of parameters for the operation.
        **execution_kwargs: Additional keyword arguments to pass to execute.
    """

fetch_all(operation, parameters=None, **execution_kwargs) abstractmethod async

Fetch all results from the database.

Parameters:

Name Type Description Default
operation

The SQL query or other operation to be executed.

required
parameters

The parameters for the operation.

None
**execution_kwargs

Additional keyword arguments to pass to execute.

{}

Returns:

Type Description
List[Tuple[Any]]

A list of tuples containing the data returned by the database, where each row is a tuple and each column is a value in the tuple.

Source code in src/prefect/blocks/abstract.py
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
@abstractmethod
async def fetch_all(
    self, operation, parameters=None, **execution_kwargs
) -> List[Tuple[Any]]:
    """
    Fetch all results from the database.

    Args:
        operation: The SQL query or other operation to be executed.
        parameters: The parameters for the operation.
        **execution_kwargs: Additional keyword arguments to pass to execute.

    Returns:
        A list of tuples containing the data returned by the database,
            where each row is a tuple and each column is a value in the tuple.
    """

fetch_many(operation, parameters=None, size=None, **execution_kwargs) abstractmethod async

Fetch a limited number of results from the database.

Parameters:

Name Type Description Default
operation

The SQL query or other operation to be executed.

required
parameters

The parameters for the operation.

None
size

The number of results to return.

None
**execution_kwargs

Additional keyword arguments to pass to execute.

{}

Returns:

Type Description
List[Tuple[Any]]

A list of tuples containing the data returned by the database, where each row is a tuple and each column is a value in the tuple.

Source code in src/prefect/blocks/abstract.py
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
@abstractmethod
async def fetch_many(
    self, operation, parameters=None, size=None, **execution_kwargs
) -> List[Tuple[Any]]:
    """
    Fetch a limited number of results from the database.

    Args:
        operation: The SQL query or other operation to be executed.
        parameters: The parameters for the operation.
        size: The number of results to return.
        **execution_kwargs: Additional keyword arguments to pass to execute.

    Returns:
        A list of tuples containing the data returned by the database,
            where each row is a tuple and each column is a value in the tuple.
    """

fetch_one(operation, parameters=None, **execution_kwargs) abstractmethod async

Fetch a single result from the database.

Parameters:

Name Type Description Default
operation

The SQL query or other operation to be executed.

required
parameters

The parameters for the operation.

None
**execution_kwargs

Additional keyword arguments to pass to execute.

{}

Returns:

Type Description
Tuple[Any]

A list of tuples containing the data returned by the database, where each row is a tuple and each column is a value in the tuple.

Source code in src/prefect/blocks/abstract.py
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
@abstractmethod
async def fetch_one(
    self, operation, parameters=None, **execution_kwargs
) -> Tuple[Any]:
    """
    Fetch a single result from the database.

    Args:
        operation: The SQL query or other operation to be executed.
        parameters: The parameters for the operation.
        **execution_kwargs: Additional keyword arguments to pass to execute.

    Returns:
        A list of tuples containing the data returned by the database,
            where each row is a tuple and each column is a value in the tuple.
    """

JobBlock

Bases: Block, ABC

Block that represents an entity in an external service that can trigger a long running execution.

Source code in src/prefect/blocks/abstract.py
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
class JobBlock(Block, ABC):
    """
    Block that represents an entity in an external service
    that can trigger a long running execution.
    """

    @property
    def logger(self) -> Logger:
        """
        Returns a logger based on whether the JobBlock
        is called from within a flow or task run context.
        If a run context is present, the logger property returns a run logger.
        Else, it returns a default logger labeled with the class's name.

        Returns:
            The run logger or a default logger with the class's name.
        """
        try:
            return get_run_logger()
        except MissingContextError:
            return get_logger(self.__class__.__name__)

    @abstractmethod
    async def trigger(self) -> JobRun:
        """
        Triggers a job run in an external service and returns a JobRun object
        to track the execution of the run.
        """

logger: Logger property

Returns a logger based on whether the JobBlock is called from within a flow or task run context. If a run context is present, the logger property returns a run logger. Else, it returns a default logger labeled with the class's name.

Returns:

Type Description
Logger

The run logger or a default logger with the class's name.

trigger() abstractmethod async

Triggers a job run in an external service and returns a JobRun object to track the execution of the run.

Source code in src/prefect/blocks/abstract.py
176
177
178
179
180
181
@abstractmethod
async def trigger(self) -> JobRun:
    """
    Triggers a job run in an external service and returns a JobRun object
    to track the execution of the run.
    """

JobRun

Bases: ABC, Generic[T]

Represents a job run in an external system. Allows waiting for the job run's completion and fetching its results.

Source code in src/prefect/blocks/abstract.py
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
class JobRun(ABC, Generic[T]):  # not a block
    """
    Represents a job run in an external system. Allows waiting
    for the job run's completion and fetching its results.
    """

    @property
    def logger(self) -> Logger:
        """
        Returns a logger based on whether the JobRun
        is called from within a flow or task run context.
        If a run context is present, the logger property returns a run logger.
        Else, it returns a default logger labeled with the class's name.

        Returns:
            The run logger or a default logger with the class's name.
        """
        try:
            return get_run_logger()
        except MissingContextError:
            return get_logger(self.__class__.__name__)

    @abstractmethod
    async def wait_for_completion(self) -> Logger:
        """
        Wait for the job run to complete.
        """

    @abstractmethod
    async def fetch_result(self) -> T:
        """
        Retrieve the results of the job run and return them.
        """

logger: Logger property

Returns a logger based on whether the JobRun is called from within a flow or task run context. If a run context is present, the logger property returns a run logger. Else, it returns a default logger labeled with the class's name.

Returns:

Type Description
Logger

The run logger or a default logger with the class's name.

fetch_result() abstractmethod async

Retrieve the results of the job run and return them.

Source code in src/prefect/blocks/abstract.py
147
148
149
150
151
@abstractmethod
async def fetch_result(self) -> T:
    """
    Retrieve the results of the job run and return them.
    """

wait_for_completion() abstractmethod async

Wait for the job run to complete.

Source code in src/prefect/blocks/abstract.py
141
142
143
144
145
@abstractmethod
async def wait_for_completion(self) -> Logger:
    """
    Wait for the job run to complete.
    """

NotificationBlock

Bases: Block, ABC

Block that represents a resource in an external system that is able to send notifications.

Source code in src/prefect/blocks/abstract.py
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
class NotificationBlock(Block, ABC):
    """
    Block that represents a resource in an external system that is able to send notifications.
    """

    _block_schema_capabilities = ["notify"]
    _events_excluded_methods = Block._events_excluded_methods.default + ["notify"]

    @property
    def logger(self) -> Logger:
        """
        Returns a logger based on whether the NotificationBlock
        is called from within a flow or task run context.
        If a run context is present, the logger property returns a run logger.
        Else, it returns a default logger labeled with the class's name.

        Returns:
            The run logger or a default logger with the class's name.
        """
        try:
            return get_run_logger()
        except MissingContextError:
            return get_logger(self.__class__.__name__)

    @abstractmethod
    async def notify(self, body: str, subject: Optional[str] = None) -> None:
        """
        Send a notification.

        Args:
            body: The body of the notification.
            subject: The subject of the notification.
        """

    _raise_on_failure: bool = False

    @contextmanager
    def raise_on_failure(self) -> Generator[None, None, None]:
        """
        Context manager that, while active, causes the block to raise errors if it
        encounters a failure sending notifications.
        """
        self._raise_on_failure = True
        try:
            yield
        finally:
            self._raise_on_failure = False

logger: Logger property

Returns a logger based on whether the NotificationBlock is called from within a flow or task run context. If a run context is present, the logger property returns a run logger. Else, it returns a default logger labeled with the class's name.

Returns:

Type Description
Logger

The run logger or a default logger with the class's name.

notify(body, subject=None) abstractmethod async

Send a notification.

Parameters:

Name Type Description Default
body str

The body of the notification.

required
subject Optional[str]

The subject of the notification.

None
Source code in src/prefect/blocks/abstract.py
 94
 95
 96
 97
 98
 99
100
101
102
@abstractmethod
async def notify(self, body: str, subject: Optional[str] = None) -> None:
    """
    Send a notification.

    Args:
        body: The body of the notification.
        subject: The subject of the notification.
    """

raise_on_failure()

Context manager that, while active, causes the block to raise errors if it encounters a failure sending notifications.

Source code in src/prefect/blocks/abstract.py
106
107
108
109
110
111
112
113
114
115
116
@contextmanager
def raise_on_failure(self) -> Generator[None, None, None]:
    """
    Context manager that, while active, causes the block to raise errors if it
    encounters a failure sending notifications.
    """
    self._raise_on_failure = True
    try:
        yield
    finally:
        self._raise_on_failure = False

NotificationError

Bases: Exception

Raised if a notification block fails to send a notification.

Source code in src/prefect/blocks/abstract.py
63
64
65
66
67
class NotificationError(Exception):
    """Raised if a notification block fails to send a notification."""

    def __init__(self, log: str) -> None:
        self.log = log

ObjectStorageBlock

Bases: Block, ABC

Block that represents a resource in an external service that can store objects.

Source code in src/prefect/blocks/abstract.py
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
class ObjectStorageBlock(Block, ABC):
    """
    Block that represents a resource in an external service that can store
    objects.
    """

    @property
    def logger(self) -> Logger:
        """
        Returns a logger based on whether the ObjectStorageBlock
        is called from within a flow or task run context.
        If a run context is present, the logger property returns a run logger.
        Else, it returns a default logger labeled with the class's name.

        Returns:
            The run logger or a default logger with the class's name.
        """
        try:
            return get_run_logger()
        except MissingContextError:
            return get_logger(self.__class__.__name__)

    @abstractmethod
    async def download_object_to_path(
        self,
        from_path: str,
        to_path: Union[str, Path],
        **download_kwargs: Dict[str, Any],
    ) -> Path:
        """
        Downloads an object from the object storage service to a path.

        Args:
            from_path: The path to download from.
            to_path: The path to download to.
            **download_kwargs: Additional keyword arguments to pass to download.

        Returns:
            The path that the object was downloaded to.
        """

    @abstractmethod
    async def download_object_to_file_object(
        self,
        from_path: str,
        to_file_object: BinaryIO,
        **download_kwargs: Dict[str, Any],
    ) -> BinaryIO:
        """
        Downloads an object from the object storage service to a file-like object,
        which can be a BytesIO object or a BufferedWriter.

        Args:
            from_path: The path to download from.
            to_file_object: The file-like object to download to.
            **download_kwargs: Additional keyword arguments to pass to download.

        Returns:
            The file-like object that the object was downloaded to.
        """

    @abstractmethod
    async def download_folder_to_path(
        self,
        from_folder: str,
        to_folder: Union[str, Path],
        **download_kwargs: Dict[str, Any],
    ) -> Path:
        """
        Downloads a folder from the object storage service to a path.

        Args:
            from_folder: The path to the folder to download from.
            to_folder: The path to download the folder to.
            **download_kwargs: Additional keyword arguments to pass to download.

        Returns:
            The path that the folder was downloaded to.
        """

    @abstractmethod
    async def upload_from_path(
        self, from_path: Union[str, Path], to_path: str, **upload_kwargs: Dict[str, Any]
    ) -> str:
        """
        Uploads an object from a path to the object storage service.

        Args:
            from_path: The path to the file to upload from.
            to_path: The path to upload the file to.
            **upload_kwargs: Additional keyword arguments to pass to upload.

        Returns:
            The path that the object was uploaded to.
        """

    @abstractmethod
    async def upload_from_file_object(
        self, from_file_object: BinaryIO, to_path: str, **upload_kwargs: Dict[str, Any]
    ) -> str:
        """
        Uploads an object to the object storage service from a file-like object,
        which can be a BytesIO object or a BufferedReader.

        Args:
            from_file_object: The file-like object to upload from.
            to_path: The path to upload the object to.
            **upload_kwargs: Additional keyword arguments to pass to upload.

        Returns:
            The path that the object was uploaded to.
        """

    @abstractmethod
    async def upload_from_folder(
        self,
        from_folder: Union[str, Path],
        to_folder: str,
        **upload_kwargs: Dict[str, Any],
    ) -> str:
        """
        Uploads a folder to the object storage service from a path.

        Args:
            from_folder: The path to the folder to upload from.
            to_folder: The path to upload the folder to.
            **upload_kwargs: Additional keyword arguments to pass to upload.

        Returns:
            The path that the folder was uploaded to.
        """

logger: Logger property

Returns a logger based on whether the ObjectStorageBlock is called from within a flow or task run context. If a run context is present, the logger property returns a run logger. Else, it returns a default logger labeled with the class's name.

Returns:

Type Description
Logger

The run logger or a default logger with the class's name.

download_folder_to_path(from_folder, to_folder, **download_kwargs) abstractmethod async

Downloads a folder from the object storage service to a path.

Parameters:

Name Type Description Default
from_folder str

The path to the folder to download from.

required
to_folder Union[str, Path]

The path to download the folder to.

required
**download_kwargs Dict[str, Any]

Additional keyword arguments to pass to download.

{}

Returns:

Type Description
Path

The path that the folder was downloaded to.

Source code in src/prefect/blocks/abstract.py
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
@abstractmethod
async def download_folder_to_path(
    self,
    from_folder: str,
    to_folder: Union[str, Path],
    **download_kwargs: Dict[str, Any],
) -> Path:
    """
    Downloads a folder from the object storage service to a path.

    Args:
        from_folder: The path to the folder to download from.
        to_folder: The path to download the folder to.
        **download_kwargs: Additional keyword arguments to pass to download.

    Returns:
        The path that the folder was downloaded to.
    """

download_object_to_file_object(from_path, to_file_object, **download_kwargs) abstractmethod async

Downloads an object from the object storage service to a file-like object, which can be a BytesIO object or a BufferedWriter.

Parameters:

Name Type Description Default
from_path str

The path to download from.

required
to_file_object BinaryIO

The file-like object to download to.

required
**download_kwargs Dict[str, Any]

Additional keyword arguments to pass to download.

{}

Returns:

Type Description
BinaryIO

The file-like object that the object was downloaded to.

Source code in src/prefect/blocks/abstract.py
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
@abstractmethod
async def download_object_to_file_object(
    self,
    from_path: str,
    to_file_object: BinaryIO,
    **download_kwargs: Dict[str, Any],
) -> BinaryIO:
    """
    Downloads an object from the object storage service to a file-like object,
    which can be a BytesIO object or a BufferedWriter.

    Args:
        from_path: The path to download from.
        to_file_object: The file-like object to download to.
        **download_kwargs: Additional keyword arguments to pass to download.

    Returns:
        The file-like object that the object was downloaded to.
    """

download_object_to_path(from_path, to_path, **download_kwargs) abstractmethod async

Downloads an object from the object storage service to a path.

Parameters:

Name Type Description Default
from_path str

The path to download from.

required
to_path Union[str, Path]

The path to download to.

required
**download_kwargs Dict[str, Any]

Additional keyword arguments to pass to download.

{}

Returns:

Type Description
Path

The path that the object was downloaded to.

Source code in src/prefect/blocks/abstract.py
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
@abstractmethod
async def download_object_to_path(
    self,
    from_path: str,
    to_path: Union[str, Path],
    **download_kwargs: Dict[str, Any],
) -> Path:
    """
    Downloads an object from the object storage service to a path.

    Args:
        from_path: The path to download from.
        to_path: The path to download to.
        **download_kwargs: Additional keyword arguments to pass to download.

    Returns:
        The path that the object was downloaded to.
    """

upload_from_file_object(from_file_object, to_path, **upload_kwargs) abstractmethod async

Uploads an object to the object storage service from a file-like object, which can be a BytesIO object or a BufferedReader.

Parameters:

Name Type Description Default
from_file_object BinaryIO

The file-like object to upload from.

required
to_path str

The path to upload the object to.

required
**upload_kwargs Dict[str, Any]

Additional keyword arguments to pass to upload.

{}

Returns:

Type Description
str

The path that the object was uploaded to.

Source code in src/prefect/blocks/abstract.py
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
@abstractmethod
async def upload_from_file_object(
    self, from_file_object: BinaryIO, to_path: str, **upload_kwargs: Dict[str, Any]
) -> str:
    """
    Uploads an object to the object storage service from a file-like object,
    which can be a BytesIO object or a BufferedReader.

    Args:
        from_file_object: The file-like object to upload from.
        to_path: The path to upload the object to.
        **upload_kwargs: Additional keyword arguments to pass to upload.

    Returns:
        The path that the object was uploaded to.
    """

upload_from_folder(from_folder, to_folder, **upload_kwargs) abstractmethod async

Uploads a folder to the object storage service from a path.

Parameters:

Name Type Description Default
from_folder Union[str, Path]

The path to the folder to upload from.

required
to_folder str

The path to upload the folder to.

required
**upload_kwargs Dict[str, Any]

Additional keyword arguments to pass to upload.

{}

Returns:

Type Description
str

The path that the folder was uploaded to.

Source code in src/prefect/blocks/abstract.py
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
@abstractmethod
async def upload_from_folder(
    self,
    from_folder: Union[str, Path],
    to_folder: str,
    **upload_kwargs: Dict[str, Any],
) -> str:
    """
    Uploads a folder to the object storage service from a path.

    Args:
        from_folder: The path to the folder to upload from.
        to_folder: The path to upload the folder to.
        **upload_kwargs: Additional keyword arguments to pass to upload.

    Returns:
        The path that the folder was uploaded to.
    """

upload_from_path(from_path, to_path, **upload_kwargs) abstractmethod async

Uploads an object from a path to the object storage service.

Parameters:

Name Type Description Default
from_path Union[str, Path]

The path to the file to upload from.

required
to_path str

The path to upload the file to.

required
**upload_kwargs Dict[str, Any]

Additional keyword arguments to pass to upload.

{}

Returns:

Type Description
str

The path that the object was uploaded to.

Source code in src/prefect/blocks/abstract.py
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
@abstractmethod
async def upload_from_path(
    self, from_path: Union[str, Path], to_path: str, **upload_kwargs: Dict[str, Any]
) -> str:
    """
    Uploads an object from a path to the object storage service.

    Args:
        from_path: The path to the file to upload from.
        to_path: The path to upload the file to.
        **upload_kwargs: Additional keyword arguments to pass to upload.

    Returns:
        The path that the object was uploaded to.
    """

SecretBlock

Bases: Block, ABC

Block that represents a resource that can store and retrieve secrets.

Source code in src/prefect/blocks/abstract.py
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
class SecretBlock(Block, ABC):
    """
    Block that represents a resource that can store and retrieve secrets.
    """

    @property
    def logger(self) -> Logger:
        """
        Returns a logger based on whether the SecretBlock
        is called from within a flow or task run context.
        If a run context is present, the logger property returns a run logger.
        Else, it returns a default logger labeled with the class's name.

        Returns:
            The run logger or a default logger with the class's name.
        """
        try:
            return get_run_logger()
        except MissingContextError:
            return get_logger(self.__class__.__name__)

    @abstractmethod
    async def read_secret(self) -> bytes:
        """
        Reads the configured secret from the secret storage service.

        Returns:
            The secret data.
        """

    @abstractmethod
    async def write_secret(self, secret_data) -> str:
        """
        Writes secret data to the configured secret in the secret storage service.

        Args:
            secret_data: The secret data to write.

        Returns:
            The key of the secret that can be used for retrieval.
        """

logger: Logger property

Returns a logger based on whether the SecretBlock is called from within a flow or task run context. If a run context is present, the logger property returns a run logger. Else, it returns a default logger labeled with the class's name.

Returns:

Type Description
Logger

The run logger or a default logger with the class's name.

read_secret() abstractmethod async

Reads the configured secret from the secret storage service.

Returns:

Type Description
bytes

The secret data.

Source code in src/prefect/blocks/abstract.py
487
488
489
490
491
492
493
494
@abstractmethod
async def read_secret(self) -> bytes:
    """
    Reads the configured secret from the secret storage service.

    Returns:
        The secret data.
    """

write_secret(secret_data) abstractmethod async

Writes secret data to the configured secret in the secret storage service.

Parameters:

Name Type Description Default
secret_data

The secret data to write.

required

Returns:

Type Description
str

The key of the secret that can be used for retrieval.

Source code in src/prefect/blocks/abstract.py
496
497
498
499
500
501
502
503
504
505
506
@abstractmethod
async def write_secret(self, secret_data) -> str:
    """
    Writes secret data to the configured secret in the secret storage service.

    Args:
        secret_data: The secret data to write.

    Returns:
        The key of the secret that can be used for retrieval.
    """