dctools.data.connection.connection_manager.S3WasabiManager

class dctools.data.connection.connection_manager.S3WasabiManager(connect_config, call_list_files=True, batch_size=64)

Specific S3 Manager for Wasabi (inheriting from S3Manager).

Parameters:
__init__(connect_config, call_list_files=True, batch_size=64)
Parameters:

Methods

__init__(connect_config[, call_list_files, ...])

adjust_full_day(date_start, date_end)

Adjust date_end to cover a full day if dates are the same at midnight.

download_file(remote_path, local_path)

Download a file from the remote source to the local path.

estimate_resolution(ds, coord_system)

Estimate resolution from dataset based on coordinates.

extract_global_metadata()

Extract global metadata (common to all files) from a single file.

extract_metadata(path)

Extract metadata combining global/file-specific info.

extract_metadata_worker(path, ...[, argo_index])

Extract metadata combining global/file-specific info.

get_config_clean_copy()

Return a clean copy of the configuration.

get_global_metadata()

Get global metadata for all files in the connection manager.

list_files()

List files matching pattern.

list_files_with_metadata()

Version with integrated Dask client and optimized configuration.

list_first_n_files(fs, remote_path[, n, pattern])

Use fsspec filesystem to quickly list up to n files recursively.

open(path[, mode])

Open a file, prioritizing local then remote access.

open_local(local_path)

Open a file locally if it exists.

open_remote(path[, mode])

Open a file remotely from an S3 bucket.

set_global_metadata(global_metadata)

Sets the global metadata for the connection manager.

supports(path)

Checks if path is supported.

open_remote(path, mode='rb')

Open a file remotely from an S3 bucket.

Parameters:
  • path (str) – Remote path of the file.

  • mode (str) – Mode to open the file (default is “rb”).

Returns:

Opened dataset, or None if remote opening is not supported.

Return type:

Optional[xr.Dataset]

classmethod supports(path)

Checks if path is supported.

Parameters:

path (str)

Return type:

bool