birdfsd_yolov5.model_utils package

Submodules

birdfsd_yolov5.model_utils.check_env_file module

check_env_file(env_file)

birdfsd_yolov5.model_utils.clear_preds_history module

clear_preds_history(model_version_to_keep: str, project_id: Optional[Union[int, str]] = None, all_projects: bool = False) → None

birdfsd_yolov5.model_utils.download_weights module

class DownloadModelWeights(model_version: str, output: str = 'best.pt')

Bases: object

get_weights(skip_download: bool = False, object_name_only: bool = False)

Get the weights for a given model version.

Parameters

• skip_download (bool) – If True, the function will return the download URL instead of downloading the weights file.

• object_name_only (bool) – If True, the function will return the object name of the weights file instead of downloading the weights file.

Returns

The path to the weights file. str: The download URL for the weights file. str: The model version.

Return type

str

exception ModelVersionDoesNotExist

Bases: Exception

Raised when the model version does not exist in the database.

birdfsd_yolov5.model_utils.handlers module

catch_keyboard_interrupt() → Callable

This function catches the keyboard interrupt handler.

Returns

A keyboard interrupt handler callable

Return type

Callable

keyboard_interrupt_handler(sig: int, _: object) → None

This function handles the KeyboardInterrupt (CTRL+C) signal.

It’s a handler for the signal, which means it’s called when the OS sends the signal. The signal is sent when the user presses CTRL+C.

Parameters

• sig (int) – The id of the signal that was sent.

• _ (object) – The current stack frame.

Returns

None

birdfsd_yolov5.model_utils.mongodb_helper module

exception MissingConnectionString

Bases: Exception

get_tasks_from_mongodb(project_id: str, db: Optional[Database] = None, dump: bool = False, json_min: bool = False, get_predictions: bool = False) → list

Get tasks from mongodb.

Parameters

• project_id (Union[int, str]) – The project id.

• db (pymongo.database.Database) – MongoDB client instance.

• dump (bool) – Dump the tasks to a json file.

• json_min (bool) – Use the minified json.

• get_predictions (bool) – Get predictions instead of tasks.

Returns

A list of tasks.

Return type

list

mongodb_db(db_connection_string: Optional[str] = None) → Database

Create a MongoDB client.

The database connection string is read from the environment variable DB_CONNECTION_STRING. The database name is read from the environment variable DB_NAME. If either of these environment variables are not set, None is returned.

Parameters

db_connection_string (str) – MongoDB connection string.

Returns

a MongoDB database object.

Return type

pymongo.database.Database

birdfsd_yolov5.model_utils.postgres_helper module

export_table_as_json(table_name, postgres_connection_string, output_fname)

jsonify_pg_tables(tables: Optional[list] = None, output_dir: str = 'tables')

birdfsd_yolov5.model_utils.relative_to_abs module

relative_to_abs() → None

Updates paths in the dataset_config.yaml file from relative to absolute.

This function is used to replace the dataset path in the dataset_config.yml file from a relative to absolute path.

birdfsd_yolov5.model_utils.s3_helper module

exception BucketDoesNotExist

Bases: Exception

Raised when the request bucket does not exist.

class S3

Bases: object

S3 client that uses MinIO as a backend.

download(bucket_name: str, object_name: str, dest: Optional[str] = None) → Object

Downloads an object from the bucket.

Parameters

• bucket_name (str) – The name of the bucket.

• object_name (str) – The name of the object.

• dest (str) – The destination path to download the object.

Returns

The destination path where the object was downloaded.

Return type

str

get_dataset(object_name: Optional[str] = None) → str

Retrieves the latest dataset from the S3 bucket.

Parameters

object_name (str) – The name of the object to retrieve. If not provided, the latest object will be retrieved.

Returns

The name of the object retrieved.

Return type

str

get_model_weights(model_version: str = 'latest') → str

Get the model weights from the model bucket.

Parameters

model_version – The version of the model to get. If ‘latest’, the latest version is returned. If a string, the model with that version is returned. If None, the latest version is returned.

Returns

The name of the model object in the model bucket.

Return type

str

upload(bucket_name: str, file_path: str, public: bool = False, scheme: str = 'https', dest: Optional[str] = None) → Union[ObjectWriteResult, str]

Uploads a file to an S3 bucket.

Parameters

• bucket_name (str) – The name of the bucket to upload to.

• file_path (str) – The path to the file to upload.

• public (bool) – True if the file is uploaded to a publicly accessible bucket.

• scheme (str) – The scheme to use for the URL.

• dest (str) – The destination path for the file.

Returns

The URL of the uploaded file.

Return type

str

birdfsd_yolov5.model_utils.update_run_cfg module

update_run_cfg(run_path: str, dataset_dir: str, dataset_name: str) → None

Adds additional metadata to a finished wandb run.

This function uploads the classes.txt and hist.jpg files to a finished wandb run. It also updates the run’s config with the dataset name, base ML framework version, and system hardware.

birdfsd_yolov5.model_utils.utils module

add_logger(current_file: str) → str

Adds a logger to the current file.

Parameters

current_file (str) – The name of the current file.

Returns

The name of the log file.

Return type

str

api_request(url: str, method: str = 'get', data: Optional[dict] = None, return_text: bool = False) → Union[dict, list, str]

Makes an API request to the given url with the given method and data.

Parameters

• url (str) – The url to make the request to.

• method (str) – The HTTP method to use. Defaults to ‘get’.

• data (Optional[dict]) – The data to send with the request. Defaults to None.

• return_text (bool) – Return the response as literal string.

Returns

The response from the API.

get_data(json_min: bool) → list

Get all tasks data from all projects.

Parameters

json_min – Whether to download the tasks in JSON_MIN format.

Returns

A list of all tasks.

Return type

list

get_labels_count() → dict

Creates a dictionary of labels and their frequency.

Returns

A dictionary of labels and their frequency.

Return type

dict

get_project_ids_str(exclude_ids: Optional[str] = None) → str

Get a comma separated string of project ids.

Parameters

exclude_ids (Optional[str]) – A comma separated string of project ids to exclude.

Returns

A comma separated string of project ids.

Return type

str

requests_download(url: str, filename: str) → None

Downloads a file from a given URL to a given filename.

Notes

tqdm integration snippet from: https://stackoverflow.com/a/63831344

Parameters

• url (str) – The URL of the file to download.

• filename (str) – The name of the file to save the downloaded file to.

Returns

None

Raises

RuntimeError – If the returned status code is not 200.

tasks_data(output_path: Optional[str]) → None

Get all tasks data of all projects.

Parameters

output_path (str) – Path to the output JSON file.

Returns

None

upload_logs(logs_file: str) → None

Uploads the logs file to S3.

Parameters

logs_file – The path to the logs file.

Returns

None

Raises

S3Error – If the upload fails.

birdfsd_yolov5.model_utils.wandb_helpers module

add_f1_score(run_path: str) → None

Adds the F1 score to the run. :param run_path: The wandb run path. :type run_path: str

upload_artifact(run_path: str, artifact_type: str, file_path: str, artifact_name: Optional[str] = None, aliases: Optional[list] = None) → None

Upload an artifact to an existing wandb run.

Parameters

• run_path (str) – The wandb run path.

• artifact_type (str) – The artifact type (e.g., dataset, model, etc.).

• file_path (str) – The path to the file on the local disk.

• artifact_name (str) – A human-readable name for the artifact. If None, the file’s base name is used.

Module contents