kfp_toolbox.pipeline_jobs module#

kfp_toolbox.pipeline_jobs.submit_pipeline_job(pipeline_file: str | PathLike, endpoint: str | None = None, iap_client_id: str | None = None, api_namespace: str = 'kubeflow', other_client_id: str | None = None, other_client_secret: str | None = None, arguments: Mapping[str, Any] | None = None, run_name: str | None = None, experiment_name: str | None = None, namespace: str | None = None, pipeline_root: str | None = None, enable_caching: bool | None = None, service_account: str | None = None, encryption_spec_key_name: str | None = None, labels: Mapping[str, str] | None = None, project: str | None = None, location: str | None = None, network: str | None = None)[source]#

Submit a pipeline job.

Submit a pipeline job to the appropriate environment. If an endpoint is specified, it is considered an instance of Kubeflow Pipelines. Otherwise, attempt to submit to Vertex AI Pipelines.

Parameters:

  • pipeline_file (Union[str, os.PathLike]) – Path of the pipeline package file.

  • endpoint (Optional[str], optional) – Endpoint of the KFP API service to connect. Used only for Kubeflow Pipelines. Defaults to None.

  • iap_client_id (Optional[str], optional) – The client ID used by Identity-Aware Proxy. Used only for Kubeflow Pipelines. Defaults to None.

  • api_namespace (str, optional) – Kubernetes namespace to connect to the KFP API. Used only for Kubeflow Pipelines. Defaults to “kubeflow”.

  • other_client_id (Optional[str], optional) – The client ID used to obtain the auth codes and refresh tokens. Used only for Kubeflow Pipelines. Defaults to None.

  • other_client_secret (Optional[str], optional) – The client secret used to obtain the auth codes and refresh tokens. Used only for Kubeflow Pipelines. Defaults to None.

  • arguments (Optional[Mapping[str, Any]], optional) – Arguments to the pipeline function provided as a dict. Defaults to None.

  • run_name (Optional[str], optional) – Name of the run to be shown in the UI. Defaults to None.

  • experiment_name (Optional[str], optional) – Name of the experiment to add the run to. Defaults to None.

  • namespace (Optional[str], optional) – Kubernetes namespace where the pipeline runs are created. Used only for Kubeflow Pipelines. Defaults to None.

  • pipeline_root (Optional[str], optional) – The root path of the pipeline outputs. Defaults to None.

  • enable_caching (Optional[bool], optional) – Whether or not to enable caching for the run. Defaults to None.

  • service_account (Optional[str], optional) – Specifies which Kubernetes service account this run uses. Defaults to None.

  • encryption_spec_key_name (Optional[str], optional) – The Cloud KMS resource identifier of the customer managed encryption key used to protect the job. Used only for Vertex AI Pipelines. Defaults to None.

  • labels (Optional[Mapping[str, str]], optional) – The user defined metadata to organize PipelineJob. Used only for Vertex AI Pipelines. Defaults to None.

  • project (Optional[str], optional) – The project that you want to run this PipelineJob in. Used only for Vertex AI Pipelines. Defaults to None.

  • location (Optional[str], optional) – Location to create PipelineJob. Used only for Vertex AI Pipelines. Defaults to None.

  • network (Optional[str], optional) – The full name of the Compute Engine network to which the job should be peered. Used only for Vertex AI Pipelines. Defaults to None.