Projects

Project functions

sync.api.projects.create_project(name: str, product_code: str, description: Optional[str] = None, job_id: Optional[str] = None, cluster_path: Optional[str] = None, workspace_id: Optional[str] = None, cluster_log_url: Optional[str] = None, auto_apply_recs: bool = False, prediction_params: Optional[dict] = None, app_id: Optional[str] = None, optimize_instance_size: bool = False) → sync.api.projects.Response[dict]

Creates a Sync project for tracking and optimizing Apache Spark applications

Parameters

  • name (str) – Project name

  • product_code (str) – Product code

  • description (str, optional) – application description, defaults to None

  • job_id (str, optional) – Databricks job ID, defaults to None

  • cluster_path (str, optional) – path to cluster definition in job object, defaults to None

  • workspace_id (str, optional) – Databricks workspace ID, defaults to None

  • cluster_log_url (str, optional) – S3 or DBFS URL under which to store project configurations and logs, defaults to None

  • auto_apply_recs (bool, optional) – automatically apply project recommendations, defaults to False

  • prediction_params (dict, optional) – dictionary of prediction parameters, defaults to None. Valid options are documented here

  • app_id (str, optional) – Apache Spark application identifier, defaults to None

Returns

the newly created project

Return type

Response[dict]

sync.api.projects.create_project_recommendation(project_id: str, **options) → sync.api.Response[str]

Creates a prediction given a project id

Parameters

project_id (str, optional) – ID of project to which the prediction belongs, defaults to None

Returns

prediction ID

Return type

Response[str]

sync.api.projects.create_project_submission(platform: sync.models.Platform, cluster_report: dict, eventlog_url: str, project_id: str) → sync.api.Response[str]

Create a submission

Parameters

  • platform (Platform) – platform, e.g. “aws-databricks”

  • cluster_report (dict) – cluster report

  • eventlog_url (str) – event log URL

  • project_id (str) – ID of project to which the submission belongs

Returns

prediction ID

Return type

Response[str]

sync.api.projects.create_project_submission_with_eventlog_bytes(platform: sync.models.Platform, cluster_report: dict, eventlog_name: str, eventlog_bytes: bytes, project_id: str) → sync.api.Response[str]

Creates a submission given event log bytes instead of a URL

Parameters

  • platform (Platform) – platform, e.g. “aws-databricks”

  • cluster_report (dict) – cluster report

  • eventlog_name (str) – name of event log (extension is important)

  • eventlog_bytes (bytes) – encoded event log

  • project_id (str) – ID of project to which the submission belongs

Returns

prediction ID

Return type

Response[str]

sync.api.projects.delete_project(project_id: str) → sync.api.Response[str]

Deletes a project

Parameters

project_id (str) – project ID

Returns

confirmation message

Return type

Response[str]

sync.api.projects.get_cluster_definition_and_recommendation(project_id: str, cluster_spec_str: str) → sync.api.projects.Response[dict]

Print Current Cluster Definition and Project Configuration Recommendation. Throws error if no cluster recommendation found for project

Parameters

  • project_id (str) – project ID

  • cluster_spec_str (str) – Current Cluster Recommendation

Returns

Current Cluster Definition and Project Configuration Recommendation object

Return type

dict

sync.api.projects.get_latest_project_config_recommendation(project_id: str) → Optional[sync.api.projects.Response[dict]]

Get Latest Project Configuration Recommendation.

Parameters

project_id (str) – project ID

Returns

Project Configuration Recommendation object

Return type

Response object or None

sync.api.projects.get_products() → sync.api.projects.Response[List[str]]

Get supported platforms :return: list of platform names :rtype: Response[list[str]]

sync.api.projects.get_project(project_id: str, params: Optional[dict] = None) → sync.api.projects.Response[dict]

Retrieves a project

Parameters

project_id (str) – project ID

Returns

project object

Return type

Response[dict]

sync.api.projects.get_project_by_app_id(app_id: str) → sync.api.projects.Response[dict]

Retrieves a project by app ID

Parameters

app_id (str) – app ID

Returns

project or error if none exists for the app ID

Return type

Response[dict]

sync.api.projects.get_project_cluster_template(project_id: str, region_name: Optional[str] = None) → sync.api.projects.Response[dict]

Retrieve a project cluster template.

Parameters

  • project_id (str) – project ID

  • region_name (str, optional) – region name, defaults to AWS configuration

Returns

project object

Return type

Response[dict]

sync.api.projects.get_project_recommendation(project_id: str, recommendation_id: str) → sync.api.projects.Response[dict]

Get a specific recommendation for a project id

Parameters

  • project_id (str) – project ID

  • recommendation_id (str) – recommendation ID

Returns

recommendation object

Return type

Response[dict]

sync.api.projects.get_project_submission(project_id: str, submission_id: str) → sync.api.projects.Response[dict]

Get a specific submission for a project id

Parameters

  • project_id (str) – project ID

  • submission_id (str) – submission ID

Returns

submission object

Return type

Response[dict]

sync.api.projects.get_projects(app_id: Optional[str] = None) → sync.api.projects.Response[List[dict]]

Returns all projects authorized by the API key

Parameters

app_id (str, optional) – app ID to filter by, defaults to None

Returns

projects

Return type

Response[list[dict]]

sync.api.projects.get_submissions(project_id: str) → sync.api.projects.Response[dict]

Get submissions for a project id

Parameters

project_id (str) – project ID

Returns

List of Submission Configuration objects

Return type

dict

sync.api.projects.get_updated_cluster_definition(project_id: str, cluster_spec_str: str) → sync.api.projects.Response[Union[AWSProjectConfiguration, AzureProjectConfiguration]]

Return Cluster Definition merged with Project Configuration Recommendations.

Parameters

  • project_id (str) – project ID

  • cluster_spec_str (str) – Current Cluster Recommendation

Returns

Updated Cluster Definition with Project Configuration Recommendations

Return type

AWSProjectConfiguration or AzureProjectConfiguration

sync.api.projects.get_updated_cluster_defintion(project_id: str, cluster_spec_str: str) → sync.api.projects.Response[Union[AWSProjectConfiguration, AzureProjectConfiguration]]

Return Cluster Definition merged with Project Configuration Recommendations.

Parameters

  • project_id (str) – project ID

  • cluster_spec_str (str) – Current Cluster Recommendation

Returns

Updated Cluster Definition with Project Configuration Recommendations

Return type

AWSProjectConfiguration or AzureProjectConfiguration

sync.api.projects.reset_project(project_id: str) → sync.api.Response[str]

Resets a project

Parameters

project_id (str) – project ID

Returns

confirmation message

Return type

Response[str]

sync.api.projects.update_project(project_id: str, description: Optional[str] = None, cluster_path: Optional[str] = None, workspace_id: Optional[str] = None, cluster_log_url: Optional[str] = None, app_id: Optional[str] = None, auto_apply_recs: Optional[bool] = None, prediction_params: Optional[dict] = None, job_id: Optional[str] = None, optimize_instance_size: Optional[bool] = None) → sync.api.projects.Response[dict]

Updates a project’s mutable properties

Parameters

  • project_id (str) – project ID

  • description (str, optional) – description, defaults to None

  • cluster_path (str, optional) – path to cluster definition in job object, defaults to None

  • workspace_id (str, optional) – Databricks workspace ID, defaults to None

  • cluster_log_url (str, optional) – location of project event logs and configurations, defaults to None

  • app_id (str, optional) – external identifier, defaults to None

  • auto_apply_recs (bool, optional) – automatically apply project recommendations, defaults to None

  • prediction_params (dict, optional) – dictionary of prediction parameters, defaults to None. Valid options are documented here

  • job_id (str, optional) – Databricks job ID, defaults to None

  • optimize_instance_size (bool, optional) – flag to turn on/off instance size recommendations, defaults to None

Returns

updated project

Return type

Response[dict]

sync.api.projects.wait_for_recommendation(project_id: str, recommendation_id: str) → sync.api.projects.Response[dict]

Get a recommendation, wait if it’s not ready

Parameters

  • project_id (str) – project ID

  • recommendation_id (str) – recommendation ID

Returns

recommendation object

Return type

Response[dict]