Skip to content

Using the interactive Dask Cluster Client

  • This example demonstrates how to connect to the Practicus AI Dask cluster we created, and execute simple Dask operations.
  • Please run this example on the Dask Coordinator (master).
import practicuscore as prt

# Let's get a Dask session
client = prt.distributed.get_client()

# And execute some code
import dask.array as da

print("Starting calculation.")

x = da.random.random((10000, 10000), chunks=(1000, 1000))
result = (x + x.T).mean(axis=0).compute()

print("Completed calculation. Results:", result)

Dask Dashboard

Practicus AI Dask offers an interactive dashboard where you can view execution details. Let's open the dashboard.

dashboard_url = prt.distributed.open_dashboard()

print("Page did not open? You can open this url manually:", dashboard_url)

# Let's execute the same code
import dask.array as da

print("Starting calculation.")

x = da.random.random((10000, 10000), chunks=(1000, 1000))
result = (x + x.T).mean(axis=0).compute()

print("Completed calculation. Results:", result)

Now you should see in real-time the execution details in a view similar to the below.

Dask Dashboard

Terminating the cluster

  • You can go back to the other worker where you created the cluster to run:

coordinator_worker.terminate()
- Or, terminate "self" and children workers with the below:

prt.get_local_worker().terminate()

Troubleshooting

If you’re experiencing issues with an interactive cluster that doesn’t run job/train.py, please follow these steps:

  1. Agent Count Mismatch: If the number of distributed agents shown by prt.distributed.get_client() is less than what you expected, wait a moment and then run get_client() again. This is usually because the agents have not yet joined the cluster. Note: Batch jobs automatically wait for agents to join.

  2. Viewing Logs: To view logs, navigate to the ~/my/.distributed folder.

Previous: Start Cluster | Next: Batch Job > Batch Job