Instruction Guide for Uploading a Google Cloud Bucket to CKAN

This guide walks through the process of replicating a public Google Cloud Storage bucket and its metadata into CKAN using a Python script.

Step 1: Prerequisites

You must have:

A public Google Cloud Storage bucket you want to replicate.
Files in the bucket must be publicly accessible via: https://storage.googleapis.com/YOUR_BUCKET_NAME/YOUR_FILE.EXT

Your environment must have:

Make the bucket public:
- Go to your bucket in the Google Cloud Console
- Navigate to Permissions > Add Principal
- Add:
```
Principal: allUsers
Role: Storage Object Viewer
```
Verify public access:
- Copy a file’s path (ex. data/sample.json)
- Visit: "https://storage.googleapis.com/YOUR_BUCKET_NAME/data/sample.json" (replace bucket name and file name)
- If it loads in the browser its public

In the gcs_to_ckan.py script:

Set the bucket name:
```
bucket_name = "YOUR_GCS_BUCKET_NAME"
```
Use your service account JSON for authentication: Make sure the file (ex. gcs-credentials.json) is in your working directory, and replace the default client call with:
```
gcs_client = storage.Client.from_service_account_json("YOUR_CREDENTIALS_JSON_PATH")
```

From the terminal:

python gcs_to_ckan.py

This will: