Google Cloud Storage is a RESTful online file storage web service for storing and accessing data on Google Cloud Platform infrastructure.
Requirements and Permissions
Collect the settings information for the source channel and add Crux’s service accounts to your bucket policy. Permission must be in place before setting up and testing your connection.
Setup Procedure
If you do not have a GCS bucket, follow these steps to create one. Note: if you already have an established GCS bucket with data, skip to the next section on Granting Crux access to a GCS bucket.
Log in to the Google Cloud console
Make sure the Cloud Storage API is enabled. Navigate to APIs & Services.
Navigate to the Cloud Storage product in the menu and click on Buckets.
Enter a name for your bucket. Note that GCS bucket names are globally unique across all organizations and projects.
Choose which region or multi-region you'd like to store your data in.
Use the Standard default storage class.
Ensure that Enforce public access prevention on this bucket as well as the Uniform radio button.
For protecting object data, use the default settings.
Next, you will need to load data into the GCS bucket accordingly so that Crux can use it as a data source.
Granting Crux Access to a GCS Bucket
The following steps are required to grant Crux read-access to your GCS bucket.
The Crux service accounts that will need access to the bucket are:
Granting access should be done on the bucket itself, not at the project or organization level. Grant the service accounts noted above with Storage Legacy Bucket Reader and Storage Legacy Object Reader roles by selecting the bucket, clicking the Permissions tab, then clicking Grant Access. Click Save when done.
Connect to Your Source
To connect to a GCS bucket after providing access above, provide the following field values:
Provide a unique connection name that you can refer to at a later time.
Select GCS in the source connection method.
Provide the GCS project name associated with the GCS bucket. This project name can be found on the top bar of the navigation.
Provide the GCS bucket name to which you are connecting.
To create a GCS source connection, configure the following connection settings:
Field | Description |
Connection name | Name for your source connection that can be quickly referenced in the future. |
Connection method | Method for source connection. |
GCS project | The name of the project in which the GCS bucket resides. |
GCS bucket name | The name of the GCS bucket to which you are connecting. This value is part of the domain name in the URL for the GCS cloud location. |