Crossplane logo
Crossplane logo
Deploy Preview for PR #572
master v1.13
Latest
v1.12 v1.11

GCP Quickstart Part 3

On this page
This document is for an older version of Crossplane.

This document applies to Crossplane version v1.11 and not to the latest release v1.13.

Important

This guide is part 3 of a series.

Follow part 1 to install Crossplane and connect your Kubernetes cluster to GCP.

Follow part 2 to create a composition, custom resource definition and a claim.

Part 2 created a composite resource definition to define the schema of the custom API. Users create a claim to use the custom API and apply their options. Part 2 didn’t show how the options set in a claim change or get applied the associated composite resources.

Prerequisites

  • Complete quickstart part 1 and Part 2 to install Crossplane and the quickstart configurations.

  1. Add the Crossplane Helm repository and install Crossplane.
1helm repo add \
2crossplane-stable https://charts.crossplane.io/stable
3helm repo update
4&&
5helm install crossplane \
6crossplane-stable/crossplane \
7--namespace crossplane-system \
8--create-namespace
  1. When the Crossplane pods finish installing and are ready, apply the GCP Provider.
1cat <<EOF | kubectl apply -f -
2apiVersion: pkg.crossplane.io/v1
3kind: Provider
4metadata:
5  name: upbound-provider-gcp
6spec:
7  package: xpkg.upbound.io/upbound/provider-gcp:v0.28.0
8EOF
  1. Create a file called gcp-credentials.json with your GCP service account JSON file.
Tip
The GCP documentation provides information on how to generate a service account JSON file.
  1. Create a Kubernetes secret from the GCP JSON file
1kubectl create secret \
2generic gcp-secret \
3-n crossplane-system \
4--from-file=creds=./gcp-credentials.json
  1. Create a ProviderConfig Include your GCP project ID in the ProviderConfig settings.
Tip
Find your GCP project ID from the project_id field of the gcp-credentials.json file.
 1cat <<EOF | kubectl apply -f -
 2apiVersion: gcp.upbound.io/v1beta1
 3kind: ProviderConfig
 4metadata:
 5  name: default
 6spec:
 7  projectID: 
 8  credentials:
 9    source: Secret
10    secretRef:
11      namespace: crossplane-system
12      name: gcp-secret
13      key: creds
14EOF
  1. Create a composition
 1cat <<EOF | kubectl apply -f -
 2apiVersion: apiextensions.crossplane.io/v1
 3kind: Composition
 4metadata:
 5  name: topic-with-bucket
 6spec:
 7  compositeTypeRef:
 8    apiVersion: custom-api.example.org/v1alpha1
 9    kind: XTopicBucket
10  resources:
11    - name: crossplane-quickstart-bucket
12      base:
13        apiVersion: storage.gcp.upbound.io/v1beta1
14        kind: Bucket
15        spec:
16          forProvider:
17            location: US
18    - name: crossplane-quickstart-topic
19      base:
20        apiVersion: pubsub.gcp.upbound.io/v1beta1
21        kind: Topic
22        spec:
23          forProvider:
24            messageStoragePolicy:
25              - allowedPersistenceRegions: 
26                - "us-central1"
27EOF
  1. Create a composite resource definition
 1cat <<EOF | kubectl apply -f -
 2apiVersion: apiextensions.crossplane.io/v1
 3kind: CompositeResourceDefinition
 4metadata:
 5  name: xtopicbuckets.custom-api.example.org
 6spec:
 7  group: custom-api.example.org
 8  names:
 9    kind: XTopicBucket
10    plural: xtopicbuckets
11  versions:
12  - name: v1alpha1
13    served: true
14    referenceable: true
15    schema:
16      openAPIV3Schema:
17        type: object
18        properties:
19          spec:
20            type: object
21            properties:
22              location:
23                type: string
24                oneOf:
25                  - pattern: '^EU$'
26                  - pattern: '^US$'
27            required:
28              - location
29  claimNames:
30    kind: TopicBucket
31    plural: topicbuckets
32EOF
  1. Create a new namespace
1kubectl create namespace test

Enable composition patches

In a composition, patches map fields in the custom API to fields inside the managed resources.

The example composition has two managed resources, a bucket and a topic.

 1apiVersion: apiextensions.crossplane.io/v1
 2kind: Composition
 3# Removed for Brevity
 4resources:
 5    - name: crossplane-quickstart-bucket
 6      base:
 7        apiVersion: storage.gcp.upbound.io/v1beta1
 8        kind: Bucket
 9        spec:
10          forProvider:
11            location: US
12    - name: crossplane-quickstart-topic
13      base:
14        apiVersion: pubsub.gcp.upbound.io/v1beta1
15        kind: Topic
16        spec:
17          forProvider:
18            messageStoragePolicy:
19              - allowedPersistenceRegions: 
20                - "us-central1"

The custom API defined a single option, location. A location can be either EU or US.

 1apiVersion: apiextensions.crossplane.io/v1
 2kind: CompositeResourceDefinition
 3# Removed for brevity
 4spec:
 5  group: custom-api.example.org
 6  names:
 7    kind: XDatabase
 8# Removed for brevity
 9      spec:
10        type: object
11        properties:
12          location:
13            type: string
14            oneOf:
15              - pattern: '^EU$'
16              - pattern: '^US$'

Creating a composition patch allows Crossplane to update the settings of a composite resource. Patches apply to an individual managed resource inside the composition.

A patch has a fromField and a toField specifying which value from the custom API should apply to a field in the managed resource.
Patches can create a transform to change the from field before it’s applied.

The transform type is what kind of change to make on the from field. Types of changes could include appending a string, preforming a math operation or mapping one value to another.

Applying a patch to the Topic uses the custom API spec.location field to use as the managed resource allowedPersistenceRegions value.

The custom API value “EU” is mapped to the value “europe-central2” and “US” is mapped to the value “us-central1.”

 1apiVersion: apiextensions.crossplane.io/v1
 2kind: Composition
 3# Removed for Brevity
 4resources:
 5    - name: crossplane-quickstart-topic
 6      base:
 7        apiVersion: pubsub.gcp.upbound.io/v1beta1
 8        kind: Topic
 9        spec:
10          forProvider:
11            messageStoragePolicy:
12              - allowedPersistenceRegions: 
13                - "us-central1"
14      patches:
15        - fromFieldPath: "spec.location"
16          toFieldPath: "spec.forProvider.messageStoragePolicy[*].allowedPersistenceRegions[*]"
17          transforms:
18            - type: map
19              map: 
20                EU: "europe-central2"
21                US: "us-central1"

Patching is a powerful tool enabling simpler or abstracted APIs. Developers aren’t required to know the specific GCP region, just the abstracted option of “EU” or “US.”

Apply the updated composition

Apply a similar patch to the Bucket managed resource and apply the updated composition.

 1cat <<EOF | kubectl apply -f -
 2apiVersion: apiextensions.crossplane.io/v1
 3kind: Composition
 4metadata:
 5  name: topic-with-bucket
 6spec:
 7  compositeTypeRef:
 8    apiVersion: custom-api.example.org/v1alpha1
 9    kind: XTopicBucket
10  resources:
11    - name: crossplane-quickstart-bucket
12      base:
13        apiVersion: storage.gcp.upbound.io/v1beta1
14        kind: Bucket
15        spec:
16          forProvider:
17            location: "US"
18      patches:
19        - fromFieldPath: "spec.location"
20          toFieldPath: "spec.forProvider.location"
21          transforms:
22            - type: map
23              map: 
24                EU: "EU"
25                US: "US"
26    - name: crossplane-quickstart-topic
27      base:
28        apiVersion: pubsub.gcp.upbound.io/v1beta1
29        kind: Topic
30        spec:
31          forProvider:
32            messageStoragePolicy:
33              - allowedPersistenceRegions: 
34                - "us-central1"
35      patches:
36        - fromFieldPath: "spec.location"
37          toFieldPath: "spec.forProvider.messageStoragePolicy[*].allowedPersistenceRegions[*]"
38          transforms:
39            - type: map
40              map: 
41                EU: "europe-central2"
42                US: "us-central1"
43EOF

Create a claim

Create a new claim and set the location to “EU.”

1cat <<EOF | kubectl apply -f -
2apiVersion: custom-api.example.org/v1alpha1
3kind: TopicBucket
4metadata:
5  name: claimed-eu-topic-with-bucket
6  namespace: test
7spec:
8  location: "EU"
9EOF

View the claim with kubectl get claim

1kubectl get TopicBucket -n test
2NAME                           SYNCED   READY   CONNECTION-SECRET   AGE
3claimed-eu-topic-with-bucket   True     True                        2m26s

The claim reports SYNCED and READY as True after Crossplane creates all the managed resources.

Describe the Topic resource to see the GCP location is europe-central2.

1kubectl describe topic | grep "For Provider" -A3
2  For Provider:
3    Message Storage Policy:
4      Allowed Persistence Regions:
5        europe-central2

Describe the Bucket resource to see the GCP location is also set to EU.

1kubectl describe bucket | grep "For Provider" -A1
2  For Provider:
3    Location:                  EU

Using location: "EU" in the claim patches the composite resource, updating the Topic GCP region from us-central1 to europe-central-2 and the Bucket from GCP region US to GCP region EU.
The developer creating the claim isn’t required to know which specific GCP region or the naming conventions. Using the abstract API options of “EU” or “US” the developer places their resources in the desired location.

In this example, patching also allows platform teams to ensure all resources are in the same location.

Deleting the claim removes the managed resources.

Note
The managed resources take up to 5 minutes to delete.
1kubectl delete TopicBucket claimed-eu-topic-with-bucket -n test

Create a Crossplane configuration package

Creating a configuration package makes your Crossplane custom APIs portable and versioned.

Crossplane configuration packages allow users to combine their custom resource definition and composition files into an OCI image.

Note
The Open Container Initiative defines the OCI image standard.
An OCI images is a standard way to package data.

You can host configuration packages in image registries like Docker Hub or the Upbound Marketplace.

Crossplane can download and install configuration packages into a Kubernetes cluster.

Building and installing configuration packages requires an OCI image compatible tool.

Note
You can use any software that builds OCI images. This includes Docker or Upbound’s Up command-line tool

A configuration package includes three files:

  • crossplane.yaml defines the metadata of the package.
  • definition.yaml is the composite resource definition for the package.
  • composition.yaml is the composition template for the package.

Create a crossplane.yaml file

Configuration packages describe their contents and requirements with a crossplane.yaml file.

The crossplane.yaml file lists the required Crossplane providers and their compatible versions as well as the required Crossplane version.

The Crossplane meta.pkg API defines the schema for a Configuration.

Inside the spec define the required Crossplane version.

The dependsOn section lists the dependencies for a package.

This package lists the Upbound provider-gcp version 0.28.0 or later as a dependency.

Tip
Crossplane automatically installs dependencies. Dependencies can include other configuration packages.
 1apiVersion: meta.pkg.crossplane.io/v1
 2kind: Configuration
 3metadata:
 4  name: crossplane-gcp-quickstart
 5spec:
 6  crossplane:
 7    version: ">=v1.11.0"
 8  dependsOn:
 9    - provider: xpkg.upbound.io/upbound/provider-gcp
10      version: ">=v0.28.0"

Create a new directory and save the crossplane.yaml file.

 1mkdir crossplane-gcp-quickstart
 2cat <<EOF > crossplane-gcp-quickstart/crossplane.yaml
 3apiVersion: meta.pkg.crossplane.io/v1
 4kind: Configuration
 5metadata:
 6  name: crossplane-gcp-quickstart
 7spec:
 8  crossplane:
 9    version: ">=v1.11.0"
10  dependsOn:
11    - provider: xpkg.upbound.io/upbound/provider-gcp
12      version: ">=v0.28.0"
13EOF

Create a definition.yaml file

A configuration package requires a composite resource definition (XRD) to define the custom API.

Save the XRD as definition.yaml in the same directory as the crossplane.yaml file.

 1cat <<EOF > crossplane-gcp-quickstart/definition.yaml
 2apiVersion: apiextensions.crossplane.io/v1
 3kind: CompositeResourceDefinition
 4metadata:
 5  name: xtopicbuckets.custom-api.example.org
 6spec:
 7  group: custom-api.example.org
 8  names:
 9    kind: XTopicBucket
10    plural: xtopicbuckets
11  versions:
12  - name: v1alpha1
13    served: true
14    referenceable: true
15    schema:
16      openAPIV3Schema:
17        type: object
18        properties:
19          spec:
20            type: object
21            properties:
22              location:
23                type: string
24                oneOf:
25                  - pattern: '^EU$'
26                  - pattern: '^US$'
27            required:
28              - location
29  claimNames:
30    kind: TopicBucket
31    plural: topicbuckets
32EOF

Create a composition.yaml file

The composition template creates the managed resources and allows patches to customize the managed resources.

Copy the composition into the composition.yaml file in the same directory as crossplane.yaml.

 1cat <<EOF > crossplane-gcp-quickstart/composition.yaml
 2apiVersion: apiextensions.crossplane.io/v1
 3kind: Composition
 4metadata:
 5  name: topic-with-bucket
 6spec:
 7  compositeTypeRef:
 8    apiVersion: custom-api.example.org/v1alpha1
 9    kind: XTopicBucket
10  resources:
11    - name: crossplane-quickstart-bucket
12      base:
13        apiVersion: storage.gcp.upbound.io/v1beta1
14        kind: Bucket
15        spec:
16          forProvider:
17            location: "US"
18      patches:
19        - fromFieldPath: "spec.location"
20          toFieldPath: "spec.forProvider.location"
21          transforms:
22            - type: map
23              map: 
24                EU: "EU"
25                US: "US"
26    - name: crossplane-quickstart-topic
27      base:
28        apiVersion: pubsub.gcp.upbound.io/v1beta1
29        kind: Topic
30        spec:
31          forProvider:
32            messageStoragePolicy:
33              - allowedPersistenceRegions: 
34                - "us-central1"
35      patches:
36        - fromFieldPath: "spec.location"
37          toFieldPath: "spec.forProvider.messageStoragePolicy[*].allowedPersistenceRegions[*]"
38          transforms:
39            - type: map
40              map: 
41                EU: "europe-central2"
42                US: "us-central1"
43EOF

Install the Crossplane command-line

To build a configuration package install the Crossplane Kubernetes command-line extension.

1wget "https://raw.githubusercontent.com/crossplane/crossplane/master/install.sh"
2chmod +x install.sh
3./install.sh
4sudo mv crossplane /usr/local/bin

Verify the Crossplane command-line installed with crossplane --help

 1crossplane --help
 2Usage: crossplane <command>
 3
 4A command line tool for interacting with Crossplane.
 5
 6Flags:
 7  -h, --help       Show context-sensitive help.
 8  -v, --version    Print version and quit.
 9      --verbose    Print verbose logging statements.
10# Ouptut removed for brevity

Build a configuration package

Use the crossplane CLI to create an .xpkg file containing the custom APIs and Crossplane configuration.

1crossplane build configuration -f crossplane-gcp-quickstart/ --name="crossplane-gcp-quickstart"

Now an .xpkg OCI image is inside the crossplane-gcp-quickstart directory.

1ls crossplane-gcp-quickstart/
2composition.yaml  crossplane-gcp-quickstart.xpkg  crossplane.yaml  definition.yaml

Next steps