Managing Jenkins X secrets

Managing Jenkins X secrets

In traditional computing infrastructures, all of the resources and components (hardware, networking, availability, security and deployment) as well as associated labor costs are locally managed. Third-party computing environments such as cloud service providers and Git hosts offer decentralized solutions with distinct advantages in service reliability and costs over the traditional solutions.

However, one issue with using cloud services, distributed storage, and remote repositories is the lack of trusted networks, vetted hardware, and other closely observed security measures practiced in locally-hosted infrastructure. For the sake of convenience, users often store sensitive information like authentication credentials in open, public repositories, exposed to potential malicious activity.

Hashicorp Vault is one tool that centralizes the management of secrets: resources that provide authentication to your computing environment such as tokens, keys, passwords, and certificates.

CloudBees Jenkins X Distribution handles security and authentication resources through the integration of Vault. Users can deploy Vault to securely store and manage all aspects of their development platform.

CloudBees Jenkins X Distribution installs and configures Vault for your cluster by default through the cluster creation process. Refer to Creating a Jenkins X cluster for more information.

Vault features

Vault is a tool for accessing and storing user secrets. It manages the complexity of secure resource access:

  • Storing secrets - Vault places secrets in an encrypted format in a remote storage bucket.

  • Secret creation and deletion - Vault creates secrets for dynamic access to storage buckets, ephemeral access that are created/destroyed as needed for temporary data access, and generating keys for database authentication.

  • Encrypting data - Vault stores secrets in a remote storage bucket in secure directories using strong encryption.

Using Vault with CloudBees Jenkins X Distribution

CloudBees Jenkins X Distribution interacts with Vault via the jx command line program. There are commands for creating, deleting, and managing secrets and vaults.

CloudBees Jenkins X Distribution uses Vault to store all Jenkins X secrets, such as the GitHub personal access token generated for the pipeline bot when creating a Jenkins X cluster. It also stores any GitOps secrets, such as passwords for storage buckets, and keys for secure server access.

Secrets can be retrieved by the pipeline or via command-line if logged into the account associated with the kubernetes service as well as any secrets stored in the jx namespace for the pipeline.

Vaults are provisioned in kubernetes using vault-operator, an open-source Kubernetes controller installed when Vault is configured during cluster creation and Jenkins X installation on the cluster.

Configuring DNS and TLS settings for Vault

For a secure CloudBees Jenkins X Distribution installation, you must enable TLS when interacting with the vault service. To configure TLS, you must first configure Zone DNS settings within Google Cloud Platform, and then configure external DNS settings for Ingress and TLS in the jx-requirements.yaml configuration file.

Configuring Google Cloud DNS

In order to configure Vault for the proper DNS and TLS access, you must configure Google Cloud DNS settings appropriately.

Using the fictional Acme organization used in Creating a Jenkins X cluter, an administrator should have the following a domain name registered with a name registrar, for example www.acmecorp.example before configuring DNS Zone settings. For more information, refer to Creating a managed public zone from the Google documentation.

  1. Navigate via browser to the Project Selector page. and choose your Google Cloud Platform project.

  2. Create a DNS zone

    1. Choose Public as your Zone Type.

    2. Type a Zone Name for your zone.

    3. Input a DNS suffix in DNS name, for example acmecorp.example.

    4. Choose your DNSSEC or DNS Security state, which should be set to Off for this configuration.

    5. (Optional) Input a Description for your DNS zone.

    6. Click Create.

Once created, the Zone Details page loads. NS (Name server) and SOA (Start of autority) records are automatically created for your domain (for example acmecorp.example)

Configuring External DNS in CloudBees Jenkins X Distribution

Once you have configured Google Cloud DNS, you can use browse the Zones page in your Google Cloud Platform project to setup your external domain.

External DNS will automatically updates DNS records if you reuse the domain name, so if you delete an old cluster and create a new one it will preserve the same domain configuration for the new cluster.
  1. Choose a unique DNS name; you can use nested domains (for example,  cluster1.acmecorp.example). Enter the name in the DNS Name field

  2. Run the jx create domain command against jx create domain gke --domain cluster1.acmecorp.example.

    1. The program prompts you to choose your Google Cloud Platform project from the available list.

    2. The program prompts you to update your existing managed servers to use the displayed list of Cloud DNS nameservers. Copy the list for use in the next steps.

  3. From the Google Cloud Platform Zones page, change the Resource Record Type to NS) and use the default values for your domain for for TTL (5) and TTL Unit (minutes).

  4. Add the first nameserver to the Name server field

  5. Click Add item and add any subsequent nameservers.

  6. Click Create.

  7. Configure CloudBees Jenkins X Distribution for the new domain names:

    • Edit the jx-requirements.yaml file and update the domain field (in ingress) to your domain name, for example cluster1.acmecorp.example

    • In the tls setting, enable TLS with enabled: true

    • The resulting jx-requirements.yaml entries for these settings should look similar to the following:

      gitops: true
        domain: cluster1.acmecorp.example
        externalDNS: true
        namespaceSubDomain: -jx.
          email: certifiable@acmecorp.example
          enabled: true
          production: true
      secretStorage: vault
    • Run jx boot for the changes to take effect in your environment.

Creating a Vault

A vault is created by default using the Cluster creation process to create your cluster, unless you specified during the cluster configuration not to create the vault. In this case, you can create one post-installation with the jx create command-line interface:

jx create vault
  1. The program will ask you the name you want for your vault (for example, acmevault)

  2. The program will ask you for your Google Cloud Zone of choice. Refer to Regions and Zones in the Google Cloud documentation for more information. In this example, us-east1-c is chosen for proximity to Acme Headquarters.

  3. If you have a storage bucket account configured from creating a cluster with jx boot, then the jx create vault command will scan your installation for Vault-related storage buckets and, if found, prompt you to approve deleting and recreating the Vault from scratch.

  4. The program will ask you the Expose type for your vault in order to create rules and routes for cluster load balancer and other services. Default is Ingress.

  5. The program will ask for a cluster domain. Default is the one created in the Cluster creation process such as

  6. The program will ask for an URLTemplate. Press Enter to use the default value.

  7. The program will verify your answers to the previous questions in summary and prompt you to approve the Vault creation (default is Yes).

Retrieving Vault secrets

If you need to recall your secrets (such as a password, keypair, or token) you can run the jx get command to find the Vault Address to open in your browser and the Vault Token for logging into the Vault.

jx get vault-config

The output is in the form of export statements. For example:

export VAULT_ADDR=
export VAULT_TOKEN=t.lBNBWR9JIMwwH9AXD95grlwmn

These export statements can be used to run Vault’s own client command-line tools, or they can be copied from the command-line and pasted in a web browser to retrieve stored secrets.

Deleting a Vault

If you need to delete your vault due to misconfiguration or changes in the authentication protocols of 3rd party resources, you can delete a Vault using the jx delete command and the name of the Vault to remove all associated secrets stored in the Vault.

jx delete vault acmevault

If you have any questions or feedback on the CloudBees Jenkins X Distribution documentation, send them to