Table of contents

CloudBees Core on modern cloud platforms administration guide


External HTTP Endpoints

External HTTP endpoints work with Cross Team Collaboration to enable external systems such as GitHub or Nexus to generate notification events for Pipelines.

Jenkins external notification webhooks are user-defined HTTP callbacks that can be triggered by external actions, such as pushing a new artifact to a Nexus repository. When this happens, the external system makes an HTTP request to this endpoint and passes a data payload in the body of the request. Jenkins broadcasts the request via the internal messaging API to all jobs that are configured with an event trigger.

Webhook events are typically secured with a shared secret that the sender uses to generate a unique message digest. The message digest is sent to the receiving system in an HTTP request header. Jenkins uses the digest and the shared secret to verify that the message has not been altered and that it originates from a trusted source.

In addition to using a shared secret, some webhooks support the use of IP address whitelisting, which allows an Administrator to specify a list of addresses (or address ranges) that are allowed to create a webhook events.

Note
Validating the contents of the incoming message using a shared secret known only by the sender and the receiver ensures that rogue users are not able to send "fake" events which would cause Jenkins to trigger unexpected actions.
high level
Figure 1. External HTTP Endpoints overview

The external HTTP endpoints feature requires the installation of the following plugins on the Operations Center instance:

Configuring

External webhook endpoints are added and deleted from Manage Jenkins  Manage Notification Webhook HTTP Endpoints.

external endpoints config
Figure 2. The external HTTP endpoints configuration page
Note
The Manage Notification Webhook HTTP Endpoints option will only appear if the external-notification-plugin plugin is installed.

The configuration page also shows the details of current endpoints.

webhooks config
Figure 3. The external HTTP endpoints configuration page

The configuration screen shows all currently enabled endpoints, with the following details:

  • Name - The user friendly name given to this endpoint

  • Endpoint URL - The URL by which this endpoint can be accessed

  • Last Event - The date/time at which this endpoint was last accessed

  • Last Status - The HTTP status code associated with the last request received by this endpoint

  • Webhook Message Format - The type of message that this endpoint accepts

  • Details - Displays the full details for this endpoint

  • Delete - Deletes this endpoint.

    Note: This action cannot be undone.

Hovering over the Last Status field or clicking on the Details button shows the HTTP status message associated with the last request received by an endpoint. This can be very helpful when troubleshooting new webhook configurations.

External HTTP notification endpoint types

Several endpoint types are already included. Custom endpoints can be defined by specifying the security type and relevant details. The following endpoint types are currently supported:

  • Custom JSON Webhook

  • Bitbucket Cloud

  • Bitbucket Server

  • Docker Hub

  • GitHub

  • Nexus 3

To configure a new endpoint, click Register a new endpoint.

Custom JSON webhooks

The Custom JSON webhook allows full control of the configuration of a new webhook endpoint. This allows integrations with systems not supported with an out of the box integration. Custom JSON webhooks support two forms of security - HMAC message signatures and IP address whitelisting. At least one of these security types must be chosen, and if desired, both can be used together.

Note
Custom JSON webhooks require the use of at least one type of security - either HMAC signatures or IP address whitelists.
Custom JSON webhooks using HMAC signature validation

The custom JSON webhook allows full control of the configuration of a new webhook endpoint. This allows integrations with systems not supported with an out of the box configuration.

custom json webhook config
Figure 4. Configuring a custom JSON webhook endpoint with HMAC signature validation
  • Webhook Name - A user friendly name for this new endpoint

  • HMAC HTTP Header - The name of the HTTP request header that will contain the message digest

  • Message Signature Validation Type - The cryptographic hash type that will be used to generate the message digest

  • Secret - The shared secret that will be used to validate the authenticity of received events. Use the Generate Secret button to automatically generate a cryptographically random secret.

Refer to HMAC (hash-based message authentication code) for more information on this widely used method of verifying the integrity and authenticity of a message.

Custom JSON webhook with whitelists

Not all webhook providers support HMAC signature validation. For those that don’t, it is possible to provide a level of security by whitelisting the IP addresses of the system that will generate the webhook events. To do this, check the box labeled "Define a list of IP addresses that are allowed to send events to this endpoint" and enter the IP addresses (one per line) of the remote host which will be generating the webhook events. You may enter both IPv4 and IPv6 addresses, as well as address ranges using CIDR notation.

custom json webhook config none
Figure 5. Configuring a custom JSON webhook endpoint with an IP whitelist
Note
When choosing NONE as the Message Signature Validation Type the whitelist entry will automatically be shown. Although it is not recommended, you may enter 0.0.0.0/0 and ::/0 which will allow access from any host. If no addresses are specified, then access from any host will be denied.

Bitbucket Cloud

The Bitbucket Cloud webhook endpoint is pre-configured to work with Bitbucket Cloud events. Bitbucket Cloud does not support HMAC signatures, and per Atlassian’s documentation, it is recommended to secure your endpoint using an IP whitelist. The Bitbucket Cloud webhook is pre-configured with the IP addresses from Atlassian’s documentation.

bitbucket cloud webhook config
Figure 6. Configuring a Bitbucket Cloud webhook endpoint

The following fields are required to create a new Bitbucket Cloud endpoint:

  • Webhook Name - A user friendly name for this new endpoint

  • Whitelist - A list of IP addresses (or address ranges) specifying the hosts that are allowed to send Bitbucket Cloud webhook events to your Jenkins instance.

  • Secret - (Optional) A unique string which will be use to validate the authenticity of incoming webhook events. Use the Generate Secret button to automatically generate a cryptographically random secret, or leave this field blank if you do not wish to perform this validation (this is not recommended).

It is also possible to specify a shared secret which will be included in the URL that is generated for a Bitbucket Cloud webhook endpoint. Although no signature validation will be performed, incoming events that do not include this secret URL parameter will be ignored. Be sure to copy the entire url shown in the Manage Notification Webhook Endpoints list to use when configuring your Bitbucket Cloud webhook.

bitbucket cloud secret url
Figure 7. Bitbucket Cloud Webhook URL With Secret

For more information about Bitbucket Cloud webhooks, please refer to Manage Webhooks documentation provided by Atlassian.

Bitbucket server

The Bitbucket Server webhook endpoint is pre-configured to work with Bitbucket Server events, and supports HMAC SHA256 message signatures.

bitbucket server webhook config
Figure 8. Configuring a Bitbucket Server Webhook Endpoint

The following fields are required to create a new Bitbucket Server endpoint:

  • Webhook Name - A user friendly name for this new endpoint

  • Secret - The shared secret that will be used to validate the authenticity of received events. Use the Generate Secret button to automatically generate a cryptographically random secret.

Refer to the Bitbucket Server documentation for more infomation on Bitbucket Server webhooks.

Docker Hub

The Docker Hub webhook endpoint is pre-configured to work with Docker Hub webhook events. Docker Hub does not support HMAC message signatures, so it is highly recommended to specify a secret which will be included with every webhook request that Docker Hub makes to your Jenkins instance. When a secret is configured then only requests which include this will be accepted.

docker hub webhook config
Figure 9. Configuring a Docker Hub Webhook Endpoint

The following fields are required to create a new Docker Hub endpoint:

  • Webhook Name - A user friendly name for this new endpoint

  • Secret - (Optional) A unique string which will be use to validate the authenticity of incoming webhook events. Use the Generate Secret button to automatically generate a cryptographically random secret, or leave this field blank if you do not wish to perform this validation (this is not recommended).

docker hub secret url
Figure 10. Docker Hub Webhook With Secret

Be sure to copy the entire url shown in the Manage Notification Webhook Endpoints list to use when configuring your Docker Hub webhook.

Nexus 3 webhook

The Nexus 3 webhook endpoint allows for integration with Nexus events. Nexus webhooks are secured with a shared secret and allow you to subscribe to certain events on the Nexus repository.

nexus3 webhook config
Figure 11. Configuring a Nexus 3 webhook endpoint

The following fields are required to create a new Nexus endpoint:

  • Webhook Name - A user friendly name for this new endpoint

  • Secret - The shared secret that will be used to validate the authenticity of received events. Use the Generate Secret button to automatically generate a cryptographically random secret.

Refer to the Nexus documentation for more information on Nexus webhooks.

GitHub webhook

The GitHub webhook endpoint integrates with GitHub events. GitHub webhooks are secured with a shared secret and allow you to subscribe to events from GitHub whenever certain actions (ie. tag/label creation, github release, etc)

github webhook config
Figure 12. Configuring a GitHub webhook endpoint

The following fields are required to create a new GitHub endpoint:

  • Webhook Name - A user friendly name for this new endpoint

  • Secret - The shared secret that will be used to validate the authenticity of received events. Use the 'Generate Secret' button to automatically generate a cryptographically random secret.

Refer to the GitHub documentation for more information on GitHub webhooks.

Integrating webhooks with pipeline event triggers

A webhook payload is used by a matching event trigger in the pipeline jobs that are interested in the message. Ensure that the following plugins are installed on the master running the pipeline:

Refer to the Cross Team Collaboration documentation to configure notifications.

When a webhook receives an event, all the Pipelines listening on events (event triggers) will be queried. The Pipelines with a trigger condition that matches the publishEvent will be executed. The trigger step uses query patterns based on jmespath syntax to match events. Triggers can be configured directly from the Pipeline job configuration page or can be written as a Pipeline step.

Example webhook event trigger

This GitHub webhook example assumes you have installed the required plugins and that you have an existing GitHub account.

1 - Create a New GitHub Webhook Endpoint

From Manage Jenkins  Manage Notification Webhook Endpoints, create a new GitHub webhook with the following values:

  • Name - My First Webhook

  • Secret - NotAGoodSecret

example github webhook
Figure 13. Create a new webhook endpoint for GitHub

2 - Create a New Webhook on GitHub

Next, from the Settings tab on your GitHub repository, select Webhooks on the left hand side and click Add webhook. Fill in the form values as shown:

  • Payload URL - Paste the Jenkins webhook endpoint URL that was generated in the first step

  • Content type - Select application/json from the dropdown list

  • Secret - Enter the shared secret value (NotAGoodSecret) from the first step

  • The rest of the form values can be left at their defaults

After filling in the details, click Add webhook to save the changes.

configure github webhook
Figure 14. Create a new webhook on GitHub

3 - Validate the New Webhook Endpoint

After the webhook is defined, GitHub will send a test message to your Jenkins endpoint. Click Edit for the webhook you created and scroll to the bottom of the page. If all went well, it will show something like:

github recent deliveries
Figure 15. Validating your new endpoint
Note
If you see a red triangle instead of a green check mark, click on the message id to see more details. Make needed changes to your configuration and click on the Redeliver button to validate again.

4 - Create a Pipeline Job Triggered by the Webhook Event

Create a Pipeline that will trigger when an event is received. We will use the GitHub test message and create a job that will build when it sees the attribute "zen" with a value of "Non-blocking is better than blocking.". Create a new Pipeline job and copy the following script:

Jenkinsfile (Declarative Pipeline)
pipeline {
    agent any

    triggers {
        eventTrigger jmespathQuery("zen=='Non-blocking is better than blocking.'")
    }

    stages {
        stage('Example') {
            steps {
                sh 'echo Hello World';
            }
        }
    }
}

Save your job and then click Build Now once to ensure the event triggers are properly registered.

5 - Test!

Clicking Redeliver from the GitHub repository webhook configuration will send an event and broadcast the message to all listening jobs. The Pipeline job created in the previous step will see this event, and trigger a build!

[Pipeline] {
[Pipeline] stage
[Pipeline] { (Example)
[Pipeline] sh
[Github Trigger] Running shell script
+ echo Hello World
Hello World
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS

This is a very simple example illustrating the basic steps needed to integrate external webhook events with Jenkins. For more details about writing build triggers, please refer to Cross Team Collaboration

Troubleshooting

There are a number of steps that can be helpful when troubleshooting webhooks:

  • Endpoint "Last Event"/"Last Status" fields do not show any webhook request was received

    • Verify the endpoint url is accessible outside the firewall

    • Verify that the external system that is generating webhook events is configured with the correct URL for your endpoint

  • Verify that the external system is using the expected mime-type of application/json

    • Click Details to view the error message associated with the most recent request

  • Endpoint receives a message, but the "Last Status" shows an error code

    • Click Details to view the error message associated with the most recent request

  • Endpoint successfully receives a message, but build jobs are not triggering as expected

    • Verify that Notification is enabled and configured

    • Verify that your JMESPath trigger query is correct

    • Create a new log recorder for com.cloudbees.opscenter.plugins.notification.endpoint.WebhookEndpoint and set the level to FINEST in order to view the content of the webhook message received by Jenkins. This can be helpful in creating and troubleshooting event trigger queries.

Note
The Details view can be very helpful when troubleshooting webhook problems. In addition to allowing you to verify the endpoint configuration, the details view will show additional information about the type of error you may be encountering.