# Deveel Webhooks The **Deveel Webhooks** framework is composed of a set of libraries that can be used, at different degrees, to implement a system that allows applications to send and receive webhooks. ## Project Direction **This framework is maintained for the long term.** In parallel, part of its functionality is being gradually migrated to [Deveel Events](https://events.deveel.org/) ([GitHub repository](https://github.com/deveel/deveel.events)), which is closer to the broader Domain-Driven Design (DDD) concept of domain events. In that perspective, webhooks are one of multiple event integration patterns. * [**Sending Webhooks**](/send_webhooks) - Using libraries of the framework you can send webhooks to receivers, based on your own logic and rules. * [**Receiving Webhooks**](/receivers) - The framework provides a set of libraries implementing the capabilities for receiving webhooks from senders and reacting to events from external systems (such as Twilio, SendGrid, Facebook, etc.). * [**Notifications**](/notifications) - The framework provides a set of libraries that can be used to manage subscriptions to events and notify subscribing applications of events that occurred in your system. *** Read more about this framework: | Topic | Description | | --------------------------------------- | ----------------------------------------------------------------- | | [**Concepts**](/concepts) | A list of basic concepts used in the framework | | [**Getting Started**](/getting-started) | A quick guide to start using the framework | | [**Sending Webhooks**](/send_webhooks) | Sending webhooks messages to receivers | | [**Receiving Webhooks**](/receivers) | Receiving webhooks from senders | | [**Notifications**](/notifications) | Notify subscribing applications of events occurred in your system | # Getting Started The overall design of this framework is open and extensible (implementing the traditional [Open-Closed Principle](https://en.wikipedia.org/wiki/Open%E2%80%93closed_principle)), which means base contracts can be extended, composed, or replaced. It is possible to use its components as they are provided or use the base contracts to extend single functions, while still using the rest of the provisioning. ### Sending and Receiving The framework provides three major capabilities to the applications using its libraries
CapabilityDescription
Send WebhooksSend a Webhook message to a receiver, enforcing formatting, integrity and retry rules
Notify WebhooksCommunicate the occurrence of an event in a system to an external application that is listening for those events
Receive WebhooksAccepts and processes a notification from an external system, to trigger any related process
The two sending capabilities (*send* and *notify*) are disconnected from the receiving capability, since they represent two different parts of the communication channel (the *Sender* and the *Receiver*): as such the architecture of the framework is designed so that they don't depend on each other's. # Concepts The following concepts are used in this project: | Topic | Description | | -------------------------------------------------------------------------------------------------------------- | ------------------------------------------ | | [**Webhook**](/concepts/webhook) | What is it a 'Webhook' and why I need it? | | [**Subscriptions**](https://github.com/deveel/deveel.webhooks/blob/main/docs/concepts/webhook-subscription.md) | How does a subscription to an event works? | | [**Senders**](/concepts/webhook-sender) | What is a sender of webooks? | | [**Receivers**](/concepts/webhook-sender) | What is a receiver of webooks? | | **Notifications** | What is a notification of events? | # What is a Webhook? Briefly, we can say that a webhook is the notification of an event that has occurred somewhere, and that is sent to a listening application that is interested in such event. ## The Event Model If we consider the overall model of *processes*, where an activity is performed, and eventually preceding other activities, until a final result, we can define the *events* as the triggers of such activities (eg. *something has occurred, and therefore we must perform some subsequent activities*). Equally we can consider that some activities *produce* the information that an event has occurred, to make aware interested actors to react (eg. *while performing this activity, this has happened: feel free to act consequentially*). If you think about it, this happens in your daily life as well: * *Today I **woke up** and I **brushed my teeth*** - You are informing the listener that two events occurred today: you (the actor) woke up and you brushed your teeth * *While I was crossing the street, a car **horned** at me* - You are informing the listener that *a car* (the actor) horned at you ### Standard Models Academic studies have analyzed this concept more than a century ago, and some formal representations of such concepts have been designed and adopted by the *Information Technology* industry as standards (eg. [*BPMN*](https://en.wikipedia.org/wiki/Business_Process_Model_and_Notation), [*Workflows*](https://en.wikipedia.org/wiki/Workflow), etc.), in order to rationalize and optimize processes within the information systems (although, not all activities are actually performed by systems: some *actors* in such processeses are humans, and such scenarios we talk about *manual events*). For example, one of the most common of these formal models, the [*Business Process Management Notation (BPMN)*](https://en.wikipedia.org/wiki/Business_Process_Model_and_Notation), that defines several types: * *Triggers* - Those who start a process or an activity(like *a message* or *a timer*), * *Intermediate Events* - Those produced during an activity of a process * *Terminal Events* - Those that cause the end of the process ## Event-Driven Design The development and operational model of systems has greatly benefit by the adoption of designs where those systems were loosely coupled with other systems, implementing asynchronous operations based on the occurrence of events produced elsewhere: this approach, that today seems a *given* for many developers, has optimized the overall performances and maintainability of services, which are able to isolate their functions, and instead of *pulling* the event at regular intervals (eg. *querying an external service every X minutes to see if anything has happened*), can be resiliently *listening* and *reacting* only when needed. This design has also produced extensibility opportunities, since it made it possible for a system to be more easily integrated (once the events and their information is known), since the dependency from the components is not direct and the load is moved to the transportation medium, rather than being provided by the service itself (eg. *the I/O to the databases is reduced or removed, since the event carrying all needed information has been produced*). ## The Webhook Model Moving out of the conceptual model, the initial problem faced to export *events* out of the boundaries of the systems of a service provider to external listening applications consisted in the format and protocol of the transportation. The best solution found along the way (and at today still the most used) to notify such events is through *HTTP callbacks*: a *POST* request through the *HTTP protocol* to a publicly exposed end-point belonging to an application *accepting* such notifications. This methodology has been commonly denominated by the industry as ***Webhook***. At today, there is still no agreed common protocol to define the format and contents of a *Webhook*, although some elements represent a pattern across the various implementations available: * The callback request is performed using the *POST* verb of the *HTTP* protocol * The payload (the content) of the request is formatted as a *JSON* or *XML* text (although some implementations use the *www-form-urlencoded* format) * The request *optionally* includes an header or a query string element that expresses a *signature* that is interpretable by the subscribing application (to be sure of the genuinity of the information) ### CloudEvents In the last years, the [*Cloud Native Computing Foundation (CNCF)*](https://www.cncf.io/) has been working on a standardization of the *Webhook* model, and has produced the [*CloudEvents*](https://cloudevents.io/) specification, that defines a common format for the payload of the *Webhook* requests, and a set of *extensions* that can be used to enrich the information carried by the event. The *CloudEvents* specification is still in its early stages, and it is not yet widely adopted by the industry, but it is a good starting point to define a common model for the *Webhook* events. # What is a Webhook Subscription? It is easy to define the concept of subscription to the notification of a certain type of events, and we have several examples of this in our daily life: * We can subscribe to a newspaper to receive it every day, * We can be opt-in to be notified when a new product is available in a shop by a phone call or an email * We can subscribe to a newsletter to be notified when a new article is published on a blog In all these cases, we are interested in being notified when something happens, and we are not interested in the details of the event itself: some of the elements of the event are relevant to us, and some of these elements are relevant for the publisher to know how and when to reach us. For the other party that has to notify us on the type of events we are interested in, it is important to know how to reach us, and it is important to know what we are interested in. In the case of a newspaper, the publisher needs to know our address, and the type of newspaper we want to receive. In the case of a shop, the publisher needs to know our phone number or email address, and the type of products we are interested in. In the case of a blog, the publisher needs to know our email address, and the type of articles we are interested in. ## Webhook Subscription Information systems work partially in the same way: as owners of a system, we can subscribe to a certain type of events ocurring in an external system, and we can specify the endpoint, reachable from a HTTP channel, where the system will be notified when these events happen. The publisher system is then requested to keep a record of all these information, so that when an event occurs, it can notify all the subscribers that are interested in that event, given a set of criteria. In fact, a system can assume both roles of publisher and subscriber, where it can be notified of events from an external system, and then notify other systems of events occurring in its own scope (or in some cases just routing the events received). Subscriptions to events are typically indicating used by publisher to control the behavior of the notification process. Although no formal specification for this type of objects was agreed and standardized, a typical pattern across the providers of services is to define the following elements: | Attribute | Description | | ------------------------ | ---------------------------------------------------------------------------------------------------------- | | **Subscription ID** | A unique identifier of the subscription, used to identify the subscription in the system | | **Subscription Name** | A human-readable name of the subscription, used to identify the subscription in the system | | **Event Type** | The type of event the subscription is interested in | | **Event Criteria** | A set of criteria that the event must satisfy in order to be notified to the subscriber | | **Destination Endpoint** | The endpoint where the subscriber will be notified of the event | | **Status** | The status of the subscription, which can be `ACTIVE`, `INACTIVE`, `EXPIRED`, `DELETED` | | **Headers** | A set of headers that will be sent to the subscriber when the event occurs (usefull to identify a context) | | **Secret Key** | A secret key that will be used to sign the notification to the subscriber | # What is a Sender of Webhooks? Considering the overall network of communications, where Webhooks are messages exchanged between two applications, a Webhook Sender is an application that sends these messages to another application (or to a multitude of applications). In real life, you might consider the *Sender* as the postman delivering a letter or a package to the destination, ensuring the address is correct and exists, that the postbox can hold the shipment, eventually coming back more than once at the address (if the recipient is not at home), accepting the signature, etc. As much as a postman wouldn't care if the letter or package was in delivery because of an agreement between the recipient and an organization, the same way a Webhook Sender doesn't have to necessarily require the knowledge that an application is expecting the message it is sending (this would be a [Webhook Subscription](/concepts/webook-subscription)). ### The Delivery Destination A Sender requires the specification of a Destination, to be able to attempt the delivery of Webhooks, which is like a postal address, following the example above. In most cases, this is simply a URL (*Universal Resource Locator*), that can be reached by an HTTP request, but in some scenarios, this can be enriched with further information instructing the sender on the behavior to have with it, when attempting to deliver a package (for example, we inform the sender to leave the package to our neighbor, if we are not at home, or to try reaching us only on Thursdays, etc.). Such destination-specific instructions would act as an exception to the regular behavior of the sender, which is described below. ### The Sender Behavior When sending a Webhook to another party, the application sending messages should ensure that the destination is reachable, the message is well-formatted, its delivery is retried (in case of failures), and the integrity of the content is assured. This is done by a configured behavior, that typically provides the following elements of configuration
AttributeDescription
Content TypeThe specification of the format of the contents of the Webhook message (eg. JSON, XML, Form-Encoded)
Retry StrategyThe methodology to deliver the message (eg. Circuit Breaker, Exponential Backoff, Timeout, etc.), and the configurations of retries (eg. a maximum number of retries, delays between attempts, etc.)
Signature MethodA methodology used to sign the Webhook payloads, so that the the receiver can ensure their integrity
ContextThe set of contextual information informing the receiver of the origination of the Webhook (eg. the machine name, the application type, etc.)
# What is a Receiver of Webhooks? As we have cleared out before, Webhooks are messages that inform an application of the occurrence of an event in a system: as such we consider the context of communications, where there are two (or more) parties sending and receiving messages between each other. In a specular way to the definition of the Webhook Sender, of which we made an analogy with a postman delivering a letter or a package, the Webhook Receiver can be considered the recipient (a person or a company), that physically receives the deliverable. Considering the real-life scenario of delivery of letters or packages, we all can be receivers, if we provide a postbox with our name on it, that we make available for postmen to access. In some special cases of secure shipments, we might also be requested to sign a receipt to ensure that the packages or envelopes were actually delivered. # What is an Event Notification? # Sending Webhooks The simple act of sending Webhooks to receivers is not dependent on the existence of subscriptions and can be executed through direct invocations to instances of the `IWebhookSender` service. In fact, the Webhook Sender component of the framework provides the following capabilities: * **Serialization of the Payload** - The content of the payload of the webhook is serialized according to the configurations and format (eg. *JSON*, *XML*, *Form-Encoded*, etc.) * **Signature** - The sender computes a signature of the webhook payload, to provide the receiver a proof of its integrity * **Delivery Retry** - The delivery of the webhooks is retried until successful, or until a breaking condition is met This component doesn't provide any capabilities for managing subscriptions to event notifications, or to automate the notification of the events: see the Webhook Notification chapter for learning how to activate it. ## Install the Required Libraries The overall set of libraries are available through [NuGet](https://nuget.org), and can be installed and restored easily once configured in your projects. ### Requirements The library currently suppots both the `.NET 6.0` and `.NET 7.0` runtimes. ### Installing the Package You can do this by using the .NET command line on the root folder of your project ```bash dotnet add package Deveel.Webhooks.Sender ``` Alternatively, you can add a reference in your project file ```xml ne7.0 ... ... ``` ### Registering the Webook Sender The most common way to use the Webhook Sender is to register it in the collection of services of your application, using the *dependency injection* pattern to obtain it. You can use one of the overloads of the extension method `.AddWebhookSender()` to the `IServiceCollection` contract, which will return a builder object that can be used to configure the service. For example: ```csharp services.AddWebhookSender() .Configure(options => { // ... }); ``` or even simplier: ```csharp // To use the default configurations services.AddWebhookSender(); ``` This method will register the default implementation of the `IWebhookSender`, returning a builder object for further configurations of the service. See the [configuration chapters](/send_webhooks/configuring-the-sender) for further information about the available options to configure the sender. ### The Webhook Scope The Webhook Sender service is scoped to the type of the webhook: this means that you can register multiple instances of the service, each one for a specific type of webhook. You will find that this is useful when you need to send different types of webhooks, with different configurations, using the same infrastructure or application. The same scoping mechanism is inherited by the `IWebhookNotifier` service: see the documentation of the [Webhook Notification](/notifications) chapter for further information. #### Example Registration Assuming you are working on a traditional [*ASP.NET application model*](https://docs.microsoft.com/en-us/aspnet/core/fundamentals/?view=aspnetcore-6.0\&tabs=windows), you can start using the sender functions of this framework through the *Dependency Injection* (DI) pattern, including a default implementation of the sender during the Startup of your application, by calling the `.AddWebhooks()` extension method provided by *Deveel Webhooks*, and optionally configuring the behavior of the delivery: ```csharp using System; using Microsoft.Extensions.Configuration; using Microsoft.Extensions.DependencyInjection; using Deveel.Webhooks; namespace Example { public class Startup { public Startup(IConfiguration config) { Configuration = config; } public IConfiguration Configuration { get; } public void Configure(IServiceCollection services) { // ... add any other service you need ... // this call adds the basic services for sending of webhooks services.AddWebhookSender(webhooks => { // Optional: if not configured by you, the service // will use the default configurations.. webhooks.Configure(options => { // ... configure the options ... }); }); } } } ``` ## Using the Webhook Sender At this point, you can obtain from the service provider an instance of the `IWebhookSender` that is configured and ready to be used. If you have not overridden the default sender service (using the `.UserSender()` method of the builder object), you will also have an instance of the `WebhookSender` client service available. #### Example Usage Assuming your application is using the *ASP.NET Core* framework, you can inject the service in your controllers, and use it to send Webhooks to the receivers: ```csharp using System; using Deveel.Webhooks; using Example.WebModels; namespace Example { [ApiController] [Route("webhook")] public class WebhookController : ControllerBase { private readonly IWebhookSender webhookSender; public WebhookController(IWebhookSender webhookSender) { this.webhookSender = webhookSender; } [HttpPost] public async Task Post([FromBody] WebhookModel webhook) { // First you must transform the model to a MyWebhook instance // that is compatible with the sender.. we assume that the // you are using the Adapt() extension method provided by // Mapperly ... var myWebhook = webhook.Body.Adapt(); var destination = webhook.Destination.Adapt(); // The service sends the webhook to the destination address // according to the configurations provided var result = await webhookSender.SendAsync(destination, myWebhook, HttpContext.RequestAborted); // The result of the send is not serializable and requires // a transformation to a compatible object var resultModel = result.Adapt(); return Ok(resultModel); } } } ``` ### Webhook Destinations While q webhook represents a message that is sent to a receiver, the Destination is the address of the receiver, complimented with additional configurations that can override the default configurations of the sender service. In the context of *simple sending* of Webhooks, the destination is represented by an instance of the `WebhookDestination` structure. ```csharp var destination = new WebhookDestination("https://my-webhook-receiver.com/events/webhooks") .WithRetry(options => { options.RetryCount = 3; options.RetryDelay = TimeSpan.FromSeconds(5); }); var result = await webhookSender.SendAsync(destination, myWebhook); ``` ### Webhook Delivery Results The result of the sending of a Webhook is represented by an instance of the `WebhookDeliveryResult` class, that provides an aggregation of the delivery attempts done by the sender. The sender can perform multiple attempts to deliver the message to the receiver (depending on the retry configuration), and the result of the delivery is represented by an instance of the `WebhookDeliveryAttempt` class: a result can be considered successful if at least one attempt was successful, and it can be considered failed if all the attempts failed. You can use the `WebhookDeliveryResult` class to check the status of the delivery, and to retrieve the results of the attempts. # Configuring the Sender ### Configuring the Delivery Behavior The framework provides a default behavior for the delivery of webhooks to the registered recipients, but you can configure the behavior of the service to suit your needs. The library `Deveel.Webhooks` depends from the `Deveel.Webhooks.Sender` library, which provides the needed functions to send webhooks to receivers: you will find the abstractions and helpers to configure the behavior of the webhook delivery to recipient systems, controlling aspects of the process like *payload formatting*, *retries on failures*, *signatures*. You have several options to configure the service, and therefore you are free to chose the methodology that suits you best. ### The WebhookSenderOptions The `WebhookSenderOptions` class provides a set of properties that can be used to configure the behavior of the webhook sender. The configurations are evolving with the versions of the framework, and the following ones apply to the current version (`1.1.6`) of the library. ```csharp var options = new WebhookSenderOptions { // When using the IHttpClientFactory, this is the name of the client // that will be used to send the webhooks HttpClientName = "my-http-client", // A set of default headers that will be added to the requests // sent to the webhook recipients DefaultHeaders = new Dictionary { // The default headers that will be added to the requests // sent to the webhook recipients ["X-Sender"] = "My-Webhook-Sender/1.0" }, // The default format of the payload that will be sent to the // webhook recipients (possible values are Json and Xml) DefaultFormat = WebhookPayloadFormat.Json, // The default timeout for the requests sent to the webhook // recipients to be completed, before being considered failed Timeout = TimeSpan.FromSeconds(30), // The default retry options for the requests sent to the webhook // recipients, that can be overridden by the specific subscription // configurations Retry = new WebhookRetryOptions { // The default number of retries that will be performed // after a failed request, before giving up MaxRetries = 3, // The default delay between retries Delay = TimeSpan.FromSeconds(5), // The default timeout for each retry request // before being considered failed Timeout = TimeSpan.FromSeconds(3) }, // The default signature options for the requests sent to the webhook // recipients, that can be overridden by the specific subscription // configurations Singature = new WebhookSenderSignatureOptions { // The default location within the request where the signature // will be added (possible values are Header and QueryString) Location = WebhookSignatureLocation.Header, // The default name of the header that carries the signature, // when the location of the signature is the Header HeaderName = "X-Signature", // The default name of the query string parameter that carries // the signature, when the location of the signature is the QueryString QueryParameter = "signature", // The default algorithm that will be used to sign the requests // sent to the webhook recipients Algorithm = WebhookSignatureAlgorithm.HmacSha256, // The name of the query string parameter that will be used to // specify the algorithm used for the signature AlgorithmQueryParameter = "alg_sig" } }; ``` Mind that every subscription can override some of the default options, and you can also provide a custom implementation of the `IWebhookSender` interface to customize the behavior of the delivery process. ### IConfiguration Pattern If you keep your configurations in an `appsetting.json` file, environment variables, secrets, implementing a typical pattern of the *ASP.NET* applications, you can invoke the overloads provided by `Deveel.Webhook` that access the available instances of `IConfiguration`. ```csharp using System; using Microsoft.Extensions.Configuration; using Microsoft.Extensions.DependencyInjection; using Deveel.Webhooks; namespace Example { public class Startup { public Startup(IConfiguration configuration) { Configuration = configuration; } public IConfiguration Configuation { get; } public void Configure(IServiceCollection services) { // The configurations are specified in the 'Webhooks:Sender' // section of the configuration instance, and the service // will find the IConfiguration instance within the // container and use it to configure the service services.AddWebhookSender("Webhooks:Sender"); } } } ``` After this, an instance of `IOptions` is available for injection in the webhook services or in your code. Given the design of the service, it will also be possible to access a webhook-specific instance of `IOptions` for a given webhook type, using the `IOptionsSnapshot` service, using the type name of the webhook as the key (eg. `IOptionsSnapshot.Get("MyWebhook")`). ### Manual Configuration If you prefer to configure the service manually, you can use the `AddWebhooks` overload that accepts an instance of `WebhookSenderOptions` as parameter. ```csharp using System; using Microsoft.Extensions.Configuration; using Microsoft.Extensions.DependencyInjection; using Deveel.Webhooks; namespace Example { public class Startup { public Startup(IConfiguration configuration) { Configuration = configuration; } public IConfiguration Configuation { get; } public void Configure(IServiceCollection services) { services.AddHttpClient("my-http-client"); services.AddWebhookSender(options => { options.HttpClientName = "my-http-client"; }); } } } ``` Like in the previous case, an instance of `IOptionsSnapshot` is available for injection in the webhook services or in your code. # Signing Webhooks A recommended practice for Webhooks is to sign the payloads of the messages so that the receiver can verify the authenticity and integrity of the message. The framework provides a mechanism to sign the payloads of the messages sent (as *Sender*) and to verify the signatures of the incoming messages (as *Receiver*). Signature providers are implementations that use the payload of a webhook message and a secret key, to compute a signature to be attached to the webhook, by implementing the `IWebhookSigner` service contract. By default, when registering a Webhook Sender service, the framework also registers an implementation of a signature provider for the '*HMAC-SHA-256'* algorithm: it is possible to add custom ones by calling the method `.AddSigner()` of the service builder. ```csharp using System; using Microsoft.Extensions.Configuration; using Deveel.Webhooks; namespace Example { public class Startup { public Startup(IConfiguration config) { Configuration = config; } public IConfiguration Configuration { get; } public void Configure(IServiceCollection services) { // ... add any other service you need ... // this call adds the basic services for sending of webhooks services.AddWebhookSender(webhooks => { // Optional: if not configured by you, the service // Add a custom signature provider webhooks.AddSigner(); }); } } } ``` # Webhook Serialization The most common format for the payloads of Webhooks is the JSON format, but it is possible to use also the XML format, as long as the receiver is able to understand it. The framework provides a mechanism to serialize the payloads of the messages sent to the receivers. The framework provides a set of default implementation of the serializers
TypeLibraryDescription
SystemTextWebhookJsonSerializerDeveel.WebhooksAn implementation that uses the System.Text.Json serialization functions (recommended)
NewtonsoftWebhookJsonSerializerDeveel.Webhooks.NewtonsoftJsonImplements the webhook serialization by using the Newtonsoft.Json serializers
SystemWebhookXmlSerializerDeveel.WebhooksSerializes webhooks to XML using the System.Xml native functions
When initializing the sender, a default serializer for each format is added to the service, but it is possible to add custom ones through the call to `.UseJsonSerializer()` or `.UseXmlSerializer()` of the service builder. ```csharp using System; using Microsoft.Extensions.Configuration; using Deveel.Webhooks; namespace Example { public class Startup { public Startup(IConfiguration config) { Configuration = config; } public IConfiguration Configuration { get; } public void Configure(IServiceCollection services) { // ... add any other service you need ... // this call adds the basic services for sending of webhooks services.AddWebhookSender(webhooks => { webhooks.UseJsonSerializer(); webhooks.UseXmlSerializer(); }); } } } ``` # Webhook Notifications The notification process of a webhook introduces elements of automation, putting together the [*subscription management*](/notifications/webhook-subscriptions-management) and the [*sending of webhooks*](https://github.com/deveel/deveel.webhooks/blob/main/docs/sending-webhooks/README.md) processes, and an optional step of [*data transformation*](https://github.com/deveel/deveel.webhooks/blob/main/docs/notifications/custom_datafactory.md) (to resolve event information in a fully formed object to be transferred). This process is dependent on some components: * **Webhook Subscriptions** - To be able to notify a webhook, the notifier must be aware of which application has subscribed to the notification of the given event. * **Webhook Factory** - The event that occurred must be converted into complete information to be notified. * **Sending Webhooks** - The webhooks must be delivered to the destination, through serialization, signature, and retries * **Logging Delivery Results** - In scenarios of usage as background service, the logging of the results of the delivery process provides observability of the performances of the notification ## Install the Required Library The overall set of libraries are available through [NuGet](https://nuget.org), and can be installed and restored easily once configured in your projects. ### Requirements To implement the core functionalities of webhook notifications, you must install the `Deveel.Webhooks` library. The library currently supports both the `.NET 6.0` and `.NET 7.0` runtimes. ### Install the Package You can install it through the `dotnet` command line, using the command ```sh $ dotnet add package Deveel.Webhooks ``` Or by editing your `.csproj` file and adding a `` entry. ```xml ne7.0 ... ... ``` ## Registering the Webhook Service To begin using the functions of the webhook service, all that you need is to invoke the `.AddWebhookNotifier()` function of the service collection in the instrumentation of your application: this will add the required default services to the container, enabling the application to start notifying events to the registered recipients. For example: ```csharp var builder = services.AddWebhookNotifier(); ``` The method above registers the default services required to notify webhooks, and returns an instance of `WebhookNotifierBuilder` that can be used to configure the behavior of the service. It also registers a default implementation of the `IWebhookNotifier` service, that can be used to trigger the notification process: please refer to the [specific chapter](/notifications/webhook_notifier) to learn more about this service. ### The Webhook Scope The Webhook Notifier service is designed to be scoped to a specific type of webhook, isolating services like the subscription resolver, the webhook factory, or the sender service to the specific type of webhook. This allows to have multiple instances of the service, each one dedicated to a specific type of webhook, and each one with its own configuration, running in the same application. #### Example Registration For example, assuming you are working on a traditional [*ASP.NET application model*](https://docs.microsoft.com/en-us/aspnet/core/fundamentals/?view=aspnetcore-6.0\&tabs=windows), you can enable these functions like this: ```csharp using System; using Deveel.Webhooks; using Example.WebModels; namespace Example { public class Startup { public Startup(IConfiguration config) { Configuration = config; } public IConfiguration Configuration { get; } public void Configure(IServiceCollection services) { // ... and use the MongoDB webhook subscription layer ... services.AddWebhookSubscriptions(subs => subs.UseMongo(Configuration.GetConnectionString("MongoWebhooks"))); // this call adds the basic services for sending of webhooks services.AddWebhookNotifier(webhooks => { // by default a IWebhookSubscriptionResolver service // is registered, that is a wrapper around the IWebhookSubscriptionRepository available // from the previous call... // Optional: add a filter engine that handles string-based "linq" filters webhooks.AddDynamicLinqFilters(); // Optional: configure the delivery behavior of the sender service webhooks.ConfigureDelivery(delivery => { ... }); }); } } } ``` ### The `IWebhookNotifier` Service The *Deveel Webhook* framework implements the notification functions through instances of the `IWebhookNotifier` service, which is designed to trigger the notification process from an Event. Registering the service also registers by default the `WebhookNotifier` service, which implements the notification process by depending on external services for the Webhook Subscription resolution, the building of Webhooks, and the logging of Delivery Results. During the configuration of the service, it is possible to replace the default implementation with a custom one, or to configure the default one with additional services, using the instance of the `WebhookNotifierBuilder` returned by the `.AddWebhookNotifier()` method. See the [specific chapter](/notifications/webhook_notifier) to learn more about the default service. ### Using the Webhook Notifier (ASP.NET Core Example) Once your application's service collection has been built, an instance of the `IWebhookNotifier` service will be available through Dependency Injection. Assuming you are running a ASP.NET Core service, using a traditional MVC model. ```csharp namespace Example { [ApiController] [Route("webhook")] public class WebhookController : ControllerBase { private readonly IWebhookNotifier webhookNotifier; public WebhookController(IWebhookNotifier webhookNotifier) { this.webhookNotifier = webhookNotifier; } [HttpPost("{tenantId}")] public async Task Post([FromRoute]string tenantId, [FromBody] EventModel webEvent) { // The service requires an instance of 'EventInfo' // to trigger the notification process and we assume // your EventModel class can create one... var eventInfo = webEvent.AsEventInfo(); // The service then tries to notify the event to the // registered subscribers, and returns an aggregation of // all the results of the delivery process var result = await webhookNotifier.NotifyAsync(eventInfo, HttpContext.RequestAborted); // The object returned by the notifier is not serializable and // to return the object through ASP.NET you need an instance // supporting serialization var resultModel = WebhookNotificationResultModel.FromResult(result); return Ok(resultModel); } } } ``` ## Notifications for Multi-Tenancy Scenarios # Webhook Subscriptions Management # Data Layers The persistence of information object for long term operations is based on the implementation of a set of contracts of the management domain of the service. Since the native support of multi-tenancy of the information, a model is in place to create tenant-specific contexts, provided throug a 'store provider' pattern. ## Storage Contracts The main contracts used to implement this persistence are the following: | Interface | Description | | ------------------------------------- | -------------------------------------------------------------------------------------------- | | `IWebhookSubscriptionStore` | Implements the functions to manage the storage of `IWebhookSubscription` information | | `IWebhookSubscriptionStoreProvider` | Provides the means to instantiate a tenant-specific context owning the webhook subscriptions | | `IWebhookDeliveryResultStore` | Implements the functions to manage the storage of webhook delivery results | | `IWebhookDeliveryResultStoreProvider` | Creates tenant-specific scopes for the storage of the webhook delivery results | ## Subscription Resolution Although the resolution of the webhook subscriptions is not directly related to the storage of the information, the framework provides a contract to implement the resolution of the subscriptions based on the storage. In fact, the `IWebhookSubscriptionResolver` interface is used by the framework to resolve the subscriptions to a specific event, and in the default implementation, it uses the `IWebhookSubscriptionStore` to retrieve the information. ## Delivery Logging Some advanced scenarios of usage may require to log the delivery results of the webhook notifications, to provide a way to track the delivery status of the notifications. Even if the logging mechanism doesn't require a specific database (it might be logging on CSV files, JSON, etc.), the framework provides the contract `IWebhookDeliveryResultStore` to implement the storage of the delivery results. ## Storage Implementations The *Deveel Webhooks* framework provides the following implementations of the storage: | Implementation | Description | Subscription Store | Delivery Logging | Multi-Tenant | | --------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ | :------------------: | :------------------: | :------------------: | | [`MongoDB Storage`](https://github.com/deveel/deveel.webhooks/blob/main/docs/notifications/advanced_usage_mongodb.md) | Implements the storage of the webhook subscriptions and delivery results using MongoDB | :white\_check\_mark: | :white\_check\_mark: | :white\_check\_mark: | | [`Entity Framework`](https://github.com/deveel/deveel.webhooks/blob/main/docs/notifications/advanced_usage_ef.md) | An implementation that stores subcriptions and delivery results using the Entity Framework | :white\_check\_mark: | :white\_check\_mark: | :x: | Refere to the specific documentation of each implementation for more details. # Entity Framework Layer The [Entity Framework](https://learn.microsoft.com/en-us/ef/) is an advanced [ORM](https://en.wikipedia.org/wiki/Object%E2%80%93relational_mapping) for the .NET enviroment that allows the abstraction of the domain model of the application from the database-specific commands. The *Deveel Webhooks* framework provides an implementation of the storage layer that uses the Entity Framework to store the data, primarily in relational in databases: it is not intended to be used with a specific SQL database vendor. The implementation of the storage layer is provided by the `Deveel.Webhooks.EntityFramework` package. ### Installation The package is available on [NuGet](https://www.nuget.org/packages/Deveel.Webhooks.EntityFramework) and can be installed using the following command from the NuGet Package Manager Console: ```powershell Install-Package Deveel.Webhooks.EntityFramework ``` Alternatively, you can use the `dotnet` CLI: ```bash dotnet add package Deveel.Webhooks.EntityFramework ``` ### Configuration Once the package is installed, you can configure the storage layer in the `Startup` class of your application, by using the `AddEntityFrameworkStorage` extension method: ```csharp public void ConfigureServices(IServiceCollection services) { // ... services.AddWebhookSubscriptions(webhook => { webhook.UseEntityFramework(ef => { ef.UseContext(options => { options.UseSqlServer(Configuration.GetConnectionString("Webhooks")); }); }); }); // ... } ``` The `UseEntityFramework` method accepts a callback that allows to configure the storage layer: the callback is provided with an instance of `EntityFrameworkStorageBuilder` that allows to configure the storage layer. One of the methods of the builder is `UseContext` that allows to configure the database context to use: the callback is provided with an instance of `DbContextOptionsBuilder` that allows to configure the database context: you can provide your own implementation of the database context, or use one of the pre-defined implementations provided by the framework. #### Pre-defined Database Context The framework provides a pre-defined database context `WebhookDbContext` that can be used to configure the storage layer: this applies the default entity configurations. # The WebhookNotifier Service As described in the [previous chapter](/notifications), when registering the Webhook Notifier service, a number of services are registered in the container, that are used to manage the subscriptions and to deliver the webhooks to the receiving end-point. An default implementation of the `IWebhookNotifier` service is also registered by the framework, which is the `WebhookNotifier` class. This implementation executes the following steps: 1. Resolution of any webhook subscriptions matching the *event type* and the *tenant identifier* (*in case of multi-tenant scenarios*). 2. Transformation of the event into an instance of the webhook, using any service implementing `IWebhookDataFactory` registered, that can transform or normalize the event information (eg. *resolving a database record from an identifier*), for each of the subscription resolved. 3. Filtering the subscriptions mathing the conditions and criteria configured, against the webhook object constructed in the previous step 4. Attempt to deliver the webhooks to the receiving end-point of the subscription, eventually retrying in case of failures 5. Optional logging of the results of the delivery of the webhooks, when a service implementing `IWebhookDeliveryResultLogger` is registered in the container ### Service Dependencies
ServiceDescription
IWebhookSubscriptionResolver<TWebhook>Resolves the subscriptions to an event. It is generally provided externally, since it's generally tied to the Webhook Subscription service. Anyway, a default implementation of this service is registered, that is a wrapper around any registered IWebhookSubscriptionRepository<TSubscription> in the container.
IWebhookFactory<TWebhook>Transforms the event into a webhook object of the type supported by the service, and that will be then notified.
IWebhookFilterEvaluator<TWebhook>The service used to evaluate the filters of the subscriptions. When none is provided at the registration, and webhook subscriptions define any filter, the notification to those subscribers will fail. See this chapter for more information
IWebhookDeliveryResultLogger<TWebhook>When available in the context of the application, logs the results of the delivery of the webhooks
## Custom Webhook Notifiers The `WebhookNotifier` service is registered as `IWebhookNotifier` in the container, and can be resolved as such: in fact it doesn't have any specific extensions to the original contract. If you plan to implement your own webhook notifier, you can do so by implementing the `IWebhookNotifier` interface, and registering it in the container, replacing the default implementation, or by inheriting from the `WebhookNotifier` class, and overriding the methods you need to customize (that will save you time and preserve from design issues). You can replace the default implementation of the `IWebhookNotifier` service by registering your own implementation in the container, as follows: ```csharp services.AddWebhookNotifier() .UseNotifier(); ``` All other default services will still be registered, and you can still use them in your custom implementation, by injecting them in the constructor of your class. # Webhook Factories In notification scenarios triggered by an event, the webhooks to be notified must be built and delivered to subscribers, and this construction process might need to *transform* the original data carried by those triggering events into a new object. In fact, sometimes events don't carry all the information that has to be transmitted to the receivers, for several reasons (eg. *privacy*, *design*, *external context*, etc.), and this requires an additional intervention for the integration of that information (eg. *resolving an entity from the database*, *appending the environment variables of the notifier*, etc.). *Note*: This is not a mandatory passage in the notification process, and can be skipped if the data carried by the event represents the information to be transferred to the receivers. ### Event Information The overall contract of an event as recognized by the system is defined by the `EventInfo` structure, that is composed of the following fields:
FieldTypeDescription
IdstringThe identifier of the event
SourcestringThe source of the event (eg. github)
SubjectstringThe subject of the event (eg. issue)
TypestringThe type of the event (eg. created)
TimeStampDateTimeOffsetThe exact time the event has occurred in the source system
DataobjectThe payload that contains the actual data of the event
This design respects a general contract of the domain-driven design, informing of the event's nature and the payload that contains the actual data. #### Cloud Events An example of the implementation of this contract is the [Cloud Events](https://cloudevents.io/) specification, that defines a standard for the representation of events in a cloud-native environment. Anyway, despite the adherence to the overall design, the Deveel Webhooks framework is not tied to this specification, but it can be used to implement it, an it requires the EventInfo structure to be provided in order to be able to send notifications. ### Event Data As mentioned before, the event data is not always in the format that is expected by the target system that will be notified, and the resons to implement a further passage of transformation are various: * The event data might contain sensitive information, that should not be exposed to the target system * The event data might contain information that is not relevant to the target system * The event data might contain information that is not available in the event, but must be retrieved from external sources (eg.*database entries*, *environment variables*, etc.) ### Transforming the EventInfo to a Webhook: the `IWebhookFactory` interface The notifier service uses instances of the `IWebhookFactory` interface to transform a triggering event, into a webhook object that is expected by the subscribing applications, using a subscription-scoped transformation logic. Transformations are specific to your use cases and they can be specific to a given event condition (eg. *only a specific type of event with a given value in its 'data' component triggers the transformation*). To implement this logic in the application, you must first create a new class that inherits from the `IWebhookFactory` contract. ```csharp public class MyWebhookFactory : IWebhookFactory { private readonly IUserResolver resolver; public MyWebhookFactory(IUserResolver resolver) { this.resolver = resolver; } public Task CreateAsync(IWebhookSubscription subscription, EventInfo eventInfo, CancellationToken cancellationToken) { // Resolve the event data var userId = eventInfo.Data.GetString("userId"); var user = await resolver.GetUserAsync(userId, cancellationToken); var webhookType = $"user.{eventInfo.Type}"; // Transform the event data return new MyWebhook { Type = webhookType, User = new UserInfo { Id = user.Id, Email = user.Email, FirstName = user.FirstName, LastName = user.LastName } }; } } ``` ### Registering the Webhook Factory To enable the implementation by your custom Webhook Factory, you can register it during the configuration of the application. ```csharp using System; using Microsoft.Extensions.Configuration; using Microsoft.Extensions.DependencyInjection; using Deveel.Webhooks; namespace Example { public class Startup { public Startup(IConfiguration config) { Configuration = config; } public IConfiguration Configuration { get; } public void Configure(IServiceCollection services) { // ... add any other service you need ... // this call adds the basic services for sending of webhooks services.AddWebhookNotifier(webhooks => { // This call registers your custom webhook factory as a // singleton service by default, but other overloads // allow controlling the lifetime webhooks.UseWebhookFactory(); }); } } } ``` ### The Default WebhookFactory When registering the Webhook Notifier service, by default the framework will register a default implementation of the `IWebhookFactory` interface, if the type of `TWebhook` is derived from the `Webhook` class: this factory will attempt to create instances of the webhook type using the default constructor, and then it will try to map the properties of the webhook object to the properties of the event data and the subscription data. This approach is useful when the webhook object is a simple POCO that can be easily mapped to the event data, and it doesn't require any further transformation logic, but it's not suitable for more complex scenarios. # Filtering Webhook Subscriptions By design, webhook subscriptions are bound to a set of *event types*, and they are resolved on the occurrence of one or more of the events matching the type subscribed. It is possible to define *second-level filtering* on the subscription, applying filters that will be evaluated, for the notification to be delivered to the subscriber. Such capability is useful for scenarios like * Avoid sending unnecessary notifications to the receiver * Reduce the load on the receiver * Routing the delivery of the notifications to different receivers ## Webhook Filter Evaluators Webhook subscriptions might include additional filters, such as IWebhookSubscriptionFilter, specifying the format in which they are expressed: a matching service supporting that format must be present in the application, for the evaluation to be performed. These filtering conditions are evaluated by services implementing the `IWebhookFilterEvaluator` interface, that is in fact a filtering engine. By default, when no filtering service is registered in the application to support a specific format of the webhook subscription's filter, the notification service will fail, and will not deliver the notification to the receiver. #### Registering the Filter Service To enable the filtering capability, the `IWebFilterEvaluator` service must be registered through the notification service builder: ```csharp namespace Example { public class Startup { public void ConfigureServices(IServiceCollection services) { services.AddWebhookNotifier(webhooks => { // Register the filter evaluator service that // is using the "linq" syntax webhooks.UseDynamicLinq(); }); } } } ``` In the above code, we registered the `DynamicLinqFilterEvaluator`, which is a service provided in the [Deveel.Webhooks.DynamicLinq](https://www.nuget.org/packages/Deveel.Webhooks.DynamicLinq) package and that uses the [DynamicLINQ](https://dynamic-linq.net/) syntax to evaluate filters. ### Evaluating Webhooks The filtering engine is invoked with the instance of the webhook object to be delivered, and the filtering expressions are evaluated against its structure and data: keep in mind this when creating the filtering conditions. The representation of the webhook is dependent on the implementation of the serialization service (eg. `System.Text.Json`, `Newtonsoft.Json` or `System.Xml`) and the format of the webhook payload (either `json` or `xml`): these serializer might have different behaviors when serializing the webhook object (such as attributes or properties, or the casing of the names), and the filtering conditions must be defined accordingly. ## LINQ Filters As mentioned above, the framework provides the `DynamicLinqFilterEvaluator` service, which provides filtering capabilities using the LINQ syntax, a very powerful and flexible syntax to define filtering conditions. You can install it by calling the following command on the root of your project: ```bash dotnet add package Deveel.Webhooks.DynamicLinq --version 2.1.5 ``` To enable it you will have to invoke the `.UseDynamicLinqFilters()`, like in the example above. #### Example Filter For example, consider a webhook that is serialized as a JSON object as the following one: ```json { "event_type": "user.created", "user_name": "antonello", "user_email": "antonello@deveel.com", "roles": ["admin", "user"], "timestamp": 1682197154054 } ``` The filtering expression ```csharp event_type == "user.created" && user_name.startsWith("anto") ``` will evaluate to `true` if the webhook is for the event `user.created` and the user name starts with `anto`. **Note** - The engine does not provide any access to an external context other than the webhook object itself: this means that is not possible to access external data or services to evaluate the filtering conditions. # Receivers ## Receiving Webhooks The framework also provides a set of services that can be used to receive webhooks from external systems and to process them. To do so, you need to add the `Deveel.Webhooks.Receiver.AspNetCore` library to your project, which must be an *ASP.NET Core* application. You can add the library to your project using the `dotnet` command line tool: ```bash dotnet add package Deveel.Webhooks.Receiver.AspNetCorebas ``` or add the following line to your `csproj` file: ```xml ``` ### Configuring the Receiver The receiver is configured using the `AddWebhooksReceiver` extension method on the `IServiceCollection` interface. To run the receiver, you need to add the `UseWebhooksReceiver` middleware to the pipeline of your application, specifying the path where the receiver will be listening for the incoming webhooks. When the service is configured with a webhook receiver, this will be invoked when a webhook is received, allowing the processing of the incoming webhook. ```csharp using System; using Microsoft.Extensions.Configuration; using Deveel.Webhooks; namespace Example { public class Startup { public Startup(IConfiguration configuration) { Configuration = configuration; } public IConfiguration Configuation { get; } public void Configure(IServiceCollection services) { services.AddWebhooksReceiver() .AddHandler(); } public void Configure(IApplicationBuilder app) { app.UseRouting(); app.UseWebhooksReceiver("/my-webhook"); } } } ``` The `AddWebhooksReceiver` method accepts a generic type parameter that specifies the type of the webhook that will be received by the receiver: this allows to accept and process multiple webhooks in the same application. ### Handling Webhooks The receiver will invoke the registered handlers in the order they are registered, that allows to process the incoming webhook in a pipeline. Handlers can use dependency injection to access the services registered in the application. ```csharp using System; using Deveel.Webhooks; namespace Example { public class MyWebhookHandler : IWebhookHandler { private readonly ILogger logger; public MyWebhookHandler(ILogger logger) { this.logger = logger; } public Task HandleAsync(MyWebhook webhook, CancellationToken cancellationToken) { logger.LogInformation("Received webhook {0}", webhook.Id); // Do something with the webhook } } } ``` The framework provides a set of libraries that can be used to receive webhooks from external sources. | Receiver | Description | | -------------------------------------------- | ------------------------------------------------------ | | [**Facebook**](/receivers/facebook_receiver) | Receive webhooks from Facebook Messenger | | [**SendGrid**](/receivers/sendgrid_receiver) | Receive webhooks and emails from SendGrid | | [**Twilio**](/receivers/twilio_receiver) | Receive webhooks and SMS/WhatsApp messages from Twilio | # Webhook Receivers The ability to receive webhooks is a core feature of the platform: webhooks are sent to your application when certain events occur in the platform. For example, when a user is created in an external system, a webhook is sent to your application with the details of the user. You can then use this information to create the user in your application. ## ASP.NET Receivers The framework provides provides an implementation of a webhook receiver for ASP.NET Core applications, available as a NuGet package: [Webhook.Receiver.AspNetCore](https://www.nuget.org/packages/Webhook.Receiver.AspNetCore/). You can use the contracts and the middlewares provided by the package to receive webhooks in your ASP.NET Core application and react to them, accordingly with the design of your application. ### Installation To install the package, use the following command in the Package Manager Console: ```powershell Install-Package Webhook.Receiver.AspNetCore ``` or use the .NET CLI: ```bash dotnet add package Webhook.Receiver.AspNetCore ``` ## Instrumenting the Application To start receiving webhooks in your ASP.NET Core application, you need to register the webhook receiver in the service collection, and add the webhook receiver middleware to the application pipeline. The following code shows how to register the webhook receiver in the service collection, and how to add the webhook receiver middleware to the application pipeline: ```csharp namespace Example { public class Startup { public void ConfigureServices(IServiceCollection services) { services.AddWebhookReceiver() .AddHandler(); } } public void Configure(IApplicationBuilder app, IWebHostEnvironment env) { // Use the registered factory handlers ... app.MapWebhook("/ids/webhooks/"); // ... or use a middleware to handle the webhook app.MapWebhook("/ids/webhooks/handled", async(IdentityWebhook webhook, ILogger logger) => { // Handle the webhook logger.LogInformation("Webhook received: {Webhook}", webhook); }); } } ``` ## Factory-Based Handlers The framework provides two alternative methods to handle webhooks, depending on the design of your application or the complexity of the webhook handling logic. The first method is to uses a factory to create the handlers registered in the service collection, and it's the most suitable for scenarios where to handle a webhook you need to depend on one or more external services. For example, consider the following webhook handler: ```csharp namespace Example { public class UserCreatedHandler : IWebhookHandler { private readonly IUserService _userService; public UserCreatedHandler(IUserService userService) { _userService = userService; } public async Task HandleAsync(IdentityWebhook webhook, CancellationToken cancellationToken) { var userInfo = webhook.Data.UserInfo; var user = new User { Email = userInfo.Email, FirstName = userInfo.FirstName, LastName = userInfo.LastName, ExternalId = userInfo.Id }; await _userService.CreateUserAsync(user, cancellationToken); } } } ``` When creating the webhook receiver, you can register the handler in the service collection using the service builder, and wire the handler to the webhook type: ```csharp namespace Example { public class Startup { public void ConfigureServices(IServiceCollection services) { services.AddWebhookReceiver() .AddHandler(); } } public void Configure(IApplicationBuilder app, IWebHostEnvironment env) { app.MapWebhook("/ids/webhooks/"); } } ``` The above code works as follow: 1. The `AddWebhookReceiver` method registers the webhook receiver in the service collection, isolating any behavior to the webhook type `IdentityWebhook` 2. The `AddHandler` method registers the handler in the service collection as a scoped service 3. The `MapWebhook` method maps the specific path (for a `POST` request) to the webhook receiver middleware, that receives webhooks of type `IdentityWebhook` 4. When a webhook is received, the middleware will create a scope and resolve any handlers associated to the webhook of type `IdentityWebhook`, passing them the webhook to handle ### Webhook Handling When a webhook is received and the handlers are resolved, their execution is performed in parallel by default, and the middleware will wait for the completion of all the handlers before returning a response to the sender. It is recommended that implementations of the handlers are designed to be executed in a non-blocking form, to avoid blocking the middleware and the sender of the webhook: currently no background process is executed to handle the webhooks, and the middleware will wait for the completion of all the handlers before returning a response to the sender. ### Execution Modes By default, the middleware will execute all the registered the handlers (fo the type of webhook) in parallel. This behavior can be changed by specifying an execution mode when registering the webhook receiver, using the `ExecutionMode` configuration property of the `WebhookHandlingOptions` class, when calling the `MapWebhook` method. ```csharp namespace Example { public class Startup { public void ConfigureServices(IServiceCollection services) { services.AddWebhookReceiver() .AddHandler(); } } public void Configure(IApplicationBuilder app, IWebHostEnvironment env) { app.MapWebhook("/ids/webhooks/", new WebhookHandlingOptions { ExecutionMode = WebhookExecutionMode.Sequential; }); } } ``` ## Convention-Based Receivers Another method to handle webhooks is to use middlewares to handle webhooks, and it's the most suitable for scenarios where the handling of the webhook is simple and doesn't require to depend on several external services (for example, when using a mediator to handle the webhook). This is done by passing a delegate to the `MapWebhook` method, that will be directly invoked by the middleware when a webhook is received, without attempting to resolve any further handler for the same type of webhook. ```csharp namespace Example { public class Startup { public void ConfigureServices(IServiceCollection services) { services.AddWebhookReceiver(); } } public void Configure(IApplicationBuilder app, IWebHostEnvironment env) { app.MapWebhook("/ids/webhooks/", async (IdentityWebhook webhook, CancellationToken cancellationToken) => { var mediator = context.RequestServices.GetRequiredService(); await mediator.Send(new CreateUserCommand(webhook.Data.UserInfo), cancellationToken); }); } } ``` As you can see the above code is simpler than the previous one, but it comes with a limitation: the first argument must always be the webhook of the type handled by the middleware. One of the arguments (in no particular position) can be a cancellation token, that can be used to cancel the execution of the middleware: this will be the same cancellation token used by the middleware to cancel the execution of the handlers. This method provides few alternative signatures to the delegate, depending on the design of your application, that can be executed synchronously or asynchronously. The following code shows the alternative signatures of the delegate: ```csharp // Async MapWebhook(string path, Func handler); MapWebhook(string path, Func handler); MapWebhook(string path, Func handler); MapWebhook(string path, Func handler); // Sync MapWebhook(string path, Action handler); MapWebhook(string path, Action handler); MapWebhook(string path, Action handler); MapWebhook(string path, Action handler); ``` Any additional parameter than the webhook will be resolved in the request scope, and passed to the delegate when invoked. ### Webhook Handling When using the delegate-based method to handle webhooks, the middleware will invoke the given delegate when a webhook is received, and will wait for the completion of the delegate before returning a response to the sender. It is recommended that implementations of the delegate are designed to be executed in a non-blocking form, to avoid blocking the middleware and the sender of the webhook: currently no background process is executed to handle the webhooks, and the middleware will wait for the completion of the delegate before returning a response to the sender. ## Webhook Types The overall design of the framework allows the segregation of the receiving functions to the webhook type, so that you can register multiple webhook receivers in the same application, each one handling a different type of webhook. Consider for example the following code: ```csharp namespace Example { public class Startup { public void ConfigureServices(IServiceCollection services) { services.AddWebhookReceiver() .AddHandler(); services.AddWebhookReceiver() .AddHandler(); } } public void Configure(IApplicationBuilder app, IWebHostEnvironment env) { app.MapWebhook("/ids/webhooks/"); app.MapWebhook("/payments/webhooks/"); } } ``` The above code registers two webhook receivers, one for the `IdentityWebhook` type and one for the `PaymentWebhook` type, and it registers a handler for each webhook type. This allows separating the behaviors in configuring and handling the webhooks, which might come from different sources and have different payloads. # Facebook Webhook Receiver This is a simple webhook receiver for Facebook Messenger. It is designed to be used with [Facebook Messenger Platform](https://developers.facebook.com/docs/messenger-platform). ## Installation To install the package in your project, use the following command: ```bash dotnet add package Deveel.Webhooks.Receiver.Facebook ``` ## Instrument the Webhook Receiver To enable your application to receive webhooks and messages from Facebook Messenger, you need to register the receiver and then wire it up to your application. The following example shows how to register the receiver in a ASP.NET Core application: ```csharp public void ConfigureServices(IServiceCollection services) { // ... services.AddFacebookReceiver() .AddHandler(); // ... } ``` The following example shows how to wire up the receiver in a ASP.NET Core application: ```csharp public void Configure(IApplicationBuilder app, IHostingEnvironment env) { // ... app.MapFacebookWebhook("/webhook/facebook"); app.MapFacebookVerify("/facebook/verify"); // ... } ``` The `MapFacebookWebhook` extension method is used to map the webhook endpoint in the application pipeline, that will be used by Facebook to send webhooks and messages. The `MapFacebookVerify` extension method is used to map the endpoint used by Facebook to verify that your application is authorized to receive webhooks. ## Configuration The receiver can be configured using the `FacebookReceiverOptions` class, that can be passed to the `AddFacebookReceiver` method during the registration process. The following code shows how to configure the receiver: ```csharp public void ConfigureServices(IServiceCollection services) { // ... services.AddFacebookReceiver(options => { options.AppSecret = Configuration["Facebook:AppSecret"], options.VerifyToken = Configuration["Facebook:VerifyToken"] options.VerifySignature = true }); } ``` As you can notice, the set of configurations provided by the `FacebookReceiverOptions` class are less than the one available from the `WebhookReceiverOptions` class, because the *Facebook Messenger Platform* has a more strict set of requirements for the webhook receiver. The following table shows the available options for the receiver: | Option | Description | | ----------------- | --------------------------------------------------------------------------------------- | | `AppSecret` | The application secret provided by Facebook | | `VerifyToken` | The token used to verify the webhook endpoint | | `VerifySignature` | A flag to indicate if the receiver should verify the signature of the incoming messages | # Receiving Webhooks The `Deveel.Webhooks.Receiver` library provides a set of components that can be used to receive webhooks from external sources, and handle them in an ASP.NET Core application. The following sections describe how to use the library to receive custom webhooks in an ASP.NET Core application, and how to handle them, but other *out-of-the-box* implementations are also provided for specific providers (eg. Facebook, Twilio, SendGrid, etc.): please check the specific section for the configuration of your application to receive webhooks from those providers. ## Installation To start receiving webhooks from external sources, you can use the `Deveel.Webhooks.Receiver.AspNetCore` library, that allows the registration of a webhook receiver in an ASP.NET Core application. Run this command on the root of your project to install the library from NuGet: ```bash dotnet add package Deveel.Webhooks.Receiver.AspNetCore ``` ## Instrumenting the Application If you are using a traditional ASP.NET Core MVC application, you can register the webhook receiver service by modifying the `Startup` class as follows: ```csharp public class Startup { public void ConfigureServices(IServiceCollection services) { services.AddWebhookReceiver(); } } ``` Alternatively, if you are using the mininal API pattern, you can use the following code: ```csharp var builder = WebApplication.CreateBuilder(args); builder.Services.AddWebhookReceiver(); ``` The above simple calls registers the webhook receiver in the application, and enable it to receive webhooks of type `MyWebhook`: receivers are segregated by the type of webhook they can handle, and you can register multiple receivers for different types of webhooks. By default the registration of the webhook receiver adds a set of default services, that are required to handle the webhooks, such as the `IWebhookReceiver` and `IWebhookHandler`, and a default set of options: you can control further the services and configurations by using the builder instance returned by the `AddWebhookReceiver` method. ## Receiving Webhooks - Using Controllers Following the registration of the webhook receiver, you can receive webhooks by using the `IWebhookReceiver` service, that is registered in the application, if you want to handle the receive process directly. This approach is typical in MVC APIs that implement the request processing in the controller, and can be used as follows: ```csharp namespace Demo { [ApiController] [Route("webhook")] public class WebhookController : ControllerBase { private readonly IWebhookReceiver webhookReceiver; private readonly IWebhookHandler webhookHandler; public WebhookController(IWebhookReceiver webhookReceiver, IWebhookHandler webhookHandler) { this.webhookReceiver = webhookReceiver; this.webhookHandler = webhookHandler; } [HttpPost] public async Task ReceiveWebhook() { var result = await webhookReceiver.ReceiveAsync(Request, HttpContext.RequestAborted); if (!result.IsValid) return BadRequest(result.Error); var webhook = result.Webhook; await webhookHandler.HandleAsync(webhook, HttpContext.RequestAborted); return Ok(); } } } ``` Mind that in the above scenario you must also inject the `IWebhookHandler` service, that is used to handle the received webhook. **Note** - The design of the receiver allows the registration of multiple handlers for the same type of webhook, which can be injected in the controller as an `IEnumerable>` service, and can be used to handle the webhook in different ways. For simplicity of the example, we are using a single handler. ## Receiving Webhooks - Using Middlewares Alternatively the `Deveel.Webhooks.Receiver.AspNetCore` library provides a middleware that can be used to receive webhooks, and handle them automatically. To use the middleware, you must first register it in the `Startup` class of your application: ```csharp public class Startup { public void ConfigureServices(IServiceCollection services) { services.AddWebhookReceiver(); } public void Configure(IApplicationBuilder app, IWebHostEnvironment env) { app.MapWebhook("/webhook"); } } ``` If you are using the minimal API pattern, you can use the following code: ```csharp var builder = WebApplication.CreateBuilder(args); builder.Services.AddWebhookReceiver(); var app = builder.Build(); app.MapWebhook("/webhook"); app.Run(); ``` The above code registers the middleware in the application, and enables the application to receive webhooks of type `MyWebhook` at the `/webhook` endpoint, using the configurations defined when registering the receiver, The middlware will automatically scan for all the registered webhook receiver service configured, and will handle the received webhooks by invoking all the `IWebhookHandler` services registere. The middleware design allows to handle the webhooks without any prior registered handler, by specifying an handling delegate in the `MapWebhook` method: ```csharp [...] app.MapWebhook("/webhook", async webhook => { // Handle the webhook here await Task.CompletedTask; }); ``` Or a alternatively a synchronous handling delegate: ```csharp [...] app.UseWebhookReceiver("/webhook", webhook => { // Handle the webhook here }); ``` ## Further Reading If you want to learn more and learn more advanced usage of the receivers, visit the [Advanced Usage of Receiver](/receivers/custom_receiver). # SendGrid Webhook and E-Mail Receiver The framework provides a set of configurations and extensions to support the capabilities for receiving and processing SendGrid webhooks and e-mails. ## Installation You can install the package from NuGet, running the following command in the console: ```bash dotnet add package Deveel.Webhooks.Receiver.SendGrid ``` ## Configuration ### Receiving Webhooks To activate the SendGrid receiver you don't need many cerimonies or configurations, just add the following line to the `ConfigureServices` method of your `Startup` class (assuming you are using a classic ASP.NET Core application): ```csharp public void ConfigureServices(IServiceCollection services) { // ... services.AddSendGridReceiver() .AddHandler(); // ... } ``` The above line will register the required services and configurations to the DI container, using the default configurations, so that the receiver can be used in the application. If you need to customize directly the configurations, you can use the following overload of the `AddSendGridReceiver` method: ```csharp public void ConfigureServices(IServiceCollection services) { // ... services.AddSendGridReceiver(options => { options.VerifySignatures = true; options.Secret = "my-secret"; }); // ... } ``` If your configurations reside in a configuration section of the `appsettings.json` file, you can use the following overload: ```csharp public void ConfigureServices(IServiceCollection services) { // ... services.AddSendGridReceiver("Webhooks:SendGrid") .AddHandler(); // ... } ``` *Note: the above overload will use the `Webhooks:SendGrid` section of the configuration file to load the configurations, but any can be used* ### Receiving E-Mails SendGrid and other providers of e-mail services support the capability to forward e-mails to a specific endpoint, so that the application can process them, using alternative methods than the classic SMTP protocol. By nature, these HTTP requests are not considered as webhooks, but they are still HTTP requests that can be processed by the framework: in fact the receiver library provides a specific handler that can be used to process e-mails. Since thes e-mails are not following the practices of webhooks, the receiver will not validate the signature of the request, but it will process it as it is, requiring no additional configurations. To activate the e-mail receiver, you can use the following overload of the `AddSendGridEmailReceiver` method: ```csharp public void ConfigureServices(IServiceCollection services) { // ... services.AddSendGridEmailReceiver("/email/sendgrid") .AddHandler(); // ... } ``` ## Mapping Webhook Events To map the events received from SendGrid to the handlers provided by the framework, you can use the `MapSendGridWebhook` and `MapSendGridEmail` extension methods of the `IApplicationBuilder` contract: ```csharp public void Configure(IApplicationBuilder app, IHostingEnvironment env) { // ... app.MapSendGridWebhook("/webhook/sendgrid"); app.MapSendGridEmail("/email/sendgrid"); app.MapSendGridWebhook("/webhook/sendgrid/handled", webhook => {}); app.MapSendGridEmail("/email/sendgrid/handled", email => {}); // ... } ``` The framework will bind the incoming webhooks and emails to instances of the `SendGridWebhook` and `SendGridEmail` classes, that can be used to process the data received from the provider. # Twilio Webhook Receiver Twilio is a cloud communications platform as a service (CPaaS) provider, allowing software developers to programmatically make and receive phone calls, send and receive text messages, and perform other communication functions using its web service APIs. In the process of messaging, Twilio sends a webhook to a configured URL: this contains incoming messages, directed to the receiver, or the status of outgoing messages, sent by an application. ## Installation To enable your ASP.NET Core application to receive Twilio Webhooks, install the Deveel.Webhooks.Receiver.Twilio library, using the NuGet package manager: ```bash dotnet add package Deveel.Webhooks.Receiver.Twilio ``` ## Configuration To configure the Twilio Webhook Receiver, you need to add the `TwilioWebhookReceiver` to the services collection of your application, in the `ConfigureServices` method of the `Startup` class: ```csharp public void ConfigureServices(IServiceCollection services) { // ... services.AddTwilioReceiver(); // ... } ``` Then, you need to configure the receiver in the `Configure` method of the `Startup` class: ```csharp public void Configure(IApplicationBuilder app, IWebHostEnvironment env) { // ... app.MapTwilioWebhook("/twilio/webhook"); app.MapTwilioWebhook("/twilio/other", webhook => { // ... }); // ... } ``` # Developer Guidelines *More content coming soon...* # Frequently Asked Questions ## Q: Why have you developed another Webhook management library A: We didn't have the ambition to develop this project, but rather to use some already available, anyway given the conditions we were in, we could not find any fitting alternative: * [**Microsoft's ASP.NET Webhook Framework**](https://github.com/aspnet/WebHooks), before being retired, supported only the .NET 4.6 framework * [**Microsot's ASP.NET Core Webhook Framework**](https://github.com/aspnet/AspLabs/tree/main/src/WebHooks) was *demoted* to an experimental project (within the scope of [AspNetLabs](https://github.com/aspnet/AspLabs) space), and anyway did not provide any capability for the management of subscriptions, or logging results of deliveries * [**ASP.NET Boilerplate (by Volosoft)**](https://github.com/aspnetboilerplate/aspnetboilerplate) provides functionalities for the management and sending of webhooks that are embedded into a more extended framework, that we didn't want to use in its entirety. ## Q: Which .NET versions are supported by Deveel Webhooks? A: Since the version 2.1.1, the framework supports both .NET 6.0 and .NET 7.0, and therefore it can be used in any .NET implementation that supports this version of the standard. Previous versions are built on the .NET 6.0 framework only. ## Q: Is Deveel Webhooks still maintained, and how does it relate to Deveel Events? A: Yes, **Deveel Webhooks is maintained for the long term**. At the same time, its functionality is being gradually migrated and expanded into [Deveel Events](https://events.deveel.org/) ([GitHub repository](https://github.com/deveel/deveel.events)), because that project is closer to the broader Domain-Driven Design (DDD) concept of domain events. In practical terms, Webhooks remains focused on webhook-specific capabilities, while Deveel Events provides a more general model where webhooks are one type of event integration mechanism. ## Q: Do you have any commercial plans for this framework? A: No. Not at the moment. The origin of this project was to support a commercial service, and we wanted to provide the community with the outcomes of our experiences and finding in this specific area. ## Q: Is your aim to replace Microsoft's Webhook Framework? A: As pointed out in the answer provided above (and on the motivations of this project), currently Microsoft provides no stable alternatives to handle webhook subscription management and notifications, but just an experimental framework to implement receivers of webhooks from major service providers. ## Q: Does Deveel Webhooks support webhook subscriptions? A: Yes. The server part of the framework provides a mechanism to manage webhook subscriptions, that can be used to register a webhook endpoint to receive webhooks from your applications (as provider). ## Q: Which persistency layers do you provide to store Webhook Subscriptions? A: The current implementation of the framework provides a MongoDB and an Entity Framework Core persistency layer, but we are open to contributions to support other persistency layers. The data model of subscriptions and webhooks is not complex and should not be a challenge to contribute with alternatives (please refer to the [contributing guidelines](https://github.com/deveel/deveel.webhooks/blob/main/CONTRIBUTING.md)). ## Q: Does Deveel Webhooks support webhook formats other than JSON? A: Yes. Since version *2.0.1*, the framework supports JSON and XML formats for sending webhooks, while for the receiving part it support JSON, XML and Forms (*application/x-www-form-urlencoded* Content-Type) formats, that is dependent on the receiver implementation. ## Q: Does Deveel Webhooks support webhook authentication? A: Yes. Since version *2.0.1*, the framework supports the validation of the signature of the webhook payload, using the *HMAC-SHA256* and *HMAC-SHA1* algorithms, and the secret key provided by the subscription. ## Q: Does Deveel Webhooks support webhook encryption? A: Not at the moment. Encrypting and decrypting messages is an intensive operation, and it is not in the scope of the framework to provide this functionality at the moment, especially in consideration of the messaging nature of webhooks. ## Q: Does Deveel Webhooks support webhook retry policies? A: Yes. The framework supports retry policies for the delivery of webhooks, that is applied when the delivery of a webhook fails. The policy can be defined at the level of the subscription, or at the level of the webhook itself. ## Q: Does Deveel Webhooks support webhook delivery scheduling? A: No. It is not in the scope of the framework to provide a scheduler for the delivery of webhooks. The framework provides a mechanism to trigger the delivery of webhooks, but it is up to the application to implement a scheduler that triggers the delivery of webhooks. ## Q: Does Deveel Webhooks support webhook delivery logging? A: Yes. The framework provides a mechanism to log the results of the delivery of webhooks, that can be used to persist the results of the delivery of webhooks. The framework provides a default implementation of the logging mechanism that uses the data layer to persist the results of the delivery of webhooks. ## Q: Does Deveel Webhooks support webhook delivery throttling? A: No. At the moment the framework does not provide any mechanism to throttle the delivery of webhooks, but this is something that we are considering to implement in the future. ## Q: Does Deveel Webhooks support webhook delivery batching? A: Not at the moment, but we have included in the issues as an idea to implement in the future. ## Q: Does Deveel Webhooks support webhook delivery deduplication? A: No. It is a good idea to explore for future implementations. ## Q: Which webhook providers are supported by Deveel Webhooks? A: At the moment the framework supports the following external webhook providers: * [**Facebook**](/receivers/facebook_receiver) * [**SendGrid**](/receivers/sendgrid_receiver) * [**Twilio**](/receivers/twilio_receiver) Follow the issues of this project to see which providers are planned to be supported in the future. The framework also provides a generic receiver that can be used to implement a webhook receiver for any other provider, including your own applications.