WebHook

From Shopify Wiki

Jump to: navigation, search

Contents

What is a WebHook?

WebHooks are a way to tell Shopify to call a script on one of your own web servers whenever a given event occurs and react in any way you want. They can be thought of as push notifications or event listeners.

Some possible uses for this feature include:

  • Notify your IM client or your pager when you are offline
  • Collect data for data-warehousing
  • Integrate your accounting software
  • Filter the order items and inform various drop shippers about the order
  • Create license keys for software sales
  • Remove customer data from your database when they uninstall your app

Please do note that you should not depend on webhooks for real-time push. The data you receive in a webhook may be delayed and out of date. For an authoritative snapshot of Shopify state, use the Shopify API.

WebHook Basics

Web hooks can be registered for the following events:

  • orders/create
  • orders/updated
  • orders/paid
  • orders/cancelled
  • orders/fulfilled
  • app/uninstalled
  • customer_groups/create
  • customer_groups/update
  • customer_groups/delete
  • products/create
  • products/update
  • products/delete

File:/upload/b/b4/Webhook.png

To create a web hook go to the Preferences / Email & Notifications area of your shop and click the 'Add a webhook subscription' button at the bottom. Select the event type you want to listen for from the drop down box, and enter the URL you want to receive notifications. Just like our regular API, you can choose to receive data formatted as either XML or JSON.

Once you register a webhook URL with Shopify we will issue a HTTP POST request to the URL specified every time that event occurs. The request's POST parameters will contain XML/JSON data relevant to the event that triggered the request.

If your server is down when Shopify POSTs to it, don’t worry; we’ll simply try again until your server confirms to us that it has successfully received the notification.

Shopify will call the URL you provide here every time an order is placed. This means that if a merchant bulk-uploads 1000 products, your webhook will be called 1000 times.

Timeout: The web hook request will time out after 10 seconds. Move long running processes (e.g. PDF generation) into an asynchronous background task and make sure that your application responds immediately to the Shopify server.

Request Data

In the POST parameters of the request Shopify passes along XML or JSON data with all of the order's details. Here is an example XML document that would be sent along with an order/* hook.

Don't assume this is exactly what is posted ... use the "test webhook" from the Shopify admin! 

POST /your-path
Content-Type: application/xml

<?xml version="1.0" encoding="UTF-8"?>
<order>
  <buyer-accepts-marketing type="boolean">false</buyer-accepts-marketing>
  <cart-token nil="true"></cart-token>
  <closed-at type="datetime" nil="true"></closed-at>
  <created-at type="datetime">2005-07-31T15:57:11Z</created-at>
  <currency>USD</currency>
  <email>bob@customer.com</email>
  <financial-status>paid</financial-status>
  <fulfillment-status nil="true"></fulfillment-status>
  <gateway>bogus</gateway>
  <id type="integer">516163746</id>
  <note nil="true"></note>
  <number type="integer">1</number>
  <shop-id type="integer">820947126</shop-id>
  <subtotal-price type="integer">10.00</subtotal-price>
  <taxes-included type="boolean">false</taxes-included>
  <total-discounts type="integer">0.00</total-discounts>
  <total-line-items-price type="integer">10.00</total-line-items-price>
  <total-price type="integer">11.50</total-price>
  <total-tax type="integer">1.50</total-tax>
  <total-weight type="integer">0</total-weight>
  <updated-at type="datetime">2005-08-01T15:57:11Z</updated-at>
  <name>#1001</name>
  <billing-address>
    <address1>123 Amoebobacterieae St</address1>
    <address2></address2>
    <city>Ottawa</city>
    <company></company>
    <country>Canada</country>
    <first-name>Bob</first-name>
    <id type="integer">943494288</id>
    <last-name>Bobsen</last-name>
    <phone>(555)555-5555</phone>
    <province>Ontario</province>
    <shop-id type="integer">820947126</shop-id>
    <zip>K2P0V6</zip>
    <name>Bob Bobsen</name>
  </billing-address>
  <shipping-address>
    <address1>123 Amoebobacterieae St</address1>
    <address2></address2>
    <city>Ottawa</city>
    <company></company>
    <country>Canada</country>
    <first-name>Bob</first-name>
    <id type="integer">943494288</id>
    <last-name>Bobsen</last-name>
    <phone>(555)555-5555</phone>
    <province>Ontario</province>
    <shop-id type="integer">820947126</shop-id>
    <zip>K2P0V6</zip>
    <name>Bob Bobsen</name>
  </shipping-address>
  <line-items type="array">
    <line-item>
      <fulfillment-service>manual</fulfillment-service>
      <grams type="integer">1500</grams>
      <id type="integer">642703538</id>
      <price type="integer">10.00</price>
      <quantity type="integer">1</quantity>
      <sku>1</sku>
      <title>Draft</title>
      <variant-id type="integer">47052976</variant-id>
      <vendor nil="true"></vendor>
      <name>Draft - 151cm</name>
    </line-item>
  </line-items>
</order>

Headers

Webhooks have some common headers that get sent out to help consumers:

  • x-shopify-topic - Has the topic of the webhook (orders/create, products/create, etc.)
  • x-shopify-test - Set to true if the webhook was triggered by the test button in the admin
  • x-shopify-shop-domain - has the x.myshopify.com domain of the store where the webhook originated.

Note that the application/uninstalled webhook does not send any data. You can get the shop it refers to from the appropriate header.

Verifying a web hook

If your webhook was created using the API you can verify it using the HMAC digital signature header. See our Verifying Webhooks page for details on this.

Examples:

Rails

There is sample application on github based on our shopify_app gem that covers setting up webhooks and properly verifying them.

PHP Example w/ SimpleXML (PHP 5+)

The script below shows you how to get the XML data in from Shopify into your script, archive the file, and send the proper headers ... 

Given that the new order subscription setup in the admin for the webhook is: http://example.com/some-script.php?key=123456789
Contents of some-script.php on http://example.com/

// Protect URL from rogue attacks/exploits/spiders
// Grab from GET variable as given in Shopify admin URL
// for the webhook
//
// NOTE: This is not necessary, just a simple verification
//
// A digital signature is also passed along from Shopify,
// as is the shop's domain name, so you can use one or both of those
// to ensure a random person isn't jacking with your script (or some
// spider just randomly hitting it to see what's there).
//
// If $key doesn't matched what should be passed in from the
// webhook url, the script simply exits
$key = $_GET['key']; 

if ($key != '123456789') {
  header('HTTP/1.0 403 Forbidden');
  exit();
}

// Variables used for processing/saving
$xmlString = ;  // Used to get data from Shopify into script
$name = ;  // Saves the billing address name to be used for later ...
$email = ;  // Save the email address of the user to be used for later ...
$productTitles = array();  // Saves all titles of products purchased to be used for later ... 

// Get XML data and read it into a string for use with SimpleXML
// Thanks to David Oxley (http://www.numeriq.co.uk) for help with this
$xmlData = fopen('php://input' , 'rb'); 
while (!feof($xmlData)) { $xmlString .= fread($xmlData, 4096); }
fclose($xmlData);

// Save order XML in file in orders directory
// This creates a file, write the xml for archival purposes, and closes the file ...
// If the file already exists, it appends the data ... this should create a separate
// file for every order but if two orders are processed the same second, they'll both
// be in the same file
file_put_contents('orders/order' . date('m-d-y') . '-' . time() . '.xml', $xmlString, FILE_APPEND);

// Use SimpleXML to get name, email, and product titles
// SimpleXML allows you to use the $xml object to easily
// retrieve the data ...
// Please note, if hyphens are used in the xml node, you must
// surround the call to that member with {'member-name'} as is
// shown below when getting the billing-address name & the
// line items
$xml = new SimpleXMLElement($xmlString);

$name = trim($xml->{'billing-address'}->name);
$email = trim($xml->email);

// Create productTitles array with titles from products
foreach ($xml->{'line-items'}->{'line-item'} as $lineItem) {
  array_push($productTitles, trim($lineItem->title));
}

// You would then go on using $name, $email, $productTitles in your script
// to do whatever the heck you please ...

// Once you are done doing what you need to do, let Shopify know you have 
// the data and all is well!
header('HTTP/1.0 200 OK');
exit();

// If you want to tell Shopify to try sending the data again, i.e. something
// failed with your processing and you want to try again later
header('HTTP/1.0 400 Bad request');
exit();

What should be returned

Shopify will simply discard the body of any webhook responses. You just have to make sure that the status code of the resulting page is between 200 and 399 ( preferably 200 OK ) which indicates success.

How to test

Shopify lets you test your web hooks. After you create a web hook you will see it in the list of order notifications. You will also see a Test link. This Test link allows you to send an example order to the address provided.

You can also use curl: 

curl -i --header "X-Shopify-Header-1: one" --header "X-Shopify-Header-2: two" -d @- http://appdev/webhooks < /tmp/requestbody

If you want to capture the contents of a webhook to examine them, the easiset way is to set up a new subscription that points to a service like RequestBin or PostCatcher which will capture the result and let you view it in a browser.

Automatic Retries and Deletion

If an error is returned or a timeout occurs when sending a webhook, Shopify will retry the same request for 48 hours using an exponential back-off approach. In total 19 attempts will be made to deliver the information.

A webhook will be deleted if there are 19 consecutive failures for the exact same webhook. So, 10 failures for orders/create and 10 failures for orders/updates would not cause the webhook to be deleted. The same goes for 10 failures for order A and 10 failures for order B, this still won't delete the webhook for this topic yet.

Notifications about pending deletions will be sent to the support email associated with the app if the subscription was created through the API, or the store owner's email if it was created through the admin dashboard.

Best Practices

When Should I Be Using Webhooks?

Let's start with the basics. The obvious case for webhooks is when you need to act on specific events. An order being placed, a product price changing, etc. If you would otherwise have to poll for data, use a webhook.

Another less obvious use-case we've seen is when you're dealing with data that isn't easily searchable though the Shopify API. Whilst we offer several filters on our index requests, there's some secondary or implied data that isn't directly covered by these. Examples on Shopify include:

  • Product SKUs
  • Shipping addresses on orders
  • Vendors 
Re-requesting the entire product catalog or order history whenever you want to search these fields requires a lot of API requests and takes time. Instead, you can use webhooks.

The first thing you should do is grab a copy of the store's product catalog, order history, or other appropriate data when your app is installed. Then you can register a webhook on the update event that captures changes and saves them to your database. Bam, now you have a fully searchable up-to-date local database that you can transform or filter any way you please.

How Should I Handle Webhook Requests?

This is a big one. As explained above, if Shopify doesn't receive a response within 10 seconds of sending a webhook, it assumes there's a problem and marks it as failed.

A common cause for this is an app that does some processing when it receives a webhook request before responding. In the normal web-app world this is desired as you need to send data back to the user. When processing webhooks on the other hand, all that's needed is a quick 200 OK response that acknowledges receipt of the data.

Here's some pseudocode demonstrating what I mean: 

func handle_webhook(request)
  process_data(request.data)
  respond(200)
end

To make sure that you don't accidentally run over the timeout limit, you need to defer any processing processing until *after* the response has been sent. In Rails, Delayed Jobs are perfect for this.

Here's how your code should look: 

func handle_webhook(request)
  schedule_processing(request.data)
  respond(200)
end

Even if you're only doing a small amount of processing, there are other factors to take into account. On-demand services such as Heroku or PHPFog sometimes need to spin up a new node to handle the request, and this action can take several seconds. Even if your app is only spending five seconds processing data it'll still 'fail' if the underlying server took six seconds to start up.

What Do I Do if Everything Blows Up?

Imagine the worst case scenario: Your hosting centre exploded and your app has been offline for more than 48 hours. Ouch. It's back on its feet now, but you've missed a pile of data that was sent to you in the meantime. Not only that, but Shopify has cancelled your webhooks because you weren't responding for an extended period of time.

How do you catch up? Let's tackle the problems in order of importance.

Getting your webhook subscriptions back should be straightforward as your app already the code that registered them in the first place. If you know for sure that they're gone you can just re-run that and you'll be good to go. One thing to consider is adding a quick check that fetches all the existing webhooks and only re-registers the ones that you need.

Importing the missing data is trickier. Shopify can't re-send old data, so you'll have to look to the API to fetch what you've missed. Fortunately, webhook request data is very similar (exactly in some cases) to the format returned by an index call to the corresponding endpoint on the API.

Using this knowledge, you can build a harness that fetches data from the time period you were down for and feeds it into the webhook processing code one object at a time. The only caveat is that you'll need the processing code to be decoupled correctly in order to do this but that shouldn't be a problem for the caliber of developers using the Shopify API, right?

References

Retrieved from "http://wiki.shopify.com/WebHook"