Introduction

Welcome to the ControlShift Labs JSON API and Webhook Endpoint documentation! Our API and Webhooks are designed to allow software engineers to create rich integrations between ControlShift content and third-party services and websites.

JSON API Endpoints

The JSONP API is a simple way to embed ControlShift petition content in external sites. It’s intended for use by a front-end developer embed content on web pages outside of the platform. For example, a developer could:

Show petitions
Allow a user to search for near by petitions
List petition categories
Show petitions within an effort
If a site visitor is logged in, show a users past actions and created petitions

All of the endpoints can be consumed as JSONP instead of JSON by adding callback or variable parameters to the URLs.

The URL slugs through the API are the same as those that are used through the web to represent specific petitions or categories. Many front-end libraries including jQuery make it easy to consume JSONP endpoints. Our examples below use jQuery to consume these resources.

These examples use the domain demo.controlshiftlabs.com. You’ll need to replace this with the hostname of your instance.

Get a single petition

$(document).ready(function(){
  var petitionSlug = 'turn-back-human-trafficking-faith-leaders';
  $.ajax({
    url: 'https://demo.controlshiftlabs.com/petitions/'+petitionSlug+'.json',
    dataType: 'jsonp',
  })
  .done(function(data) {
    console.log(data);
  });
});

The above code would display your returned petition data in the console. The JSON response data would be structured like this:

{
  "id": 107370,
  "title": "Pledge to TURN BACK human trafficking Now - Faith or Community Leaders",
  "who": "President John Dramani Mahama",
  "why": "Today there are over 192,000 people trapped in modern slavery in Ghana [1]. Denied of their rights to education and often lacking parental care or support, over 1.8 million Ghanaian children are working rather than in school (21.8% of the population). Of these an estimated 1.2 million are forced to undertake hazardous child labour including mining, fishing and domestic slavery [3].\r\n\r\nThe experience of Kwesi is a typical example. Following the death of his father, Kwesi’s mother borrowed money to pay for the funeral. Faced with a debt she was unable to pay her creditor threatened her with jail unless she gave up her son as a labourer.  Kwesi was trafficked to Lake Volta to work as a “fisher boy”. and for the next three years, Kwesi worked for an average of 18 hours per day with little rest, no pay and only enough food to give him the energy for more work. By the time we found Kwesi and rescued him, he had suffered extensive emotional and physical abuse and partially lost sight in his left eye. \r\n\r\nOver 21,000 children like Kwesi, many of them trafficked, are trapped in hazardous labour on Lake Volta in Ghana[2]. We are working hard to bring them home, but want to ensure no one is trafficked in the first place. Please help fulfil the wish made by Kwesi when he gained his freedom: “I want the world to know that no child should be sent into slavery”.\r\n\r\n[1]. http://www.globalslaveryindex.org/ 2014\r\n[2].International Labour Organisation/International Programme on Elimination of Child Labour (ILO/IPEC) Analytical Study on Child Labour In Lake Volta Fishing in Ghana\r\n[3].Ghana Living Standard Survey 6 (GLSS 6): Child Labour Report by Ghana Statistical Services, 2014",
  "what": "I know that globally millions of people are trafficked for forced labour and sexual exploitation and that Ghana is a source, transit and destination country for this social evil. The concerns raised by Parliament recently, about men, women, and children who are trafficked and subjected to forced labor and sexual exploitation human points to the need for everyone to take action.\r\n\r\nI hereby pledge to TURN BACK human trafficking. \r\n•\tI will be vigilant of suspicious movement of children and vulnerable people, victims of bonded labour and of children working when they should be in school \r\n•\tI promise to always report my concerns to the appropriate authorities as quickly as possible.\r\n\r\nAs a faith or community leader, I promise to warn my congregation of the dangers of child trafficking, and support them in reporting any suspicions they have.",
  "created_at": "2015-06-01T15:37:47.621-04:00",
  "updated_at": "2015-10-14T08:45:52.396-04:00",
  "slug": "turn-back-human-trafficking-faith-leaders",
  "delivery_details": null,
  "administered_at": "2015-07-29T09:32:08.258-04:00",
  "source": "",
  "location_id": null,
  "alias": null,
  "bsd_constituent_group_id": null,
  "categories": [{
    "name": "Civil Rights",
    "slug": "civil-rights-1"
  }],
  "goal": 100,
  "effort": null,
  "group": "turn-back-human-trafficking-now",
  "resized_image_url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/107370/hero/Untitled.png?1433187465",
  "image_url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/107370/original/Untitled.png?1433187465",
  "creator_name": "Kristyn Arrighi",
  "signature_count": 6
}

This retrieves a single petition object.

HTTP Request

GET https://demo.controlshiftlabs.com/petitions/<slug>.json

Query Parameters

Parameter	Default	Description
slug	null	string - required - The petition’s unique identification slug. If none is provided, you will get a 404 error. Note: submitted as a part of the endpoint path, not as a separate URL parameter

Working Example

View and edit a working example on codepen.io:

    $(document).ready(function(){
      var petitionSlug = 'turn-back-human-trafficking-faith-leaders';
      $.ajax({
        url: 'https://demo.controlshiftlabs.com/petitions/'+petitionSlug+'.json',
        dataType: 'jsonp',
      })
      .done(function(data) {
        $('#title').html(data.title);
        $('#signature_count').html(data.signature_count);
        $('#creator_name').html(data.creator_name);
        $('#what').html(data.what);
        $('#who').html(data.who);
        $('#why').html(data.why);
        $('#js-content-wrapper').removeClass('hidden');
      });
    });

Get featured petitions

$(document).ready(function(){
  $.ajax({
    url: 'https://demo.controlshiftlabs.com/featured.json',
    dataType: 'jsonp',
  })
  .done(function(data) {
    console.log(data);
  });
});

The above code would display your returned featured petitions data in the console. The JSON would be structured like this:

[{
  "slug": "release-pro-democracy-youth-activists-in-azerbaijan",
  "title": "Release pro-democracy youth activists in Azerbaijan",
  "url": "http://demo.controlshiftlabs.com/petitions/release-pro-democracy-youth-activists-in-azerbaijan",
  "admin_status": "awesome",
  "image_url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/28870/hero/Screen_Shot_2014-11-14_at_3.57.21_PM.png?1415999011",
  "additional_image_sizes_url": [{
    "style": "form",
    "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/28870/form/Screen_Shot_2014-11-14_at_3.57.21_PM.png?1415999011"
  }, {
    "style": "large",
    "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/28870/large/Screen_Shot_2014-11-14_at_3.57.21_PM.png?1415999011"
  }, {
    "style": "open_graph",
    "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/28870/open_graph/Screen_Shot_2014-11-14_at_3.57.21_PM.png?1415999011"
  }, {
    "style": "original",
    "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/28870/original/Screen_Shot_2014-11-14_at_3.57.21_PM.png?1415999011"
  }],
  "who": "President Ilham Aliyev of Azerbaijan",
  "what": "Immediately release the imprisoned youth activists from the NIDA civic movement.\r\n\r\nShahin Novruzlu, Rashadat Akundov, Mammad Azizov, Bakhtiyar Guliyev, Zaur Gurbanli, Rashad Hasanov, Uzeyir Mammadli and Ilkin Rustamzade were arrested on criminal charges of possessing explosives and intent of public disorder. Human rights organizations believe these charges have been fabricated. The actual reason for their arrest was their criticism of the Azerbaijani authorities and their online activism, calling for peaceful demonstrations.\r\n\r\nShahin Novruzlu, who was only 17 at the time of his arrest, has lost his front teeth as a result of torture in detention. Mammad Azizov and Bakhtiyar Guliyev have complained of torture too.\r\n\r\nFrom 14 May 2014, Azerbaijan is chairing the Committee of Ministers of the Council of Europe. This organization is responsible for promoting and protecting human rights. I am deeply concerned that there are civil society activists in Azerbaijan tortured and imprisoned solely for expressing their views and organizing peaceful protests.\r\n\r\nAzerbaijan must live up to the high expectations of the chair of the Committee of Ministers of the Council of Europe. The NIDA activists must be immediately released.",
  "goal": 200,
  "signature_count": 119,
  "creator_name": "Kristyn Arrighi",
  "created_at": "2014-07-24T20:07:14.607-04:00",
  "updated_at": "2016-02-26T14:30:57.330-05:00",
  "why": "Shahin Novruzlu and fellow youth activists from the pro-democracy movement NIDA campaigned for democracy and against human rights abuses and widespread corruption in Azerbaijan. In February 2013 th..."
}, {
  "slug": "give-us-more-icecream",
  "title": "Give us more icecream",
  "url": "http://demo.controlshiftlabs.com/petitions/give-us-more-icecream",
  "admin_status": "awesome",
  "image_url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/139482/hero/10982478_634745530004482_279346285079955699_n.jpg?1455250695",
  "additional_image_sizes_url": [{
    "style": "form",
    "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/139482/form/10982478_634745530004482_279346285079955699_n.jpg?1455250695"
  }, {
    "style": "large",
    "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/139482/large/10982478_634745530004482_279346285079955699_n.jpg?1455250695"
  }, {
    "style": "open_graph",
    "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/139482/open_graph/10982478_634745530004482_279346285079955699_n.jpg?1455250695"
  }, {
    "style": "original",
    "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/139482/original/10982478_634745530004482_279346285079955699_n.jpg?1455250695"
  }],
  "who": "Cook",
  "what": "Cook and provide more icecream",
  "goal": 100,
  "signature_count": 6,
  "creator_name": "Tudor Bradatan",
  "created_at": "2016-02-11T23:17:42.782-05:00",
  "updated_at": "2016-02-26T14:20:46.987-05:00",
  "why": "Because we like and need it",
  "location": {
    "query": "Cluj",
    "latitude": "46.7712101",
    "longitude": "23.6236353",
    "street": "",
    "postal_code": "",
    "country": "RO",
    "region": "CJ",
    "street_number": "",
    "venue": "",
    "created_at": "2016-02-11T23:17:42.769-05:00"
  }
}, {
  "slug": "turn-back-human-trafficking-faith-leaders",
  "title": "Pledge to TURN BACK human trafficking Now - Faith or Community Leaders",
  "url": "http://demo.controlshiftlabs.com/petitions/turn-back-human-trafficking-faith-leaders",
  "admin_status": "awesome",
  "image_url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/107370/hero/Untitled.png?1433187465",
  "additional_image_sizes_url": [{
    "style": "form",
    "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/107370/form/Untitled.png?1433187465"
  }, {
    "style": "large",
    "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/107370/large/Untitled.png?1433187465"
  }, {
    "style": "open_graph",
    "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/107370/open_graph/Untitled.png?1433187465"
  }, {
    "style": "original",
    "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/107370/original/Untitled.png?1433187465"
  }],
  "who": "President John Dramani Mahama",
  "what": "I know that globally millions of people are trafficked for forced labour and sexual exploitation and that Ghana is a source, transit and destination country for this social evil. The concerns raised by Parliament recently, about men, women, and children who are trafficked and subjected to forced labor and sexual exploitation human points to the need for everyone to take action.\r\n\r\nI hereby pledge to TURN BACK human trafficking. \r\n•\tI will be vigilant of suspicious movement of children and vulnerable people, victims of bonded labour and of children working when they should be in school \r\n•\tI promise to always report my concerns to the appropriate authorities as quickly as possible.\r\n\r\nAs a faith or community leader, I promise to warn my congregation of the dangers of child trafficking, and support them in reporting any suspicions they have.",
  "goal": 100,
  "signature_count": 6,
  "creator_name": "Kristyn Arrighi",
  "created_at": "2015-06-01T15:37:47.621-04:00",
  "updated_at": "2015-10-14T08:45:52.396-04:00",
  "why": "Today there are over 192,000 people trapped in modern slavery in Ghana [1]. Denied of their rights to education and often lacking parental care or support, over 1.8 million Ghanaian children are wo..."
}]

This retrieves a JSON array of featured petitions objects.

HTTP Request

GET https://demo.controlshiftlabs.com/featured.json

Working Example

View and edit a working example on codepen.io:

Featured Petitions

    $(document).ready(function(){
      $.ajax({
        url: 'https://demo.controlshiftlabs.com/featured.json',
        dataType: 'jsonp',
      })
      .done(function(data) {
        var $placeholder = $('#featured-petitions');
        $.each(data, function(index, petition){
          output = ''
          output += ''+petition.title+'';
          output += ''+petition.signature_count+' signatures out of '+petition.goal+' needed ';
          output += ''+petition.why+'';
          output += 'Learn More';
          output += '';
          $placeholder.append(output);
        });
      });
    });

Get list of categories

$(document).ready(function(){
  $.ajax({
    url: 'https://demo.controlshiftlabs.com/categories.json',
    dataType: 'jsonp',
  })
  .done(function(data) {
    console.log(data);
  });
});

The above code would display a list of categories in the console. The JSON would be structured like this:

[
  {
    "category_name": "Animal Rights",
    "category_count": 7,
    "slug": "animal-rights-2",
    "url": "https://demo.controlshiftlabs.com/categories/animal-rights-2.json",
    "signature_count": 8,
    "locales": {
      "es": "Derechos Animales"
    }
  }, {
    "category_name": "cats",
    "category_count": 31,
    "slug": "cats",
    "url": "https://demo.controlshiftlabs.com/categories/cats.json",
    "signature_count": 270,
    "locales": {
      "es": "gatos"
    }
  }, {
    "category_name": "Civil Rights",
    "category_count": 23,
    "slug": "civil-rights-1",
    "url": "https://demo.controlshiftlabs.com/categories/civil-rights-1.json",
    "signature_count": 165,
    "locales": {
      "es": "Derechos Civiles"
    }
  }, {
    "category_name": "Culture",
    "category_count": 3,
    "slug": "cultura",
    "url": "https://demo.controlshiftlabs.com/categories/cultura.json",
    "signature_count": 10,
    "locales": {
      "es": "Cultura"
    }
  }
]

This retrieves a JSON array of category objects.

HTTP Request

GET https://demo.controlshiftlabs.com/categories.json

Working Example

View and edit a working example on codepen.io:

Categories List

    $(document).ready(function(){
      $.ajax({
        url: 'https://demo.controlshiftlabs.com/categories.json',
        dataType: 'jsonp',
      })
      .done(function(data) {
        var $placeholder = $('#categories');
        $.each(data, function(index, category){
          output = ''+category.category_name+'';
          output += ''
          output += 'Slug: '+category.slug+'';
          output += 'Signatures:'+category.signature_count+'';
          output += 'API URL:'+category.url+'';
          output += '
';
          $placeholder.append(output);
        });
      });
    });

List petitions in a category

$(document).ready(function(){
  $.ajax({
    url: 'https://demo.controlshiftlabs.com/categories/mice.json',
    dataType: 'jsonp',
  })
  .done(function(data) {
    console.log(data);
  });
});

The above code would return petitions data from the category with the slug mice. The JSON would be structured like this:

{
  "current_page": 1,
  "total_pages": 1,
  "previous_page": null,
  "next_page": null,
  "name": "mice",
  "results": [{
    "slug": "apple-stop-your-comic-sans-snobbery",
    "title": "Apple: Stop your Comic Sans snobbery",
    "url": "http://demo.controlshiftlabs.com/petitions/apple-stop-your-comic-sans-snobbery",
    "admin_status": "good",
    "image_url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/24551/hero/helveticacomicsans.gif?1398764339",
    "additional_image_sizes_url": [{
      "style": "form",
      "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/24551/form/helveticacomicsans.gif?1398764339"
    }, {
      "style": "large",
      "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/24551/large/helveticacomicsans.gif?1398764339"
    }, {
      "style": "open_graph",
      "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/24551/open_graph/helveticacomicsans.gif?1398764339"
    }, {
      "style": "original",
      "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/24551/original/helveticacomicsans.gif?1398764339"
    }],
    "who": "Tim Cook, CEO Apple",
    "what": "Use Comic Sans for the default font on the iPhone 6.",
    "goal": 100,
    "signature_count": 7,
    "creator_name": "John Wood",
    "created_at": "2014-04-29T05:35:47.693-04:00",
    "updated_at": "2016-01-06T11:02:17.617-05:00",
    "why": "Apple occupy a key position in the hearts of typeface poseurs everywhere. If they were to make a bold step and switch to using the people's font, Comic Sans on the new iPhone, this would cause mass..."
  }, {
    "slug": "clean-up-the-himalayas",
    "title": "Clean up the Himalayas",
    "url": "http://demo.controlshiftlabs.com/petitions/clean-up-the-himalayas",
    "admin_status": "good",
    "image_url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/6359/hero/gorak_shep_thumb.jpg?1431734031",
    "additional_image_sizes_url": [{
      "style": "form",
      "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/6359/form/gorak_shep_thumb.jpg?1431734031"
    }, {
      "style": "large",
      "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/6359/large/gorak_shep_thumb.jpg?1431734031"
    }, {
      "style": "open_graph",
      "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/6359/open_graph/gorak_shep_thumb.jpg?1431734031"
    }, {
      "style": "original",
      "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/6359/original/gorak_shep_thumb.jpg?1431734031"
    }],
    "who": "Nepal Ghurung",
    "what": "Clean up the Khumbu region.",
    "goal": 100,
    "signature_count": 1,
    "creator_name": "Grant Cox",
    "created_at": "2013-06-03T17:13:41.399-04:00",
    "updated_at": "2015-07-17T15:11:48.665-04:00",
    "why": "It's quite dirty, and I don't like it.",
    "location": {
      "query": "Nepal",
      "latitude": "28.394857",
      "longitude": "84.124008",
      "street": "",
      "postal_code": "",
      "country": "NP",
      "region": "",
      "street_number": "",
      "venue": "Nepal",
      "created_at": "2013-06-03T17:13:41.057-04:00"
    }
  }]
}

This retrieves a paginated list of petitions in a category.

HTTP Request

GET https://demo.controlshiftlabs.com/categories/<category slug>.json

Query Parameters

Parameter	Default	Description
category slug	null	string - required - submitted as a part of the endpoint path, not as a separate URL parameter
page	1	integer - optional - The page number of results for the specified category. Minimum of 1.

Working Example

View and edit a working example on codepen.io:

Petitions in category “”

    $(document).ready(function(){
      $.ajax({
        url: 'https://demo.controlshiftlabs.com/categories/mice.json',
        dataType: 'jsonp',
      })
      .done(function(data) {
        // Populate global category data
        $('#name').text(data.name);

        // Populate petitions data
        var $placeholder = $('#petitions');
        $.each(data.results, function(index, petition){
          output = ''+petition.title+'';
          output += '';
          output += 'Image: '+petition.image_url+'';
          output += 'Who: '+petition.who+'';
          output += 'What: '+petition.what+'';
          output += 'Why: '+petition.why+'';
          output += 'Signatures: '+petition.signature_count+'';
          output += 'URL: '+petition.url+'';
          output += '
';
          $placeholder.append(output);
        });
      });
    });

List petitions in an effort

$(document).ready(function(){
  var effortSlug = 'forecast-the-facts';
  $.ajax({
    url: 'https://demo.controlshiftlabs.com/efforts/'+effortSlug+'.json',
    dataType: 'jsonp',
  })
  .done(function(data) {
    console.log(data);
  });
});

The above code would return petitions data from the effort with the slug forecast-the-facts. The JSON would be structured like this:

{
  "title": "Forecast the Facts",
  "slug": "forecast-the-facts",
  "description": "It is imperative that weather reporters fulfill an important part of their job, and explain the ways that climate change has influenced the freakish \"Frankenstorm\" Sandy. Below is a list of the meteorologists in your area, with telephone numbers and e-mail addresses. Please call or email them a message in your own words, asking them to connect the dots between global warming pollution and disasters like Sandy. Take a look at the talking points below if you need any help.",
  "goal": 100,
  "signature_count": 11,
  "image_url": "https://d8s293fyljwh4.cloudfront.net/efforts/images/16/hero/forecast.png?1351625402",
  "petitions": [
    {
      "slug": "forecast-the-facts-of-frankenstorm-wmtv-madison-wi",
      "title": "Forecast the Facts of Frankenstorm: WMTV Madison, WI",
      "url": "http://demo.controlshiftlabs.com/petitions/forecast-the-facts-of-frankenstorm-wmtv-madison-wi",
      "admin_status": "good",
      "image_url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/102705/hero/forecast.png?1429199472",
      "additional_image_sizes_url": [{
        "style": "form",
        "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/102705/form/forecast.png?1429199472"
      }, {
        "style": "large",
        "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/102705/large/forecast.png?1429199472"
      }, {
        "style": "open_graph",
        "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/102705/open_graph/forecast.png?1429199472"
      }, {
        "style": "original",
        "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/102705/original/forecast.png?1429199472"
      }],
      "who": "WMTV Madison, WI",
      "what": "It is imperative that weather reporters fulfill an important part of their job, and explain the ways that climate change has influenced the freakish \"Frankenstorm\" Sandy. Below is a list of the meteorologists in your area, with telephone numbers and e-mail addresses. Please call or email them a message in your own words, asking them to connect the dots between global warming pollution and disasters like Sandy. Take a look at the talking points below if you need any help.",
      "goal": 100,
      "signature_count": 1,
      "creator_name": "Kristyn Arrighi",
      "created_at": "2015-04-16T11:51:12.700-04:00",
      "updated_at": "2015-08-03T20:23:06.034-04:00",
      "why": "We want TV meteorologists to understand that the public is depending on them for information about climate change, especially as it increasingly impacts our weather patterns. We know that meteorolo...",
      "location": {
        "query": "WMTV-TV (Madison), Forward Drive, Madison, WI, United States",
        "latitude": "43.05091",
        "longitude": "-89.486738",
        "street": "Forward Dr",
        "postal_code": "53711",
        "country": "US",
        "region": "WI",
        "street_number": "615",
        "venue": "WMTV-TV (Madison)",
        "created_at": "2015-04-16T11:51:11.222-04:00"
      }
    }, {
      "slug": "forecast-the-facts-of-frankenstorm-woak",
      "title": "Forecast the Facts of Frankenstorm: WOAK",
      "url": "http://demo.controlshiftlabs.com/petitions/forecast-the-facts-of-frankenstorm-woak",
      "admin_status": "good",
      "image_url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/115310/hero/forecast.png?1439846499",
      "additional_image_sizes_url": [{
        "style": "form",
        "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/115310/form/forecast.png?1439846499"
      }, {
        "style": "large",
        "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/115310/large/forecast.png?1439846499"
      }, {
        "style": "open_graph",
        "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/115310/open_graph/forecast.png?1439846499"
      }, {
        "style": "original",
        "url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/115310/original/forecast.png?1439846499"
      }],
      "who": "WOAK",
      "what": "It is imperative that weather reporters fulfill an important part of their job, and explain the ways that climate change has influenced the freakish \"Frankenstorm\" Sandy. Below is a list of the meteorologists in your area, with telephone numbers and e-mail addresses. Please call or email them a message in your own words, asking them to connect the dots between global warming pollution and disasters like Sandy. Take a look at the talking points below if you need any help.",
      "goal": 100,
      "signature_count": 2,
      "creator_name": "Adam Klaus",
      "created_at": "2015-08-17T17:21:39.791-04:00",
      "updated_at": "2015-08-17T17:29:02.816-04:00",
      "why": "We want TV meteorologists to understand that the public is depending on them for information about climate change, especially as it increasingly impacts our weather patterns. We know that meteorolo...",
      "location": {
        "query": "Oklahoma City, OK, United States",
        "latitude": "35.4675602",
        "longitude": "-97.5164276",
        "street": "",
        "postal_code": "",
        "country": "US",
        "region": "OK",
        "street_number": "",
        "venue": "",
        "created_at": "2013-12-17T14:49:51.324-05:00"
      }
    }
  ]
}

This retrieves a list of petitions in an effort.

HTTP Request

GET https://demo.controlshiftlabs.com/efforts/<effort slug>.json

Parameter	Default	Description
effort slug	null	string - required - submitted as a part of the endpoint path, not as a separate URL parameter

Working Example

View and edit a working example on codepen.io:

Petitions in effort “”

    $(document).ready(function(){
      var effortSlug = 'forecast-the-facts';
      $.ajax({
        url: 'https://demo.controlshiftlabs.com/efforts/'+effortSlug+'.json',
        dataType: 'jsonp',
      })
      .done(function(data) {
        // Populate global effort data
        $('#title').text(data.title);
        $('#description').text(data.description);

        // Populate petitions data
        var $placeholder = $('#petitions');
        $.each(data.petitions, function(index, petition){
          output = ''+petition.title+'';
          output += '';
          output += 'Image: '+petition.image_url+'';
          output += 'Who: '+petition.who+'';
          output += 'What: '+petition.what+'';
          output += 'Why: '+petition.why+'';
          output += 'Signatures: '+petition.signature_count+'';
          output += 'Goal: '+petition.goal+'';
          output += 'URL: '+petition.url+'';
          output += 'Location: '
          output += 'Query: '+petition.location.query+'';
          output += 'Latitude: '+petition.location.latitude+'';
          output += 'Longitude: '+petition.location.longitude+'';
          output += '';
          output += '
';
          $placeholder.append(output);
        });
      });
    });

Find near by petitions

$(document).ready(function(){
  var effortSlug = 'forecast-the-facts',
    formatted_address = '123 Main Street Fall Church, VA, USA',
    latitude = 38.88233400000001,
    longitude = -77.17109140000002;
  $.ajax({
    url: 'https://demo.controlshiftlabs.com/efforts/'+effortSlug+'/'+'near.json',
    dataType: 'jsonp',
    data: {
      'location[query]': formatted_address,
      'location[latitude]': latitude,
      'location[longitude]': longitude
    }
  })
  .done(function( data ) {
    console.log(data);
  });
});

The above code would return petitions data from the effort with the slug forecast-the-facts. The JSON would be structured like this:

{
  "closest_target": {
    "slug": "doug-kammerer-nbc4",
    "name": "Doug Kammerer - NBC4",
    "location": "Washington D.C., DC",
    "status": "target_petition_created",
    "petition": {
      "slug": "forecast-the-facts-of-frankenstorm-doug-kammerer-nbc4",
      "title": "Forecast the Facts of Frankenstorm: Doug Kammerer - NBC4",
      "url": "https://demo.controlshiftlabs.com/petitions/forecast-the-facts-of-frankenstorm-doug-kammerer-nbc4",
      "image_url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/2139/hero/forecast.png?1351625677",
      "who": "Doug Kammerer - NBC4",
      "goal": 100,
      "signature_count": 1
    }
  },
  "other_targets": [
    {
      "slug": "linda-church-wpix",
      "name": "Linda Church - WPIX",
      "location": "New York, NY",
      "status": "target_petition_created",
      "petition": {
        "slug": "forecast-the-facts-of-frankenstorm-linda-church-wpix",
        "title": "Forecast the Facts of Frankenstorm: Linda Church - WPIX",
        "url": "https://demo.controlshiftlabs.com/petitions/forecast-the-facts-of-frankenstorm-linda-church-wpix",
        "image_url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/2137/hero/forecast.png?1351625520",
        "who": "Linda Church - WPIX",
        "goal": 100,
        "signature_count": 1
      }
    },
    {
      "slug": "wnyc",
      "name": "WNYC ",
      "location": "Manhattan, New York, NY, United States",
      "status": "target_awaiting_petition",
      "create_petition_url": "https://demo.controlshiftlabs.com/efforts/forecast-the-facts/petitions/creating?target_id=5148"
    },
    {
      "slug": "whdtv-boston",
      "name": "WHDTV Boston",
      "location": "Boston, MA, United States",
      "status": "target_petition_created",
      "petition": {
        "slug": "forecast-the-facts-of-frankenstorm-whdtv-boston",
        "title": "Forecast the Facts of Frankenstorm: WHDTV Boston",
        "url": "https://demo.controlshiftlabs.com/petitions/forecast-the-facts-of-frankenstorm-whdtv-boston",
        "image_url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/28044/hero/forecast.png?1405015225",
        "who": "WHDTV Boston",
        "goal": 100,
        "signature_count": 1
      }
    },
    {
      "slug": "wmtv-madison-wi",
      "name": "WMTV Madison, WI",
      "location": "WMTV-TV (Madison), Forward Drive, Madison, WI, United States",
      "status": "target_petition_created",
      "petition": {
        "slug": "forecast-the-facts-of-frankenstorm-wmtv-madison-wi",
        "title": "Forecast the Facts of Frankenstorm: WMTV Madison, WI",
        "url": "https://demo.controlshiftlabs.com/petitions/forecast-the-facts-of-frankenstorm-wmtv-madison-wi",
        "image_url": "https://d8s293fyljwh4.cloudfront.net/petitions/images/102705/hero/forecast.png?1429199472",
        "who": "WMTV Madison, WI",
        "goal": 100,
        "signature_count": 1
      }
    }
  ]
}

Search for closest petition in an effort

This JSON endpoint allows you to reproduce the “search for the nearest petition in an effort” interface. Use this endpoint to build a place for users to enter their location. Once they’ve searched for a location, we’ll return the nearest petition. This endpoint can most easily be used with a Google Maps integration to help your users locate their addresses or other points of interest. For an example, click on the working example below.

HTTP Request

GET https://demo.controlshiftlabs.com/efforts/<effort slug>/near.json

Query Parameters

This endpoints accepts a few nested attributes in a location object, as described below.

Parameter	Default	Description
effort slug	null	string - required - submitted as a part of the endpoint path, not as a separate URL parameter
location[latitude]	null	float - required - a float representing the queried latitude
location[longitude]	null	float - required - a float representing the queried longitude
location[country]	null	string - optional - for efforts configured to use the “by country” search strategy, you can pass in the target country as a two letter ISO 3166-1 code.
location[region]	null	string - optional - for efforts configured to use the “by state” search strategy, you can pass in the target state as a two letter state abbreviation.

Working Example

View and edit a working example on codepen.io:

Find the closest petitions to you

    function buildPetition(target, description){
      output = ''+description+' Petition: '+target.name+'';
      output += ''+target.location+'
 ';
      output += 'Title: '+target.petition.title+'';
      output += 'Who: '+target.petition.who+'';
      output += 'Goal: '+target.petition.goal+'';
      output += 'Signatures so far: '+target.petition.signature_count+'';
      output += '
';
      return output;
    }

    function getPetitionsCallback(place) {
      // Get the place details from the autocomplete object.
      var place = autocomplete.getPlace();

      // get the targets and petitions (if they exist) for the effort near the place specified.
      $.ajax({
        url: 'https://demo.controlshiftlabs.com/efforts/forecast-the-facts/near.json',
        dataType: 'jsonp',
        data: {
          'location[latitude]': place.geometry.location.lat(),
          'location[longitude]': place.geometry.location.lng()
        }
      })
      .done(function(data) {
        $placeholder = $('#petitions');
        //populate closest target
        $placeholder.append( buildPetition(data.closest_target, 'Closest') );
        $.each(data.other_targets, function(index, other_petition){
          $placeholder.append( buildPetition(other_petition, 'Near By') );
        });
      });
    };

    $(document).ready(function(){
      $('form').on('submit', function(e){e.preventDefault()});
      var input = document.getElementById('effort-lookup-input');
      var options = { };
      autocomplete = new google.maps.places.Autocomplete(input, options);
      autocomplete.addListener('place_changed', getPetitionsCallback);
    });

Logged in user

$(document).ready(function(){
  $.ajax({
    url: 'https://demo.controlshiftlabs.com/api/graph/me.json',
    dataType: 'jsonp',
  })
  .done(function(data) {
    console.log(data);
  });
});

The above code would user details for a signed in user. The returned JSON would be structured like this:

{
  "status":"authenticated",
  "me":{
    "first_name":"John",
    "last_name":"Smith",
    "email":"john@smith.com"
  },
  "petitions":[
    {
       "slug":"sample-campaign",
       "title":"Sample Campaign",
       "created_at":"2016-03-25T15:37:28.849-04:00",
       "updated_at":"2016-03-25T15:37:35.304-04:00"
    }
  ],
  "events":[
    {
       "slug":"sample-campaign",
       "title":"Sample Campaign",
       "created_at":"2016-03-25T15:37:28.849-04:00",
       "updated_at":"2016-03-25T15:37:35.304-04:00"
    }
  ],
  "chapters":[
    {
       "name":"Brooklyn bruisers",
       "slug":"brooklyn-bruisers",
       "created_at":"2015-05-14T15:53:03.901-04:00",
       "updated_at":"2016-03-25T15:40:44.661-04:00"
    }
  ]
}

Response for a visitor who is not logged in:

{
  "status":"not_authenticated",
  "message":"user is not signed in"
}

Response for a hostname that is not whitelisted:

{
  "status":"error",
  "errors":[
    {
      "field":"referer",
      "messages":[
        "host name not whitelisted"
      ]
    }
  ]
}

We provide a specialized jsonp endpoint to return information about the currently signed in user. Customers can use this endpoint to power features where information about the current user’s petitions, sign in status, and other information about their account is displayed on external sites. Some examples of the sorts of functionality that this could be used to provide include:

Conditionally show either a login link for unauthenticated users or a link to the user’s my account page in the header of external pages.
Show information about a user’s campaigns when someone visits an external site, or a prompt to start a campaign if the currently signed in user has none.
Show a direct link to a user’s local group from an external site rather than linking to a generic search for your group page.

HTTP Request

GET http://demo.controlshiftlabs.com/api/graph/me.json

Working Example

View and edit a working example on codepen.io. Note: codepen.io is whitelisted for the demo account.

Logged in User

    $(document).ready(function(){
      $.ajax({
        url: 'https://demo.controlshiftlabs.com/api/graph/me.json',
        dataType: 'jsonp',
      })
      .done(function(data) {
        if(data.status == 'authenticated'){
          // Populate data for logged in user
          $('#first_name').html(data.me.first_name);
          $('#last_name').html(data.me.last_name);
          $('#email').html(data.me.email);

          var $petitionsPlaceholder = $('#petitions');
          $.each(data.petitions, function(index, petition){
            $petitionsPlaceholder.append(''+petition.title+'');
          });

          var $eventsPlaceholder = $('#events');
          $.each(data.events, function(index, event){
            $eventsPlaceholder.append(''+event.title+'');
          });

          var $chaptersPlaceholder = $('#chapters');
          $.each(data.chapters, function(index, chapter){
            $chaptersPlaceholder.append(''+chapter.name+'');
          });

          $('.js-logged-in').removeClass('hidden');
        }else if(data.status == 'not_authenticated'){
          // Show "not logged in" for users who aren't logged in
          $('.js-not-logged-in').removeClass('hidden');
        }else if(data.status == 'error'){
          // Show known error messages
          $('body').append('Errors
')
          $.each(data.errors, function(index, error){
            $('body').append(''+error.field+' '+error.messages.join('; ')+'')
          });
          $('body').append('')
        }
      });
    })
    .error(function(data){
      $('body').append('Errors
Unknown Error');
    });

Whitelisting a Hostname

We authenticate requests to the current user endpoint by validating the hostname set in the HTTP referrer header against a whitelist. The whitelist is set of hostnames that scripts can be served from. If you request the endpoint from a script served from a non-whitelisted hostname you’ll get an error message. This is necessary to prevent cross site scripting attacks that would allow someone to display logged in user information on unauthorized sites.

You can add hostnames to the whitelist through a self-serve interface in the “Settings” area of your instance. Settings > Add Hostname > enter a Hostname, and click save.

The hostname must be a hostname string like “localhost” “www.google.com” or the like. Wildcards or full HTTP URLs are not supported.

When you access the graph API endpoint from a page served from a whitelisted hostname the browser will automatically set a referrer header on those requests. We use this referrer header to validate the server the script was served from against the whitelist. You can verify that a whitelisted hostname is working properly with the curl command line tool:

curl --referer http://localhost/ http://demo.controlshiftlabs.com/api/graph/me.json

The above should return a JSON blob indicating that the user is not signed in if localhost is whitelisted. Otherwise you’ll get a not whitelisted error.

For authenticated users we currently return their first and last names, email addresses, and lists of petitions they’ve created, events they’ve created and local groups they are a member of.

Webhook Endpoints

Webhooks can be used by software engineers to integrate ControlShift with third-party systems. They allow engineers to build software that is triggered by events that take place within ControlShift. ControlShift would execute HTTP calls when certain event happen (e.g. a petition is launched; a category is changed).

To begin using these new webhooks, go to the admin homepage and click “Settings” under “Manage.” Then choose the “CRM Integration & Webhooks” tab and click “Configure Webhook Endpoints.” To begin sending data, you’ll need to add a “New Webhook Endpoint.” Each Webhook Endpoint URL receives a full firehose of all hooks that occur within ControlShift.

We recommend using a tool like RequestBin to help debug and develop your webhook integration. RequestBin allows engineers to see all of the webhooks generated by the application and sent to an endpoint.

If you need additional information about webhooks or how to use them, please send us a support email support@controlshiftlabs.com. We’d also love stories about how customers are using webhooks.

Identifying Unique Webhook Events

In the extermely unlikely event that a webhook is double set for some reason (error on our end, Internet gremlins, etc.), you can always identify webhooks by their unique jid (job id). If an incoming webhook has the same jid as a webhook that has already been processed, you can safely ignore the new one.

Webhook types summary

You can configure ControlShift Labs Webhooks to return the following event types:

Type	Description
blast_email.created	A new blast email is ready for moderation
category.created	A new category is created
category.updated	A category is changed
data.full_table_exported	A full-table dump is ready for retrieval at S3
data.incremental_table_exported	An incremental CSV update is ready for retrieval at S3
event.created	A new event is published
event.updated	An event is updated
flag.created	Someone flags a problem for admin attention
local_chapter.last_organiser.deleted	A local chapter’s last organiser leaves the group
local_chapter.organiser_request.created	A user applies to be a local chapter organiser
member.deleted	A member is deleted
member.deleted.resources_transferred	Resources previously owned by a deleted member have been transferred to another
petition.flagged	A petition is flagged for the first or fifth time
petition.inappropriate.creator_message	An inappropriate petition’s creator writes a message to admins
petition.launched	A new petition is launched
petition.launched.ham	A petition passes the post-launch spam check
petition.launched.requires_moderation	A new petition requires moderation
petition.reactivated	A hidden or ended petition is reactivated
petition.updated	A petition is updated
petition.updated.requires_moderation	An updated petition requires moderation
signature.created	Member signs a petition
signature.deleted	Signature permanently deleted. Signature deletions are distinct from unsubscribes and should be treated as if the signature never happened
unsubscribe.created	Member unsubscribes
locale.created	New i18n instance created
forum.message.requires_moderation	A new message requires moderation

blast_email.created

Example payload for blast_email.created:

{
  "type": "blast_email.created",
  "data": {
    "id": 100,
    "from_name": "Jane Doe",
    "from_address": "jane@example.com",
    "subject": "Please help spread the word",
    "recipient_count": 990
  },
  "jid": "030ee43a4fa9788e084d9cfc"
}

A new blast email is ready for moderation

category.created

Example payload for category.created:

{
  "type": "category.created",
  "data": {
    "id": 22,
    "name": "Environment",
    "slug": "environment",
    "external_id": null,
    "created_at": "2015-08-03T12:38:46-04:00",
    "updated_at": "2015-08-03T12:38:46-04:00"
  },
  "jid": "df67126973794c2efde76cc4"
}

A new category is created

category.updated

Example payload for category.updated:

{
  "type": "category.updated",
  "data": {
    "id": 22,
    "name": "Environment",
    "slug": "environment",
    "external_id": null,
    "created_at": "2015-08-03T12:38:46-04:00",
    "updated_at": "2015-08-03T12:38:46-04:00"
  },
  "jid": "df67126973794c2efde76cc4"
}

A category is changed

data.full_table_exported

Example payload for data.full_table_exported:

{
  "type": "data.full_table_exported",
  "data": {
    "url": "https://agra-data-exports.s3.amazonaws.com/default/petitions_20150803180434.csv?AWSAccessKeyId=XXX&Expires=1438711490&Signature=ABCDE%3D",
    "table": "petitions"
  },
  "jid": "cf0f50d7d225e20a188be328"
}

A full-table dump is ready for retrieval at S3

data.incremental_table_exported

Example payload for data.incremental_table_exported:

{
  "type": "data.incremental_table_exported",
  "data": {
    "url": "https://agra-data-exports.s3.amazonaws.com/default/petitions_20150803180434.csv?AWSAccessKeyId=XXX&Expires=1438711490&Signature=ABCDE%3D",
    "table": "petitions"
  },
  "jid": "cf0f50d7d225e20a188be328"
}

An incremental CSV update is ready for retrieval at S3

event.created

Example payload for event.created:

{
  "type": "event.created",
  "data": {
    "slug": "petition-hand-in",
    "title": "Petition Hand In",
    "url": "https://demo.controlshiftlabs.com/events/petition-hand-in"
  },
  "jid": "030ee43a4fa9788e084d9cfc"
}

A new event is published

event.updated

Example payload for event.updated:

{
  "type": "event.updated",
  "data": {
    "slug": "petition-hand-in",
    "title": "Petition Hand In",
    "url": "https://demo.controlshiftlabs.com/events/petition-hand-in"
  },
  "jid": "030ee43a4fa9788e084d9cfc"
}

An event is updated

flag.created

Example payload for flag.created:

{
  "type": "flag.created",
  "data": {
    "id": 7,
    "flagged": {
      "type": "ForumMessage",
      "id": 224
    },
    "flagged_by": {
      "email": "bobby@example.com",
      "name": "Bobby Singer"
    },
    "reason": "This violates our community commitments",
    "status": "unreviewed",
    "created_at": "2015-08-05T13:17:46-04:00",
    "updated_at": "2015-08-05T13:17:46-04:00"
  },
  "jid": "f848f4b38b9125f7089d59ad"
}

Someone flags a problem for admin attention

local_chapter.last_organiser.deleted

Example payload for local_chapter.last_organiser.deleted:

{
  "type": "local_chapter.last_organiser.deleted",
  "data": {
    "slug": "springfield-members",
    "name": "Springfield Members",
    "url": "https://demo.controlshiftlabs.com/local_chapters/springfield-members"
  },
  "jid": "030ee43a4fa9788e084d9cfc"
}

A local chapter’s last organiser leaves the group

local_chapter.organiser_request.created

Example payload for local_chapter.organiser_request.created:

{
  "type": "local_chapter.organiser_request.created",
  "data": {
    "id": 1234,
    "local_chapter": {
      "slug": "springfield-members",
      "name": "Springfield Members",
      "url": "https://demo.controlshiftlabs.com/local_chapters/springfield-members"
    },
    "user": {
      "id": 505,
      "email": "jane@example.com",
      "full_name": "Jane Goodall",
      "first_name": "Jane",
      "last_name": "Goodall",
      "phone_number": "12345-1",
      "postcode": "34058-2356"
    },
    "reason": "I want to be part of this amazing group",
    "created_at": "2015-08-03T12:38:46-04:00"
  },
  "jid": "030ee43a4fa9788e084d9cfc"
}

A user applies to be a local chapter organiser

member.deleted

Example payload for member.deleted:

{
  "type": "member.deleted",
  "data": {
    "member": {
      "id": 100,
      "email": "jane@example.com"
    },
    "resources": {
      "new_owner": {
        "id": 888,
        "email": "richard@example.com",
        "full_name": "Richard Chamberlain",
        "first_name": "Richard",
        "last_name": "Chamberlain",
        "phone_number": "12345-1",
        "postcode": "34058-2356"
      }
    }
  },
  "jid": "030ee43a4fa9788e084d9cfc"
}

A member is deleted

member.deleted.resources_transferred

Example payload for member.deleted.resources_transferred:

{
  "type": "member.deleted.resources_transferred",
  "data": {
    "member": {
      "id": 100,
      "email": "jane@example.com"
    },
    "resources": {
      "new_owner": {
        "id": 888,
        "email": "richard@example.com",
        "full_name": "Richard Chamberlain",
        "first_name": "Richard",
        "last_name": "Chamberlain",
        "phone_number": "12345-1",
        "postcode": "34058-2356"
      }
    }
  },
  "jid": "030ee43a4fa9788e084d9cfc"
}

Resources previously owned by a deleted member have been transferred to another

petition.flagged

Example payload for petition.flagged:

{
  "type": "petition.flagged",
  "data": {
    "slug": "don-t-destroy-our-park",
    "title": "Don't destroy our park!",
    "url": "https://demo.controlshiftlabs.com/petitions/don-t-destroy-our-park",
    "image_url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/hero/imag0945.jpg?1438621934",
    "additional_image_sizes_url": [
      {
        "style": "form",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/form/imag0945.jpg?1438621934"
      },
      {
        "style": "large",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/large/imag0945.jpg?1438621934"
      },
      {
        "style": "open_graph",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/open_graph/imag0945.jpg?1438621934"
      },
      {
        "style": "original",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/original/imag0945.jpg?1438621934"
      }
    ],
    "who": "Anytown City Council",
    "what": "The park on Main Street is an important community space.  Don't approve the proposal to sell it to office building developers.",
    "goal": 100,
    "signature_count": 0,
    "creator": {
      "id": 1,
      "email": "agra.user.adm@gmail.com",
      "full_name": "Tyshawn Morissette",
      "first_name": "Tyshawn",
      "last_name": "Morissette",
      "phone_number": "12345-1",
      "postcode": "34058-2356"
    },
    "created_at": "2015-08-03T13:11:02-04:00",
    "updated_at": "2015-08-03T13:12:15-04:00",
    "why": "Studies have shown that green space is important to child development and community well-being."
  },
  "jid": "9fccc1957d7d7ebc93fb0f05"
}

A petition is flagged for the first or fifth time

petition.inappropriate.creator_message

Example payload for petition.inappropriate.creator_message:

{
  "type": "petition.inappropriate.creator_message",
  "data": {
    "slug": "don-t-destroy-our-park",
    "title": "Don't destroy our park!",
    "url": "https://demo.controlshiftlabs.com/petitions/don-t-destroy-our-park",
    "image_url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/hero/imag0945.jpg?1438621934",
    "additional_image_sizes_url": [
      {
        "style": "form",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/form/imag0945.jpg?1438621934"
      },
      {
        "style": "large",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/large/imag0945.jpg?1438621934"
      },
      {
        "style": "open_graph",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/open_graph/imag0945.jpg?1438621934"
      },
      {
        "style": "original",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/original/imag0945.jpg?1438621934"
      }
    ],
    "who": "Anytown City Council",
    "what": "The park on Main Street is an important community space.  Don't approve the proposal to sell it to office building developers.",
    "goal": 100,
    "signature_count": 0,
    "creator": {
      "id": 1,
      "email": "agra.user.adm@gmail.com",
      "full_name": "Tyshawn Morissette",
      "first_name": "Tyshawn",
      "last_name": "Morissette",
      "phone_number": "12345-1",
      "postcode": "34058-2356"
    },
    "created_at": "2015-08-03T13:11:02-04:00",
    "updated_at": "2015-08-03T13:12:15-04:00",
    "why": "Studies have shown that green space is important to child development and community well-being."
  },
  "jid": "9fccc1957d7d7ebc93fb0f05"
}

An inappropriate petition’s creator writes a message to admins

petition.launched

Example payload for petition.launched:

{
  "type": "petition.launched",
  "data": {
    "slug": "don-t-destroy-our-park",
    "title": "Don't destroy our park!",
    "url": "https://demo.controlshiftlabs.com/petitions/don-t-destroy-our-park",
    "image_url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/hero/imag0945.jpg?1438621934",
    "additional_image_sizes_url": [
      {
        "style": "form",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/form/imag0945.jpg?1438621934"
      },
      {
        "style": "large",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/large/imag0945.jpg?1438621934"
      },
      {
        "style": "open_graph",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/open_graph/imag0945.jpg?1438621934"
      },
      {
        "style": "original",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/original/imag0945.jpg?1438621934"
      }
    ],
    "who": "Anytown City Council",
    "what": "The park on Main Street is an important community space.  Don't approve the proposal to sell it to office building developers.",
    "goal": 100,
    "signature_count": 0,
    "creator": {
      "id": 1,
      "email": "agra.user.adm@gmail.com",
      "full_name": "Tyshawn Morissette",
      "first_name": "Tyshawn",
      "last_name": "Morissette",
      "phone_number": "12345-1",
      "postcode": "34058-2356"
    },
    "created_at": "2015-08-03T13:11:02-04:00",
    "updated_at": "2015-08-03T13:12:15-04:00",
    "why": "Studies have shown that green space is important to child development and community well-being."
  },
  "jid": "9fccc1957d7d7ebc93fb0f05"
}

A new petition is launched

petition.launched.ham

Example payload for petition.launched.ham:

{
  "type": "petition.launched.ham",
  "data": {
    "slug": "don-t-destroy-our-park",
    "title": "Don't destroy our park!",
    "url": "https://demo.controlshiftlabs.com/petitions/don-t-destroy-our-park",
    "image_url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/hero/imag0945.jpg?1438621934",
    "additional_image_sizes_url": [
      {
        "style": "form",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/form/imag0945.jpg?1438621934"
      },
      {
        "style": "large",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/large/imag0945.jpg?1438621934"
      },
      {
        "style": "open_graph",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/open_graph/imag0945.jpg?1438621934"
      },
      {
        "style": "original",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/original/imag0945.jpg?1438621934"
      }
    ],
    "who": "Anytown City Council",
    "what": "The park on Main Street is an important community space.  Don't approve the proposal to sell it to office building developers.",
    "goal": 100,
    "signature_count": 0,
    "creator": {
      "id": 1,
      "email": "agra.user.adm@gmail.com",
      "full_name": "Tyshawn Morissette",
      "first_name": "Tyshawn",
      "last_name": "Morissette",
      "phone_number": "12345-1",
      "postcode": "34058-2356"
    },
    "created_at": "2015-08-03T13:11:02-04:00",
    "updated_at": "2015-08-03T13:12:15-04:00",
    "why": "Studies have shown that green space is important to child development and community well-being."
  },
  "jid": "9fccc1957d7d7ebc93fb0f05"
}

A petition passes the post-launch spam check

petition.launched.requires_moderation

Example payload for petition.launched.requires_moderation:

{
  "type": "petition.launched.requires_moderation",
  "data": {
    "slug": "don-t-destroy-our-park",
    "title": "Don't destroy our park!",
    "url": "https://demo.controlshiftlabs.com/petitions/don-t-destroy-our-park",
    "image_url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/hero/imag0945.jpg?1438621934",
    "additional_image_sizes_url": [
      {
        "style": "form",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/form/imag0945.jpg?1438621934"
      },
      {
        "style": "large",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/large/imag0945.jpg?1438621934"
      },
      {
        "style": "open_graph",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/open_graph/imag0945.jpg?1438621934"
      },
      {
        "style": "original",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/original/imag0945.jpg?1438621934"
      }
    ],
    "who": "Anytown City Council",
    "what": "The park on Main Street is an important community space.  Don't approve the proposal to sell it to office building developers.",
    "goal": 100,
    "signature_count": 0,
    "creator": {
      "id": 1,
      "email": "agra.user.adm@gmail.com",
      "full_name": "Tyshawn Morissette",
      "first_name": "Tyshawn",
      "last_name": "Morissette",
      "phone_number": "12345-1",
      "postcode": "34058-2356"
    },
    "created_at": "2015-08-03T13:11:02-04:00",
    "updated_at": "2015-08-03T13:12:15-04:00",
    "why": "Studies have shown that green space is important to child development and community well-being."
  },
  "jid": "9fccc1957d7d7ebc93fb0f05"
}

A new petition requires moderation

petition.reactivated

Example payload for petition.reactivated:

{
  "type": "petition.reactivated",
  "data": {
    "slug": "don-t-destroy-our-park",
    "title": "Don't destroy our park!",
    "url": "https://demo.controlshiftlabs.com/petitions/don-t-destroy-our-park",
    "image_url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/hero/imag0945.jpg?1438621934",
    "additional_image_sizes_url": [
      {
        "style": "form",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/form/imag0945.jpg?1438621934"
      },
      {
        "style": "large",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/large/imag0945.jpg?1438621934"
      },
      {
        "style": "open_graph",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/open_graph/imag0945.jpg?1438621934"
      },
      {
        "style": "original",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/original/imag0945.jpg?1438621934"
      }
    ],
    "who": "Anytown City Council",
    "what": "The park on Main Street is an important community space.  Don't approve the proposal to sell it to office building developers.",
    "goal": 100,
    "signature_count": 0,
    "creator": {
      "id": 1,
      "email": "agra.user.adm@gmail.com",
      "full_name": "Tyshawn Morissette",
      "first_name": "Tyshawn",
      "last_name": "Morissette",
      "phone_number": "12345-1",
      "postcode": "34058-2356"
    },
    "created_at": "2015-08-03T13:11:02-04:00",
    "updated_at": "2015-08-03T13:12:15-04:00",
    "why": "Studies have shown that green space is important to child development and community well-being."
  },
  "jid": "9fccc1957d7d7ebc93fb0f05"
}

A hidden or ended petition is reactivated

petition.updated

Example payload for petition.updated:

{
  "type": "petition.updated",
  "data": {
    "slug": "don-t-destroy-our-park",
    "title": "Don't destroy our park!",
    "url": "https://demo.controlshiftlabs.com/petitions/don-t-destroy-our-park",
    "image_url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/hero/imag0945.jpg?1438621934",
    "additional_image_sizes_url": [
      {
        "style": "form",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/form/imag0945.jpg?1438621934"
      },
      {
        "style": "large",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/large/imag0945.jpg?1438621934"
      },
      {
        "style": "open_graph",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/open_graph/imag0945.jpg?1438621934"
      },
      {
        "style": "original",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/original/imag0945.jpg?1438621934"
      }
    ],
    "who": "Anytown City Council",
    "what": "The park on Main Street is an important community space.  Don't approve the proposal to sell it to office building developers.",
    "goal": 100,
    "signature_count": 0,
    "creator": {
      "id": 1,
      "email": "agra.user.adm@gmail.com",
      "full_name": "Tyshawn Morissette",
      "first_name": "Tyshawn",
      "last_name": "Morissette",
      "phone_number": "12345-1",
      "postcode": "34058-2356"
    },
    "created_at": "2015-08-03T13:11:02-04:00",
    "updated_at": "2015-08-03T13:12:15-04:00",
    "why": "Studies have shown that green space is important to child development and community well-being."
  },
  "jid": "9fccc1957d7d7ebc93fb0f05"
}

A petition is updated

petition.updated.requires_moderation

Example payload for petition.updated.requires_moderation:

{
  "type": "petition.updated.requires_moderation",
  "data": {
    "slug": "don-t-destroy-our-park",
    "title": "Don't destroy our park!",
    "url": "https://demo.controlshiftlabs.com/petitions/don-t-destroy-our-park",
    "image_url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/hero/imag0945.jpg?1438621934",
    "additional_image_sizes_url": [
      {
        "style": "form",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/form/imag0945.jpg?1438621934"
      },
      {
        "style": "large",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/large/imag0945.jpg?1438621934"
      },
      {
        "style": "open_graph",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/open_graph/imag0945.jpg?1438621934"
      },
      {
        "style": "original",
        "url": "https://d34smfggpfnvat.cloudfront.net/petitions/images/788/original/imag0945.jpg?1438621934"
      }
    ],
    "who": "Anytown City Council",
    "what": "The park on Main Street is an important community space.  Don't approve the proposal to sell it to office building developers.",
    "goal": 100,
    "signature_count": 0,
    "creator": {
      "id": 1,
      "email": "agra.user.adm@gmail.com",
      "full_name": "Tyshawn Morissette",
      "first_name": "Tyshawn",
      "last_name": "Morissette",
      "phone_number": "12345-1",
      "postcode": "34058-2356"
    },
    "created_at": "2015-08-03T13:11:02-04:00",
    "updated_at": "2015-08-03T13:12:15-04:00",
    "why": "Studies have shown that green space is important to child development and community well-being."
  },
  "jid": "9fccc1957d7d7ebc93fb0f05"
}

An updated petition requires moderation

signature.created

Example payload for signature.created:

{
  "type": "signature.created",
  "data": {
    "id": 645,
    "first_name": "Jennifer",
    "last_name": "Goines",
    "email": "jenny@example.com",
    "phone_number": null,
    "postcode": "23456",
    "country": "",
    "join_organisation": null,
    "join_group": null,
    "locale": "en",
    "token": "2919ec5f25d493c19690a21135d7fd902a336927",
    "source": "",
    "bucket": "",
    "user_ip": "127.0.0.1",
    "user_agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/43.0.2357.130 Chrome/43.0.2357.130 Safari/537.36",
    "daisy_chain_used": null,
    "from_embed": null,
    "confirmed_at": null,
    "last_signed_at": "2015-08-03T13:30:01-04:00",
    "additional_fields": {
    },
    "member": {
      "id": 747,
      "created_at": "2015-08-03T13:30:01-04:00"
    },
    "petition": {
      "url": "http://localhost/petitions/don-t-destroy-our-park",
      "slug": "don-t-destroy-our-park"
    },
    "created_at": "2015-08-03T13:30:01-04:00",
    "updated_at": "2015-08-03T13:30:01-04:00"
  },
  "jid": "1ebf2eb10c3d8939f42dff68"
}

Member signs a petition

signature.deleted

Example payload for signature.deleted:

{
  "type": "signature.created",
  "data": {
    "id": 645,
    "first_name": "Jennifer",
    "last_name": "Goines",
    "email": "jenny@example.com",
    "phone_number": null,
    "postcode": "23456",
    "country": "",
    "join_organisation": null,
    "join_group": null,
    "locale": "en",
    "token": "2919ec5f25d493c19690a21135d7fd902a336927",
    "source": "",
    "bucket": "",
    "user_ip": "127.0.0.1",
    "user_agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/43.0.2357.130 Chrome/43.0.2357.130 Safari/537.36",
    "daisy_chain_used": null,
    "from_embed": null,
    "confirmed_at": null,
    "last_signed_at": "2015-08-03T13:30:01-04:00",
    "additional_fields": {
    },
    "member": {
      "id": 747,
      "created_at": "2015-08-03T13:30:01-04:00"
    },
    "petition": {
      "url": "http://localhost/petitions/don-t-destroy-our-park",
      "slug": "don-t-destroy-our-park"
    },
    "created_at": "2015-08-03T13:30:01-04:00",
    "updated_at": "2015-08-03T13:30:01-04:00"
  },
  "jid": "1ebf2eb10c3d8939f42dff68"
}

Signature permanently deleted. Signature deletions are distinct from unsubscribes and should be treated as if the signature never happened

unsubscribe.created

Example payload for unsubscribe.created:

{
  "type": "unsubscribe.created",
  "data": {
    "id": 3,
    "email": "river@example.com",
    "unsubscribe_organisation": true,
    "unsubscribable_type": null,
    "created_at": "2015-08-03T13:38:49-04:00"
  },
  "jid": "030ee43a4fa9788e084d9cfc"
}

Member unsubscribes

locale.created

Example payload for locale.created:

{
  "type": "locale.created",
  "data": {
    "id": 22,
    "name": "es",
    "action_kit_external_id": null,
    "created_at": "2015-08-03T12:38:46-04:00",
    "updated_at": "2015-08-03T12:38:46-04:00"
  },
  "jid": "df67126973794c2efde76cc4"
}

New i18n instance created

forum.message.requires_moderation

Example payload for forum.message.requires_moderation:

{
  "type": "forum.message.requires_moderation",
  "data": {
    "id": 10,
    "thread_title": "Hello everyone",
    "content": "I just wanted to say hi",
    "admin_status": "unreviewed",
    "sent_at": "2015-08-05T13:17:46-04:00",
    "member": {
      "id": 10829,
      "created_at": "2015-08-05T13:17:46-04:00"
    },
    "discussable": {
      "discussable_type": "LocalChapter",
      "discussable_id": 56
    }
  },
  "jid": "f848f4b38b9125f7089d59ad"
}

A new message requires moderation

Bulk Data

ControlShift’s Bulk Data Webhooks make it easy to pull your data into external services. \

Webhooks

We provides two special bulk data webhooks to help you keep an external reporting or analytics database server up to date with information from ControlShift’s internal tables. The data.full_table_exported and data.incremental_table_exported webhooks can be consumed to keep an external database mirror containing ControlShift data up to date. This service was built in a database agnostic way, but it should be possible to build a ControlShift -> Amazon Redshift data pipeline using the following technique.

ControlShift to Redshift Pipeline

Setting up an Amazon Redshift integration should take one to two hours. We’ll use a custom AWS Lamda to receive data from ControlShift’s webhook and Amazon’s Lamda Redshift Loader to load our data into Redshift. For this example, we’ll focus on the daily full_table dump, and truncating the previous days data. You could also add a second pipeline to add incremental data if you need more frequent updates.

Example Flow

Pending…

Redshift Pipeline Setup

The core steps to get this up and running are:

Setting up S3 buckets to retain file storage
Setting up your Redshift database cluster and preloading a schema.
Setting up your custom lambda webhook receiver
Setting up your aws-lambda-redshift-loader.
Configure your bulk data webhook in the Control Shift settings console.

Note: All resources should be set up in the same AWS region (ex: us-east-1) so they can access each other. “US Standard” and “us-east-1” are the same thing.

Before starting: You’ll need git, npm, and access to a PostgreSQL connection tool(Redshift is based on PostgreSQL). These instructions use psql on the command line.

These instructions were prepared for aws-lambda-redshift-loader v2.4.0.

Setting Up S3 Buckets

Login to you AWS console and create two s3 buckets for future use.
1. controlshift-receiver - will hold received csv files
2. controlshift-manifests - will hold resulting manifest files from your aws-lambda-redshift-loader
That’s it. You’re done here.

Setting Up Custom Lambda Receiver

Lambda Receiver Code

var https = require('https');
var url = require('url');
var AWS = require('aws-sdk');

var s3 = new AWS.S3();
var targetBucket = 'controlshift-receiver'; // receiver bucket name

exports.handler = function(event, context) {
  var receivedJSON = JSON.stringify(event, null, 2);
  console.log('Received event:', receivedJSON);
  if(event.type == 'data.full_table_exported'){
    var downloadUrl = event.data.url;
    var fileName = url.parse(downloadUrl).pathname.replace('/','')
    https.get(downloadUrl, function(httpResponse) {
      var upload_details = {Bucket: targetBucket, Key: fileName, Body: httpResponse};
      s3.upload(upload_details, function(err, data) {
        if (err){
          // an error occurred
          console.log(err, err.stack);
          context.fail(err);
        }else{
          // successful response
          console.log(httpResponse);
          context.succeed({"status": "success", "payload": receivedJSON});
        }
      });
    });
  }else{
    context.succeed({"status": "skipped", "payload": receivedJSON});
  }
};

From the Lambda management console, you’ll need to do the following.

Create the Receiver Lambda

Click to create a new Lambda function.
Skip the existing blueprints (button towards the bottom of page)
Name your function something like “controlShiftWebhookReciever” and leave the runtime to node default.
Copy and paste the “Lambda Receiver” function from the right into the inline lambda code editor.
Leave the handler as the default.
For role, select “S3 Execution role”. This will open a new window.
In this new window, called “IAM Management Console”, name your role something relevant, like “controlShiftReceriver_s3.” Then click “Allow” in the far bottom left corner.
Next is setting the memory size. You should be fine with 128MB, but you can always update the memory size later.
Set the Timeout a little higher - 30 seconds should be fine.
Use “No VPC”.

Setup the API Endpoint

Now that we have our Lambda setup, we need to be able to POST to it from your Control Shift instance.

If not already there, go back to the Receiver Lambda we just created.
Click on the “API Endpoints” tab and click to add a new endpoint.
Select an API Gateway.
Set the Method to POST and Security to Open.
Click create. Done!

Setting Up RedShift Database

Click on over to the Redshift service in the AWS console.

Launch Your Cluster

Click to create your cluster.
Setup a identifier, database name, master user name and password as you desire and click next. Let’s use these values for this example:
- Cluster: control-shift-cluster
- Database name: controlshiftdb
- Port: Leave the default
- Master Username: masteruser
Use the defaults on the next two screens, “Node Configuration” and “Additional Configuration, just click next.
Then Launch!
Once the creation and launch process finishes, you can click to view your new cluster’s properties, which will include an endpoint address. Save that address somewhere. Example: control-shift-cluster.xXxXxX00Xx.us-east-1.redshift.amazonaws.com

Prepare Redshift Connection Security

We need to whitelist our IP address for our default security group in order to connect to our Redshift DB. While we’re managing our security through a VPC (Virtual Private Cloud), The best UI for our purposes is actually in the EC2 configuration console.

Go to the AWS Console > EC2
Click to "Security Groups” in the left hand menu.
Click on the default Security Group (it should say default in the “Group Name” column,
Click on the “Inbound Rules” tab at the bottom and click edit.
Click “Add Rule.”
Select “All Traffic” for “type” and for “source” select “My IP.” This should prefill your CIDR. Save the new rule.
Click “Add Rule” again.
Select “Redshift” for “Type.” Enter your cluster’s port (typically 5439). Type “sg-” into the source field, and it should pull up your default security group.

Note: If you’re having trouble getting your own IP/CIDR setup, try either looking up your subnet mask and using this table. Your CIDR will almost always be your id address followed by /32 or /64 for ipv6. Example: 123.123.123.123/32. If all else fails, use the global 0.0.0.0/0 - opening your cluster to the public Internet. The latter isn’t recommended. If you do have to do this, make sure you remove the rule when you’re done. –>

Prepare Redshift Schema

We need to prep our tables data schema to receive our ConrtolShift Data. We’ll use psql for this.

Install psql you don’t have it already.
Download the current schema for our petitions table as petitions_schema.sql.
Finally, you’ll need to load that schema into your table with the following command.

psql -h <cluster_endpoint> -U <database_user> -d <database_name> -p <cluster_port> -f petitions_schema.sql

Note: When connecting, you should almost immediately get a request for your password. If you’re connection is timing out before you enter your password, there is a authorization issue. If you are timing out after connecting, you can extend your keep alive timeout with the following shell command.

sudo /sbin/sysctl -w net.ipv4.tcp_keepalive_time=200 net.ipv4.tcp_keepalive_intvl=200 net.ipv4.tcp_keepalive_probes=5

Creating a Database User & Granting Privileges

Creating a user and granting INSERT Privileges

CREATE USER redshift_user PASSWORD PlainTextPassword123;
ALTER TABLE petitions OWNER TO redshift_user;
GRANT INSERT ON petitions TO redshift_user;

We need to allow our user to access our aws-lambda-redshift-loader to access our Redshift database. Connect via psql and create your user and grant permissions. See code example.

psql -h <cluster_endpoint> -U <database_user> -d <database_name> -p <cluster_port>

Security Notes:

Typically, we’d like to limit the privileges we’re granting our redshift_user to just INSERT and TRUNCATE. Unfortunately, Redshift doesn’t allow us to specifically grant TRUNCATE permissions. Thus, our redshift_user must own the table.
In PostgreSQL, we wouldn’t need to grant INSERT privileges to the owner, but Redshift seems to need this.
For Redshift cluster versions > 1.0.1046, you can store your password as a MD5 hash instead of cleartext. At the time of this writing, most clusters are on version 1.0.1044. See the md5 section of the create user documentation.

Setting up aws-lambda-redshift-loader

Deploying the aws-lambda-redshift-loader Function

AWS Lambda Execution Role

{
  "Version":"2012-10-17",
  "Statement":[
    {
      "Sid":"Stmt1424787824000",
      "Effect":"Allow",
      "Action":[
        "dynamodb:DeleteItem",
        "dynamodb:DescribeTable",
        "dynamodb:GetItem",
        "dynamodb:ListTables",
        "dynamodb:PutItem",
        "dynamodb:Query",
        "dynamodb:Scan",
        "dynamodb:UpdateItem",
        "sns:GetEndpointAttributes",
        "sns:GetSubscriptionAttributes",
        "sns:GetTopicAttributes",
        "sns:ListTopics",
        "sns:Publish",
        "sns:Subscribe",
        "sns:Unsubscribe",
        "s3:Get*",
        "s3:Put*",
        "s3:List*",
        "kms:Decrypt",
        "kms:DescribeKey",
        "kms:GetKeyPolicy"
      ],
      "Resource":[
        "*"
      ]
    },
    {
      "Effect":"Allow",
      "Action":[
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource":"arn:aws:logs:*:*:*"
    },
    {
      "Effect":"Allow",
      "Action":[
        "ec2:CreateNetworkInterface",
        "ec2:DescribeNetworkInterfaces",
        "ec2:DetachNetworkInterface",
        "ec2:DeleteNetworkInterface"
      ],
      "Resource":"*"
    }
  ]
}

Go to the AWS Lambda Console in the same region as your S3 bucket and Amazon Redshift cluster.
Select Create a Lambda function and enter the name controlShiftRedshiftLoader (for example).
Under Code entry type select ‘Upload a zip file’ and upload the AWSLambdaRedshiftLoader-2.4.0.zip from your local dist folder or download it.
Use the default values for the handler, and in the Role drop-down, select “* Basic Execution Role.” A IAM creation wizard will open in a new window.
Follow the wizard, selecting to “Create a new IAM Role” and name it as you like. Click to “View the Policy” and then click edit.
Copy and paste the the AWS Lambda Execution Role permissions from the example “AWS Lambda Execution Role” or from the official readme.
Then click “Add Policy.” Select the AmazonRedshiftFullAccess role and add it, then add another role - AmazonDMSRedshiftS3Role.
Navigate back to your Lambda setup tab and set the max timeout (5 minutes) to accommodate potentially long COPY times.
From the VPC dropdown, select your default VPC.
Leave the rest of the settings alone.
Click next and then click to create your Lambda.

Establishing an Event Source

Once deployed, you need to add an event source:

Select “Event Source” tab and click to “Add event source.”
The event source type should be set to S3.
Select the S3 bucket you want to use for input data - controlshift-receiver.
Select the 'Object Created (All)’ notification type.

Configuration Prep

If you have already installed npm’s aws-sdk with your credentials, great, but if not, you can always export your credentials as environment variables before running setup node.js. export AWS_ACCESS_KEY_ID=XXXXXXXXX AWS_SECRET_ACCESS_KEY=XXXXXXXXXXX
You’ll need to export your region as well export AWS_REGION=us-east-1.

Running setup.js

After setting up your environment, head to your local copy of aws-lambda-redshift-loader.
run node setup.js
The script will request various configuration options. Some sensible responses for our use case are below. Empty responses were just skipped.


Enter the Region for the Configuration > us-east-1
Enter the S3 Bucket & Prefix to watch for files > controlshift-receiver
Enter a Filename Filter Regex >
Enter the Cluster Endpoint > control-shift-cluster.xXxXxX00Xx.us-east-1.redshift.amazonaws.com
Enter the Cluster Port > 5439
Does your cluster use SSL (Y/N) > Y
Enter the Database Name > controlshiftdb
Enter the Table to be Loaded > petitions
Enter the comma-delimited column list (optional) > (Copy and paste from our petitions_headers.csv)
Should the Table be Truncated before Load? (Y/N) > Y # Don’t truncate if this lambda will handle incremental builds.
Enter the Database Username > masteruser # Or any user you created in psql
Enter the Database Password > SuPeRs8CuR1tY # your password
Enter the Data Format (CSV, JSON or AVRO) > CSV
Enter the CSV Delimiter > ,
Enter the S3 Bucket for Redshift COPY Manifests > controlshift-manifests
Enter the Prefix for Redshift COPY Manifests > manifests
Enter the Prefix to use for Failed Load Manifest Storage > failed
Enter the Access Key used by Redshift to get data from S3. If NULL then Lambda execution role credentials will be used >
Enter the Secret Key used by Redshift to get data from S3. If NULL then Lambda execution role credentials will be used >
Enter the SNS Topic ARN for Successful Loads >
Enter the SNS Topic ARN for Failed Loads >
How many files should be buffered before loading? > 1
How old should we allow a Batch to be before loading (seconds)? > 120
Additional Copy Options to be added >
If Encrypted Files are used, Enter the Symmetric Master Key Value >

Configure ControlShift’s webhook

Finally, you’ll need to log into your admin panel. Settings > CRM Integrations & Webhooks > Configure Webhook Endpoints. Then add a bulk data webhook pointing at your Lambda endpoint.

Troubleshooting Errors

Example LambdaRedshiftBatches error

{
  "control-shift-cluster.ckcgajpfnj82.us-east-1.redshift.amazonaws.com":{
    "status":-1,
    "error":{
      "name":"error",
      "length":143,
      "severity":"FATAL",
      "code":"28000",
      "file":"/home/awsrsqa/padb/src/pg/src/backend/libpq/auth.c",
      "line":"402",
      "routine":"auth_failed"
    }
  }
}

A few tips if things aren’t working:

Most errors will probably occur when attempting to load data from Lambda into Redshift. The DynamoDB batch history captures errorMessages in the the LambdaRedshiftBatches table. Click on the entries field. You can also check AWS Cloudwatch > Logs and then click on the appropriate cloud stream.
Since Redshift is based on PostgreSQL 8.0.2, there is a healthy amount of overlapping error codes. You can lookup errors codes here. Example errors:
- 28000 - Unable to connect to the database. Is the username/password/database name entered correctly when running setup.js? Is the user granted proper permissions on your database?
- 42P01 - Redshift doesn’t have the table you’re trying to import data into. Did you import the schema as described above?
- XX000 - This is an internal error. In our case, the most likely causes are either an invalid schema usually caused by an unsupported data type. Double check that your imported schema is correct. If you aren’t using the provided schema double check that you’re using the [supported types] and that your imported data matches those types.
- 42601 - Syntax error - This is most likely a typo when running setup.js. If you’re entering your CSV columns during setup, be sure they match the schema you’ve loaded into the database. Make sure your CSV Delimiter is set correctly and your CSV is properly formated.
- 42501 - Insufficient permissions errors.
Be sure to check the logging messages you see in your Lambda logs.
Try manually uploading a petitions export to your controlshift-receiver S3 bucket, and check DynamoDB batch history for errors. You can fetch these exports from your ControlShift Labs instance, in Settings > Exports.