Integrations? We can help.

by Reid M. Porter

Over the past few months, we’ve done some serious spring cleaning — by southern hemisphere standards — of our API endpoints and documentation, all in an effort to make our API more user-friendly and accessible to our clients. [I’m sure that most of you reading this are already bored. Don’t stop reading just yet! There’s something here for everyone, whether you know what an API is or not!]

DevResults has always had a public API, and we’ve always encouraged our clients to make use of it — to build integrations, to exchange data across systems, or to power external dashboards and websites. Occasionally we’ve built our own integrations, most recently with Dropbox, Power BI, and SurveyCTO.

To date, we’ve only had a few takers who used our public API to develop their own integration workflows. Even so, we believe in — and our company exists to — reduce data friction and solve the software puzzle for teams. When data can flow easily from user to user and system to system, busy people are more likely to consult that data to make decisions or judge their effectiveness. To that end, we committed to making our API easier to use, writing documentation that’s easier to follow, and providing more support to users who want to build their own integrations.

Throughout our sector, there has been a shift toward adopting technology solutions throughout the project lifecycle. From providing enumerators with data collection tools, to creating a single organizational source of truth, to providing visualizations that tell compelling stories from across the globe, the digital shift requires that technology tools communicate with one another to ensure the flow of data.

What even is an API?

An API is an Application Programming Interface. It’s become trite to say “APIs are how computer programs talk to other computer programs,” and such a simplification obscures both the power and the intricacies of what an API actually does. Unless you live in a science fiction movie, computers are actually pretty bad at talking to each other: they speak mutually incomprehensible dialects, they don’t know what to say unless they’re told, and they don’t know to listen unless they’re asked.

A slightly more useful metaphor to understand APIs might be the dinner table. Each person at your table wants to accomplish something similar (eat food), but each has different needs at different times. One person might be sitting in front of the potatoes, and doesn’t have to ask anyone else to help them get some on their plate. That said, they might offer the potatoes to someone on the other side of the table by saying, “Hey, would you like some potatoes?” while holding the bowl aloft. Someone else might say “I’d like seconds, would you pass them here?” and respond with “Thanks!” when they’ve been successfully handed over. There’s no need for a non-eating third-party server or waiter (i.e. a human user) to pass the bowls around; everyone can serve themselves and pass dishes back and forth as requested. (Can you see where I’m going with this?)

How does an API work?

An API works in the same way. Each platform has a list of requests that it understands — pass the potatoes, here are some potatoes, are there any potatoes left — helpfully written down in its API documentation. When a programmer knows what kinds of requests different platforms understand and what ‘dishes’ they have available, they can script out a conversation that can occur repeatedly (at a certain time each night) or as needed (when a user clicks on something).

In fact, this is exactly how every app that you use on a regular basis works: you push a button, and a flurry of requests are initiated to retrieve or write data in various places. It’s how the front-end user interface that you see interacts with the back-end database that you don’t see. (If you’d like to learn more, there’s a great video by technology communicator and YouTube celebrity Tom Scott that explores the promise and pitfalls of APIs, especially from 1:43 to 3:10.)

Is it really that simple?

Yes and no. It’s not wrong to say “APIs are how programs talk to programs,” it’s just not the full story. In order for one program to talk to another via their APIs, a programmer (read: human) has to write out the dialog. That script turns out to be pretty brittle; if one speaker suddenly starts speaking in a different language or even deviating ever so slightly from the words on paper, the other conversants won’t know how to respond. That’s when your API integration breaks. A programmer (a human, again) has to be alerted that something’s not working the way it should, and rewrite the dialog to accommodate the shift in language. Now the scripted dialogue may proceed.

This is exactly why we at DevResults have neither committed to build nor maintain client-specific integrations — it's a significant investment and risk to rely on a third-party’s API infrastructure that we have little control over. Even when we add “baked-in” integrations to our app, we are selective about who we integrate with — ideally, someone that has a well-documented API, that knows not to change that API unexpectedly, and is friendly and easy to work with if something unexpected happens. In return, we endeavor to also have a well-documented API that changes rarely with much notice, and failing that, to be a good neighbor when something breaks.

So how does an organization build integrations with DevResults?

Most users won’t ever read or even be aware of our API documentation, but we’ve invested time in documenting and maintaining it so that your IT teams, consultants, and programmers (if you’re lucky enough to have any) can tie your internal systems into our platform. However, given the complexities involved — some of which we’ve discussed, others we have not — building an integration with our API is probably best thought of as a last resort.

We recommend starting the integration process by asking a few key questions:

  1. What problem needs to be solved? What task needs to be accomplished?
  2. How often, and by how many people?
  3. How tedious, complicated, or difficult is it to do manually?

Knowing the answer to #1 is important. People ask us all the time if we can integrate with another system. We ask, “what information do you want to move or update, in which system, from where, and based on what criteria?” Most people aren’t sure. If you’ve narrowed it down to some specific needs — for instance, “Program managers need to update their activities with information from the grant management system” — then you’re ready to proceed.

If the answer to #2 is “rarely by several people” or “regularly, but only by one person,” you might want to consider options that are easier and cheaper than building an API integration. Remember: the goal is not necessarily “integrate all the things!” The goal is to be good stewards of scarce resources, chiefly: time, money, skilled personnel, and high-quality data. It wouldn’t make sense to replace a 20-minute-per-quarter task with an automated API integration that could take months to build.

Here’s some alternatives to API integrations to try first, in order of the expense to your organization in terms of your staff’s time and effort ($-$$):

  • Shared key values or unique IDs ($) can join or link two or more records in different systems if and when data need to be combined. The joining/linking process is somewhat manual, but can still be done systematically with just a little bit of expertise (for instance, using the VLOOKUP function in Excel). There’s lots of ways to do this in DevResults: a few purpose-built fields are activity short names or reference numbers, indicator codes, and organization short names or organization identifiers.
  • Cross-referenced hyperlinks ($) can ensure that users can jump directly from a record in one system to a related record in another system quickly and easily. It’s only slightly more “expensive” than shared key values because links can and do break from time to time, but luckily they’re easy to fix. Even with a team of developers in our company, we regularly use this integration method internally to link help tickets to development tasks and back again. That way, we never have to hunt for the right email thread to reply back to the user that we’ve finished a bug fix or a feature improvement. You can add links to most text fields in DevResults wherever they may be relevant, such as activity or indicator description text fields — use Markdown formatting like this: [link text](www.website.com) — or in data table cells.
  • Structured imports and exports ($$) require a bit more coordination between different systems upfront, but then the export/import process itself is just a few clicks for each user. For instance, let’s say you want to take your grant management system records and use them to populate a batch of new activities in DevResults. If you can get your grant management system to generate a spreadsheet-based export of new grants similar to DevResults’ activity upload template, a quarterly, monthly, or even weekly update process will be a cinch — less time than it takes to write an email to tell an intern to do it for you. If you’re going the other way — from DevResults to another system — you can either use one of our many built-in exports, or request a custom query that structures the data in the format that your other system is expecting.

If information needs to be exchanged frequently or by many people, or the process is very tedious, with lots of possibilities for human error, then it might be time to build a true, API-based integration. Actually, building it might be the easiest and cheapest part. The full cost of an API-based integration includes:

  • Building it which typically requires hiring a developer or engaging a consultant.
  • Testing it to make sure that it holds up under real world conditions with millions of records, not just a handful of test cases.
  • Maintaining it over time (again, via an on-staff developer or third-party contractor) as systems change, security patches are required, or user needs evolve.
  • Socializing it when unfamiliar users may prefer to manually transfer data themselves, rather than relying on an invisible robot to do it for them. (You’d be surprised how many people still prefer to import data from survey platforms manually, despite the fact that we’ve had multiple automated integrations over the years.)

If you’ve considered these elements and done the cost-benefit analysis, then we’re ready to help you use our API to its fullest extent! Start by familiarizing yourself with our API documentation, either at that link or at [your_site].devresults.com/api-help and get in touch if you have any questions. We’d like to hear how you’re using our API and how helpful you find the documentation so that we can publish new endpoints or add more detail where needed!

Let’s have some fun...

Want to earn some bragging rights? Let’s unlock your API achievement! All you need for these examples is your web browser. No coding required!

First, log into your DevResults site. This gives your browser the necessary “authentication token” to not only use the site, but also to use the API.

Then, enter the following GET request just like you would a website URL, being sure to replace the text in brackets with your site’s subdomain/name:

https://[your_site_name_here].devresults.com/api/awards

Next, hit enter. Congratulations - you’ve just made a GET request (your first?) from our activities endpoint! You told the API you wanted to get a list of activities (née awards in our database and API). Now let’s look at what you got back. It should look something like this:

[{"AwardID":38004,"Title":"Access to School Lunches","ShortTitle":"ASL","Code":"ED-08","ReferenceNumber":"01-519-HBD-00-5414","AwardingOrganizationID":27501,"StatusOptionID":2998,"MechanismID":8730,"PrimaryUrl":"awards/Award/38004","ShortTitleAndTitle":"Access to School Lunches (ASL)","PrimaryOrganization_OrganizationID":27481},{"AwardID":38013,"Title":"Addressing Corruption in Local Government","ShortTitle":"ACLG","Code":"DR-05","ReferenceNumber":"01-172-FEE-00-6674","AwardingOrganizationID":27501,"StatusOptionID":2998,"MechanismID":8727,"PrimaryUrl":"awards/Award/38013","ShortTitleAndTitle":"Addressing Corruption in Local Government (ACLG)","PrimaryOrganization_OrganizationID":27493},{"AwardID":38003,"Title":"AgriMAX","ShortTitle":"AgriMAX","Code":"AG-07","ReferenceNumber":"01-950-GDC-00-4751","AwardingOrganizationID":27501,"StatusOptionID":2998,"MechanismID":8729,"PrimaryUrl":"awards/Award/38003","ShortTitleAndTitle":"AgriMAX (AgriMAX)","PrimaryOrganization_OrganizationID":12345} …]

Note: if you forgot to log in to your site first, or if you don’t have permission to view your activity index, you will instead see this:

{"Message":"Authorization has been denied for this request."}

Behold, this is what your activity index looks like in JSON (JavaScript Object Notation, a common machine-readable language and data interchange format). You might see some familiar names and numbers, but on the whole this output is illegible and unusable… unless you’re a computer!

If you were a computer, you would be able to recognize, deconstruct, and parse this blob of text into useful things, like activity reference numbers or partner organization IDs. Let’s see how! This time we’re going to make a more-specific GET request — not just a data dump — based on the first.

To get started, find your favorite activity name in the text blob from earlier, then very carefully start reading from that name until you see "PrimaryOrganization_OrganizationID:" The number that follows is the ID that DevResults uses to uniquely identify which partner organization goes with which activity.

Copy or write that number down and stick it in the following request (don’t forget to replace your site subdomain/name again too!):

https://[your_site_name_here].devresults.com/api/organizations/[that_ID_number]

What you should get back is another (much shorter) blob of text, this time with the name and short name of the primary partner organization for your favorite activity. It should look something like:

{"OrganizationID":12345,"Title":"Caudill Associates","ShortTitle":"Caudill"}

These two GET requests show you existing information in your DevResults site. To make changes to DevResults via API, like creating a new user, editing an activity, or writing an indicator result to the database, you would use a more sophisticated tool like Postman to make POST, PUT, and DELETE requests. A computer could run a complex series of requests on a schedule (or when a human clicked a button in another system) to do things like upload data to DevResults, export data from DevResults and import it somewhere else, update an activity’s budget, generate a report, add new users and assign them to activities, and pretty much anything else you can do in DevResults.

How can we help?

As a company, we provide an evolving platform that enables our partners to advance as quickly as technology changes. We commit our time and resources to create end-to-end continuity of data utilization. One of our key values as a SaaS company is focusing on “what’s next” in order to ensure that our platform evolves along with the sector. It’s this concept that has made us successful for 10 years running.

If you can think of a way that our API could help your organization connect and integrate your data systems more seamlessly, please reach out to our Helpdesk at help@devresults.com and we’ll guide you through the process!