Website Design & Development Blog

We recently wrote some code to interact with an online service via their API. The service provider handles donation gifts and donor records. Their API is lacking in several areas. I want to highlight some of the more problematic issues.

Data Passing Madness
Normally what you find with an API is that the provider is using SOAP or REST, or sometimes something in between. Generally it involves passing name-value pairs or an XML document to the service and expecting a response. Not so with this provider. All data passed to the provider was sent through a single parameter. Where you traditionally would send param1=value1, param2=value2, paramN=valueN. This provider required all data passed in a single parameter like this: params=value1,value2,valueN.

This is irritating for a few reasons. First, I can't tell what was passed based on the contents of the call. All I see are a bunch of values. Second, the position of the values matters. It must, since there is no other way to know which parameter the value represents. If no value is provided, you must pass null. As in, the string "null". Request parameters typically look like this, but are usually longer:

...?login=username&pass=password&action=request_type ¶ms=10,N,null,null,null,Joe,Smith,null,1,N,null,null,null,null, null,null,null,10.00,null,null,null,null

It's just a bad idea to try to roll your own API when there are existing specifications like SOAP and REST. I just don't know why they would choose to do it this way.

I'm Looking to Escape...
Another issue we found was that some requests were failing. The error message from the service was an ODBC SQL error message from the database layer complaining about special characters in the value. We had an apostrophe in the value. At first I thought, well we should be escaping the input before we send it over to the API, our bad! Although after a while I decided that wasn't right. Why should I escape special characters on my end? What if the provider decides they're going to move to from MS SQL to another database one day. Now I have to update my code?

Sorry, but no, the service provider should take the input, escape it and apply value requirements to each parameter, and then perform the database action as required. I should only need to pass the data using the correct credentials and in the required formats. Something as trivial as escaping output before a database action shouldn't be up to the end-user/developer.

Just Let 'Em Query Directly! Why Not?
Finally, one of the "features" this provider espouses is that they offer "advanced" developers greater access to the database by allowing them to run queries directly. Yep, I can send pass through an SQL select statement and it will return an data set in XML format. Nevermind that I don't know how the data is stitched together on their end. Forget that there is no documentation for this "feature".

My guess is that this provider was getting requests from developers to open up their data to a wider variety of calls. This was the providers response. Perhaps someone needs to explain the idea of an API to the development team.

Conclusion
So in summary, when planning your API service, don't:

• Lump all your values into a single parameter. It's annoying for your developers
• Make me escape values so that they can be inserted into your database. It's your database, you escape it before it is inserted.
• Ignore existing best practices and specifications for creating an API service by creating your own. Use REST or SOAP.
• Give your developers carte blanche access to issue select queries, but don't provide them with any documentation.

When planning your API, DO:

• Use an existing specification like SOAP or REST so that your API is more widely accepted by the development community.
• Make your API as easy as possible for your developers by providing clear data requirements, providing concise and understandable error messages, and handling the petty little things like escaping output for them.
• Inform your developers when there is a change to your API.

The word "techie" gets tossed around these days as if it were a proper designation. It's not. I think it's a derogatory term which indicates that the person is ignorant of what their programmer, web developer, or IT professional actually does for a living.

Wikipedia, in part, defines techie as:

"Techie is a term, derivative of the word technology, for a person who displays a great, sometimes even obsessive, interest in technology, high-tech devices, and particularly computers. Some techies express a disregard for any technology that could possibly be considered obsolete, even if the technology or process in question might only be considered obsolete within the techie community."

I read that definition and understand that "techies" are condescending, narrow-minded technocrats obsessively interested in technology. Not a pretty picture. Certainly not one that describes most of the so-called "techies" that I know.

It's a label chosen because you know that your deskside support guy, Jim, knows how to make your e-mail start flowing again by doing things you don't understand. You don't want to call him a nerd or because you've heard they don't like being called that, so you just call him a "techie". Techie fits the bill nicely. It almost sounds cute. A word of advice: Jim doesn't like techie either, he thinks you're being condescending, and most likely, he thinks your foolish for having called him that.

Your "techie" is a professional. They probably went to university for their training, spending many late hours studying, just like you did for your MBA. Or spent a lot of time in the trenches over many years gaining experience to do what they do today. Some of us are computer scientists -- yeah, you know, a scientist like this guy -- or even software engineers. These people do incredibly complex things with software, logic, math, and more.

I recently read an article in Canadian Business magazine that, in my mind, made egregous use of the term "Techie".

"Cormack hopes TREC Legal Track’s research will lead to a more precise search protocol.This year, the workshop’s dozen or so academics, lawyers and techies will perform mock discovery requests using Enron litigation. Participants will search nearly 500,000 e-mail messages and 800,000 online documents for topics such as “stock transactions.” These digital findings are submitted to TREC, which then measures the percentage of relevant documents found using varying search methodologies."

Do you see the problem? Here: "the workshop's dozen or so academics, lawyers, and techies". I guarantee you the individuals programming this system are computer scientists, likely with Doctorates in their field. The author of the article doesn't understand what it is that these people do and in their ignorance have put these highly skilled individuals on the same level as the computer technician who works at Best Buy. It's a shame really.

So the next time you think to call someone a techie, give another thought to what you're really trying to say. I'm certain most people don't mean anything bad by the term. Some people who work in technical fields do take offence to it, though.

/rant

Most Mac users know there are a ton of awesome Mac apps out there just waiting for you and I to find them. There in lies the problem, however: you have to find them. Bodega is an app store like the iTunes Store, but for desktop applications. You can use it to find applications that you might not have otherwise found by pumping keywords into Google.

It's more than that, though. One of the features that is awesome about Bodega is that it keeps track of what apps you have on your system, notifies you when those apps need to be updated, and provides a simple interface for installing those updates.How many times have you opened an application while being in the "groove" of your workflow, only to be interrupted by an update notification. Then you have to decide whether you want to update the app, or leave it till later. If you choose to update, you have to wait for it to relaunch. Then... wait, what were you doing again? With Bodega you can keep your apps up to date and never get bothered by annoying update notifications while you're working.

Apple's iTunes Application Store is widely popular because its so easy to use. Bodega, I think, strives to replicate some of that. Finding apps is pretty easy, you can search by keyword and browse by category as you'd expect. You can also see the top paid and top free apps just like the iTunes Store.

Bodega is developed by friends of mine, so maybe I'm biased. I think if it wasn't a very good tool I'd just tell them and not use it. The fact is that simply isn't the case, it's a great application and it has made my life just a little easier. If you're interested, you can download Bodega here.

I've always believed that it is immensely important for any service provider to truly understand their client's business. What I see in the web design industry today is far too many web designers offering more form than function. Often there is a disconnect between their clients goals and aspirations and the end result. I attribute this to not enough time spent understanding the client. How can one solve any problem when they fail to understand the reasons the problem exists in the first place?

With this in mind, I was excited to be offered a chance to fly to Churchill, Manitoba to join one of our clients, Frontiers North Adventures Inc. (FNA) in seeing how their flagship product is delivered to their customers.

FNA is a much esteemed adventure tourism company. They have received a great number of awards and accolades, including a few for being an ecologically friendly company. Their primary product is the Tundra Buggy Adventure®, a tour which takes adventurous individuals out on a tundra buggy to view wild polar bears near Churchill.

If you are unfamiliar with Churchill, know that it is a small remote town located on the coast of Hudson's Bay in northern Manitoba. It is only accessible via rail or chartered air. Polar Bears migrate from the south through Churchill.

Offering a service of the magnitude that FNA does to their guests is no small accomplishment. It is a huge logistical undertaking. Prior to the season launch in late October, FNA brings everything they need into Churchil via rail or air. The manage a large operation. In Churchill they operate a giftshop, hotel, and the overall buggy tours that go out onto the trails. They have their own garage for continued work on one of their 12 tundra buggies.

Tundra buggy maintenance makes up a lot of the logistical issues. These behemoths are built with giant tires (think monster truck tires or large tractor tires), the frame and differential of an airport firetruck, a giant diesel engine from a semi-trailer, and an all aluminium frame. Read about the history of tundra buggies here.

I had the privileged to sit down for a discussion with Merv Gunter, President and CEO of FNA while we trundled around on the frozen tundra in search of polar bears. We covered some of the logistical issues and past weather-related and technical-related troubles he and his team have experienced in the past. Every minute was an interesting insight into what their company does.

We saw around 30 polar bears on our early November trip. I saw how politely FNA staff treated their guests, how seriously their drivers took the security and safety of every guest on the buggy, and I saw how smoothly their entire operation ran.

Every experience I had on the tundra helped me understand our client a little better. In the future I am sure that we will be able to offer services to FNA which are grounded in the understanding of the fundamentals of their business.

So you're starting to work with a new class structure and you need to know what parameters you can pass through a method. You decide to check the API documentation and you come across this:

This kind of thing is quickly becoming a major pet peeve of mine. Often what I find is that $options is an array. The keys in the array are not defined in the reference documentation. My only recourse is then to open the class file and dig through the method in question looking for the allowed keys for the $options array. Also of note is the fact that the majority of the time the allowed keys are not defined in the doc block for the method and subsequently do not appear in the generated API documentation.

When possible, I will avoid this kind of thing by creating an abstract data structure, that is to say, a class which represents the data I'm trying to pass to my method. The concept is nothing new, but the "let's stuff everything into an array and pass it through" approach is far too prevalent in my experience so I thought I'd write about it.

There are many strengths in taking this approach:

• The type of data is clearly defined. In the method declaration the class can be specified:

  ...
  public function __construct(MyDataType $data)
  ...
  

• Required data types can be enforced in the data class through the constructor or setter methods. In a language where there is no built-in strict data typing, this kind of allows us to provide psuedo-strict data typing.
• Developers can use accessor methods to more easily retrieve parameters when they are using the data type, leading to more concise and readable code.
• When documentation is generated using phpDocumentor the data class will be referenced in the generated documentation whenever the data class is required.
• Better documentation leads to better long term maintainability, of course, but this approach also provides a good deal of flexibility if you need to refactor in the future.

Example
Let's look at a very basic example of this. We'll create a palette, like a painters palette, which can have any number of colors. The colors have to be in RGB format. We'll start by defining the Color class:

class Color
{
    protected $r;
    protected $g;
    protected $b;
    
    /**
     * Constructor
     * 
     * @param int $r
     * @param int $g
     * @param int $b          
     */
    public function __construct($r, $g, $b)
    {
        $this->setColor('r', $r);
        $this->setColor('g', $g);
        $this->setColor('b', $b);
    }

    /**
     * Retrieve a color, values can be r,g, or b.
     * 
     * @param string $color
     * @return int|boolean
     */
    public function getColor($color)
    {
        $color = strtolower($color);
        if (is_set($this->{$color})) {
            return $this->{$color};
        } 
        return false;
    }

    /**
     * Validate the color as a whole, all colors 
     * present is valid
     * 
     * @returns boolean
     */
    public function isValid()
    {
        if (!isset($this->r) || 
             !isset($this->g) || 
             !isset($this->b)) {
            return false;
        }
        return true;
    }
    
    /**
     * Render the color as a string
     * 
     * @returns string
     */
    public function __toString()
    {
        return "R:{$this->r} G:{$this->g} B:{$this->b}";
    }
    
    /**
     * Set the value for a color, check that the 
     * value is an integer. $color can be r, g, or b
     * 
     * @param string $color
     * @param int $value     
     */
    protected function setColor($color, $value)
    {
        $color = strtolower($color);
        if (is_int($value)) {
            $this->{$color} = $value;
        }
    } 
}

I've defined a few properties and methods here. First, I've defined three protected properties, one for each color value. The constructor accepts three values and then in turn calls the setColor method which checks that the value is an integer and assigns it if it is, otherwise it ignores the value.

Also of note is the isValid method, which defines whether this color is valid. In this case, a "valid" color is one that has all three values: r, g, and b.

The Palette class is pretty simple:

require('Color.php');

class Palette
{
    protected $colors = array();
    
    /**
     * Add a color to the palette
     * 
     * @param Color $color
     * @returns boolean
     * @throws Exception
     */
    public function add(Color $color)
    {
        if (!$color->isValid()) {
            throw new Exception('Color provided does not have valid color data: '.$color);
        }
        
        $colors[] = $color;
        
        return true;
    }
    
    /**
     * Simply display a list of colors
     * 
     */
    public function listColors()
    {
        foreach ($this->colors as $color) {
            echo $color . "\n";
        }
    }
}

The add method is used to add a color to the palette. It accepts one clearly defined data type, a Color object. PHP takes care of throwing an exception if anything else is provided. We call isValid which tells the palette that the passed in value is correctly formed and if so it adds it to our array of colors.

We can write a simple test to make sure that this works:

require('Palette.php');

$c1 = new Color(50, 50, 50);
$c2 = new Color('x', 255, 128);
$c3 = new Color(128, 255, 80);

$p = new Palette();
$p->add($c1);
$p->add($c2);
$p->add($c3);

This throws the exception "Color provided does not have valid color data: R: G:255 B:128", which indicates that our second color is the culprit.

Obviously this is a very simple example but I think it illustrates the point effectively. Some might balk at the extra code that needs to be written in order to have this work, but I think the trade-off is more than worth it in the long run. Hopefully you find it worth it too.