About a year ago I wrote a blog post about scriptable Hypermedia Clients i.e. the role of the Code on Demand hypermedia constraint - the one from the seminal Fielding dissertation.

The post was hopefully OK, but much more interesting was the discussion that ensued, in the comments section, with Mike Amundsen about the future role of Code on Demand in APIs. By the end of the debate, we sortof partied at: “it’s probably too early for CoD in APIs”.

A lot of things have happened since the blog post. Among other things: I got an exceptional privilege of spending last year working right next to Mike and the rest of the amazing team of API experts at the API Academy. I also got much more time to research and think about API design, experiment with various things, since APIs are now my primary job. Independent from me, Hypermedia architectural style has also become significantly more accepted in the industry. We very rarely run into API teams, at this point, who are not either building Hypermedia APIs, have considered it or at least have heard of it. This was definitely not the case year and a half ago.

The need to possibly revisit the question of Code on Demand in Hypermedia, has been brewing in my mind…

Yesterday we had a wonderful API360 Summit in Dallas. Since we were fortunate to have some of the brightest minds as guest speakers and panelists, Summit ended-up into very insightful API design discussions. Furthermore, in the evening Mike, Ronnie, Jason Harmon and I met for drinks where Jason brought the topic of Code on Demand up, again.

While the impressions from the Summit and the following alcohol-enlightened discussion are still fresh in my mind, I want to share with you where I think the status quo of Code on Demand is in APIs now.

Semantic Gap Is Still Very Real

Hypermedia API design is all about bringing the benefits of the universal interfaces, which we enjoy in the Human Web, to the world of the “machine web”: the world of APIs. If we can eliminate the spread of wasted effort that stems from each API provider designing essentially the same things in completely incompatible ways and everybody having to write individual clients for each of the thousands of web APIs - we can spend time/talent on more productive work.

But we have a significant problem: Human Web can enjoy a “single” kind of client (a web browser) only because browsers are always operated by humans, and because humans are incredibly skilled in making semantic sense of most unstructured data displayed by web browsers. Machines are virtually incapable of understanding the meaning (“semantics”) of anything that is not preprogrammed into them. However “preprogramming” creates tight coupling between an API client and the server, and leads to the waste and breaking changes that we have already mentioned as a significant problem.

The missing ability of machines to understand semantics of the information received is what we commonly refer to as “Semantic Gap” in Hypermedia API design.

Semantic Gap

There are really only two ways we can address Semantic Gap:

1. Make clients “smarter”
2. Keep clients “dumb” but send them instructions they need on-demand so that preprogramming (and resulting coupling) is still avoided. Mike Amundsen aptly calls this: “late binding in APIs”.

Choice number one is what we have been doing to date. While creating sufficient artificial intelligence is not an immediately viable solution, we can fake “enough” intelligence by limiting and standardizing common semantic vocabularies. Efforts such as: IANA Link Relations repository, microformats and schema.org are all geared towards creating such vocabularies. Once and if we have enough of such vocabularies, we can use them in APIs via profile link relation [RFC6906] and machine-readable profile definition standards such as: ALPS.

While significant amount of work is put in this direction, it is not without challenges. For one: due to human nature, it is quite hard to make people use common anything, let alone vocabularies or standards. And to be fair, it is also hard to reach a point where enough vocabularies are available.

Let’s talk about the second choice:

Dumb Clients, Smart Endpoints

A seminal breakthrough in Information Theory and modern communications came from Shannon–Weaver model of communication, which is based on the idea of “dumb pipes and smart endpoints”. This model has had vital consequences in everything information-exchange-related, including even things like: Microservices.

I believe that a similar approach of: Dumb Clients, Smart Endpoints can alleviate a lot of challenges of the Semantic Gap. If we can come up with the model where the clients don’t need to understand the semantics of communication and can remain largely “dumb”, by just understanding processing rules, we can get benefits very similar to those that Shannon and Weaver achieved by keeping transport layer completely agnostic of the information being transmitted.

So, What Are We Waiting For?

I do strongly believe that Code on Demand in APIs will happen. There really is no avoiding it, but I also am very accepting of the idea that timing has to be right.

The analogy to this is Human Web again. In Human Web the “code on demand” is: Javascript-based scripting of HTML pages (really: messages).

(Sideline: I am well-aware of the correct capitalization of “JavaScript” but I refuse to imply that JavaScript is anything like Java scripting, so I never capitalize “S” on purpose).

For better or worse, Javascript is almost ubiquitous nowadays, but a lot of us still remember the time from web development when it was a badge of honor to build a website/webapp without using any Javascript. In early 2000s, a lot of organizations had coding guidelines prohibiting use of Javascript or at least requiring that everything still worked with Javascript turned off. Eventually those requirements turned into “graceful degradation” requirement for Javascript, but most teams today will think you are crazy if you ask whether their web app works without Javascript.

While most of anti-Javascriptism could be attributed to problems with Internet Explorer, I believe we had to go through the phase where using Javascript was a ‘bad taste’ for us to develop things like HTML and CSS properly. I also agree with Mike Amundsen when he thinks we are still abusing Javascript and using it where it shouldnt be used. For instance: there’re many components and elements implemented in Javascript which really should just be provided by HTML. In HTML5 the spec authors finally added some of those (e.g. date picker) but it’s quite abysmal how archaic, say, select boxes are in HTML.

If we jump into code-on-demand too early, we can hurt ourselves

That said, now that Hypermedia API design is much more widely accepted, now that we have many more people understanding it, I think we can start having intelligent discussions about what Code on Demand is for APIs, where it is appropriate, and how it may look if implemented outside the scope of a traditional web browser.

Is This All “Just Talk”?

So, do we have anything even remotely like Code On Demand for APIs? Obviously Javascript is CoD for HTML, but what about “real APIs?” Is this all theoretical, or is there any substance?

We do, actually. And not just things archaic and ugly like Java applets or Flash/Silverlight etc. apps. I believe we can easily call Javascript embed codes from Google Analytics, Chartbeat etc. very successful examples of CoD targeted at APIs. These Javascript snippets are only embedded in your pages to interface with an API and they certainly have no intention of rendering any UI (traditional use of JS for HTML).

Granted, Google Analytics or Chartbeat’s Javascript is still running in a browser, but the client that first enabled such interaction is largely irrelevant for the underlying principle. Point is: these dynamically loaded code snippets serve as the glue between the client (web browser) that doesn’t understand the semantic of the interaction and the server (Google Analytics). This glue code can also easily change if server-side logic changes, providing high degree of backwards and future compatibility.