Every time you visit a Web page URL (e.g. - www.codylindley.com) you are making an HTTP (Hypertext Transfer Protocol) request to a server. The response to the request is the page that loads in the browser. Open a browser, go to codylindley.com, and there you have it -- you made an HTTP request and got an HTTP response (shameless self promotion). This is the Web paradigm that most Web developers are familiar with, regardless of whether or not you know all the fine details of HTTP.
AJAX, when you boil it down, is about manually making "micro" HTTP requests using the XMLHttpRequest (XHR) JavaScript object. It mimics the behavior of the user agent, but instead of using the browser interface, you are using JavaScript to make a programmatic HTTP request (which also handles the response). This technique actually has many names: AJAX, scripting HTTP, XHR, or remote scripting, but personally I prefer the term XHR.
The nuts and bolts of XHR are rather simple when compared to the workings of browsers like Firefox or Internet Explorer. Truthfully, the basic HTTP life cycle is the same. What is not the same, is that the browser was not designed for this type of hidden (hidden from the user) micro request and response. The page refresh is the most common indicator that something is going to happen, and XHR eliminates it altogether. So, XHR from the browser's perspective is really quite a hack. This is where our hardships begin.
I have put together this comprehensive guide in order to help you get a better foothold on XHR, and to help diminish some of these hardships. The following is a long list of concepts surrounding the use of the XMLHttpRequest JavaScript object for the purpose of handling HTTP requests and responses.
Remote scripting techniques pre-date XHR
Former solutions, some actually still in use today, include:
- Applet, ActiveX control, Flash
- XML-RPC
- Simple HTTP via a hidden iframe
Each of these techniques has advantages and disadvantages, but more than anything, each was a sign that the needs of Web developers were not being adequately met by browser makers. This is the primary reason XHR came into existence.
Multi-page architecture (Web 1.0) vs. single-page architecture (Web 2.0)
The traditional web, or web 1.0, facilitates the default functionality of the Web browser. When you request a page, the browser refreshes, and a new page is displayed. This is the intended functionality of a Web browser. XHR works outside of the browser model. Single-page architecture, or Web 2.0, uses micro HTTP requests to update the page without a page reload, and really without the browser's knowledge. Web 2.0 is about a lot of things (social networking, tagging, community, etc.), but in my opinion it all boils down to making micro HTTP requests outside of the traditional HTTP model used by Web browsers. This is helping to bring the desktop to the Web.
Why develop using a single-page architecture?
- XHR allows us to build more desktop-like applications in the web browser.
- XHR avoids a complete page reload when only a small portion of the UI needs to change, which allows for immediate access to server-side logic and data.
- By updating chunks of an interface on demand, the application can inform the user immediately (e.g - live validation or live searches).
- If the request stalls, the user can perform other tasks while waiting, unlike the multi-page architecture.
- Page size can be managed by triggering the addition of content when needed, and not on initial page load.
Why build browser based applications?
- There is nothing to install on the user's computer.
- The data hosted on a network is available from anywhere.
- The Web is OS-agnostic, so it has a larger reach.
- Upgrades can happen without user intervention.
- Most users already have what they need -- a Web browser!
Why use XHR and JavaScript instead of Flash, Applets, Iframes, etc.?
- Iframes are a hindrance to SEO and accessibility.
- No third-party plugins are required.
- It is the simplest of all the remote scripting techniques.
- XHR is universally much easier to support given the popularity of Web 2.0 applications.
But wait, Remote Scripting breaks the browser
Rich Internet Applications (RIAs) using XHR should be built specifically with the Web browser in mind, since the browser and the OS are not created equal. Developers must take into account the limitations of this unique delivery mechanism when planning to build a Web site or Web application. In reality, remembering this concept is crucial during all phases of the project -- the visual design, interaction design, and technical design -- even quality assurance.
RIAs that implement remote scripting techniques break the traditional model that browsers support. This effects the user's expectations and experience. Specifically, XHR breaks the history log, back button, bookmarking, and the URL itself. Applications using XHR or some flavor of remote scripting will require unique URLs to access specific states of the application. Think Google maps -- they do this by providing a link to a specific map. In a traditional page model this is not a problem because the URL shown in the browser and managed by the browser represents the exact state of the page. Some of these hurdles have been overcome, but not without a heavy reliance on JavaScript.
- History -- Because the browser will not monitor micro HTTP requests and responses, they are not accounted for in the browser history log.
- Back Button -- Because there is no history log for XHR, the back button will only return to URLs that the browser manages. If you are using both a traditional page architecture and single page architecture together, the back button will work differently depending upon what is managing the HTTP.
- URL -- The URL becomes broken when using XHR because the URL no longer represents a single state of an interface. The response of the URL that appears in the browser no longer represents what you will actually see.
Visit the following links for more information on fixing a few of these behaviors:
Because remote scripting breaks the browser, it also breaks SEO
In simple terms, if a URL no longer represents a search-able and bookmark-able page, then search engines will have trouble indexing the content. Remember, search engines do not care about JavaScript, other than the fact that it can become a stumbling block. This means that there is great potential that your search ranking will suffer. Web developers should become familiar with the means by which this can be overcome, as well as communicating the pitfalls to clients.
The client/server relationship (tight coupling vs. loose coupling)
How dependent (intertwined) is the client on the server when using XHR? In tight coupling the client is dependent on the server in a very specific manner, whereas in a loose coupling the handshake between the server and the client only cares about the format of the request, and the structure of the returned data. Most web services are good examples of loose coupling.
The JavaScript / interface relationship (tight coupling vs. loose coupling)
How dependent (intertwined) is the interface on the javascript? Loose coupling typically involves using a generic library of JavaScript code to create your own AJAX enabled interfaces -- interfaces that you build from the ground up. Tight coupling here means that the JavaScript is intertwined with the interface or the server, or both. Any time the server-side logic or JavaScript is creating the interface (XHTML and CSS), you are dealing with tight coupling. This can have consequences if you are concerned about accessibility.
Web 2.0 Accessibility
Building accessible web pages in the traditional Web world is difficult; building accessible Web 2.0 Web pages is even more difficult. The bottom line is that true accessibility can be expensive and time consuming, though this varies greatly depending on the level of access you want to provide to your visitors. If it is an all-encompassing audience, then you must spend even more money and time. If you have to build an accessible Web 2.0 application, then you will need to first build an accessible Web 1.0 application, and then enhance it (via progressive enhancement) with remote scripting. In my opinion, "graceful degradation" is more of an afterthought than a real approach to the accessibility issue as it pertains to XHR. The best approach is Hijax, first introduced by Jeremy Keith.
Typical browser support for XHR development and rich interfaces
- IE 6.x on Windows 2000, XP, and Vista
- IE 7 on Windows XP and Vista
- Firefox 1.5+ on Windows and Mac
- Camino 1.x on Mac
- Safari 2.x on Mac
- Safari 3.x on Windows XP, Vista and Mac
- Opera 9.x on Windows and Mac
RESTful (Representational State Transfer) Web service principles
Simply stated, REST is an architecture style. In practice, when you hear the term REST, it typically refers to a service call on the Web that you connect with using a unique URL (sometimes including an API key). This call returns a stateless chunk of data in XML, JSON, or text format. The REST call is "dialed" with XHR.
Example:
You can almost read this URL and know what you are going to get back. We can discern from the URL that:
- It is a Web search service from Yahoo!.
- It is version 1 of this REST architecture.
- I am requesting the top 2 results of a Web search using the keyword "madonna".
JSONP
JSON with Padding (JSONP) is a concept that allows the sender of an HTTP request (On-demand Script Tag), where JSON is returned, to provide a name for a function that is invoked with the JSON object as a parameter of the function. This technique does not use XHR, and is an alternative approach that has gained some traction among developers.
Example:
Client storage options
Cookies are a small amount of named data stored by the Web browser, and associated with a particular Web page or Web site. Cookie data is automatically transmitted between the Web browser and Web server, so that server-side languages can read and write cookie values that are stored on the client. JavaScript can also be used to manipulate cookies using the cookie property of the document object. Although it has been discouraged in the past, the practice of grabbing data with XHR, and storing it in a cookie is gaining popularity.
Flash SharedObjects enable Flash movies to save data which persists across multiple sessions. SharedObject is sometimes referred to as flash cookies. To read more, please visit the following URLs:
- http://www.niallkennedy.com/blog/2007/01/ajax-performance-local-storage.html
- http://codinginparadise.org/weblog/2005/10/amass-ajax-massive-storage-system.html
HTTP post and get methods
XHR uses the HTTP get and post methods for sending data. Get retrieves information from the server by attaching parameters to the URL. Get is limited in the amount of data it can send to the server. It should be used when the request is idempotent, meaning that multiple requests will return the same results. Post on the other hand does not have a limitation on payload sent to the server. It should be used when you want to conceal information, and actually change the state of the server with a request. Post requests are appended within the request body instead of being appended to the URL.
HTTP Response Headers
HTTP headers contain meta-data about HTTP requests and responses that will further help describe the message, the client, and the server expectations. By using the HTTP head method you can forgo the message and get only the head meta-data of an HTTP request and response. Additionally, it is possible to set and get the values contained in headers by using the following methods of the XMLHttpReqest object: getAllResponseHeaders, getResponseHeader, and setRequestHeader.
The browser caches XHR responses
Under certain circumstance the browser will cache the responses from multiple requests to the same URL. For this reason it is useful to add a timestamp to most XHR requests as a query parameter. This will ensure that the browser identifies the URL as a unique un-cached request.
XHR responseText and responseXML properties
JavaScript can access the HTTP response to an HTTP request using the responseText and the responseXML properties of the XMLhttpRequest object. The responseText property holds the response as a text string representing the response. The responseXML property holds the response as a JavaScript object that represents an XML DOM tree. The responseXML object can be traversed, manipulated, and accessed by using W3C DOM methods.
Sending parameters as an XML string
Using the XMLHttpRequest, the client can send parameters (POST) in the form of a text string or an XML document. However, creating an XML document has browser support issues, so sending data to the server as XML requires sending it as a text string representing XML nodes. This technique is best used in a closed environment (such as an Intranet) with limited browser support.
The server's role
Because you are making an HTTP request to the server, the requested information can first be processed by a server side language before the response is returned to the client. Remember, just like in the traditional Web model, XHR requests send parameters to the server. The response can be contingent upon these query string parameters. A simpler request would be one that just requests a static (no server-side scripting) HTML file on the server, and does not send any parameters along with the call. This pattern has been labeled AXAH.
innerHTML vs. W3C DOM methods
Once you have a response from the server, regardless of the data interchange format (string or XML object), you will want to add portions of the response to the current page. This can be done in two ways. The first, innerHTML, is the fastest and simplest manner in which to change the DOM. This is where we rely on the server to completely handle our request by outputting a DOM ready text string that we can simply add to the DOM. It replaces one snippet of HTML with a new snippet. The second relies on DOM methods for manipulating, creating, and adding new nodes to a DOM tree. This is more complicated, but is often accepted as the "standards compliant" technique.
Creating HTML elements in the client
When you have your XHR response from the server you need to intermingle the data with the DOM.
First, you use a bit of JavaScript to extract the data you want. Now that you have the data, be it XML or JSON, it needs to be formatted to XHTML so that it can be added back into the document. At this point you have to use DOM methods to create the appropriate HTML elements. The process of creating HTML in the client using JavaScript is rather painful. Thankfully there are several scripts out there that can aid us in this process.
Changing a XML DOM tree into a JSON object in the client
When you do not have the choice of getting an XML DOM tree, but want to work with a JavaScript object, then you can objectify an XML tree into JSON.
JSON, ResponseText, and InnerHTML
The fact of the matter is that using DOM methods on an XML DOM tree is a laborious and slow task. Another pattern exists the seems to be the current standard. It uses the responseText property to get a string version of a JavaScript object that is then converted from a string to an actual JavaScript object. This object can easily (unlike XML data) be manipulated using JavaScript, and then added to the DOM with the innerHTML property.
JSON and the eval method
The eval method alone should not be used to convert a string to an object when the string or response is coming from a third party resource. There are scripts that will side step the direct use of eval to objectify a JSON string to a JavaScript object. As well, these same scripts can take a JSON object and turn it into a JSON string. This can be useful if you would like to send a JSON string as a parameter with a request.
Same source policy for XHR
The XHR object is subject to the browsers security "sandbox". Any resources requested by the XHR object must reside within the same domain from which the calling script originated. Servers can talk to other servers. So, if you have your server request a Web service from a different server using a proxy you can in fact access information on another domain that can then be sent back to the original client seeking the information.
Using the script element (Dynamic Script Tag) to work around the same source policy
XHTML script elements can access JavaScript on other domains, just like images or iframes. Because of this, one domain can request a JSON object from another domain. If that domain is really thoughtful, they might even return the JSON object as a parameter of a function you declare (JSONP).
Example:
Read more about these techniques here:
