Suppose the business requirements document or other test basis information for change to an insurance company’s website contains the following:

Requirement: the user should be able to discover the current status of an existing claim

Before starting to plan testing to assure this, consider first the fundamental functions. This will give rise to questions that should be asked. If these questions can be answered, the test objectives can be made more detailed. If information is not yet available (because some design decisions have not yet been taken), the test objective should still refer to it.

Load: should this function be accessible directly, ie by navigating directly to a URL rather than navigating from other locations within the site? By what name(s) will the function be referred to in navigation mechanisms, eg menus and breadcrumb trails? What should happen if required browser features are not available or are disabled, eg JavaScript, pop-up windows, Flash?

Confirm: are all pages visited when using the function compliant with prevailing content and style guidelines/constraints?

Navigate: can all pages visited when using the function be bookmarked (added to favourites) conveniently? Can the pages be accessed via the browser history? Can the pages be navigated between using the browser’s forward and back commands/buttons?

Generate message: is it required to email the user or any other parties to notify them that someone has attempted to discover the claim status, or system administrators if the status information cannot be retrieved?

Accept and forward order: is it required to record that the status of the claim has been discovered by a user?

A careful consideration of the possible meaning of each of the fundamental functions to this requirement reduces the risk of important features and properties not being tested.

A set of test objectives is then written, which depending upon the answers to the questions above may include the following:

TO1: To show that a page enabling the user to discover the status of an existing claim loads when the user directs the browser to the URIs:

Of course when this activity is being carried out, these URIs many not yet be decided or known. In that case, a generic phrase such as “the applicable URIs” is used. Thus, this fundamental function is captured for testing - that is our most important aim. The testing itself will be developed and detailed with the required specific information when available.

TO2: To show that a page enabling the user to discover the status of an existing claim loads when the user chooses the navigation option “Track a claim” or “how to track your claim”

If the specific navigation is not yet decided/known, the phrase “when the user navigates to it” can be used.

TO3: To show that all pages displayed when a user attempts to discover the status of a claim are compliant with prevailing content and style guidelines

Note that this differs from the requirement statement which refers to “an existing claim”. The test objective is worded so that it applies when the user attempts to discover the status of any claim including one which does not exist. This adds the “error” pages and messages to the test path: those are a very common source of failure because they are often omitted from testing.

What the guidelines are and how they are defined varies between organizations; some include them in the requirements document, in a section sometimes called “confirm”, meaning that the identity of the site or its parent organization is confirmed and reinforced sufficiently in the mind of the user as they use the site. This tends to give rise to test objectives defining properties the site should have, eg:

TO4: To show that the site logo (full definition in appendix) is displayed consistently during navigation

Other organizations refer to a set of constraints, sometimes called a “stylebook” etc, defining properties the site should not have. This gives rise to “negative” test objectives such as:

TO5: To show that no font other than Arial or Helvetica appears on the site

Now we come to the navigation fundamental function.

TO6: To show that all pages displayed when a user attempts to discover the status of a claim create a unique bookmark/favourite

This could also be written “To show that all pages … have a unique title” since this is used by the browser to create the bookmark. It also raises a question whether error pages (eg in this case a “claim not found” page) should be bookmarkable in the same way.

Now the “accept and forward order” fundamental function. The “order” in this case is the user’s request for information on a claim. If details are known of what information will be captured and how it will be stored this should be included in the test objective, eg:

TO7: To show that when a user discovers the status of an existing claim, the date and time of discovery and the user’s ID are added to the claim’s audit trail file.

If these details are yet to emerge from the design and development work, a less detailed TO can be written. But this should still ensure that the feature remains in the test scope so a test for it will be created when possible, eg:

TO8: To show that when a user discovers the status of an existing claim, details of that discovery are recorded appropriately

What is “appropriate” can be decided later when more design information is available.

Having made every possible effort not to miss out any important considerations that should be tested, we can now state the obvious test objective(s) arising from the original requirements statement:

TO9: To show that a user can discover the correct current status of an existing claim

A firm basis for confidence-increasing testing

Analysis of a typical customer-facing business site generates 100-200 test objectives of this type (sites that generate more than this have very many functions and might be better considered as multiple applications with navigation or a “portal” in common). When the method is applied - as is more often the case - to a change to an existing application, the number is typically lower.

The special circumstances and contexts under which web applications are created, operated and used have tended to hamper the efforts of testers to apply skills appropriate to other development types and so make a good contribution to web projects. As with so many other testing issues, the answer is to do the hardest and most critical work as early as possible. Using this method unlocks the benefit of VELT for web, without needing good or detailed requirements. It:

• helps to verify the requirements information that is available and detect potentially severe defects before they cause damage

• completes understanding and documentation of the true requirements, so that important properties of the application or change are not omitted from testing

• focuses effort throughout the rest of the lifecycle on those properties of the site that relate directly to its business objectives and are most likely to impact on its business success.

Edward Bishop is author of many training courses on web testing

Fundamental functions of virtually any public-facing website

These are derived directly from the fundamental business objectives: presence, advertising, providing information, facilitating communication and facilitating operations respectively. During very early lifecycle test design, they should be considered in addition to the available business requirements.

Load: the most fundamental function of any application. In web terms, it means what the user must do in order to see the first document needed to access content or a specific function, and what should happen under the various circumstances that might impact that initial access

Confirm: relates to the content and design aspects of the site that help the user to identify it and/or its parent organization

Navigate: the function provided by the mechanisms and choices offered to enable the user to access desired content and specific functionality

Generate message: the function that operates whenever the site sends output to anywhere other than the client workstation or its own components (including physically or commercially external components such as web services). For example, an email to an appropriate person when the user submits an inquiry or feedback form, or a warning message to system operators when unusual user activity (eg three failed login attempts) is detected

Accept and forward order: the function that operates whenever a user event is recorded in persistent data. The term “order” in this context is very broad; it could be a sales transaction on a commercial site, or a user downloading a file for free, or just accessing information. Any user event is called an order if it should be recorded.