Topic: Black Box testing for web-based application

Black Box testing for web-based application:

1. Browser functionality:

• Is the browser compatible with the application design?
• There are many different types of browsers available.

 GUI design components

• Are the scroll bars, buttons, and frames compatible with the browser and functional?
• To check the functionality of the scroll bars on the interface of the Web page to make sure the user can scroll through items and make the correct selection from a list of items.
• The button on the interface need to be functional and the correct hyperlink should go to the correct page.
• If frames are used on the interface, they should be checked for the correct size and whether all of the components fit within the viewing screen of the monitor.

2. User Interface:
one of the reasons the web browser is being used as the front end to applications is the ease of use. Users who have been on the web before will probably know how to navigate a well-built web site. While you are concentrating on this portion of testing it is important to verify that the application is easy to use. Many will believe that this is the least important area to test, but if you want to be successful, the site better be easy to use.

3. Instructions:
You want to make sure there are instructions. Even if you think the web site is simple, there will always be someone who needs some clarification. Additionally, you need to test the documentation to verify that the instructions are correct. If you follow each instruction does the expected result occur?

4. Site map or navigational bar:
Does the site have a map? Sometimes power users know exactly where they want to go and don't want to wade through lengthy introductions. Or new users get lost easily. Either way a site map and/or an ever-present navigational bar can help guide the user. You need to verify that the site map is correct. Does each link on the map actually exist? Are there links on the site that are not represented on the map? Is the navigational bar present on every screen? Is it consistent? Does each link work on each page? Is it organized in an intuitive manner?

5. Content:
To a developer, functionality comes before wording. Anyone can slap together some fancy mission statement later, but while they are developing, they just need some filler to verify alignment and layout. Unfortunately, text produced like this may sneak through the cracks. It is important to check with the public relations department on the exact wording of the content.
You also want to make sure the site looks professional. Overuse of bold text, big fonts and blinking (ugh) can turn away a customer quickly. It might be a good idea to consult a graphic designer to look over the site during User Acceptance Testing. You wouldn't slap together a brochure with bold text everywhere, so you want to handle the web site with the same level of professionalism.
Finally, you want to make sure that any time a web reference is given that it is hyperlinked. Plenty of sites ask you to email them at a specific address or to download a browser from an address. But if the user can't click on it, they are going to be annoyed.

6. Colors/backgrounds:
Ever since the web became popular, everyone thinks they are graphic designers. Unfortunately, some developers are more interested in their new backgrounds, than ease of use. Sites will have yellow text on a purple picture of a fractal pattern. (If you've never seen this, try most sites at GeoCities or AOL.) This may seem "pretty neat", but it's not easy to use.
Usually, the best idea is to use little or no background. If you have a background, it might be a single color on the left side of the page, containing the navigational bar. But, patterns and pictures distract the user.

7. Images:
Whether it's a screen grab or a little icon that points the way, a picture is worth a thousand words. Sometimes, the best way to tell the user something is to simply show them. However, bandwidth is precious to the client and the server, so you need to conserve memory usage. Do all the images add value to each page, or do they simply waste bandwidth? Can a different file type (.GIF, .JPG) be used for 30k less?
In general, you don't want large pictures on the front page, since most users who abandon a page due to a large load will do it on the front page. If you can get them to see the front page quickly, it will increase the chance they will stay.

8. Tables:
You also want to verify that tables are setup properly. Does the user constantly have to scroll right to see the price of the item? Would it be more effective to put the price closer to the left and put miniscule details to the right? Are the columns wide enough or does every row has to wrap around? Are certain columns considerably longer than others?

9. Wrap-around :
Finally, you will want to verify that wrap-around occurs properly. If the text refers to "a picture on the right", make sure the picture is on the right. Make sure that widowed and orphaned sentences and paragraphs don't layout in an awkward manner because of pictures.

10. Functionality:
The functionality of the web site is why your company hired a developer and not just an artist. This is the part that interfaces with the server and actually "does stuff".

11. Links:
A link is the vehicle that gets the user from page to page. You will need to verify two things for each link: that the link brings you to the page it said it would and that the pages you are linking to actually exist. It may sound a little silly but I have seen plenty of web sites with internal broken links.

12. Forms:
When a user submits information through a form it needs to work properly. The submit button needs to work. If the form is for an online registration, the user should be given login information (that works) after successful completion. If the form gathers shipping information, it should be handled properly and the customer should receive their package. In order to test this, you need to verify that the server stores the information properly and that systems down the line can interpret and use that information.

13. Data verification:
If the system verifies user input according to business rules, then that needs to work properly. For example, a State field may be checked against a list of valid values. If this is the case, you need to verify that the list is complete and that the program actually calls the list properly (add a bogus value to the list and make sure the system accepts it).

14. Cookies:
Most users only like the kind with sugar, but developers love web cookies. If the system uses them, you need to check them. If they store login information, make sure the cookies work. If the cookie is used for statistics, verify that totals are being counted properly. And you'll probably want to make sure those cookies are encrypted too; otherwise people can edit their cookies and skew your statistics.

15. Interface Testing:
Many times, a web site is not an island. The site will call external servers for additional data, verification of data or fulfillment of orders.

16. Server interface:
The first interface you should test is the interface between the browser and the server. You should attempt transactions, then view the server logs and verify that what you're seeing in the browser is actually happening on the server. It's also a good idea to run queries on the database to make sure the transaction data is being stored properly.

17. External interfaces:
Some web systems have external interfaces. For example, a merchant might verify credit card transactions real-time in order to reduce fraud. You will need to send several test transactions using the web interface. Try credit cards that are valid, invalid, and stolen. If the merchant only takes Visa and MasterCard, try using a Discover card. (A script can check the first digit of the credit card number: 3 for American Express, 4 for Visa, 5 for MasterCard, or 6 for Discover, before the transaction is sent.) Basically, you want to make sure that the software can handle every possible message returned by the external server.

18. Error handling:
One of the areas left untested most often is interface error handling. Usually we try to make sure our system can handle all of our errors, but we never plan for the other systems' errors or for the unexpected. Try leaving the site mid-transaction - what happens? Does the order complete anyway? Try losing the internet connection from the user to the server. Try losing the connection from the server to the credit card verification server. Is there proper error handling for all these situations? Are charges still made to credit cards? Is the interruption is not user initiated, does the order get stored so customer service reps can call back if the user doesn't come back to the site?

19. Compatibility:
You will also want to verify that the application can work on the machines your customers will be using. If the product is going to the web for the world to use, you will need to try different combinations of operating system, browser, video setting and modem speed.

20. Operating systems:
Does the site work for both MAC and IBM-Compatibles? Some fonts are not available on both systems, so make sure that secondary fonts are selected. Make sure that the site doesn't use plug-ins only available for one OS, if your users will use both.

21. Browsers:
Does your site work with Netscape? Internet Explorer? Linux? Some HTML commands or scripts only work for certain browsers. Make sure there are alternate tags for images, in case someone is using a text browser. If you're using SSL security, you only need to check browsers 3.0 and higher, but verify that there is a message for those using older browsers.

22. Video settings:
Does the layout still look good on 640x400 or 600x800? Are fonts too small to read? Are they too big? Does all the text and graphic alignment still work?

23. Modem/connection speeds:
Does it take 10 minutes to load a page with a 28.8 modem, but you tested hooked up to a T1? Users will expect long download times when they are grabbing documents or demos, but not on the front page. Make sure that the images aren't too large. Make sure that marketing didn't put 50k of font size -6 keywords for search engines.

23. Printers:
Users like to print. The concept behind the web should save paper and reduce printing, but most people would rather read on paper than on the screen. So, you need to verify that the pages print properly. Sometimes images and text align on the screen differently than on the printed page. You need to at least verify that order confirmation screens can be printed properly.

24. Combinations:
Now you get to try combinations. Maybe 600x800 looks good on the MAC but not on the IBM. Maybe IBM with Netscape works, but not with Lynx.
If the web site will be used internally it might make testing a little easier. If the company has an official web browser choice, then you just need to verify that it works for that browser. If everyone has a T1 connection, then you might not need to check load times. (But keep in mind, some people may dial in from home.) With internal applications, the development team can make disclaimers about system requirements and only support those systems setups. But, ideally, the site should work on all machines so you don't limit growth and changes in the future.

25. Load/Stress:
You will need to verify that the system can handle a large number of users at the same time, a large amount of data from each user, and a long period of continuous use. Accessibility is extremely important to users. If they get a "busy signal", they hang up and call the competition. Not only must the system be checked so your customers can gain access, but many times crackers will attempt to gain access to a system by overloading it. For the sake of security, your system needs to know what to do when it's overloaded and not simply blow up.
Many users at the same time
If the site just put up the results of a national lottery, it better be able to handle millions of users right after the winning numbers are posted. A load test tool would be able to simulate large number of users accessing the site at the same time.
Large amount of data from each user
Most customers may only order 1-5 books from your new online bookstore, but what if a university bookstore decides to order 5000 different books? Or what if grandma wants to send a gift to each of her 50 grandchildren for Christmas (separate mailing addresses for each, of course.) Can your system handle large amounts of data from a single user?
Long period of continuous use
If the site is intended to take orders for flower deliveries, then it better be able to handle the week before Mother's Day. If the site offers web-based email, it better be able to run for months or even years, without downtimes.
You will probably want to use an automated test tool to implement these types of tests, since they are difficult to do manually. Imagine coordinating 100 people to hit the site at the same time. Now try 100,000 people. Generally, the tool will pay for itself the second or third time you use it. Once the tool is set up, running another test is just a click away.

26. Security:
Even if you aren't accepting credit card payments, security is very important. The web site will be the only exposure some customers have to your company. And, if that exposure is a hacked page, they won't feel safe doing business with you.

27. Directory setup:
The most elementary step of web security is proper setup of directories. Each directory should have an index.html or main.html page so a directory listing doesn't appear.
One company I was consulting for didn't observe this principal. I right clicked on an image and found the path "...com/objects/images". I went to that directory manually and found a complete listing of the images on that site. That wasn't too important. Next, I went to the directory below that: "...com/objects" and I hit the jackpot. There were plenty of goodies, but what caught my eye were the historical pages. They had changed their prices every month and kept the old pages. I browsed around and could figure out their profit margin and how low they were willing to go on a contract. If a potential customer did a little browsing first, they would have had a definite advantage at the bargaining table.
SSL Many sites use SSL for secure transactions. You know you entered an SSL site because there will be a browser warning and the HTTP in the location field on the browser will change to HTTPS. If your development group uses SSL you need to make sure there is an alternate page for browser with versions less than 3.0, since SSL is not compatible with those browsers. You also need to make sure that there are warnings when you enter and leave the secured site. Is there a timeout limit? What happens if the user tries a transaction after the timeout?

28 Logins:
In order to validate users, several sites require customers to login. This makes it easier for the customer since they don't have to re-enter personal information every time. You need to verify that the system does not allow invalid usernames/password and that it does allow valid logins. Is there a maximum number of failed logins allowed before the server locks out the current user? Is the lockout based on IP? What if the maximum failed login attempts is three, and you try three, but then enter a valid login? What are the rules for password selection?

29. Log files:
Behind the scenes, you will need to verify that server logs are working properly. Does the log track every transaction? Does it track unsuccessful login attempts? Does it only track stolen credit card usage? What does it store for each transaction? IP address? User name?

30. Scripting languages:
Scripting languages are a constant source of security holes. The details are different for each language. Some exploits allow access to the root directory. Others allow access to the mail server. Find out what scripting languages are being used and research the loopholes. It might also be a good idea to subscribe to a security newsgroup that discusses the language you will be testing.

31. Web Server Testing Features:

• Feature: Definition
• Transactions: The nunber of times the test script requested the current URL
• Elapsed time: The number of seconds it took to run the request
• Bytes transferred: The total number of bytes sent or received, less HTTP headers
• Response time: The average time it took for the server to respond to each individual request.
• Transaction rate: The average number of transactions the server was able to handle per second.
• Transference: The average number of bytes transferred per second.
• Concurrency: The average number of simultaneous connections the server was able to handle during the test session.
• Status code nnn: This indicates how many times a particular HTTP status code was seen.