NASA is developing a new iOS application that will provide tools for Alaskan aviators to navigate difficult terrain and weather. The North Pole High School software test team has been tasked with testing the new system.
The new application will do the following:
§ Provide quality assurance testing on the various options.
§ Provide user interface and functionality feedback.
§ Print various reports.
§ Create project documentation that illustrates team process, activities and simulation environments.
§ Provide the QA information to posted on the America Bridge website, and for distribution to NASA.
This Test Plan for the TAIGA iOS application supports the following objectives:
§ Define the activities required to prepare for and conduct TAIGA application Alpha, Beta and User Acceptance testing.
§ Communicate to all responsible parties the Application Test strategy.
§ Define deliverables and responsible parties.
§ Communicate to all responsible parties the various Dependencies and Risks
2. Scope
2.1. Quality Assurance
The application will be tested for complete interaction for each menu item. The feedback will also relate to functionality and user interface suggestions.
The NPHS team will provide (may be combined into one report):
§ A monthly QA report
§ A functional specifications report
§ A user interface report
The NPHS team will deliver the reports to Trillium Learning, who will place them into the America Bridge website, which will be customized and organized specifically for this project. The NPHS team will also provide presentation media which reflects their project process and activities.
The NPHS team is requested to protect the alpha and beta applications from distribution outside of their testing environment. NASA will provide guidance for eventual distribution requirements.
The test strategy consists of a series of different tests that will fully exercise the TAIGA system. The primary purpose of these tests is to uncover the systems limitations and measure its full capabilities. A list of the various planned tests and a brief explanation follows below.
The System tests will focus on the behavior of the iOS. User scenarios will be executed against the system as well as screen mapping and error message testing. Overall, the system tests will test the integrated system and verify that it meets the requirements defined in iOS.
Performance test will be conducted to ensure that the TAIGA response times meet the user expectations and does not exceed the specified performance criteria (TBD). During these tests, response times will be measured and recorded; local bandwidth will be factored in.
All menu and submenu items will be tested to determine appropriate responses and interactivity.
A series of functional specifications will be reviewed based on Alaskan pilots’ requirements to fly in various weather and terrain scenarios.
The application will be tested from the perspective of user interface, ease of use, and relationship to functionality, based on pilots’ requirements to utilize the application within the cockpit.
Various Alaskan flight scenarios will be tested on the NPHS flight simulators to provide feedback to NASA for refining the TAIGA application functional specifications and user interface.
The team will provide language which may help development of a user manual. Testing will be documented and written to provide a first draft of the user documentation. These tests will ensure that no features are missing, and the contents can be easily understood.
Whenever possible, flight operations on the TAIGA application will be matched and compared against flight simulator runs. Project documentation will record these activities for NASA development and user interface teams to review.
Once the TAIGA application is flight worthy, any possible testing in actual flight conditions would be of great interest (if possible) and of course with licensed pilot(s).
4. Environment Requirements
§ iPad systems (Wifi) (newer versions)
§ iOS 7.04 or newer
§ 512mb min, 1GB preferred RAM
§ 16 GB Hard Drive min
5. Test Schedule
§ Ramp up / System familiarization 1/15/14 – 1/31/14
§ Test Requirements Definition 2/1/14 – 2/14/14
§ Alpha/Beta Test ing 2/14/14 – 5/23/14
§ User Acceptance Test TBD
6. Control Procedures
The project team will perform status reviews which include each Phase. (i.e. Requirements Review, QA Review, Functionality Review, User Interface Review). A meeting notice, with related documents, will be emailed to each participant. Project Documentation is responsible for recording and archiving weekly and monthly status reviews.
Regular weekly meeting will be held to discuss reported QA issues. The development team will provide status/updates on all issues reported and the test department will provide additional defect information if needed. All members of the project team will participate.
Once testing begins, suggested changes to the TAIGA application will be recorded and documented as part of the Monthly Status Review provided to NASA.
When defects are found, the testers will complete a defect report on the defect tracking system devised by the team. The defect tracking system should be accessible by testers, developers & all members of the project team. The team will monitor new versions/builds of TAIGA to determine changes to the defects. Once a defect is verified as FIXED by the testers, the testers will close the defect report.
The following is a list of functions that will be tested:
§ TBA by NPHS team
A Requirements Validation Matrix will “map” the test cases back to the requirements. See Deliverables.
8. Resources and Responsibilities
The Test Lead and Project Manager will determine when system test will start and end. The Test lead will also be responsible for coordinating schedules, equipment, & tools for the testers as well as writing/updating the Test Plan, Weekly Test Status reports and Final Test Summary report. The testers will be responsible for writing the test cases and executing the tests. With the help of the Test Lead, the Payroll Department Manager and Payroll clerks will be responsible for the Beta and User Acceptance tests.
The test team will consist of:
§ A Project Manager
§ A Test Lead
§ x Testers
§ Project Documentation
Project Manager
Responsible for Project schedules and the overall success of the project.
Team Lead(s)
Serve as a primary contact/liaison between Trillium Learning and the project team.
Test Lead
Ensures the overall success of the test cycles. He/she will coordinate weekly meetings and will communicate the testing status to the project team.
Testers
Responsible for performing the actual system testing.
Project Documentation Manager
Serves as lead for acquiring all project media including status reviews, communications and activities recording. Construct presentations and informational media about the project.
10. Temporary Test Suspension Criteria
If any issues are found which seriously impact the test progress, the QA manager may choose to pause testing. Criteria that will justify test suspension are:
§ Hardware/software is not available at the times indicated in the project schedule.
§ New software builds not available
§ Assigned test resources are not available when needed by the test team.
If testing is temporarily paused, resumption will occur when the problem(s) that caused the suspension has been resolved. When a critical issue is the cause of the suspension, the “resolution” must be verified by the test department before testing is resumed.
The test team requires team testers to develop, perform and validate tests. The test team may also need additional resources TBD.
The NASA TAIGA alpha/beta software builds must be delivered to the iPads on (at least) a monthly basis.
The iPads and flight simulators are required hardware resources. Additionally, multimedia and project documentation will require laptops and/or desktop hardware.
The following documentation will be available at the end of the test phase:
§ Test Plan
§ Status Reviews
§ Website Media
§ Defect reports
§ Final Test Summary Report
Name (Print) Signature Date
1.
2.
3.