Universal Logging System

This is the test suite for version 1.8.0 of ULS.

A number of administrative tasks and entries must be done to set up the necessary settings for all tests. This is also a test of the ULS administration itself.

A number of well-defined basic values containing all supported datatypes are sent to the ULS and the correct transmission and presentation is checked thru a manual basic analysis.

boundary tests are made to verify the correct processing of e.g. very small and very large values.

The general tests are based on values derived from the current number of seconds

• of the current hour
• of the current day and
• of the current week (since 00:00 of last monday)

These values are specifically converted and calculated to cover the most common scenarios. In combination with threshold definitions they are used to test the firing of notifications for threshold violations The notifications will contain replaced static and dynamically calculated embedded placeholders in their message bodies.

These values will also be used for the check of the aggregation of values.

Reports are used to check the different expected results and to check the report functionality itself.

• do the necessary Preparations

This section describes all test cases.

• administrative settings and tests in Administrative Settings
• basic values, one execution in Basic Values
• continuous values generation in Continuous Values
• aggregations in Compressions & Aggregations
• notifications in Notifications

Threshold violations

Define various possible notification scenarios and verify the correctly generated notifications

ULS ticket tracking: threshold violations generate notifications and tickets. ULS users can work on those tickets and document their actions.

Threshold violations

Define various possible notification scenarios on aggregated values and verify the correctly generated notifications

ULS ticket tracking: threshold violations generate notifications and tickets. ULS users can work on those tickets and document their actions.

Continue here

the correct storage of all value types:

• including boundary tests to assure the covered value ranges
• checked by interactive analysis
• checked by reports
• deletion of values after the retention time has elapsed

values, thresholds, aggregation, …

• threshold violations

• numeric, string and file-like values
• different functions combined with number of values and/or time range
• including message bodies of notifications with replaced static and dynamically calculated embedded placeholders

• aggregation of values

• check the different possible aggregation functions on different data types

• reports

• check the expected results
• check the creation and modification of reports
• user-defined headers, separators, descriptions and logo
• PDF generation

detailing how to run each test, including any set-up preconditions and the steps that need to be followed

Test Script is a program written to test the functionality of the application. It is a set of system readable instructions to automate the testing with the advantage of doing repeatable and regression testing easily.

It is the Hardware and Software Environment where is the testing is going to be done. It also explains whether the software under test interacts with Stubs and Drivers.

Create a mock business. Treat it as real and process its data.

Test Procedure is a document with the detailed instruction for step by step execution of one or more test cases. Test procedure is used in Test Scenario and Test Scripts.

Test Log contains the details of test case execution and the output information.

recording which tests cases were run, who ran them, in what order, and whether each test passed or failed

detailing, for any test that failed, the actual versus expected result, and other information intended to throw light on why a test has failed. This document is deliberately named as an incident report, and not a fault report. The reason is that a discrepancy between expected and actual results can occur for a number of reasons other than a fault in the system. These include the expected results being wrong, the test being run wrongly, or inconsistency in the requirements meaning that more than one interpretation could be made. The report consists of all details of the incident such as actual and expected results, when it failed, and any supporting evidence that will help in its resolution. The report will also include, if possible, an assessment of the impact of an incident upon testing.

A management report providing any important information uncovered by the tests accomplished, and including assessments of the quality of the testing effort, the quality of the software system under test, and statistics derived from Incident Reports. The report also records what testing was done and how long it took, in order to improve any future test planning. This final document is used to indicate whether the software system under test is fit for purpose according to whether or not it has met acceptance criteria defined by project stakeholders.