Project #4

Purpose:

One focus area for this course is understanding how to structure and implement big software systems. By big we mean systems that may consist of hundreds or even thousands of packages¹ and perhaps several million lines of code. We won't be building anything quite that large, but our projects may be considerably bigger than anything you've worked on before. In order to successfully implement big systems we need to partition code into relatively small parts and thoroughly test each of the parts before inserting them into the software baseline². As new parts are added to the baseline and as we make changes to fix latent errors or performance problems we will re-run test sequences for those parts and, perhaps, for the entire baseline. Managing that process efficiently requires effective tools for code analysis as well as testing. How we do that code analysis is illustrated by the projects for this year. The projects this Fall focus on building software testing tools. We will emphasize C++ code but want our tools to be easily extendable to other similar languages like C# and Java. You will find it useful to look at the SW Testing blog for a brief introduction to production testing.

In this fourth project we will build and test the functionality for remote multi-user testing. Most of the functionality has already been developed in the first three projects. Here, we will focus on the Client Graphical User Interface (GUI). Things now get interesting - in both senses of the word - because C++ and its standard libraries give us little help for building graphical user interfaces. Instead, we are going to reach out to the libraries supported by two other languages for that help: The .Net libraries provide a very well engineered facility for building these interfaces, e.g., Windows Presentation Foundation (WPF). We will elect to use C# with that framework, as shown in the accompanying diagram. C# and WPF will be discussed in class, and we will show you a lot of example code. To alay your concerns, many students in this class in previous semesters have used this approach successfully. We also need a "converter shim" to allow the C# code to communicate with our native Communication Channel code. The shim is built using C++/CLI, a managed .Net language with the same object model as C#, but much of its syntax like C++. You may use a Shim I built for last year's project, probably with no changes. The only constraints for this implementation is that we need to build the Shim, and native Channel code, as a dynamic link library, which the GUI loads at startup. We will give you alot of help with that. You will spend most of your time, in Project #4, working on the GUI to provide facilities for the user to select one or more TestRequests to send, and to display the test results. Display is the most interesting, because we want to allow the test driver to log information that is relevant for a particular test, as well as show the status: pass or fail. We will leave it up to you to design all of the GUI controls and displays to support testing.

As for the earlier projects, the TestHarness must support:

Loading dynamic link test libraries:
This requires:
1. clients send one or more test libraries to the TestHarness Server
2. clients pass test request messages to define which libraries are to be executed
Executing tests:
This requires:
1. TestHarness executes each test request in a separate process, provided by a process pool.
2. The process pool is hosted by the TestHarness, which manages child test processes using message-passing communication.
3. When the test execution completes, the child test process sends, to the requesting client, a test report.
Reporting results in a Graphical User Interface:
The client displays test information based on the return message and formatting parameters set by user actions in the client GUI.

The good news is that you have already implemented all of that. Now, we are focusing on the Client. It must support:

Selecting Test Libraries to send:
The Client displays all the dynamic link libraries in a specified directory. It allows the user to make multiple selections, and passes that information to a sender package.
Sending the selected libraries to the TestHarness:
There are two ways to do that:
- The client uses Comm's sendfile mechanism to transparently send each message. We will discuss, in class, how the Comm framework does that during one of the Sockets lectures.
- The sender reads each binary file and stores it in the body of a message, one message for each file, and sends the message to the TestHarness.
The TestHarness will need to use code dedicated to whichever of the two alternatives you choose.
Displaying the results of testing:
The user has to be able to tell which results belong to which requests. With asynchronous message passing, it is possible for the results to arrive out-of-order, because each test is likely to take a different amount of time to complete and is executed concurrently with other testing. You can handle this by giving each TestRequest a unique identifier, and embed that identifier in the results message. Each results message will have a status: pass or fail, and additional information that is likely to vary in contents and quantity from test to test. You should design a structure for reporting test results, and design the UI display to accomodate that structure, perhaps as a list of textual information items.

The TAs and I will give you some help during Help Sessions and Office Hours, but we won't layout the entire GUI design for you.

Requirements:

Your TestHarness Solution:

Shall use Visual Studio 2017 and its C++ Windows Console Projects, as provided in the ECS computer labs.
Shall use Windows Presentation Foundation (WPF) for all user display.
(3) Shall assemble the working parts from Projects #1, #2, and #3 into a Client-Server configuration. Unlike conventional Client-Server architectures, the Client need not wait for a reply for a TestRequest from the TestHarness before sending additional TestRequests.
(3) Shall provide a Graphical User Interface (GUI) for the client that supports selecting test libraires to send for testing, and supports displaying the test results.
(2) Shall provide message designs appropriate for this application. All messages are instances of the same Message class, but have a specified set of attributes and body contents suited for the intended task.
(2) Shall support sending TestRequest messages from Client to TestHarness Server.
(2) Shall support correctly executing each TestRequest, as a sequence of tests, with each TestRequest running in its own Process Pool child process.
(2) Shall support sending test results, via results messages, from the child process executing the tests to the Client that made the request.
(2) Shall demonstrate correct test operations for two or more concurrent clients. You need to ensure that testing takes enough time to ensure that more than one child process is running at the same time.
(4) Shall include an automated unit test suite that demonstrates your Clients and TestHarness satisfy all of the functional requirements, above.

In C#, a package is a single source code file that contains:
- prologue, providing a name, brief descriptive phrase, author information, and environment information
- description of the package's responsiblities and required files
- maintenance history
- class definitions
- a main function that implements construction tests for all the defined code
A software baseline is the set of all code considered to be part of the current system, excluding experimental code that individual developers are working on.

Project #4 - Remote TestHarness

Purpose:

Requirements:

What you need to know: