Project #3

Purpose:

One focus area for this course is understanding how to structure and implement big software systems. By big we mean systems that may consist of hundreds or even thousands of packages¹ and perhaps several million lines of code. We won't be building anything quite that large, but our projects may be considerably bigger than anything you've worked on before. In order to successfully implement big systems we need to partition code into relatively small parts and thoroughly test each of the parts before inserting them into the software baseline². As new parts are added to the baseline and as we make changes to fix latent errors or performance problems we will re-run test sequences for those parts and, perhaps, for the entire baseline. Managing that process efficiently requires effective tools for code analysis as well as testing. How we do that code analysis is illustrated by the projects for this year. The projects this Fall focus on building software testing tools. We will emphasize C++ code but want our tools to be easily extendable to other similar languages like C# and Java. For this third project, you will find it useful to look at:

Message-Passing Communication
Blog on asynchronous communication.
Comm diagrams
Diagrams used in the second Sockets Lecture.
Class diagram
Class diagram for the Comm Channel.

You will find a relatively complete implementation of the Comm Channel in the Cpp Repository. You will find other help in the CSE687/code/Project3HelpF18 folder.

In this project we will build and test prototypes for a Process Pool and Message-Passing Communication channel, used for sending messages between a client process, the TestHarness process, and each of the child processes spawned by the TestHarness. All of the processes: Client, TestHarness, and Children, encorporate a communication package used for message and file transfers. The Client process builds a test library³ and sends that, along with a TestRequest message, to the TestHarness. Eventually it will receive a reply from the TestHarness with a test log and status for that request. The TestHarness contains a TestRequest queue and a Ready queue. These are thread-safe blocking queues⁴ used to control its process pool children. When the TestHarness starts up, it spawns a specified number of child tester processes⁵. Each child sends a ready message to its TestHarness host, indicating that it is ready to receive a request for testing. The main thread of the TestHarness blocks on the TestRequest queue, until its Comm component receives and posts a client's TestRequest message. It then attempts to dequeue a ready message, and if one is available, it sends the TestRequest to the child that posted the ready status⁵. Otherwise, it blocks until a child is ready. The main thread for each Child Tester Process blocks on its Comm receive queue until a TestRequest message is posted. It then loads the library, named in the TestRequest, and executes all the tests in its Single-User TestHarness, just as you did in Project #1. When its testing is complete, it sends its test log, in a message, to the original client posting the request. This works, because each message sent over the Comm Channel contains both source and destination addresses.

Implementing Project #3 entails:

Defining messages:
Messages are used for communication between client and test server. Each message contains: source and destingation addresses, a specified request for processing or results of processing, and may contain other attributes you deem important or useful for your design. In this project, we will implement HTTP "style" messages. Each message consists of a header with a sequence of attributes expressed as a text line with name and value, and may include a block of bytes in an optional message body.
Erecting a Communication Framework:
Erect a framework for passing addressible messages used for requests, replies, and notifications. This should be relatively straight-forward because you will find a nearly complete implementation of the Comm Channel in the Cpp Repository.
Creating a Framework for Hosting a Process Pool:
The TestHarness host will need to incorporate the Comm Channel and carry out the following activities:
1. Create a specified number of Child Processes which execute all the tests. You will find a Process Class in the Repository/Cpp folder. That shows you how to create a Win32 Process.
2. Post client TestRequest messages to the TestRequest Queue and Child Ready messages to the Ready Queue with a receive thread.
3. Dequeue, on its main thread, TestRequests and Ready messages from its internal queues. It may block on either the TestRequest queue or the Ready queue, but when it eventually returns, it has a TestRequest and a child ready to process it. Note that the only purpose of the Ready Queue is to block the TestHarness main thread until a Child Process is ready. Consequently, the Ready messages are used only to find the address of the ready Child, and are then discarded.
Demonstrating operation with concurrent clients:
Show that clients can send requests and receive replies to those requests concurrently with messaging between other clients and the server.

Requirements:

Your Communication Solution:

Shall use Visual Studio 2017 and its C++ Windows Console Projects, as provided in the ECS computer labs.
Shall use the standard streams libraries for all I/O and operators new and delete for memory management.
(2) Shall provide a ProcessPool Framework that operates as described in the Purpose section, above.
(2) Shall provide a TestHarness and Tester Children that operate as described in the Purpose section, above.
(1) Shall provide logging, as implemented in Project #1, in each Tester Child. The results of logging are sent by the child to the Client that sent the TestRequest⁶.
(2) Shall provide a Client process that sends HTTP style TestRequest messages and receives and displays test results in the form of HTTP style messages. Note that your Client process does not need to build test libraries. You will do that as described below.
(5) Shall include an automated unit test suite that exercises all of the special cases that seem appropriate for the packages in this Project. In order to support testing you shall prepare Single-User test sequences, as you did in Project #1, and build them as libraries. They will be executed in Child Tester processes, using the Single-User TestHarness you developed in Project #1. Each of these test sequences are used to demonstrate that you meet all the functional requirements of Project #3.

In C#, a package is a single source code file that contains:
- prologue, providing a name, brief descriptive phrase, author information, and environment information
- description of the package's responsiblities and required files
- maintenance history
- class definitions
- a main function that implements construction tests for all the defined code
A software baseline is the set of all code considered to be part of the current system, excluding experimental code that individual developers are working on.
A test library is a Win32 dynamic link library that contains all of the tests you want to run, exposed through an ITest interface.
The thread-safe blocking queues are those discussed during the Threading lectures. You will find an implementation of the Blocking Queue in the Repository/Cpp folder.
All messages contain both source and destination attributes, so the TestHarness knows to which child the TestRequest must go, e.g., the child that sent the dequeued ready message.
Each tester child is required to provide the entire test report at the end of its testing, before sending a ready message to the TestHarness mother process. You may also provide an option for streaming a sequence of logged report message to the client, but non-streamed reports must be the default.

Project #3 - Process Pool with Interprocess Communication

Purpose:

Requirements:

What you need to know: