Compressing Test and Evaluation by Using Flow Data for Scalable Network Traffic Analysis
Kevin Buell, Mustafa G. Baydogan, Burhan Senturk, and James P. Kerr
The specialized nature of technology-based programs creates volumes of data of a magnitude never before seen, complicating the test and evaluation phase of acquisition. This article provides a practical solution for reducing network traffic analysis data while expediting test and evaluation. From small lab testing to full integration test events, quality of service and other key metrics of military systems and networks are evaluated. Network data captured in standard flow formats enable scalable approaches for producing network traffic analyses. Because of its compact representation of network traffic, flow data naturally scale well. Some analyses require deep packet inspection, but many can be calculated/approximated quickly with flow data, including quality-of-service metrics like completion rate and speed of service.
Improving Statistical Rigor in Defense Test and Evaluation: Use of Tolerance Intervals in Designed Experiments
Leveraging the use of statistical methods is critical in providing defensible test data to the Department of Defense Test and Evaluation (T&E) enterprise. This article investigates statistical tolerance intervals in designed experiments for the T&E technical community. Tolerance intervals are scarcely discussed in extant literature as compared to confidence/prediction intervals. The lesser known tolerance intervals can ensure a proportion of the population is captured in the design space, and have the ability to map the design space where factors can be reliably tested. Further, the article investigates several two-sided approximate tolerance factors estimated by Monte Carlo simulation and compares them to the exact method. Finally, the applicability of tolerance intervals to the defense T&E community is presented using a simple case study.
A Comparative Analysis of the Value of Technology Readiness Assessments
Reginald U. Bailey, Thomas A. Mazzuchi, Shahram Sarkani, and David F. Rico
The U.S. Department of Defense endorsed and later mandated the use of Technology Readiness Assessments (TRA) and knowledge-based practices in the early 2000s for use as a tool in the management of program acquisition risk. Unfortunately, implementing TRAs can be costly, especially when programs include knowledge-based practices such as prototyping, performance specifications, test plans, and technology maturity plans. What is the economic impact of these TRA practices on the past and present acquisition performance of the U.S. Army, Navy, and Air Force? The conundrum today is that no commonly accepted approach is in use to determine the economic value of TRAs. This article provides a model for the valuation of TRAs in assessing the risk of technical maturity