By: Capt Scott Wyman and Dr. Seong Jong Joo
US Air Force Airman First Class Emma Davis, 28th Maintenance Squadron B-1 avionics back shop technician, adjusts an oscilloscope at Ellsworth Air Force Base, South Dakota, May 19, 2025. (US Air Force photo by Senior Airman Dylan Maher)
I. Introduction
It is difficult to overemphasize the importance of effective and efficient aircraft maintenance in the Air Force. Without proper maintenance, aircraft safety and serviceability plummets, negatively impacting Air Force missions on both a large and small scale.[1] While many aspects of aircraft maintenance can be considered for analysis, the maintenance of Line-Replaceable Units (LRUs) is a common choice in previous studies due to the significant impact on overall cost and timeliness when problems arise.[2] [3] [4] An LRU is a modular component of a complex system—such as an aircraft—that can be quickly removed and replaced to simplify maintenance and reduce downtime. LRUs are designed to be swapped out at the operational level without requiring extensive repairs or specialized facilities. Once removed, faulty LRUs are typically sent to a specialized maintenance facility (backshop) for in-depth troubleshooting and repair.
Considering the criticality of both LRUs and avionics in aviation, this research investigates the maintenance efficiency of a top-priority LRU, the Band 6 Transmitter. The efficiency and production of maintenance operations related to LRU production from January 2022 to December 2023 was examined using Data Envelopment Analysis (DEA). DEA is an excellent choice for this efficiency problem because of its capability to analyze multiple variables and provide benchmarks that each month can be compared to.[5]
II. Literature Review
In the following review of existing literature, themes emerged such as DEA’s usefulness for the USAF and novel uses of DEA for various aircraft maintenance measures. First, Bowlin (1987),[6] is a canonical work that evaluates DEA as a new tool to measure and assess the operational efficiency of USAF organizations. By conducting DEA and validating their results with various USAF officials to ensure accuracy, Bowlin (1987)[7] concluded that DEA has great value for the Air Force, where it can serve as a guide to auditors, budget programmers, managers, and more; to measure, evaluate and enhance operational efficiency. Building off this research, Bowlin (1998)[8] published another influential work that emphasizes the usefulness of DEA due to its ability to include multiple inputs and outputs without knowing the specific relationships between inputs and outputs; this is a positive compared to regression analysis models.
Next, Rouse et al. (2002)[9] describe the development and application of a performance monitoring system using DEA for the engineering services division of an international airline. They used DEA to quantify any productivity changes and praised it for its ability to track continuous improvement over time. Additionally, the DEA model provided a holistic overview capable of providing multiple measures and perspectives. The final aspect their paper emphasizes is that the DEA models can easily be “modified” through the thoughtful selection of relevant inputs and outputs to better measure the specific areas the airline wants to focus on. Overall, the airline was pleased with DEA as a performance feedback mechanism for aircraft maintenance workers.
Hadi Vencheh et al. (2005)[10] discuss the occurrence of production processes with negative and positive inputs/outputs and propose a DEA model to address this. Their paper has a heavy focus on mathematics and contributes considerably to DEA’s power and utility since there are very few scenarios where less input and more outputs are always good with no exceptions.
Sheikhalishahi (2014)[11] and Wei et al. (2015)[12] both use DEA to learn more about aircraft maintenance practices. First, in Sheikhalishahi’s work, a simulation is used to investigate different avenues for maintenance activities planning and then uses DEA to evaluate the different scenarios. Through this process, DEA was used to select the best activities for planning policy. On the other hand, Wei et al. (2015)[13] used DEA to address subjectivity in aircraft maintenance supervision evaluation and inspection processes. This is a more novel use of DEA, but it follows the same theme of problems identified through observation and DEA being used to achieve objective conclusions about efficiency.
Next, Liu et al. (2015)[14] uses DEA to analyze the efficiency of civil aircraft maintenance.
Considering lifecycle cost, the authors chose the cost of the acquisition, operation and maintenance, and retirement disposal as the input variables and Wing Loading and Thrust-to-Weight ratio as the output variables. These output variables were chosen because they are two of the main performance measurements of civil aircraft. Using DEA, the authors identified efficient aircraft, compared them to inefficient aircraft, and utilized this information to make suggestions on how to make all the processes efficient.
De Souza et al. (2018)[15] discussed using DEA complemented with Tobit regression to analyze whether continuous improvement and learning processes improve the efficiency and production volume of an armament manufacturer. The authors observed processes over time by using a longitudinal case study methodology to gather data to use for DEA and the Tobit regression. They cited the addition of the Tobit regression as a key addition to the DEA models and stated that it was helpful in identifying the specific factors that were affecting efficiency/inefficiency.
Finally, O’Neal et al. (2021)[16] also combined DEA with Tobit regression to measure the relative efficiency of maintenance performances for a USAF aircraft in an effort to enhance aviation safety and combat readiness. In this study, the authors found that an increased number of cannibalization (extracting necessary parts from the existing aircraft) practices tended to reduce maintenance efficiency, and the number of Mission-Capable (MC) aircraft was the most important factor for maintenance performance efficiency. This paper is one of the first studies on aircraft maintenance that considered popular but neglected cannibalization practices as a new variable for assessing maintenance efficiency. Given the importance of the repair and maintenance process in keeping equipment and facilities ready for use, it is necessary to regularly assess the process and performance of maintenance indicators so that the repair and maintenance are planned and upgraded.
As seen through the review of the existing literature, DEA is frequently used to analyze aircraft aviation practices and make policy recommendations. The articles above emphasize DEA’s usefulness because of its ability to do multicriteria analysis and to measure a variety of different aspects, from continuous improvement to organization-specific goals.
Staff Sgt. Walker Hickey, 723d Aircraft Maintenance Squadron electrical and environmental journeyman, removes a distributor from an HH-60G Pave Hawk, Feb. 12, 2020, at Moody Air Force Base, Georgia. Airmen with the 723d AMXS remove serviceable components from the ‘CANN Bird’ in order to return aircraft back to mission-capable status. (US Air Force photo by Airman 1st Class Elijah M. Dority)
III. Methodology
All production data used in this study was received directly from the field maintenance unit extracted from the Reliability, Availability, and Maintainability for Pods (RAMPOD) database. This database is a logistics engineering support system for electronic combat pods and integrated systems.[17] The initial data received contained the production information for the Band 6 Transmitter; however, maintenance hours, Maintenance Actions (MA) taken, and available hours were not included. The production data did include whether the component departed the shop Full Mission-Capable (FMC) or Non-Mission Capable (NMC). The production data was filtered and sorted to determine the number of LRUs produced (FMC or NMC) each month for a 2-year period (January 2022 through December 2023).
Next, without actual maintenance data, the organization again utilized RAMPOD to provide a by-year “crewman summary.” These datasets are standalone spreadsheets containing every Band 6 Transmitter that was produced. First, each summary dataset is compared with the production data to ensure that output production numbers matched. Each summary dataset contains the start and end date for each LRU produced in conjunction with logged MA. A count function is then used to determine how many LRUs entered and exited the shop each month, and a summation function is used to determine the total maintenance hours for each LRU. Finally, a count of logged MA provides the total MA for each LRU. The final step is to combine the two maintenance datasets with the production data. Through examination of all datasets, it was determined that the use of assigned document numbers would ease the combination of datasets.
Lastly, available maintenance hours are calculated. Following field interviews, it is determined that the organization’s typical operation is five days a week and 24 hours each day. In addition to workdays, the unit maintains the Band 6 LRU on two identical test stations. To compute the available working hours, a ‘Net-workdays’ function was used in conjunction with an array of federal holidays and family days to determine the total available workdays per month. Next, the total workdays were multiplied by 24 to provide the total hours available per month. Finally, the total hours available was multiplied by two since there are two test stations to determine the available working hours per month. With all data amalgamated, all actions to prepare for analysis using DEA solver were completed.
With all data compiled, 13 different variables were utilized in model selection. Following variable testing, it was determined that the input variables used in the analysis would be (1) available working hours (Available Hrs), (2) actual hours worked (Labor Hrs), and the output variables would be (3) MA taken and the (4) quantity of LRUs that were repaired on site (Qty MC). We selected the variables related to working hours and maintenance activities because of the emphasis on measuring manpower efficiency and effectiveness in the existing literature about aircraft maintenance .[18] The variable relating to the LRUs repaired on site is chosen from the data because of the criticality of LRU repair and replacement to maintenance duration and, thus efficiency (Dreissen et al. 2023).[19]
Between these four variables, we selected “Qty MC” as an output variable because the number of operational LRUs is the main product and purpose of the shop that is being analyzed. Table 1 summarizes the descriptive statistics for all variables, whereas Table 2 shows the correlation coefficients between the inputs and output. The unit had a maximum Decision Making Unit (DMU) output of 12 units and a minimum of 1.
DMUs are any entity or unit being evaluated for efficiency in a DEA. It can be an organization, department, person, or any system that uses resources to produce outputs. DMUs are compared to one another to determine which ones are performing efficiently and which need improvement. For this study, DMUs are months, where each month is being evaluated as an individual unit of analysis. In this case, we assessed how efficiently resources are used in each month to produce outputs and compared the performance of different months. This allows us to identify which months are performing most efficiently and which ones may need improvements in resource allocation or process management.
Correlation coefficients help us avoid redundancy in our model. There is a high correlation (0.9718) between labor hours and (MA) in particular which intuitively makes sense. Moreover, (MC) outputs have a low correlation with labor hours (0.3642) and a moderate correlation with MA (0.4699). Conversely, there is a negligible correlation between MC and available hours (-0.0313) which indicates that changes in hours do not predict changes in MC. To explore the trends of the variables, we plot the variables against time (Figure 1). Lastly, to fully visualize the MC output over time, Figure 2 shows a downward trend across the 24 DMUs.
For our analysis, the DEA models used were Charnes-Cooper-Rhodes (CCR)[20], Banker-Charnes-Cooper (BCC), Slacks-Based Measure of efficiency (SBM), all output-oriented, meaning that models evaluate each month by whether they maximized output for their given level of inputs. On the other hand, input-oriented is when inputs are minimized while the output level is held constant.[21] Due to a lack of need to decrease inputs in our models (ex: available hours do not decrease, the shop is open no matter what), we decided to use the output-oriented models. Furthermore, because flight-level output has decreased over the last two years, an output-oriented model is appropriate for understanding how to maximize output.
CCR, BCC, SBM are foundational models in DEA and are used for this study. CCR measures (TE) and assumes constant returns to scale (as inputs change, outputs change in a proportional manner). TE refers to the ability of a DMU to produce the maximum output from a given set of inputs, or alternatively, to use the minimum amount of inputs to produce a specific level of output. A DMU is considered technically efficient if it operates on the production frontier, with no waste or inefficiencies in resource use. This is simpler and more computationally efficient than varying returns to scale, but it is less realistic.
US Air Force Senior Airman Cassandra Kaehr, 28th Maintenance Squadron B-1 avionics back shop technician, connects a wire onto a line-replaceable at Ellsworth Air Force Base, South Dakota, May 19, 2025. (US Air Force photo by Senior Airman Dylan Maher)
On the other hand, BCC or Pure Technical Efficiency (PTE) accounts for the efficiency of a DMU’s use of resources while allowing for variable returns to scale. This means it recognizes that efficiency may change as the size or scale of the operation changes, allowing for a more nuanced evaluation. Unlike TE, which assumes constant returns to scale, PTE focuses only on how well a DMU uses its inputs, independent of its scale. Finally, SBM incorporates unused resources and in the case of output-oriented models, unused inputs that could be used to maximize outputs. Overall, these were chosen due to their use in previous studies and their ability to address different aspects of inefficiencies.[22] Additionally, SCALE scores (TE/PTE) were included in our analysis because they evaluate operating conditions and determine whether a firm is operating at its optimal scale size. Furthermore, MIX scores (SBM/TE) are included as these scores represent the overall efficiency of a DMU by considering both minimized inputs and excess outputs. A DMU with a MIX score of 1.00 indicates it is operating on what is called the production frontier and is utilizing inputs and outputs efficiently.
IV. Results and Discussion
As mentioned earlier, this paper evaluates how parameters chosen as inputs and outputs can affect efficiency scores regarding the results obtained from the maintenance efforts. Three DEA models (BCC, CCR, SBM) were applied to calculate the efficiency scores, as recapitulated in Table 3. Three out of twenty-four months registered 100% efficiency scores across DEA models and became the benchmark for the other twenty-one inefficient months, which was identified using the SBM model.
PTE scores were measured using a BCC model, and TE scores were measured using a CCR model. To compute the scale efficiency scores (SCALE), which represent operating conditions, TE was divided by PTE. Lastly, the MIX scores were computed by dividing SBM by TE. Doing this, sources of inefficiency could be categorized as PTE for internal operations, SCALE for operating conditions, TE for overall efficiency, and MIX for the optimal mix of outputs for an output-oriented model inputs for an input-oriented model. In Table 3, there are fundamental and composite scores. TE and SBM scores are composite scores and can be broken down to the fundamental scores as stated before. By looking at the fundamental scores such as PTE, SCALE, and MIX scores, we can figure out a source(s) of inefficiency. Among the three fundamental scores, the mean MIX score is the lowest, which implies that the imbalance between two outputs (the number of MA taken and the MC rate) is the primary source of inefficiency as measured with the SBM scores. This finding is supported by Table 5, Potential Improvement by Percentage. The mean potential improvements for MC are much higher than that of MA. Compared to MA, MC is low. Thus, MC needs managers’ attention. June 2023 received the overall lowest scores across all three models, whereas March, April, and July of 2022 were fully efficient across all measures. These months provide the maintenance leaders with starting points to evaluate the factors that led to the performance of each respective month. Table 4 captures the rank order of DMUs by SBM scores.
Interestingly, July 2022, one of the three fully efficient DMU utilized 217 labor hours and 42 MA to produce four LRUs; figures that are almost half that of other DMUs with identical production numbers. Figure 3 visualized the efficiency scores (PTE, TE, SBM, Scale, & MIX) as presented in Table 3 with an illustrated decreasing trend on the SBM model.
This trend might be caused by limited maintenance capacity and delayed spare part supply. Even though Awaiting Parts (AWP) data could not be obtained, we know from field interviews that a high number of LRUs are AWP with the latest data reflecting almost 70 LRUs AWP. Since that data could not be included in the model, it could be translated as inefficiencies with repair capabilities when the inefficiency is due to spare part production and arrival, which is out of the unit’s control. When DEA models were run, potential improvements were obtained on variables for inefficient DMUs or DMUs that were months in this study. Table 5 presents the potential improvements computed with the SBM model.
US Air Force Senior Airman Maxim Cojocaru, 28th Maintenance Squadron B-1 avionics back shop technician, takes notes at Ellsworth Air Force Base, South Dakota, May 19, 2025. (US Air Force photo by Senior Airman Dylan Maher)
Negative numbers on the input variables implied the underutilization or where a reduction of the variables could occur. Positive numbers on the output variables meant a desired increase in the variables. The input variable of available hours had a moderate reduction at 21.64% percent, whereas labor hours had smaller amounts of reduction, at 4.12% percent. For the output, MA showed a moderate increase at 13.17% percent. Furthermore, the other output, Total MC output, should be boosted by 160.64%, which may be impossible due to the supply issue mentioned earlier. For each DMU, the excess of available hours was the highest (-85.08%) in December 2023. This indicates that the unit could have decreased the number of available hours for that month. The excess labor hours reached their peak (-21.32%) in October 2022 and November 2023 posted the highest (41.63%) desired increase of MA. Lastly, June 2023, as previously discussed, recorded a staggering desired output increase (536%). This indicates that with its [June 2023] inputs, approximately five LRUs could have been produced. This was expected as this DMU ranked last in efficiency scores (.2618 SBM) across all 24 DMUs.
V. Limitations and Future Research
While we can make inferences and recommendations from our results, there are several limitations and avenues for future research. First, there was a lack of further investigation into alternative inputs such as authorized manning and assigned personnel by technical skill level in the research. This would be an excellent avenue for future research if more information could be obtained. Regarding assigned personnel by technical skill level, this could be a useful input because it could shine more light on whether this is a manpower or skill problem.
Another limitation does not include a breakout of units awaiting parts (AWP) versus unit awaiting maintenance (AWM). Gathering this information and including it as inputs in a DEA model would help explain the difference between when a transmitter is simply waiting on parts, or on the shelf in the queue for MA. This would help flush out the problem areas of concern: technical proficiency versus supply chain concerns. Moreover, networked DEA models could further examine the unit’s LRU production concerns since outputs and excess from previous DMUs could be used as inputs into the subsequent DMU. This is especially applicable in the case of units returning from depot maintenance, and the carryover of units requiring maintenance from previous month(s). Lastly, as discussed in our methodology section, our available calculations are not entirely accurate because we did not have total information about any exceptions to their general rule of being available (five days a week, 24 hours a day excluding holidays).
VI. Conclusion
In this article, we used three DEA models (CCR, BCC, SBM) to investigate and evaluate the maintenance efficiency regarding a top priority LRU repaired at an Avionics backshop. To cage our research, we focused on what has caused the decrease in production of a high priority LRU within an avionics field maintenance unit. After running three DEA models for 24 months and analyzing the data, we found that only three of the 24 months were efficient across all three models.
Among the three fundamental scores, the mean MIX score is the lowest, which implies that the imbalance between MA and the MC produced is the major source of inefficiency as measured with the SBM scores. This finding is supported by the potential improvements and compared to MA, the MC quantity is low, thus, MC needs managers’ attention. Additionally, benchmarked months and poorly performing months can be used to guide management towards specific aspects to focus on. Since there are three months that were fully efficient across all measures, we also recommend that management consider doing a deep dive into those months, the circumstances surrounding them, and whether there was any change in internal operating processes during that time. On the other hand, analyzing the circumstances and processes from some of the worst performing months such as June 2023 could be enlightening as well.
Although there were limitations in our research due to the inability to get more in-depth data, there are many opportunities for future research. Overall, DEA is an excellent tool for future research in this area as it allows us to make recommendations in this case and can be expanded and used in a variety of aircraft maintenance contexts.
The views expressed in this article are those of the author and do not necessarily reflect the official policy or position of the Air Force, the Department of Defense, or the US Government.
About the Authors
Capt Scott Wyman is a Missile and Munitions Maintenance Officer currently pursuing his master’s degree in Logistics and Supply Chain Management at the Air Force Institute of Technology, Wright-Patterson AFB. Upon graduation, he will join the Nuclear Weapons Center at Hill AFB. He can be contacted at scott.wyman@us.af.mil.
Dr. Seong-Jong Joo is a Professor of Logistics & Supply Chain Management in the Department of Operational Sciences at the Air Force Institute of Technology where he teaches graduate-level courses in Logistics and Supply Chain Management. Before returning to academia, he served in the Republic of Korea Air Force (ROKAF) for 21 years as a Supply Officer and retired honorably with the final rank of Lt Col. He can be contacted at seong-jong.joo@us.af.mil.
Works Cited
[1] O’Neal, T., Min, H., Cherobini, D., & Joo, S.-J. (2021). Benchmarking aircraft maintenance performances using data envelopment analysis. International Journal of Quality & Reliability Management, 38(6), 1328–1341. https://doi.org/10.1108/IJQRM-05-2020-0157
[2] Gill, M. S., & Fay, A. (2023). Utilisation of semantic technologies for the realisation of data-driven process improvements in the maintenance, repair, and overhaul of aircraft components. CEAS Aeronautical Journal. https://doi.org/10.1007/s13272-023-00696-5
[3] Jin, T., Li, H., & Sun, F. (2021). System Availability considering Redundancy, Maintenance and Spare Parts with Dual Repair Processes.
[4] Raza, A., & Ulansky, V. (2021). Through-Life Maintenance Cost of Digital Avionics. Applied Sciences, 11(2), 715. https://doi.org/10.3390/app11020715
[5] Charnes, A., & Cooper, W. W. (1984). Preface to topics in data envelopment analysis. Annals of Operations Research, 2(1), 59–94. https://doi.org/10.1007/BF01874733
[6] Bowlin, W. F. (1987). Evaluating the Efficiency of US Air Force Real-Property Maintenance Activities. Journal of the Operational Research Society, 38(2), 127–135. https://doi.org/10.1057/jors.1987.25
[7] Bowlin, W. F. (1987). Evaluating the Efficiency of US Air Force Real-Property Maintenance Activities. Journal of the Operational Research Society, 38(2), 127–135. https://doi.org/10.1057/jors.1987.25
[8] Bowlin, W. F. (1998). Measuring Performance: An Introduction to Data Envelopment Analysis (DEA). The Journal of Cost Analysis, 15(2), 3–27. https://doi.org/10.1080/08823871.1998.10462318
[9] Rouse, P., Putterill, M., & Ryan, D. (2002). Integrated performance measurement design: Insights from an application in aircraft maintenance. Management Accounting Research, 13(2), 229–248. https://doi.org/10.1006/mare.2002.0180
[10] Hadi Vencheh, A., Kazemi Matin, R., & Tavassoli Kajani, M. (2005). Undesirable factors in efficiency measurement. Applied Mathematics and Computation, 163(2), 547–552. https://doi.org/10.1016/j.amc.2004.02.022
[11] Sheikhalishahi, M. (2014). An integrated simulation-data envelopment analysis approach for maintenance activities planning. International Journal of Computer Integrated Manufacturing, 27(9), 858–868. https://doi.org/10.1080/0951192X.2013.869832
[12] Wei, N.-C., Liao, T.-J., Chang, C.-F., & Cham, S.-C. (2015). Using Data Envelopment Analysis to Evaluate Technical Training Program for Maintenance Units. International Journal of Engineering Research, 4(5), 264–267. https://doi.org/10.17950/ijer/v4s5/511
[13] Wei, N.-C., Liao, T.-J., Chang, C.-F., & Cham, S.-C. (2015). Using Data Envelopment Analysis to Evaluate Technical Training Program for Maintenance Units. International Journal of Engineering Research, 4(5), 264–267. https://doi.org/10.17950/ijer/v4s5/511
[14] Liu, W., Chang, W., & Zhou, S. (2015). Evaluation of Engineering Asset Acquisitions in EAM Based on DEA. In P. W. Tse, J. Mathew, K. Wong, R. Lam, & C. N. Ko (Eds.), Engineering Asset Management—Systems, Professional Practices and Certification (pp. 739–745). Springer International Publishing. https://doi.org/10.1007/978-3-319-09507-3_64
[15] De Souza, I. G., Lacerda, D. P., Camargo, L. F. R., Dresch, A., & Piran, F. S. (2018). Do the improvement programs really matter? An analysis using data envelopment analysis. BRQ Business Research Quarterly, 21(4), 225–237. https://doi.org/10.1016/j.brq.2018.08.002
[16] O’Neal, T., Min, H., Cherobini, D., & Joo, S.-J. (2021). Benchmarking aircraft maintenance performances using data envelopment analysis. International Journal of Quality & Reliability Management, 38(6), 1328–1341. https://doi.org/10.1108/IJQRM-05-2020-0157
[17] Mills, P., & Feinberg, A. (2001). Supporting expeditionary aerospace forces: Evaluation of the RAMPOD database. RAND.
[18] Datta, P. P., Srivastava, A., & Roy, R. (2013). A simulation study on maintainer resource utilization of a fast jet aircraft maintenance line under availability contract. Computers in Industry, 64(5), 543–555. https://doi.org/10.1016/j.compind.2013.02.011
[19] Driessen, J., De Kruijff, J., Arts, J., & Van Houtum, G. (2024). Optimal design of line replaceable units. Naval Research Logistics (NRL), 71(2), 189–209. https://doi.org/10.1002/nav.22146
[20] Charnes, A., & Cooper, W. W. (1984). Preface to topics in data envelopment analysis. Annals of Operations Research, 2(1), 59–94. https://doi.org/10.1007/BF01874733
[21] Bowlin, W. F. (1998). Measuring Performance: An Introduction to Data Envelopment Analysis (DEA). The Journal of Cost Analysis, 15(2), 3–27. https://doi.org/10.1080/08823871.1998.10462318
[22] Bowlin, W. F. (1998). Measuring Performance: An Introduction to Data Envelopment Analysis (DEA). The Journal of Cost Analysis, 15(2), 3–27. https://doi.org/10.1080/08823871.1998.10462318