- Radio hour
- About us
THE BUSINESS OF GOVERNMENT HOUR,
A CONVERSATION WITH AUTHORS
TREVOR BROWN, DAVID VAN SLYKE
Coauthors of Complex Contracting: Government Purchasing in the Wake of the US Coast Guard’s Deepwater Program
Interviewer: Michael Keegan
Michael Keegan: Welcome to a special edition of The Business of Government Hour, a Conversation with Authors. I'm Michael Keegan, your host and Managing Editor of The Business of Government Magazine.
For 15 years, The IBM Center for the Business of Government has sought to connect research to practice, sponsoring third-party research on a broad range of public management issues facing us today. In fiscal year 2012, the federal government acquired $517 billion worth of products through contracts. Contract expenditures amount to 16 percent of total federal spending. Purchases range from simple products like office supplies or landscaping to more complex products like advance weapon systems and program management services.
As the difficulties confronting the federal government become increasingly complicated, so too will the types of services and goods that they need to address these challenges. The federal government is increasingly acquiring products that have qualities that cannot be clearly or easily defined in advance and that are difficult to verify after the product or service has been delivered. These products are called “complex products.” The acquisition of complex products requires more sophisticated contracting approaches.
What are the challenges of acquiring complex products? What lessons can be learned from the Coast Guard’s Deepwater Program? And how can government executives most effectively manage complex acquisitions? We will explore these questions and so much more through the works of the research team, Trevor Brown, Matt Potoski and David Van Slyke, authors of the recent book, Complex Contracting: Government Purchasing in the Wake of the US Coast Guard’s Deepwater Program. Brown, Potoski and Van Slyke discuss the promise and perils of government contracting while providing wide-ranging, practical advice on complex acquisition.
I’m happy to welcome to our show two members of the research team: Trevor Brown.
Trevor Brown: Thank you. It’s a pleasure to be here.
Michael Keegan: And David Van Slyke.
David Van Slyke: Thank you.
Michael Keegan: So, before we get into specifics, I’d like to explore the importance of acquisition and the pursuit of government agency objectives. A lot of folks who listen to the show may be familiar with other mission support functions like financial management or HR. But what is acquisition in the federal context? And how is it a key strategic enabler for government to get its missions accomplished?
Trevor Brown: So, I think most people, when they think of acquisition, think more simply of purchasing, just buying simple stuff like paperclips and copy paper or office supplies. But the reality is that increasingly, federal agencies need critical goods and services to be able to perform their core missions. So, in the report I wrote for IBM, I highlighted the Black Hawk helicopter in the interdiction of Osama Bin Laden and the ultimate taking his body out of the compound. Without the Black Hawk, the mission doesn’t succeed.
Today, the thing that’s probably most on people’s mind is the healthcare.gov website. In the absence of that website working successfully, the Affordable Care Act doesn’t work successfully. And part of that is because you need the website to work so people can access the market. I think that’s the whole underlying premise. But even more narrowly, with the website, you’re trying to target a specific group of people. In the case of the Affordable Care Act, it’s healthy young people. You need them to enter the insurance pool. And the way you’re going to get them to enter the insurance pool is to go on a slick, fancy website. And if it’s not slick, fancy and doesn’t work, they’re going to check out. And then you lose your critical – you lose a critical component of that insurance pool. And then prices are going to rise.
Now, you probably don’t think about that when you’re buying a website. You think, oh, I’m just buying a website. But now, you’re buying a critical, integral part of your program. And that’s what acquisition is now. It’s acquiring essential goods and services to be able to perform basic mission functions inside organizations.
Michael Keegan: David?
David Van Slyke: And I would say, as Trevor is mentioning, it’s really important to get those purchases right. But another key aspect, really, for the listenership of this show is really thinking about being strategic as well. So, it’s not just buying stuff, but it’s really thinking through what the products are, who the potential market is, what those vendors are like. Are there options? Are there good tradeoffs?
And as Trevor was mentioning, with the healthcare.gov website, here, for example, you can see where some tradeoffs were made in terms of decision making, and others were not. So, some decisions were made to try to get a lot of commercial products, but some of the decisions associated with who was actually going to put this entire system together were perhaps given a little less thought or were thought that they might be best accomplished within the government itself. And that raises the question of: what are you buying from the market? Are you buying just products, or at times, are you also buying something else like the ability to do something that the government itself lacks the expertise or capability or the capacity to be able to execute?
Michael Keegan: Well, before I get into the actual process – because I’d like you gentlemen to walk us through the federal acquisition life cycle – I did have a quick question for you. Since you brought up the healthcare.gov, we’re going to talk about a little later, and that is the Deepwater Program. And that’s basically what your latest book, Complex Contracting out of Cambridge University Press – mentions and uses Deepwater as something you can get – lessons learned. Would you focus on the healthcare.gov acquisition as a way to learn about what you shouldn’t do or what you should do?
Trevor Brown: Absolutely focusing on what you should not do. I think it is clearly an example of a failed purchase. There are a whole host of things that could have been done differently to make that acquisition successful. And many of them are the same lessons we draw from. And we’ll get into all of those. You’ll see a lot of similarities between what’s going on with healthcare.gov and the kinds of products we talk about in the Coast Guard’s acquisitions.
Michael Keegan: Well, wonderful. That’s good to know. When we chat about that, you can draw those parallels for us or give us a sense of what it is. So what is the federal acquisition process? How does it work? Can you tell us a little bit more about it?
Trevor Brown: So, an expert would tell you there are hundreds of steps in this process. I’ll break it down simply into three phases.
The first is the pre-award phase. That’s everything that happens before you buy the product. So, that’s determining whether you want to make the product internally or you want to go out to market. If you decide you’re going to go out to market, that’s surveying the market to see what’s available. That’s meeting with the ultimate consumers of that product within the agency to see what is it they need, which ultimately ends with you, the purchaser, defining what it is you want to the degree that you can. We want it to be a certain size, shape and do all sorts of different things. Federal contract officials call that the “requirements definition portion” of the pre-award phase.
The second – once you’ve decided what you want to buy – is the aware phase. And you can think of that as the literal transaction. That’s putting the RFP out, the request for proposal that describes the product and the way it will be purchased. And then it’s meeting with potential vendors, recruiting people. It’s almost a sales pitch to the potential vendors to come forward with proposals. Then it’s selecting who the winner is going to be.
Then the third phase and final phase is the post-award phase. It’s everything that happens now that we’ve purchased the product. For some kinds of products, the post-award phase is literally just the delivery of, here’s your box of paperclips. For other kinds of products, things like information technology systems that take a long time to produce, the post-award phase, once we’ve selected the vendor, means managing a relationship with the vendor. It means working with that organization to make sure they’re following the steps that they said they would in the contract. It means monitoring their performance. It means making determinations about whether to compensate them with a performance bonus or not in some cases. And then ultimately, it means testing and evaluating the product you get, to see that it meets the terms that you specified in the contract and then potentially renegotiating that contract.
Michael Keegan: David, did you want to add anything?
David Van Slyke: Yeah. I was just going to say, as Trevor was mentioning, one part that we so often miss here is how critical the management part is once the contract has been awarded. So, for a lot of individuals, doing those first two steps is critically important. And when we think about the Federal Acquisition Regulation, there’s a lot of guidance right up until the time you make the award. But then post-award is where things become a little less clear and there’s much more discretion. And that’s where government managers, themselves, have a wider continuum in terms of how they engage the awardee, how often they engage them, how they begin to set and negotiate the rules of the relationship itself, the rules of the exchange and how they plan to work together. And I think that is a really critical and too-often missed perspective on the acquisition process.
Trevor Brown: Just to follow up with the good points Dave just made – so of these three phases, the one that perhaps gets the most attention is that middle one – the award phase. So, we hear a lot about the RFP, the bid, the decision about which vendor is selected and how that contract is structured. We tend to hear less about the pre-award phase, all the stuff that goes on beforehand and the post-award phase. And both of those are the phases where management is the most critical.
And as I say in this report – and I’m not the first to say it – buying is managing. So, the whole procurement process is a management activity. So, we often think of that middle phase and we think, oh, well, that’s the responsibility of lawyers. They write the contract. That’s right. They write the FAR; they write the contract – or economists who help us think about how to structure a marketplace. But when you look at the process from start to finish, it’s managing a series of relationships and potential relationships. And those are our management competencies and requirements.
Michael Keegan: And I want to get a sense – we understand you guys have done a wonderful job of explaining the strategic importance of contracting to mission attainment for government. You’ve given us a sense of the phases at a high level. But when you talk about selection or the award, what are the criteria used to really make that judgment? And if you can tell us a little bit more about this criteria, how does it signal oversight, too. What are the three areas that we’re talking about, and why are they important?
Trevor Brown: So, there’s a document that perhaps we’ll talk about in a minute called the Federal Acquisition Regulations that, basically, set the rules on what’s permissible in contracting. And one of the things that the FAR does is specify what the goals of federal acquisition are. And buried in there are two sort of approaches to setting the criteria by which we would judge an acquisition nor evaluate it. One is what’s called “best value.” And best value is a broader interpretation of what we’re getting out of the exchange.
And it typically involves three criteria, the sort of holy trinity of contracting. That’s cost, performance and schedule or delivery. Cost is – how much does it cost? Did it come in at the price we expected? Performance – does it do the things we want it to do? And then schedule is did it come in on time? In a best value acquisition, a procurement official is allowed to balance each proposal along those three criteria and make tradeoffs of, oh, well, this one’s more expensive. Its cost is higher. But the quality that we see in the proposal is higher. So, one approach is to balance those three criteria.
The other approach that’s specified in the FAR is called the LPTA approach which is lowest price technically acceptable. So, all those three criteria are still in play: cost, quality and schedule, but here, the argument is, if we can precisely define the product, we can say, as specifically as possible, here are the performance criteria. It’s technically acceptable. Well, then, we’re going to focus on price. So, we’re going to minimize our selection to: does it cost the lowest amount to produce it? So there, it’s a narrower set of criteria that define why we select one bid over another.
And depending on what we’re purchasing, it may make more sense to use one or the other. If we’re buying copy paper, we use the LPTA approach. That’s the guidance of the FAR. Just focus on cost. You can specify the size of the paper, its durability, how many pieces, et cetera. Whereas, if we’re buying information technology where we’re not so clear what it is that we want and we’ve got these competing criteria at play, then you are to pursue the best value approach.
Michael Keegan: Dave, did you want to add anything to that?
David Van Slyke: One of the things that Trevor mentions in terms of setting out the two approaches – best value versus lowest price technically acceptable – is that embedded there are also a set of values and with those values, a set of priorities. And so in some cases, with best value, what you’re emphasizing is perhaps a longer-term relationship or something that has a greater degree of complexity to it. You may be looking at issues that have been proven in an evidence-based way to be more effective relative to some other options. And from a cost/value perspective, it may be worth the extra cost to get a bit more on the value side.
But with the lowest price technically acceptable, there’s been some controversy around this particular selection method because what you’re really prioritizing is cost efficiency. And people then sometimes leave off the TA – the technically acceptable and focus more on the lowest price. And so this is where – even congressionally – you can begin to see certain political priorities bubble up when people say, well, you know we only want commercial products, or we’re only going to use government-furnished equipment and we want to employ a lowest price technically acceptable. You begin to see some of these values playing out.
Michael Keegan: Trevor, yes.
Trevor Brown: One other important point in light of your earlier questions about why procurement is important for the strategic management of organizations, one of the challenges in procurement is that both of these approaches become highly technical at some point. Procurement officials – particularly because they fear Congressional oversight if things go
wrong – are very concerned about following the rules and making sure that they don’t wind up in a situation where a product is over budget, delayed or doesn’t work.
Often, what’s missing in that is, does the product ultimately fulfill the mission requirements of the agency that’s purchasing it? The entry point into that and those three criteria is typically the quality of the product. So, does it help us do the things that we want to do? That’s very hard to write down in a contract sometimes. And so, where management and integrating the procurement function into the core of the management becomes so critical is making sure that when procurement officials are specifying those things, they aren’t just thinking about following the rules; they’re thinking about, well, how do these three criteria help us fulfill our mission requirements? How do they help the acquisition of this product?
I don’t want to just make sure I’m following the rules. I want to make sure that I’m following the rules and I’m delivering something at the time that it’s needed, that helps the other people in the organization fulfill the agency’s functions and does so in a way that’s resource efficient so we can buy more of it or use our resources in ways to enhance the overall performance of the organization.
David Van Slyke: And that can be more challenging whenever the government is trying to purchase, perhaps for the first time, a product that doesn’t currently exist or is trying to engage the market around the production of a service, for example, that they have not bought in the past.
So, as Trevor makes good mention of, it’s important to establish the rules. And it’s important to think about the relationship and some of those features in terms of the tradeoffs between quality and schedule and performance. Some of that becomes much harder to specify and write down where there may be some uncertainty in terms of the requirements both for the government as well as for the potential vendors themselves who are competing to potentially that product and service.
Michael Keegan: Well, Trevor, you mentioned the FAR, the Federal Acquisition Requirements. Do you have anything more you want to say about it, particularly – like what are the key components?
Trevor Brown: So, the FAR is a phone book. It is a big, thick document. And it’s not easily accessible to a layman. It is designed to be written in layman’s terms so that it’s accessible to the average person, but like any regulatory document, over time, it grows; it becomes cumbersome. And I’ve interviewed any number of federal procurement executives, and it’s fascinating the kind of divergent views on: is this a good regulatory regime, or is this a cumbersome one?
One school of thought about the FAR from those who use it to guide their acquisition is that it is Byzantine, overly procedural. It’s not clear how to get the information one needs about how to guide procurement.
The other view, from perhaps the more creative and crafty procurement officials, is it’s more of a recipe book. It gives you – you can find what you need in there. And these rules are more suggestive than prescriptive. And that divergence of view, I think actually, on the one hand, is healthy to have different views about things, but one would hope there would be some conciseness and clarity to those who use it and say, oh, no, it gives me clear guidance about what I need to do.
David Van Slyke: One of the challenges with the FAR, obviously, is every time the government purchases something new or wants to enter in the market for a new product or something to that extent, it begins to add more rules. And so part of the reason this becomes viewed as Byzantine and binding is that some individuals look at it much less analytically, much more like a checklist. I’ve got to check these things; I’ve got to make sure these I’s are dotted, these T’s are crossed.
And part of the criticism is that the FAR becomes an excuse for people perhaps to think less analytically about some of the tradeoffs, some of the options, alternative ways of doing things. As Trevor makes note, there are a number of very effective federal acquisition leaders who are thinking creatively about how to use the FAR and take some guidance and not view it as necessarily binding.
But there are a lot of individuals who also look at it as authoritative, the last rule, and in agency cultures that may be fairly risk averse, and for individual public managers who may fear making the wrong decision in some level of public scrutiny – either Congressional oversight or third party or the media – they hue closer to, “did I dot this I?”, “did I cross this t?”, and using the FAR as a checklist. And I think that’s where – when you’re purchasing fairly simple products, that may be fine. But when you’re getting into something far more complex and when you think that the federal government purchases on average about $550 billion a year, a lot of those purchases are, in fact, more complex.
Michael Keegan: Trevor.
Trevor Brown: Now, here’s one thing that’s very interesting about the FAR though that’s important to note, and that is there’s no one single FAR. Each agency has the authority to supplement the FAR with its own FAR, its own acquisition regulations. So, for example, the Department of Defense has the DFAR – the Defense Federal Acquisition Regulations. The Department of Homeland Security has its supplements to the FAR. So, this is a positive in the sense that different purchasers, different agencies, can tailor the document to the needs of the kinds of goods and services that they buy.
So, in that way, the FAR gives flexibility and discretion to different agencies to shape the purchasing process to meet their needs. So, that’s a positive. The resulting negative side of that, though, is that excludes the providers, the suppliers, the vendors who may want to offer the same kind of product across different agencies but now have to tailor not so much the product but the way they interact with these agencies to sell the product different, in a different way for each agency.
And there are many vendors who have said, I would love to sell my latest information technology system, program management service, office supply, whatever it is to the federal government, but I don’t want to deal with all this incredible variation and regulatory rules and approaches across these different agencies. So, I’m just not going to enter the market, which then diminishes the overall supply and choice available to federal agencies and has sometimes negative effects in terms of the quality of the products, the cost, et cetera.
Michael Keegan: Throughout the conversation and your insights, I’ve noticed there are some challenges to this whole process of acquisition within the federal space. Could you highlight maybe the top three challenges that are being faced and maybe some strategies on how folks could overcome them?
Trevor Brown: Well, the first is, as I just mentioned, the variability in the FAR, the regulatory process across federal agencies. On the one hand, this creates flexibility for the agencies, but it does have some diminishment of competition effects in the marketplace. It just makes it harder to sell to the federal government. So, there are sort of two things that federal managers can do as they work with the procurement personnel in their agency. One is to take a review of the FAR regulations in their agency and see first off, are there any opportunities to streamline this and make it easier for both the people on our end that engage in the procurement function and then the vendors who sell to us and then to look, how do our regulations stack up to other agencies. Is there any way we can streamline these and make them more similar to other agencies so we expand the marketplace. So, that’s one big challenge.
A second one that we haven’t touched on yet is – and it’s huge, and it’ll span everything we talk about here – is the state of the procurement work force. I think there’s this belief that when you go to the market and you buy stuff, you don’t need people anymore. But the reality is, particularly given both the scale of purchasing that goes on at the federal government and the kinds of things that the federal government buys, you need – you first off need a lot of people to handle those processes, and you need an incredibly well trained, sophisticated group that understands how they perform their technical function but do so within the overall management structure of the organization.
And most of the federal government is just not set up yet to effectively harness the upside of contracting because of the poor state of the work force. And that’s not a criticism of the people that either choose or wind up in those positions. It’s more a criticism of the system itself which is woefully underfunded – the contract management profession. And many agencies have not taken the steps to professionalize it within the agency as a pathway for career advancement, as a pathway for career development, et cetera.
Oh, a third one is that contracting is risky, and depending on the kinds of goods and services, the products that you’re buying, some of them are riskier than others. And we talked about the three criteria: cost, quality and schedule. The risk is that you end up purchasing a product that costs you way more than you thought it would, doesn’t work and comes in way late. And understanding, from a management perspective, what are the sources of those risks is critical for, again, a manager to understand how they can use procurement to achieve mission success.
In the report, I focus on elements of the design of the contract as a source of risk. A big debate in the federal procurement world is about the relative benefits of what are called “fixed price contracts” which set the price at the point of purchase and therefore, put the risk of any cost overrun on the provider, so the vendor has to make sure that if you said it was going to cost $100 and it ended up costing them $110, they’re eating that extra $10 versus “cost reimbursement contracts” which – there’s an agreement up front about what the cost elements will be, but the ultimate price of the product is a function of how much of it gets consumed, how much of it gets purchased and how much it ends up costing to make it. And so in that situation, the purchaser bears the risk of a cost overrun. And so in 2009, the Obama Administration, through the Office of Management and Budget, encouraged all agencies to review their use of cost reimbursement contracts and look for opportunities to turn some of those into fixed price arrangements because the idea was that would transfer risk to the vendors rather than to the purchasing government.
Michael Keegan: David, did you have anything to add?
David Van Slyke: One of the things that you can see, Michael, is that agencies are, in fact, able to use the FAR and some of the flexibility inside the FAR to undertake transformational efforts within their own acquisition processes. And so Undersecretary Kendall at the Department of Defense has been working on this with an initiative that he and others have been advancing on the Better Buying Power Initiative, and here, as where Trevor is suggesting, every agency can make some amendments itself. But at the same time, while that may be transformational and can help improve cost, schedule and performance, at the same time, it becomes ever more challenging in terms of the lack of standardization or the continuity of standardization.
And if you think about how challenging this can be even at the agency level – something like DOD is very large, and you have all the major services. So, as Trevor is saying, there have been lots of efforts within the federal government to try to get this better, I think too often, critics paint the acquisition process as unreformed and lacking leadership and vision in terms of how to make it more effective as government becomes ever more dependent on the market and buying products and services but at the same time how to make that more accessible to the supplier community, how to take their feedback. And so I think like so many things – and this is where a lot of the IBM reports are so valuable – it unpacks that the picture is a bit more complex and nuanced than how we often see this portrayed in the kind of broader, more generalist media.
Michael Keegan: What lessons can be learned from the Coast Guard’s Deepwater Program? We will explore this question and so much more when our special edition of The Business of Government Hour, a Conversation with Authors returns.
Welcome back to a special edition of The Business of Government Hour, a conversation with authors exploring ideas for improving government effectiveness in complex contracting with professors Trevor Brown and David Van Slyke.
So, Trevor and David, you folks have mentioned in our last segment, complexity and complex products when you were describing the federal acquisition process and I would like to get a handle. Let’s talk more about the complex nature of some of the products that the federal government is procuring because it needs to procure these things given the complex nature of its mission and varying nature of this mission.
So, what type of products are we talking about? What are some of the basic options for procuring them?
Trevor Brown: Forgive me, but I’ll do what academics do and I’ll define things first. There is a distinction made in the procurement community between goods and services. Goods are the hard stuff in terms of literal hard things. You can touch them and feel them, whatever. Services are the services that people provide.
We think that’s the wrong distinction. The important distinction, we think, is between, as you mentioned, complex products and simple products and think of products as inclusive of goods and services.
Simple products are easy to describe and easy to make. Complex products are hard to describe and hard to make and by hard to describe, we mean that they are hard to literally write down in a contract what it is that I want to buy. It is very difficult for me to specify everything about that product that I – that I want.
Things that are hard to make are things that require what academics would call specialized investments, investments that are unique to the production of that product that can’t easily be transferred to some other product if people stop buying it. So, if I am making a widget and people stop buying that widget, how easy is it for me to transfer my production product investments to create some other product?
So, I’ll give you some examples. So, a product that is very hard to describe, so these are both going to be complex products that we often purchase a lot of in the public sector, are mental health services. So, there is disagreement, for example, about what it means to suffer from some mental affliction whether it is addiction or some other more clinically diagnosed mental disorder.
And so similarly, there is disagreement about what is the best way to treat that mental affliction. So, to write a contract to help somebody who needs to rehabilitate, we are going to buy services to help people who suffer from this, is very hard to specify this is what I am buying. I am buying this kind of mental health service.
On the difficult to make side, you can think of – we’ll stick within the healthcare world – flu vaccines. Flu vaccines are – the process of making a flu vaccine is one that requires a very heavy investment in a specialized labor force and in a production process that can’t easily be transferred to something else.
So, if the federal government says, we’re going to get out of the business of buying flu vaccines, the producers of those vaccines can’t go make other stuff that they can easily sell to someone else. They have invested a lot in the production of that flu vaccine and they need the federal government to buy it or else they’ve lost their investment.
Michael Keegan: David, did you want to add anything? No? So I would be remiss in not mentioning and congratulating you gentlemen on your latest book that is out by Cambridge University Press called – its title is Complex Contracting: Government Purchasing in the Wake of the US Coast Guard’s Deep Water Program, and also your coauthor, Matt Potoski.
So, I read this whole thing in a couple of days and I have to say it is probably one of the most readable and approachable books narratively on contracting and I really suggest folks grab a copy of it because it is chock full with really good information.
But I want to talk about why you picked the Deep Water Program, what lessons you – what lessons you drew off from that program in terms of understanding strategies to better procurement, but more importantly, what were some of the assets and services that were being procured by the Coast Guard under Deep Water?
David Van Slyke: Sure. So, the United States Coast Guard is the largest coast guard in the world. There are about thirty-nine major coast guards in the world and the United States Coast Guard in terms of its assets, its shore stations, its boats and ships, its helicopters and its planes and the thirty-seventh oldest fleet out of the thirty-nine major coast guards around the world.
And the Coast Guard has a very complicated set of missions. I mean, in times of war, it works interoperably with the United States Navy and the military branches. In times of domestic peace, it has a range of different missions that it pursues. You know, you think about migrant smuggling or drug smuggling. That is one. It has kind of a law enforcement perspective. If you think about things like the Exxon Valdez or Hurricane Katrina, it has these other kinds of environment stewardship, search and rescue functions, and a variety of different missions.
And to carry out those missions, it really needed a system of assets, a system that was modernized, that was integrated, that could work hand in hand and really could create the kind of value that a small coast guard, about forty thousand people uniformed and civilian, need to be able to fulfill the series of missions that they have to perform on a daily basis.
Trevor Brown: I’ll tell you why we chose it. In addition to all of the good points that Dave said, it was a sexy case. It was sexy because the Coast Guard does all of this cool stuff that Dave just said and it is fun to learn about all of the really fascinating things that the Coast Guard does to keep our coast safe and all of that, but also because the means by which the Coast Guard approached this purchase was thought to be innovative and new and in the wake of it, the popular perception was it didn’t go well. So, it was littered and – the press was littered with all sorts of instances where, as part of the acquisition, there were problems.
So, for academics like us, inquisitive minds, we wanted to know if the sexiness was really, was really true. Was there something that was really fascinating in there that reflected some set of problems that were underlying?
And so it was a great journey of inquisitiveness into something that on the face of it looked one way, like a failed procurement, but in reality was reflective of a bunch of people trying to do their job as well as they could, fulfilling their mission requirements, and adopting some innovative practices to try and make sure that that occurred and some of it worked.
Some of the things they did really, really well so there are some very positive lessons that come out of the Deep Water case. And then there were the things that made their way into the press, the things that appeared not to go well. And so for us, it was an opportunity to sort through the – the what is really true here, what really went wrong, what really went well. So, it presented a great opportunity to do an objective analysis of something that was clearly on the forefront of the acquisition community at the time of this writing.
Michael Keegan: And it’s safe to say that the two reports you’ve done for the IBM Center, sort of the genesis for this book, correct? I mean, if you think of it as a gestation period, if you will?
Trevor Brown: We owe IBM nothing but thanks. There is no way we would have written this book without the support of IBM. Well, no, but it’s true.
Michael Keegan: But I did – off mike we were talking about it and it was well received by the Coast Guard, correct? I mean –
David Van Slyke: That’s correct. I mean, the 2008 report which was the first look at the Deep Water case really built on doing a fairly broad investigation of not just the Coast Guard but also looking at the supplier and vendor community, looking at the oversight community, looking at the vendor community that was not an active participant in the Deep Water Program itself as a procurement, looking at a whole series of third parties.
And, you know, doing about one hundred and fifty different interviews with individuals, we were able to put together what we think was some objective presentation and one that was not as start and black and white as some people may have liked and how so much of the media portrayed this. And that was really helpful because when we did the 2008 report, the Coast Guard, the chief acquisition officer at the time was Admiral Gary Blore and he went on Admiral Thad Allen’s website and posted a blog and basically said this is a good report. These guys got it right. And IBM was kind enough to say, you know, would it be helpful to host a program for the acquisition community?
We participated in that, the GAO participated , Admiral Blore participated, and so I think it was true that with the Coast Guard, perhaps there was some initial skepticism about who are these guys, what are they doing, and what do they really want? Are we going to look better or are we going to look worse or what is going to be the outcome?
And I think to some extent, there was validation that, you know, we were not trying to be on the Geraldo Rivera show, that we were in fact trying to present, you know, an accurate picture of how complex this procurement was.
And as Trevor mentioned, you know, when you think about this historically during the Clinton administration, a lot of effort around the whole idea of the new public management and government engaging industry and business and thinking about the idea of the private sector as our partner, and then really transitioning and thinking about agency transformational efforts, both when, you know, the Coast Guard was in the Department of Transportation and Admiral Jim Loy was the commandant and was thinking strategically about, okay, how do we really upgrade and modernize this system because we need to?
And then thinking post 9/11 and the Coast Guard being reassigned to a brand new agency, the Department of Homeland Security which has worked hard over the last ten or twelve years to develop its own acquisition processes and practices both in concert with the far (ph) and some that are more specialized to the agency.
So, this case had a lot of complexity to it but as noted, there were some innovative things on contract design, some innovative things in terms of agency transformational efforts, some innovative things in terms of how to engage the supplier market, but some things that did not work out as well, and there are some reasons for that that we can talk about.
Michael Keegan: Yeah, and I want to get into those innovative concepts that we’re talking about and define your terms as you were saying, Trevor. So what is – before we get into some of the lessons that were learned from this procurement, what is this system of systems contracting strategy and what does it mean to be a lead systems integrator? What are the positives and potential negatives?
David Van Slyke: Sure. So, I think, you know, with the system of systems, so, are we okay talking about healthcare now or no? So, when –
Michael Keegan: Yeah, yeah.
David Van Slyke: So, when we think about a system of systems, one of the real criticisms right now with healthcare.gov is that, you know, you had more than fifty different contractors each working on their own component piece.
The system of systems is knitting all of those different systems together, all of those different components. And so, for the Coast Guard’s Deep Water Program, this was all of their mission activity that happens on shore, all of their mission activity that happens in the water with boats and ships, all of their mission activity that happens in the air with helicopters and airplanes.
And so, when you think about, you know, a distressed boater who has perhaps, you know, been – the boat has capsized and they are several thousand miles out in the ocean, how do you find them? Right? You don’t just happen upon them in your patrol boat or at a ship, you know, with binoculars. It is using all of these systems of technology and logistics and trying to integrate it and create what is called a common operating picture so that all of these assets can actually work together.
So, a system of systems is getting these systems to actually work together. The lead systems integrator is essentially think of, you know, a general contractor. They’re the ones that manage all of the prime contractors. They’re the ones that manage all of the subcontractors. They have technical expertise. They have contract management expertise and they really understand the procurement processes and they are the direct liaison to working with the client, in this case the Coast Guard.
And, you know, where this works well is being able to capitalize on their ability to know the market and to manage the market and to higher the right prime contractors and the right subcontractors, and especially when a product is complex, to work very hard with the client in trying to define the requirements, a lot of which are uncertain because some of these things do not currently exist and have to be built, and at other times helping the client to understand where some of the cost value tradeoffs are.
So, you know, you want to put a stronger engine in a helicopter. Is that because you want it to be able to lift more capacity, more weight out of the water when you rescue a distressed boater, or is it because you want it to be able to stay in the air a longer period of time? And so, this is thinking about, well, what does the market currently offer? Are there options out there that are affordable and which of these tradeoffs do we make? And so, the lead systems integrator plays that role for a traditional client like a government agency and really trying to provide that expertise.
Where it can go wrong and one of the places where it was less effective in the Coast Guard’s Deep Water Program is when the lead systems integrator is also a vendor, when they are also a contractor, when they also have something that they want to sell. And then you begin to get into questions about, you know, conflict of interest and are they really representing the client’s interests? When they think about managing the market, do they often initially default to the range of products that they themselves may be – or subsidiaries produced, or do they really help the client say even though we produce it and in this case we’d be happy to provide that product, there are, in fact, these other providers as well and one of them may be cheaper and do the job just as effectively even though our product or someone else’s product may be a bit more expensive.
So, there are real plusses and minuses. Unfortunately, as a result of the Coast Guard’s Deep Water Program, Congress took a one hundred and eighty degree turn and when Congress turned over in 2006, sort of took an approach of throw the baby out with the bath water; no longer will the government be using lead systems integrators for these kinds of systems of systems approaches. In fact, the government will become the integrator and as we’re seeing with healthcare.gov, government, even when it is very well intentioned, lacks the capability, the experience and expertise, the technology and the resources, and the actual capacity to play that integration role. And that, I think, has been one of the big lessons that has come out of the Deep Water and we’re seeing it play out right now in real-time.
Michael Keegan: Trevor, did you have anything to add?
Trevor Brown: Yeah. I’ll just give a very sort of simple conception of a system of systems. Just think of it as an integrated or interconnected product with different pieces and parts that are put together. So, when most of us go buy a car, we don’t buy tires, an engine, a radio, an air conditioning system, and they don’t all come in a box with some assembly required. We buy a car. That is a system of systems. Somebody has to put all of that stuff together for us and that brings us to the second acronym, the LSI, the lead systems integrator or the general contractor.
Some of us are very capable of putting cars together. I’m not one of those people so I need to buy somebody, General Motors, Ford, Honda, to serve as that lead systems integrator, the person that puts all of that stuff together for us.
Michael Keegan: So, you’ve already kind of alluded to it, so the Coast Guard shows the system of systems strategy and they also had a single lead system integrator. I would like to talk about something that is misleading in that conversation was around that this was one big contract and it actually wasn’t just one big contract. I’d like for you guys to tell us a little bit more about the three tiered pyramid that basically is reflective of the complexity of the Deep Water Program. Could you tell us a little bit about that?
Trevor Brown: So, it’s the government so we’re going to use lots of acronyms, right? IDIQ, we used LIS, we used SOS. So the three tiers are at the top level. There was a master contract called an IDIQ, an indefinite delivery, indefinite quantity contract, and the best way to think about that is it was a menu. It was a menu of all of the things that the Coast Guard and ICGS agreed could be purchased over a long-term period.
So at the outset, the Coast Guard and ICGS, Integrated Coast Guard Systems, entered into this IDIQ, basically agreed – the Coast Guard said we’re going to buy our products from you and they will be from this list. How much they are, what they specifically will be, we’ll determine that later. That takes us to the second layer of this contract.
That is where individual task orders were written. Think of them as individual contracts for the different items off of the menu. So we’re going to buy some boats. We’re going to buy some helicopters. We’re going to buy some information technology. We’re going to buy some program support and each one of those is a different task order.
So, the Coast Guard still enjoyed some discretion within their big contract about the different items that they were going to purchase. The third layer down was where they nailed down all of the details. We’ve decided we’re going to buy X number of boats. We’ve decided we’re going to buy X number of planes. What specifically are those things and what are they going to look like?
That got hammered out in what were called integrated product teams, another acronym, IPTs. And that was the venue in which the details about each contract, each task order, would be specified.
To be fair, it was clear at the outset that the Coast Guard and ICGS were embarking on a long-term relationship so the expectation was if all had gone according to plan, it would have been a twenty-five to thirty-year relationship because, now we’re going back to that top level, the initial IDIQ was for a five year period that could be renewed up to five total times, each of those in five year increments. So, there was some expectation on behalf of the parties to the exchange that it might last for twenty-five to thirty years. But the reality was, and this came to pass, that each side could exit that relationship at any one of those exit points. There were exit ramps built in.
So, it would be unfair to say that the contract that ICGS and the Coast Guard entered into was a thirty year contract. It was potentially a thirty year relationship but contractually it was not.
Michael Keegan: So how – I mean, in a sense, how is it a prototype of relationship contracting in a sense? You mentioned that in your research in your book and that there were two important attributes. I think you may have alluded to just one of them in terms of the contractual structure that really gave the impression that this could be a win/win.
David Van Slyke: I think you’re right, Michael. I mean, when you think about this, what you’re signaling to your potential partner is that if you’re happy and we’re happy, there is potentially five more or four more past the initial relationship and that this relationship could last potentially for twenty-five years and even beyond that. I mean, when you think about what the suppliers were producing, these are things that are going to need to be replaced and modernized and upgraded as we go along. So, this is potentially a very long relationship.
And the other place where you could really see this relational aspect is as Trevor described. It’s on the third layer of the contract, the integrated product teams, because this is where there is acknowledgement in the master contract that there is a lot of things we simply just don’t know yet. We have a general concept of what we want and you the supplier have told us here are the kinds of things that we can do. But as Trevor already noted, this was really like a menu of options and now it’s at the integrated product team that you really put acquisition people in a room with vendor representatives, with users of the products and services themselves, and they begin to talk and they really begin to say well, we need this, you know, helicopter or this boat to be able to do these things. We need to be – for it to be able to do these things in these kinds of conditions.
We need to be able to operate it in these ways and that’s where there is then a lot of discussion and information exchange and what I would call really playing out a lot of options, alternatives and scenarios, trying to price those against difference performance scenarios, and then trying to come up with something that best represents, you know, what the Coast Guard needs not just today but in the immediate future as well. So, not planning for thirty years out but at the same time not saying simply this is what our mission has been like in the past, really trying to think about, you know, how are we going to get the best performance for the most reasonable prize on a schedule that really then does match with the mission work that we have to do on a day to day basis.
Michael Keegan: Sure. Trevor?
Trevor Brown: Michael, would you allow me to sort of step back and sort of paint the kind of dilemma that the Coast Guard faced in acquiring a complex product? And this is a dilemma that any agency faces when they go to buy something complex and again, think of complex as hard to define and hard to make, and then I’ll explain how the contracting approach tried to address some of those challenges.
So, the challenge when you go to buy something that is complex, hard to describe and hard to make, is first off at the outset, you the purchaser have some notion of what you want to buy but you can’t articulate that precisely. So it’s hard for you to specify these are the evaluative criteria by which I will judge the product when I receive it.
That poses a challenge for the producer of that product because they don’t know exactly what it is that you want so they don’t know exactly how to make it and they don’t know how much it’s going to cost and that’s what everybody wants to figure out. How are we going to make this thing and how much is it going to cost?
So, you can’t know that at the very beginning. That is going to get figured out; sort of the research and development will occur as you are producing the product. So that is challenge one.
The second is because it is hard to make, requires specialized investments; both sides are going to invest in the production of the product in such a way that if either one decides to exit; they’re going to lose a lot. They have made these big investments that will be lost if they exit and that leads to a problem that is called lock-in. Both sides are now locked into a relationship that is very expensive for them to exit. That’s risky not just for the purchaser but also for the vendor.
So, the purchaser doesn’t quite know what they want, doesn’t quite know how much it’s going to cost them, and once they choose the vendor, they’re not going to be able to get out of that relationship.
Similarly, the vendor doesn’t know how much it’s going to cost them and also doesn’t know if – if we don’t like this relationship with our purchaser, who else are we going to sell this thing to? So how are we going to recoup the investments we’re making? So that is your big challenge.
What has to happen for a deal to come off is that you write a contract that specifies as much as you can about the product but then moves from specifying what it is we’re going to buy to how it is we’re going to interact as buyer and seller. How are we going to govern the relationship because we’re going to have to figure a lot of this stuff out as we go along?
And so the contract structure that the Coast Guard setup was designed to try and address that problem. That top level contract, the IDIQ portion, couldn’t specify at year one what I am going to want by the end of year thirty. I kind of have a notion of what I want, this integrated system of boats, planes, helicopters, information technology. It’s going to have to adapt because something might have merged in the middle of this purchase like a 9/11, like a Hurricane Katrina, that makes me want to pivot and change the assets to adapt the mix because I can’t predict the future.
And similarly then, the vendor doesn’t know how long am I going to have this production facility here producing this stuff and you purchasing it? So, we agree at the outset on what the range of stuff is but then we’re going to hammer out the detail as we decide what we purchase off of that menu. And that detail involves both deciding the literal assets, the products, and then how we’re going to make them and how much they’re going to cost.
The idea was, let’s build in some rules that setup a healthy relationship between the buyer and seller that promote cooperation and we can get into the specifics of that in a minute if you’re so interested. But – so, part of it was building a set of rules into the contract structure that encouraged both sides to cooperate.
The other thing was building a relationship that similarly encouraged cooperation and that meant we use the term repeated play, the sense that – that we’re going to keep playing with each other. As long as you play fairly, I’m going to keep playing, but if you start playing unfairly, well then I do have an exit – I have an exit ramp out of this contractually.
But we both hope that because the relationship is potentially long and the value to us is – is really determined in the long run, that because we keep interacting with each other, that we will be encouraged to play fairly, that we will choose to cooperate rather than to make choices that might be harmful to the other participant.
Michael Keegan: So, you basically described the concept of the prisoner’s dilemma right, is that what you were doing? Because you used that metaphor, if you will, in the book, and I want to get into the fact, before we get into how you guys describe the phases in which the relationship sort of melted down. Why did it fail to – to get that win/win dynamic in place and could you identify some of the problems that ultimately derailed it, the program?
Trevor Brown: Well, let’s start by just – I like to define things. It’s a flaw of mine perhaps. So, the win/win is a great colloquial term that we all understand but it’s important to specify because it sets up what the lose/lose might be. And the win/win is the sense that both parties, the buyer and the seller; leave the exchange better off than they were before.
Most people are familiar with the term economists use which is paired to optimality which is any kind of exchange or relationship between two parties is paired to optimal if at a minimum, both parties to the transaction are at least the same as they were when they entered.
Well, the win/win is a step above that. Both parties are made better off by the transaction as a result of entering into an exchange. A lose/lose or a win/lose is when one party is made worse off than they were before as a result of the transaction, thinking colloquially. They might spend a bunch of money on something and get a broken product in return so now I’ve used some of my money, I was at a certain level, I had all of this money, and now I don’t have the money anymore and I don’t have the thing that works. I don’t have anything that reflects the value of what I have purchased.
The prisoner’s dilemma is that when you buy a complex product, the stuff I just walked you through just a minute ago, you wind up at this point where you can’t specify everything in a contract. It’s what we call incomplete. I can’t write everything down and so that means that within that contract, the behavioral choices I make, the decisions I make, are going to have a lot of influence on whether we wind up in a win/win where we’re both better off or a lose/lose.
And the nature of a complex product is I can’t easily tell whether you are doing things that are benefitting the product or the relationship until the very end, until I actually get the product. Because it is hard to define, it is difficult for me to know whether I’m getting the win/win and so I have to have some mechanisms in place to promote your cooperative behavior and I have to ultimately have some trust and faith that you will do things that are beneficial to the – to the relationship and ultimately the delivery of the product in order for us to continue and carry on.
So, you asked about the three phases.
Michael Keegan: Actually, that would be a good transition to that. You guys do a wonderful job of explaining sort of the meltdown if you will, or the separation. Could you tell us a little bit about those phases that you kind of highlight and then maybe draw some lessons from those phases, each of the phases?
Trevor Brown: So, the three phases are – the first stage was what we called the honeymoon phase. Both parties were really excited. ICGS, Coast Guard, this was this promising new potentially twenty-five year, thirty year relationship. The Coast Guard was going to get all of this value out of it and ICGS thought that they were going to get a stream of income and furnish their reputation with a new client.
So, at the outset, both parties entered into it in good faith thinking really good things were going to happen. Then, phase two, they get into the actual now we’ve agreed on what the menu looks like. Now, we’re going to start buying stuff off of the menu. Some of those things worked well. Some of those things didn’t work well and then it gets to the second phase, it’s foggy.
Dave can talk in a minute about some of those particular instances where things got foggy but it was unclear when things didn’t work well whether it was as a result of the purchaser. The Coast Guard maybe wasn’t clear. ICGS maybe made a decision to cut a corner because it wasn’t clear. It was foggy. They didn’t know which side was the one causing whatever the negative outcome was.
Over time, there were enough accumulations of those uncertain outcomes and who was ultimately responsible that led to a divorce. They ultimately in the third phase said there is just enough bad stuff on both sides that we decided we’re going to terminate this relationship.
I’ll give you a simple analogy and then let Dave dive into some of the specifics. Imagine you and I decide to exchange gifts this holiday. We say whether it is Christmas, Hanukkah, whatever, hey, we’re going to share gifts. We’re excited and maybe I go up to you and say, Michael, what do you want for Christmas? And you say I want something fancy that goes around my neck, like I’m really excited about something to show my style. Okay.
Then it comes time to give you the gift. Now we’re in phase two. I deliver you a box and there is a lovely scarf in there, but you were thinking I really wanted a tie and I thought that’s what I told you when I described this thing that went around my neck that was going to be – give me a little flair. I was thinking you were going to get me a tie.
Well, I give you this – this scarf and you’re feeling a little upset like, wait a minute. Didn’t you hear me? Didn’t you hear what I said I wanted? And I’m thinking, why are you so upset? I gave you what you thought you wanted which is some adornment around your neck. Ultimately, at the end of this, we’re upset because we don’t know. Were your intentions wrong? Were my intentions wrong? I’m mad because maybe you didn’t tell me as specifically as I thought and then next year we agreed not to give each other gifts.
That’s sort of the process that went on here. They went in with good intentions. They both wanted to exchange gifts and then they didn’t know didn’t we get the things we wanted. Let’s not give each other gifts anymore.
Michael Keegan: And, Trevor, that simple sort of anecdote that you just shared or crafted, it just goes to the core that this is a management process and a leadership process and if you just communicate it a little better and a little bit more often, maybe things would have been a little different.
But, David, do you want to dig deep into the –
Trevor Brown: Tell some of the good examples.
David Van Slyke: The thing about the Deep Water story, for that example itself, so many of the Coast Guard officials thought because they were using an LSI they didn’t have some of those same management and leadership responsibilities, that they didn’t have the responsibility, the coordinating function, the ongoing communication function, the information exchange function, or even some of the ongoing monitoring functions. And so it’s easy to misunderstand one another.
So, for example just, you know, at the IPT level, there would be a lot of information exchange but it turns out that decision authority was never sort of really specified. So, you know, you and I have this really great meeting. We do all of this great brainstorming. I think you’re capturing the notes. You think I’m capturing the notes and then we both leave the party – we both leave the meeting and we don’t know, well, who is going to take care of it, over what period of time, what is going to move the ball forward?
And you know, when we think about the suppliers involved in this relationship in this contract, time is money, right? I mean, they are deploying their people and their resources and their specialized investment processes, and when they’re deploying it for the Deep Water purpose, that means they cannot use those same resources for some other productive purpose. And so when they leave the meeting and they think that, you know, perhaps the Coast Guard official is now going to move this forward and they’re waiting for some formal decision action and it doesn’t come, then they begin to wonder, well I thought we agreed to that. Why haven’t we seen a decision action?
And sometimes, you know, it can be a little bit like both of us smiling and looking at each other very positively but we don’t know what the smiles mean, right? So, in this particular case, you know, the supplier in this case ICGS or Lockheed or Northrop in their individual production roles, may have in fact thought a decision was forthcoming in five business days and that’s when they say, well, we need a decision made quickly.
But for the Coast Guard, quickly may mean thirty business days and that may be quick by their standard and so the Coast Guard might not understand very well the processes or the preferences or how their suppliers see this in the same way that suppliers say, well, look. We said we needed this quickly. What don’t you understand? We need this within five days, not thirty days. And this is where it can lead to some other kinds of miscommunication.
So, one thing that the Coast Guard used because they were – they had a more limited acquisition shop for the Deep Water Program is they used something called these undefinitized contract actions and what that meant was that every time, you know, the supplier came to them and said, you know, Coast Guard, we need a decision made on this action, the Coast Guard if it did not make a timely decision within a certain period of time, the vendor would just go ahead and make a decision. But that meant that because the Coast Guard had not objected to the decision, then the decision becomes definitized and it goes forward.
But, you know, for something so big in terms of scale and complexity, that decision is probably not just a once in time decision. It has a lot of multiplier effects, spillovers, and it can drive up costs pretty significantly.
And so when the Coast Guard got a bill for undefinitized contract actions, they thought it was going to be something small. Instead, it turned out to be north of $350 million. You know, you end up having pretty different expectations. So, as Trevor used the metaphor before of a menu, you say to me I want dessert. I say no problem. We have a lot of dessert. What kind do you want? You say, well, you know, whatever you have. I bring you out the most expensive thing or I just bring it out to you and then I, you know, I give you a bill at a later point and the bill is much more than you thought it was likely to be.
So, I think the biggest area where there was really some – some mixed understanding was around a critical asset which was called the patrol boat and it was – no, not the national security cutter. It was what’s called the P1-10 and it was a boat that was one hundred and ten feet long. It was like the workhorse of the Coast Guard but some other things were needed on it and it was going to be a long time into the contract before a brand new boat was developed to do all of the things that the 1-10 did.
So, rather than wait for the new boat, the Coast Guard said let’s simply modify the existing 1-10. Let’s make it thirteen feet longer, right, which is not uncommon to expand the hull of a ship or a boat in this particular case and that will do some of the new things that we need it to do.
So they went forward with that and upgraded, you know, several of these 1-10s to become 1-23s. The numbers are –
Trevor Brown: You’ve got to pause because for me, a laymen, this is just crazy stuff. Like, I don’t know how to build anything. But the notion is you’re taking a boat that is one hundred and ten feet long and you literally crack it in half and then you add thirteen feet of hull.
Now at the outset, I would think that is ridiculous. How do you do that? But we came to learn – this is one of the joys of doing this – that this is a fairly straightforward exercise. So at the outset, the expectation was no problem, we’ll be able to do it.
David Van Slyke: So, do you want me to break that out and just go back and break that out into a piece?
Michael Keegan: Oh no, I think – no. This is good.
Trevor Brown: Keep going.
Michael Keegan: Yeah.
Trevor Brown: But I just wanted to highlight how crazy I think that sounded at the outset and yet for the average maritime person who builds and designs boats, they do that with giant cruise liners all the time.
Michael Keegan: Right. So please, we can take care of that too.
David Van Slyke: So, when the Coast Guard decided, okay, we have this one hundred and ten foot boat, it does a lot of things. It does, you know, interdiction of migrants and drug smugglers, but we needed to have some other functions, right, they simply cut the boat. They added a thirteen foot section to the boat and they added some of the other new devices that they needed.
Well, they put the boats into operation and the hulls on these boats began to buckle. Long story short, they begin to underperform. A congressman from Mississippi deems this, “the boat that don’t float”. There is a 60 Minutes story. There is a whistle blower and all of a sudden, you know, what was a well-intentioned short-term fix, now everyone is pointing fingers at one another and this is where you really get into the whole accountability regime and what you – what you end up losing is the ability to start talking with one another because now everyone is looking at blame attribution.
And so, I think this is one of the areas in which the relationship had really begun to change because there is this powerful external oversight community. There is this powerful third party called the media. There is a lot of things happening and right away at the beginning of a relationship. You have the first major product, so to speak, roll off the production process except that it doesn’t. This isn’t a new product. This is something old that they had that was already really beyond its lifespan.
They do some modernization; they do some upgrades to it. It doesn’t really work like they intend it to and people then begin to think that this is the picture that is going to emerge for the entire project and rather than both parties being able to then speak with one voice, you know, probably this is not the approach we should have gone; we had a lot of mission enhancement we had to do post 9/11 and some other things; this probably wasn’t the smartest approach, you know, they end up pointing fingers at each other and as Trevor was saying before, there is this challenge of well, were you acting in my best interest, you know, because it looks to me like you weren’t.
And from the supplier community saying look, you’re the one who told us to do this. The client is always right. We did what you asked us to do. You used it in a way that is different than you suggested to us and now you’re blaming us for it and this is where you can begin to see the relationship deteriorate.
Michael Keegan: The third phase, right?
Trevor Brown: Can I dive into this one more because this is such a great microcosm of what the challenges and some of the things that went wrong in Deep Water. So, I may tread over some of the ground that Dave mentioned. You can extract whatever.
So, you’ve got a boat that was old. It was a twenty-five-year-old boat that, in a perfect world, would have been sent out to pasture. But the (Indiscernible) seizes the moment. It is post 9/11. They need the boat now.
So, ICGS in some ways does a favor by saying, “Yeah. We can adapt the contract to give you those boats now”. So that looks positive. That is that honeymoon phase. We’re going to adjust and give you that boat. We’ll do this thing where we crack it in half and we’ll – we’ll even add on all of those toys that Dave mentioned, all of the fancy information technology. We’ll put guns on it so it can do all of the new things that you want it to do.
That appears like a good faith gesture. Well, then as Dave says, when the boat finally arrives, it starts buckling and literally cracking while coasties, Coast Guard personnel, are using the boat. And here is where the stories get just too rich not to be told.
So, most people would say, you gave me a broken boat. It’s cracking. I’m going to send it back. Well, the Coast Guard folks are committed to their mission. They want to save sailors lost at sea so they are going to do whatever they can to make that thing work and what they did was they literally brought on buckets of cement that they mixed as they are racing to save someone lost at sea, pouring it into the cracks that are emerging to make this thing keep going.
So eventually, as we heard, this is revealed. Oversight community doesn’t like it. It leads to the, well, somebody is to blame here. This is where we get into the fog. What – who is ultimately responsible? And this is worth diving into because it is a great microcosm, again, of what is happening here.
Coast Guard says, “You all agreed to make us this boat that would function in the ways we wanted. We told you how we were going to use it and cracks emerged. Your fault”. ICGS says, “No, no, no, no, no. It’s your fault because you gave us an already broken boat, twenty-five years old, and said put some scotch tape on it and make it work. Well, if you’d given us good product to work with at first, we could have, but we did our best to give you what you wanted”.
This ultimately leads to one of – not the exclusive issue in the Deep Water Program but one that is very important to highlight is both sides when you went and actually – and this is what we did – look in, well what were they each saying to each other? They would often use exactly the same language and it was very specific.
So, there is a category of the quality of the sea called the sea state, how high the waves and at what ferocity they are going. Sea state five is a really tumultuous storm, a big tempest. The Coast Guard and ICGS agreed we will give you a boat. We will build a boat that will operate in sea state five written into the task order. This boat will meet the standard of being operational in sea state five.
ICGS which composed of Northrop Grumman and Lockheed Martin, two of the biggest defense contractors in the world, had been selling product to other branches of the military for years, in particular the department of the Navy. When the department of the Navy says we’re going to operate a boat in sea state five, that means if we’re entering into a conflict and there’s a big storm that is causing waves to ride up and down, we’re going to go around it. We’re going to go around it because we’re concerned about the longevity of our assets and we want to make sure that when we get to the conflict, they work and function.
So, as we’re going around that storm, the big wave may crash over us and so we want to make sure that the boat functions in that but we’re going to try and do our best to go around it. Coast Guard says there is a sailor lost at sea in the middle of that storm. We’re not going around it. We’re going right into it. We are going to drive that boat into the heart of the storm because we have got to get that person that is lost at sea.
So, both understood, “Oh, sea state five we agree is a certain category of wave height and force”, but both did not understand how the asset would be used. So, Coast Guard thought, “I’m going to ride this boat like a mule into this big storm”, whereas ICGS is sitting there thinking “Oh no, no, we’ve just got to make sure that it can operate around the storm to get beyond”, totally different ideas about literal, precise specification in the contract.
So, at the end, each side had the best of intentions but this misunderstanding led to the finger pointing enhanced by oversight and press scrutiny that led to that third phase of divorce. There were other assets that failed or didn’t meet the standards. There were some that worked very well but this case, more than any other, exemplified the challenges of acquiring a complex product and how you wind up in the situation, trying to figure out who is responsible for the thing not working. Was it because Coast Guard didn’t treat the asset with care or was it because ICGS sold them a bill of goods, sold them a product that wasn’t going to work?
Michael Keegan: I – I mean, we’ve really – I would like to talk about given your book and your research, what are some of the things that people, government executives, can learn to make these complex contracting decisions and efforts successful, sort of turn the page to the more positive highlights?
What did you gentlemen learn from your work that you offer in your book and outline in your book that would be very helpful to government executives who are facing these kinds of decisions?
David Van Slyke: I think one of the challenges often is people, you know, there is often this notion that on the government side, that we’re not supposed to talk to the supplier community or we’re not supposed to reveal too much and there is always this question about competition and conflict of interest and what is fair and how do you create a level playing field.
In fact, I think one of the things that we learned the most was how important it is to talk to other parties, right, how important it is to learn, how important it is to understand each other’s preferences and motivations and making sure that each other really is on the right page, and at the outset, that can look very costly and time consuming. You know, when you immediately want to get to the work and everything, you know, you’re always behind. But one of the challenges is if you really don’t understand each other, you really do then have these very different expectations.
The other thing is one of the things that we heard from the supplier community over and over and over again is the client is always right. The client is always right but is the client always right when the client doesn’t have good capacity, when the client doesn’t have experience?
Let’s face it; Northrop and Lockheed have more experience in the federal procurement process than most federal agencies have and they know the regulations and they know these processes and they know the appropriation and authorization process perhaps better than a lot of agency officials.
I think one of the real positive things is agencies for sure are taking the acquisition function more seriously. They’re not seeing it as just some backwater clerical operation. They’re seeing it as strategic management, responsibility. But even so, is the client always right? When do you engage the client? When do you really try to educate the client? When does the client say, you know, but here is how I’m thinking about it?
And again, I think this is one of these areas where you have to have more communication up front to make sure that both of your goals are aligned in terms of mission fulfillment and the role, the performance aspect that that acquisition serves for the agency itself.
Michael Keegan: Trevor, did you want to add anything?
Trevor Brown: I’ve got plenty. So, one of the things we put in the book at the very outset is there have historically been two approaches to acquiring complex products. One is a very rule-driven approach. We talked about the far at the very beginning. It’s follow the checklist and be more precise about writing those rules.
The other approach which is seen as sort of diametric opposite is don’t worry so much about the rules. In fact, we should relax the rules and what we should do instead is build trusting relationships, the whole relational contracting notion. Vendors are good. Government is good. Let’s just all get along. We can worry about the details later. So, it’s the rules versus relationships.
Some people looked at Deep Water and said here’s an example where we did too much of the relationship route. We trusted industry and now what we should do is go back the rules direction. Our view on these things that are very high level is both. You’ve got to have rules that promote cooperation. They can’t be overly specified because of the nature of the product you’re buying. You can’t write everything down at the outset but you can put in certain kinds of governance rules that promote both sides getting along in those gray areas of the contract.
Secondly, you need to structure a relationship that promotes cooperation; that encourages both sides to continue to work with each other. It doesn’t necessarily presume at the outset that the other side is trustworthy. Instead, it creates opportunities for that trust to be built up over time.
It’s the notion of repeated play in the sense that if I can educate you, the other side, about the long-term nature of this relationship and the value that will come at the end, that will hopefully remind you when you are faced with those choices where you might make a choice that might be harmful to me, the purchaser, you’re going to make a choice that you think preserves the relationship rather than delivers you a short-term benefit that may come at the expense of me.
So, what we do in the book is lay out a series of criteria for what makes for good rules that promote cooperation and what makes for good relationships that, again, enhance that cooperation.
The third part is creating the conditions under which both parties understand each other and understand those rules. So, I gave you the example of they both wrote sea state five but they interpreted it differently. It takes time to – to understand that the Coast Guard is different than the Navy. You can’t just walk in and presume, “Oh, I see flags on their shoulders or their chest. They’re going to be just like the Navy”. No, they’re different. They understand these things differently. You have to build a mutual understanding of what it means to be cooperative.
And on the flipside what it means to not be cooperative so that if I receive something that doesn’t meet what I thought I was going to get, I can figure out whether you were actually just trying to do a good thing and you made an error and gave me a product maybe that wasn’t quite what I wanted, or you were literally making a choice that you thought was going to benefit you and harm me and then I’ll be upset and we can pursue those exit opportunities or incentives or whatever.
So, the book guides the reader through this sort of general framework of crafting the right rules, setting up the right relationships, and building that mutual understanding that can only be born over a series of interactions over time
Now within that, there are lots of specifics. So, for example, one that connects to things we were talking at the very beginning, LSIs are not a bad thing, right? I mean, we heard stories in the wake of Deep Water and there were congressional efforts to literally banish the word LSI from – from the procurement regulations and guidelines. Governments can’t buy LSIs. That’s foolish.
You need a general contractor. You need integrators to put things together. A great example of this is healthcare.gov the website. One of the principle failures of this is the absence of an LSI. Not one of those fifty-five vendors was specified as the one who was going to have to put all of that stuff together. The presumption of all of the vendors was that’s the Department of Health and Human Services’ job. They’re the integrator.
But they don’t have the capacity to perform those integration functions. Maybe in the future, we’ll live in a world in which the federal government will build that capacity, the systems integrators, the program managers to be able to perform those functions. But in the absence of building that internal capacity, they are going to have to buy it. The Deep Water case is a great example of how you need an integrator to pull all of these things together. They’re not necessarily bad things.
And there are plenty of positive examples. We highlight the Nimitz, the US military’s successful acquisition of aircraft carriers. And here you have a very challenging market situation in which there is only one purchaser. The Department of the Navy is the only one that buys Nimitz-class aircraft carriers and there is only one provider. Historically, it was Newport News which has been bought by Northrop Grumman.
So, there’s not a lot of buyers and sellers in this marketplace. They are in a long-term relationship but it has been a very, very successful long-term relationship in the sense that the Department of Defense has acquired an asset that allows it to project its strength and fulfill its mission requirements over a hundred year period now, and the vendor has successfully been able to remain profitable and continue to produce a product for a single buyer and that is because there has been a lot of effort devoted to getting the rules right, setting up contractual vehicles that promote a cooperative relationship, entering into a relationship, building that relationship, that notion of if we do this right, we will have a long-term relationship. And then third, through that comes that mutual understanding that is essential for the two parties to know, “Oh, you’re doing me a favor” or “you gave me a necktie when I wanted a scarf”, right, that we can figure out what we each mean to make sure we each get value out of this relationship.
Michael Keegan: One of the things about your book that I took away from it is exactly what you just illustrated. You do lay out the right way to do this given your research, but you also give very clear success case studies too that are very spot on. I mean, the Nimitz is a great example of that and how government executives can learn from what didn’t work in the past.
But let’s talk about the future. What does the future hold? It is sort of a wide, open-ended question, but what does the future hold in federal contracting that you gentlemen see and what particularly – what about complex contracting in particular? What do you see, changes – any changes down the road? Any enhancements to the way the – the capacity issue? What do you think?
Trevor Brown: I’ll start and you jump in, Chief, how does that sound? So, one easy way in the future to reduce the risk of complex contracting that is an easy solution; stop buying the stuff. The easiest way to minimize the risk of acquiring complex products is to stop buying it. That’s not going to happen.
The US federal government is continuing to push the edge of the envelope on the acquisition of sophisticated information technology, advanced weapon systems, integrated healthcare systems. All of the mission requirements that face us, all of the wicked problems that are out there demand the acquisition of complex stuff. So, you – you can’t stop buying it. In fact, I think it’s only going to accelerate and it’s going to cascade down to lower levels of government.
We used to think of things like aircraft carriers and the Deep Water system as kind of boutiquey exotic products that only the Department of Defense buys. But the reality is every government buys information technology. The lifecycle on the average information technology system is eighteen months and that’s probably going to get faster. So, everybody is going to need to figure out how do we acquire this complex stuff with rapid future change. That is uncertainty about what we’re buying.
Interestingly, I gave a talk to a bunch of local purchasers once and I said – these are people at localities, counties, and I said, “What is the most exotic thing you bought?” Three or four of them said drones, unmanned aerial vehicles. This is something we think of, “Oh no, that’s just the Department of Defense prosecuting the war in Afghanistan”. No, everybody is buying this crazy stuff.
So, it’s not going away so the risks are not going to go away. I think the open question is how do you respond to those risks? One thing that I specified in the report that is a theme that runs throughout our book is that you’ve got to invest in the work force. So, the question before policy makers, particularly before Congress, is, are you going to put the resources forward to invest in the acquisition work force? Are you going to allow agencies to buy employees, to be able to staff their procurement function?
From the agency’s standpoint, the open question is, are they going to take the steps to integrate what used to be a back-office function into the central management core? Clearly, our recommendations are you’ve got to do that. To buy is to manage and so agencies should follow the pathway of the big purchasers like the Department of Defense, a stellar example of how you professionalize the procurement core that other agencies need to follow suit. So, that is a big one.
Michael Keegan: David, did you want to – nothing to add?
Trevor Brown: Well, I mean, I think some of the other things are at some point someone is going to have to revisit the far, that the regulatory regime needs to be at least examined. In the debate about whether we should keep government running, our encouragement is we should move that debate to how do we make the government run well?
And unfortunately, politically, that debate seems to be about whether the private sector should perform functions, i.e. we should minimize the size of government, versus whether the public sector should perform functions, i.e. we should increase the size of government.
That’s the wrong debate. Both of those sectors are involved in performing public activities. Contracting is an example of where the public and the private sector interact to generate value for all of us. So, rather than have a debate about which one is better, we should have a debate about how do we make them work together and that is what contracting really is about.
How do we create a system in which the government can acquire those services and goods, those products, so that they get things delivered at cost, on time, that work for them? How do we get stuff that works? And that means that I would hope that members of congress, policy makers, would start to hold hearings not on whether we should get rid of stuff but how do we make it work better?
So, on the healthcare.gov, I’ve been actually pleasantly surprised that some of the more recent testimony has begun to focus on what can the – what can the Department of Health and Human Services do to – to not just make the website work better, but how can they improve their acquisition function?
David Van Slyke: I think this is where, you know, some of the people who listen to your show like members of the professional services council, NCMA, NIGP, one of the things that comes out in all of their reports is more skills, not just the skills of compliance and following regulations, but skills like negotiation, relationship management skills, managing the market skills, getting out and actually talking to vendors.
And it’s not only just having some technical expertise, right? You know, this is what a certain type of IT system would have and it’s not just kind of contract acquisition skills. It’s a broader set of analytical skills, how to think about some of these tradeoffs ahead of time, and I think that’s an investment that is not going to be easy to come but is much necessary.
One of the things that I’ve heard people at DOD say is that they’ve worked long and hard with the Defense Acquisition University to move away from just far compliance skills, to really thinking about, you know, negotiation skills and how to manage the market and understanding how, you know, the breadth and depth of the market on some components and how thin the market is in other areas.
So I think, you know, this is where the acquisition work force has to be a priority and I think, you know, until, you know, we’ve seen kind of fits and starts in every administration and this administration has not made acquisition a priority. It has made the notion of insourcing a priority but then even there, there has been a lot of variability and, you know, if you’re going to make insourcing a priority, you also have to make resources for training and development and capacity building a priority and you just can’t say we’re bringing it all in house. You have to have the people in house to do it.
Michael Keegan: So either way the capacity issue is really a big issue?
David Van Slyke: It’s an enormous issue.
Michael Keegan: So, David Van Slyke from the Maxwell School at Syracuse University and Trevor Brown at the Glenn School at the Ohio State University, I want to thank you gentlemen for joining me today. This has been a lot of information. The book is excellent. I’m going to make a big plug for it later when we do our closes but thanks for coming in and thanks for being here.
Trevor Brown: Thanks. It was great. Always a pleasure.
David Van Slyke: Michael, thank you very much for having us.
Michael Keegan: This has been a special edition of The Business of Government Hour, a conversation with authors exploring ideas for improving government effectiveness, with professors Trevor Brown and David Van Slyke, coauthors of Complex Contracting: Government Purchasing in the Wake of the US Coast Guard’s Deep Water Program.
You can order or download a free copy of their IBM Center report, The Challenge of Contracting for Large Complex Projects at businessofgovernment.org. Be sure to join us next week for another informative, insightful, and in-depth conversation on improving government effectiveness.
For The Business of Government Hour, I’m Michael Keegan, and thanks for joining us.