Make-or-buy decisions in research and research software

Make-or-buy decisions are common in business and research. They often involve an analysis of the pros and cons of creating something (e.g., a physical item, software, a service) vs. acquiring it from someone else. Factors that are considered can include cost, trust in suppliers, and current internal knowledge and skills as well as desired future knowledge and skills.

In some sense, such a decision comes down to defining goals and then determining how best to achieve them. If the item itself is a small part of the overall goals, it’s likely a buy decision will be best, but if the item or the ability to create it is the end goal, a make decision is more likely best.

Some examples I’ve been involved with regarding these decisions for research software follow.

When I worked at JPL, we were developing electromagnetics simulation software that needed to factor very large dense matrices. Because our team felt we were experts in electromagnetics and that this was our core knowledge and skill, we developed most of the simulation software ourselves. But of course, software is often built with a complex dependency stack, so we used existing elements where possible, such as compilers and I/O libraries. When it came to the matrix solver, we were not the experts on building such software, particularly so that it would run efficiently at extremely large scale, so we worked with another group who did this very well. In other words, we made a make decision for the software we thought of as core to our expertise and also core to what our sponsors wanted, and made a buy decision for the software where we didn’t need to feel ownership and didn’t think the details of this choice would harm our relationship with our sponsor, but just needed something that worked.

In a later project at JPL, I was on a team that was developing a prototype for a spaceborne cluster computer that we intended to test in space. We contracted with a number of university groups to build software libraries for a number of the functions of the computer system. However, our plan was not to fly this university-developed software, but to re-write it in-house based on the university ideas, algorithms, and demonstrations. In this case, we were making a make decision, because this software was so important to the project that we needed to ensure we fully understood and controlled it.

In a later project I was involved in while at LSU, I was on a team that was developing software that logically could be separated into functions, where another team had developed software that implemented one of the functions we needed. We initially decided to try using that functionality from the other team rather than (re)building it ourselves, because it again wasn’t our focus but just something we needed to use. However, we were unable to cleanly extract this software functionality from the other team’s overall software, as they hadn’t designed their work with this use case in mind, so we ended up reimplementing it as a library that we then used. In this case, we tried to make a buy decision but were forced into a make decision because the product we wanted to buy didn’t meet our needs.

This last example is a little tricky. If I imagine an alternate world where our team did want to make this functionality our focus, and where the other team had designed their software to be separable, I could also imagine us finding a reason that their software didn’t quite meet our needs and we had to rebuild it. I could imagine incentives that would make this the best choice for us, e.g., we wanted to build up our expertise in this area, we wanted recognition for this expertise, we wanted to compete with the other group in this area, or we felt we could get funding for this work while we couldn’t get funding for making minor changes to their software in collaboration with them. However, this wouldn’t have been good for the overall software ecosystem, in comparison with us either just using their software or collaborating with them on a new version of it.

At one time when I was thinking about this, I wanted to say that industry favors buy decisions and academia favors make decisions, or that production leads to buy decisions, and research leads to make decisions, but I don’t think this is so straightforward. It all seems to me now to be a matter of pros and cons, just not only the technical pros and cons that someone on the outside might see, but also those related to incentives and other social factors.

I would love to see a better study of this in the case of research software, looking at academia, national laboratories, and industry, perhaps starting with more detailed case studies and deriving general rules, not for what should happen but for what actually does happen. Any takers?

And this could lead to ideas about how to make changes that would be better for the overall software ecosystem, which could be things like funders more strictly reviewing proposals to ensure that they weren’t building new software when existing software could be used or modified, or creating new incentives for contributing to existing software.

This post has also been published as https://doi.org/10.59350/6q2e5-hhs87 on the Rouge Scholar.