Relying on open source components is now standard practice. But how do you truly understand the risks associated with these crucial dependencies? This is where Bitergia’s Risk Radar, and specifically its Total Risk Score, comes in.
What Is the Total Risk Score?
Bitergia Risk Radar approaches risk from an important perspective: that of project health risk. Rather than scanning for known vulnerabilities that already exist, Bitergia Risk Radar proactively assesses development activity within projects to identify weaker dependencies that pose risk. And it does this at scale.
Running your entire supply chain through a risk model could produce an overload of information that would suck up your time and attention.
But our risk model does not.
That’s because we simplify the information into a simple Total Risk Score for each dependency. A total risk score which directs your attention to exactly where it needs to be. And which is a starting point for taking action.
Our Baseline Risk Model: How Well Is the Project Maintained?
Each metric we use for the risk model analyzes the developer activity in maintaining the OSS dependency from a particular point of view. We think of the metrics as part of 3 main categories of maintenance issues to watch out for.
Here are those 3 categories and the metrics they encompass:
- The community cannot handle demand:
- Backlog Management Index (BMI)
- Review Efficiency Index (REI)
- The community lacks sufficient attention:
- Median Lead Time for Issues
- Median Lead Time for Pull Requests
- The balance of developers is not adequate:
- Retention Rate
- Growth of Active Contributors
- Bus Factor (Contributor Absence Factor)
What About Other Metrics, Like Financial Risk?
We can also work with you to apply other categories of metrics, depending on your particular concerns. For example, we could apply “Financial Risk” metrics to your model. This would assess risk based on the likelihood that a dependency will make costly relicensing decisions down the road. Here are some of the metrics we’d look at:
- The dependency poses financial risk from relicensing potential:
- Copyright Concentration Risk Factor: This measures how many companies own copyright. When one company owns all of the copyright of a project, the likelihood of relicensing increases.
- Ecosystem Maturity Level: This addresses incentives to relicense. For example, investors may push for relicense if they feel that open source is giving away something for free that they could monetize. Competitors may be making money off the software but are not contributing to it. The metric itself is an analysis of the economics of a project, including aspects like the business model, vendor space, and competition.
- Organizational Landscape: This looks at the user space, adoption rate, and governance of the project. When one organization has centralized governance, for example, it can create a mindset in that the project is theirs to control. We would look at the track record of the project to see how they have treated the project in the past to identify whether there are early indicators for a likely relicense move.
- SBOM Analysis: This metric delves into the dependencies of the dependencies. The project may depend on software licenses that are copyleft, which would make it harder to relicense.
- Project Maintainability: This metric goes beyond the copyright to look at the people and organizations. It is similar to the analysis we discussed in the other project-related metrics above, and it serves to better understand the stakeholders behind the copyright.
For whatever types of risk your organization is measuring, the risk model’s combines the metric results to create the simple number we call the Total Risk Score.
How Is the Total Risk Score Calculated?
Bitergia uses an approach to calculating the Total Risk Score that is pragmatic and transparent. It’s not rocket science–it’s just a thoughtful aggregation of individual risk metrics.
Based on Bitergia’s extensive research and expertise, each risk metric has specified thresholds to determine the risk level. For example:
- Bus Factor (number of people making up 50% of the code):
- More than 5 people = Low Risk
- Between 2 and 5 people = Medium Risk
- Less than 2 people = High or Very High Risk
- Lead Time for Pull Requests (median time to close a PR):
- Less than 14 days = Low Risk
- Between 14 and 30 days = Medium Risk
- More than 30 days = High Risk
We assign a numerical risk score to each metric (e.g., Low risk gets a lower score, High risk gets a higher score) and then calculate an average for metric risk score. The model assumes, at least for the initial calculation, that all metrics are equally important. However, we can customize the weights for each customer, depending on their risk appetite and profile.
We then combine individual risk scores for each dependency into a Total Risk Score from 0-10, or from low to very high risk.
What Does “No-Activity” Tell Us?
We want to acknowledge that the significance of “no-activity” can be challenging to determine. Let’s take a look at two scenarios. On the one hand, “no-activity” could mean that a project has been abandoned and will not receive any further updates. On the other hand, it could indicate that a project is feature-complete, but a maintainer is still available for when a bug or vulnerability is discovered. It is often impossible to know which of these scenarios is the case.
This presents a challenge because a great many open source projects lack activity. “No-activity” could be a useful signal or a distraction from other issues. With this dilemma in mind, we work with customers to establish the best approach for their particular concerns.
When a lack of a metric means “no-activity,” there are two common approaches for a project:
- Option 1: “No-activity” is treated as a high risk signal, and therefore the dependency is assigned very high risk. This has the downside of potential signal-overload.
- Option 2: “No-activity” is omitted from the model all together, and will therefore not influence the risk score.
When it comes to “no or low-activity,” however, there are some definitive signals to watch for. For example, it is worrying when a project that was once active but has now become inactive. This change suggests that the project should be explored, as something has changed since the project was initially assessed.
While there is no perfect solution yet, the Risk Radar addresses this issue by recognizing that “no-activity” needs to be considered.
Why Is the Total Risk Score Useful?
The Total Risk Score is easy to read, interpret, and communicate.
If we were to look at all metrics results of the risk model, it would be overwhelming to sift through for what matters. And this approach would certainly not scale to tens of thousands of dependencies.
The Total Risk Score, however, gives users a good overview fast. It helps direct attention to where it needs to be. You get a single, easily digestible categorization: Low, Medium, High, or Very High. This immediate assessment helps you quickly grasp the overall risk profile of a dependency. That way, you can prioritize your attention and effort. Then, users can drill down into the individual risk metrics to analyze how each social aspect of the developer community informs the riskiness.
A very high risk score, for example, may draw your attention to a dependency with very little maintainer activity–or no activity at all. This would be important to know and act on, since zero maintenance can be very risky in and of itself. It can leave a dependency open to vulnerabilities and attacks.
It’s all about reducing the cognitive load and making the data easy to act on.
How Can You View the Total Risk Score?
Bitergia Risk Radar offers two ways to consume the risk analysis. From a management and risk controller perspective, we offer a dashboard. The Risk Radar Dashboard allows users to see the aggregate risk evaluation for the entire set of dependencies. They can then filter down to focus on those dependencies with the highest Total Risk Score.
The second way to consume the risk score is for developers. Through an API, the risk scores can be presented during the development workflow as part of the CI/CD pipeline. Risk is most effectively managed by surfacing it early (by “shifting left”). The earlier risk is detected, the better able organizations are to get ahead of costly attacks and to ensure the sustainability of their software.
A third (bonus) option is to integrate the Total Risk Score and related data in other compliance tooling. The data API of Bitergia’s solution enables such integration.
You Have a High Total Risk Score, Now What?
Receiving a Total Risk Score is just the first step. The real value comes from understanding what actions to take based on that score:
- Very High Risk: This serves as a red flag. A “Very High” score often signifies a repository with very little to no activity. In our research, we were surprised by how many commonly used dependencies were actually very high risk due to “no-activity.” What to do about it?
- Contact maintainers: Reach out to the project’s maintainers. Is the project still active? Do they need help or other resources?
- Get a service contract: Hire the maintainer or someone else to maintain the library.
- Consider alternatives: Is there another, more actively maintained library that can fulfill the same function?
- Fork the project: If the component is critical to your operations and no alternatives exist, you may choose to fork the repository and maintain it yourself.
Again, a very high risk score that comes from “no-activity” can be hard to interpret. It could be a maintained and feature-complete dependency, but it could also be one that is abandoned and open to attacks. Based on your organization’s priorities, we can work with you to adjust the model.
- High Risk: A “High” score indicates significant issues, but the project is likely still active. Your next step is to drill down into the individual metrics that contributed to this score. With that information, you can direct your investments–time, talent, or treasure–to where they’re really needed. In addition to the options already presented above, here are some actions you may want to take:
- Identify specific problems: Is the Bus Factor too low? Is the backlog growing uncontrollably? Is the lead time for pull requests excessive?
- Resource and investment allocation: If a critical component has a high-risk metric (e.g., low Bus Factor), consider allocating resources to support its development to improve that specific metric. These are investment decisions that can stave off costly attacks down the road.
- Process review: For issues like a growing backlog, review the project’s housekeeping rules. Are old bugs being closed? Are issues being triaged effectively? These metrics can help you invest your effort where it’s actually needed.
- Prioritize based on criticality: Consider how critical the component is to your own operations. A high risk in a non-essential component might be tolerable, but a high risk in a core dependency demands immediate attention. These decisions to invest in what’s most important can help you to proactively save time and money in the long term.
- Medium Risk: This suggests areas for improvement, but not an immediate crisis. It may be best to monitor these components regularly and keep an eye on trends. Small issues, if left unaddressed, can escalate into higher risks over time. Many executives may ignore anything that is not Very High or High Risk to focus their resources and investment where they need to make the most impact.
- Low Risk: Congratulations! These are healthy components. Continue to monitor them periodically to ensure they remain well-maintained.
Final Thoughts
Bitergia Risk Radar already provides a powerful snapshot for proactive and cost-saving risk management. Companies have already leveraged these insights, discovering critical weaknesses in widely-used but sparsely-maintained components. Features like “wait and see” monitoring with trend arrows (showing if a score is improving or worsening over time) are on Bitergia’s wishlist. We’d love to hear what feature you’d like to use.
The Bitergia Risk Radar’s Total Risk Score is more than just a number; it’s a call to action. By understanding what it means and what to do with the results, organizations can make confident decisions, avoid costly disruptions, and invest in the long-term resilience of their software supply chain.
Visit the Bitergia website to request a demo of Bitergia Risk Radar!
