Trust, But Validate: Building Confidence in Assessment Results
by Dave Stapleton
Security assessments come in all shapes and sizes. Some have tightly defined scopes and others encompass an entire enterprise. Others are based on a specific security standard versus a tailored set of controls. Some are simple self-assessments while others require independent validation. This last distinction is the focus of this blog post. Validation benefits the consumers of security assessments by allowing them to establish a level of confidence in the assessment results.
Independent validation of third-party risk assessments may be requested for a number of reasons. Validating may simply be a blanket policy requirement of the customer ordering the assessment. Also, a customer may have reason to apply particular scrutiny to a selection of their vendors. Whatever the reason, validation provides value that a self-assessment simply cannot.
Despite the value that independent validation provides, it is exceedingly difficult, costly, and time-consuming to implement. Effective assessment validation requires extensive knowledge across all cybersecurity domains. In addition to the skills requirements, the level of effort and number of hours necessary to conduct validation can be prohibitive for many organizations.
Related: 3 Steps to Streamline The Vendor Risk Assessment Response Process
The good news is that CyberGRX offers validation as an included service for our Tier 1 and Tier 2 Validated assessments. This, coupled with the cost savings offered by our Exchange model provides exceptional value to our customers. The following information is intended to describe the value that our validating process delivers.
Independent validation of CyberGRX risk assessments comes in several formats, as described below.
Automated validation is based on identifying contradictions in the way that a third party completes their assessment. A straightforward example of this would be if the third party indicated that they have a highly effective patching program in place, but have not implemented an asset inventory. The concern in this scenario is that the patching program is likely to be less effective if the third party has a limited understanding of the assets requiring security updates. CyberGRX includes automated validation with all Tier 2 assessments.
Remote and On-Site Validation
Remote and On-Site validation requires a third party to provide CyberGRX Analysts with evidence artifacts that support their assessment answers. This validation process proceeds as follows:
1. Selection of Controls
The first step in this process is selecting the controls that we would like to evaluate. This selection includes a set of critical controls that are always included in the validation process. Our critical controls are loosely based on the SANS Top 20, along with an evaluation of trends in cyber-related technologies and attack scenarios. We then select additional controls to ensure broad coverage of all assessment areas.
2. Evidence Request and Collection
Third parties are provided the selected list of controls and we ask that they prepare the evidence necessary to support these controls. This preparation may include things like gathering documentation, preparing demonstrations, or taking screenshots of security tools.
3. Evidence Submission
The submission of evidence can take place via documents shared in an online repository, web conferences, or an on-site visit. The validation methodology and objectives remain the same, regardless of the format the third party uses to provide evidence. One advantage of remote or on-site evidence submission is that third parties can demonstrate the existence and effectiveness of controls in a live setting. An example of this might be a third-party security engineer who executes a series of queries in a log aggregation tool to demonstrate the source, type, and content of logs being collected by the organization.
4. Evidence Evaluation
CyberGRX Analysts apply a significant amount of rigor in the evaluation of third-party evidence. We employ a number of standards in the evidence evaluation process. For example, self-attested evidence such as written notations or un-validated assessment reports are not adequate for validation in any circumstance. Policy documents may be used to validate assessment answers, but only if the associated control is specifically related to policies and governance. The vast majority of the evidence requested by CyberGRX is of a technical nature. The most useful evidence artifacts include things like screenshots of configuration settings, exports from SIEM tools, and reports from vulnerability scans.
Independent validation is not a requirement for all security assessments. However, when validation is necessary it is imperative that it is done correctly, to ensure true confidence in overall assessment outcomes. CyberGRX is dedicated to providing world-class independent validation at a fraction of the cost that is typically associated with this quality and value of service.