Effective collection management is integral to the success of an intelligence operation. What is it and how does it work? Thomas Hofmann, the Vice President of Intelligence at Flashpoint offers a crash-course in creating an effective collection management program.
Collection management is one of the most overlooked and misunderstood aspects of an intelligence operation. And yet, it is also one of the most important. Not only does collection management enable intelligence practitioners to identify and access the critical data and information on which their operations rely, it can provide invaluable insight into how each component of an intelligence operation is functioning and performing.
Here’s what you need to know:
Collection management comes into play during the planning and direction phase of the intelligence cycle. After you’ve established intelligence requirements (IRs), collections management will enable you to determine and access the resources needed to satisfy your IRs. This entails two distinct functions:
Collection requirements management (CRM)
Collection requirements management identifies, prioritizes, and documents the observables and inputs needed to fulfill your operation’s IRs. These observables and inputs are known as collection requirements (CRs) and should map to and reflect the priority of each IR.
For example, let’s say the objective of your intelligence operation is to reduce fraud losses and your highest-priority IR is to determine which tactics, techniques, and procedures (TTPs) fraudsters are using to commit account takeover fraud at your organization. This IR could map to CRs such as:
- Monitor Russian-language deep & dark web forums X, Y, and Z where fraudsters have been known to develop and discuss account takeover TTPs.
- Engage in information sharing with trusted anti-fraud teams and industry peers to gain insight into emerging TTPs and mitigation tactics for account takeover fraud.
- Monitor reputable open-web resources such as sites X, Y, and Z, for new research findings pertaining to emerging TTPs and mitigation tactics for account takeover fraud.
And because no intelligence operation has limitless resources, prioritization is essential. Determining which CRs are most critical to your highest-priority IRs can help you to allocate your resources effectively. The example CRs outlined above are ranked from estimated highest to lowest priority based on their corresponding IR.
Collection operations management (COM)
Collection operations management specifies how to satisfy CRs and operationalizes collection resources and activities. This function typically entails:
- Auditing existing resources to identify gaps that could impede operations with respect to CRs. Satisfying the first example CR specified above would require access to the right deep & dark web forums, as well as personnel with fluency in Russian, an intimate familiarity with the Russian-language underground, and expertise in account takeover fraud TTPs.
- If no such sources are accessible and/or no such personnel available, COM would likely need to re-evaluate the CR’s priority and, if necessary, seek out suitable third-party vendors to provide the source accessibility and subject matter expertise needed to satisfy the CR.
- Monitoring for source outages. While any online source can experience an outage, deep & dark web sources tend to be particularly volatile. Such sites have been known to crash while other underground marketplaces shut down, or forum URLs changed without warning. In these situations, COM needs to identify and provide access to alternative sources and help teams adjust course to satisfy their CRs accordingly.
- Identifying redundancy in source coverage, which tends to be more common when an intelligence operation works with third-party vendors to augment its collection sources. Different vendors may offer access to the same or similar sources, but regardless of whether redundancy occurs with in-house or third-party sources, COM needs to identify it to determine whether resources need to be reallocated.
But in many cases, redundancy can be beneficial—especially for sources that satisfy high-priority CRs deemed integral to the lifeblood of your organization. These CRs should be supported by more than one source when possible; if an outage occurs, having a suitable backup source can help keep your operation on track.
Working with the OPSEC team to ensure in-house collection personnel can safely access and gather observables from the sources specified by the CRs. OPSEC controls can include persona management, privacy and obfuscation measures, and awareness training for personnel who access deep & dark web sources, as well as service-level agreements and traffic light protocol (TLP) classifications for those who engage in external information sharing, for example.
Both the CRM and COM functions are typically overseen by a collection manager. While this role is often perceived as little more than administrative oversight, an effective collection manager is integral to the success of an intelligence operation. Not only does this person plan and integrate all collection processes, they are also your best resource to make an informed assessment of your effectiveness in both CRM and COM and can help to inform:
Health and Performance
An effective collections manager can provide fast and accurate insight into the health of an intelligence operation by evaluating each IR and CR with respect to collection resources and activities. This process typically aims to determine:
- The status and progress to-date of the operation’s IRs and corresponding CRs
- How well the progress of each CR maps to its IR
- How well collection resources and activities were able to fulfill each CR
- The extent to which resource gaps and/or redundancy were present and, if so, how they impacted the operation
Using the previously mentioned account-takeover fraud IR and corresponding CRs for reference, an operational health assessment might seek to answer questions such as:
- How many new TTPs have been identified since the operation began?
- How does this number compare to the operation’s and anti-fraud team’s expectations?
- What collection resources and activities were most valuable for identifying the specified TTPs? Least valuable?
- What additional collection resources and activities could have enabled the operation to identify more TTPs more efficiently and effectively?
- Answering these types of questions enables the collection manager to update stakeholders and decision-makers on not just the overall health of the intelligence operation, but also on the individual value and performance of each collection resource and activity.
Return on investment
The results of this health assessment can then help the collection manager address what has become an age-old and widely contested question among intelligence practitioners and the C-suite alike: What is the ROI of an intelligence operation? Pinpointing the exact ROI is complex, but an effective collection manager armed with insights gleaned from the operational health assessment should be able to estimate the value provided by individual collection resources and activities.
For example, let’s say an intelligence operation invests $50,000 in a third-party vendor to gain access to a dark web forum deemed critical to identifying the emerging account-takeover fraud TTPs needed to satisfy a high-priority IR. If data obtained from this forum does satisfy the IR, the collection manager can then estimate whether $50,000 is more or less than the reduction in fraud losses that resulted from satisfying the specified IR.
This approach can also help justify budgetary allocations and expansions. If an operational health assessment reveals a gap in a critical resource, the collection manager can estimate the potential impact of that gap and use it to justify the additional budget needed to procure said resource.
It’s important to remember that regardless of the size, objective, or maturity of your intelligence operation, collection management is a must-have. And while the above crash-course introduces some of the key components of an effective collection management program, it is neither comprehensive nor prescriptive. Collection resources and activities can and do vary substantially from operation to operation and organization to organization. Given the complexity and resource-intensive nature of initiating and developing a collection management program, less-experienced teams looking to do so are advised to seek the support of trusted peers and reputable third-party vendors accordingly.