If Milton Friedman Created Your SIEM Team
When you mix an economist with the Godfather, you get an offer you can’t understand. But when you mix the philosophy of a famous economist with your SIEM team, you can create a high-performing team that continuously improves the environment, plans accordingly, creates better use cases, and ultimately reduces the probability of your phone ringing on a Friday afternoon for a SIEM issue.
Milton Friedman was one of the 20th Century’s most influential economists. Without going into detail or starting a debate on economic policy, he argued that a single owner would take better care of something than multiple entities or an unclear entity. The single owner likely has a direct interest in the value of it and will maintain it better than an entity that doesn’t. And thus his famous quote:
“When everybody owns something, nobody owns it, and nobody has a direct interest in maintaining or improving its condition.”
– Milton Friedman
A SIEM is likely one of your more complicated security products to manage, and needs extensive customization over the other black-boxed security applications your vendors manage for you. Not only do you need to manage the content and use cases, you need to manage the data feeds, ensure data is parsing correctly, troubleshoot issues with the application, support SIEM end-users, and plan for growth. All this effort requires input from various teams within your organization. Given the multiple teams involved, it’s critical to establish accountability and know who is responsible for what part of the environment.
SIEM Environment Requirements
The first requirement of any SIEM solution is clear, single ownership; an entity that has a direct interest in improving and maintaining the overall SIEM environment, and is ultimately accountable for its entire operation. Without clear ownership, staff and end users will be discouraged from escalating issues. Teams will not have a dispute mechanism, and instead of resolving issues, they will point the finger at each other. Those issues will then be brushed under the rug, and will result in a major outage or security issue down the road for leadership to deal with. Work will not be distributed accordingly, and highly-skilled staff that are overworked will leave, taking valuable knowledge and training investments with them. Relationships between the teams will be strained, and ultimately entropy will overrun your environment, in which significant investment will need to be made in order to turn it around.
The second requirement of a SIEM solution is a healthy, teamwork-oriented environment. Given that many teams are involved in the implementation and operation of your organization’s SIEM, positive and open communication between the teams is required for issues to be raised, work to be assigned to the appropriate teams, and for knowledge to be shared. Healthy teams will raise pertinent issues to leadership and resolve them quicker than teams that don’t. Healthy teams share valuable knowledge and train each other. All of this contributes to a work environment that retains staff, and attracts new talent into the team.
The third requirement of a SIEM solution is a strong skillset. SIEM environments are complicated, and you’ll need many skills to manage it from architecture and design planning, parser development, rule logic development, to social skills required to obtain and maintain data from other teams. Before making investments in your SIEM skillset, the first two requirements should be met, or else you risk losing highly skilled staff that are hard to find and retain.
The fourth requirement of a SIEM solution is documented roles and responsibilities. Many mistake this as the first requirement, but a RACI, for example, will not be followed or enforced if the first three requirements are not met. If your staff don’t have the proper skillset, one or two employees may end up doing everyone else’s work, and leave when they burn out. If your teams have poor communication with each other, issues may end up going unresolved and unnoticed by leadership, leading to an outage down the road.
Where practical, entire SIEM teams should be under one VP or line of business. Having one VP accountable for the implementation and operation of your SIEM gives the VP incentive to ensure the solution isn’t rushed into production, and that it has adequate resources for operations. The single VP will have more of an incentive to ensure the health of the SIEM environment than another organization that makes one VP accountable for the implementation only, and another VP for the support of it. Such a situation can incentivize the implementer VP to get it in as soon and cheap as possible and leave the support VP with a mess. Given that SIEMs can take years to fully implement, this should be avoided at all costs. The single VP also acts as a single escalation point and can’t deflect the issue to another VP or line of business. When there are 2 VPs and the roles and responsibilities aren’t clear, disputes can arise or the issue can be ignored. Again, it’s ideal to have your entire SIEM environment under a single VP, but in organizations with a good working environment, different parts of it owned by different VPs or lines of business can work out well. There are also some roles and responsibilities, such as server and storage administration, that are common to be outside of your security organization.
RACI Matrix Overview
One of the industry’s most common roles and responsibilities document is a RACI Matrix, which stands for Responsible, Accountable, Consulted, and Informed. The goal of a RACI is to list all stakeholders involved in the solution and the required tasks, and then assign one of the following values to a stakeholder(s) for each task.
While a RACI is designed to document roles and responsibilities, it has another valuable benefit: quantifying work efforts. Once you see all the various tasks involved in your SIEM environment, you can see how much work effort the various stakeholders are assigned. For example, if Engineering is responsible for Parser Management, and they spend 20 hours per week maintaining the 40 custom parsers, they can justify the half of an FTE they’re requesting.
It’s easy for a SIEM RACI to span several hundred lines given the amount of tasks and teams involved, and I’d thus recommend to keep it as high level as possible. The objective should be to assign tasks to the teams, and then leave the teams responsible for figuring out how work is managed. This avoids the SIEM Owner having to resolve disputes within teams. The SIEM owner should have a single point of contact within each of the teams to work with directly.
A SIEM environment should have at minimum an overall RACI that defines the roles and responsibilities of all stakeholders. Additionally, each team may want to create an internal RACI that clarifies who within the team is doing what. This is optional, but highly recommended, as it can help employees understand their tasks, assist management in understanding the required tasks and work efforts, and most important establishes accountability. For example, if you have 100 correlation rules and leave it up to “the team” to manage it, you may find that the task of keeping the rules relevant is being ignored. When you break up the rules, the first 40 to be “owned” by Bill, the next 40 “owned” by Bob, and the final 20 to be “owned” by Joe, who also gets to own reporting, you may find rule updates happening more frequently. There is accountability and you can follow up with Bill, Bob, and Joe to check the status of the tasks. If there isn’t progress, you can further narrow down the issue, whether it’s a skillset gap or work overload, and then coach the employee accordingly.
Many argue that assigning work to an individual rather than a team introduces a skillset gap when that employee leaves. The advantages of assigning it to an individual are a better understanding of the task via specialization and repetition, better documentation of the task as a result of the understanding, and ultimately a position for the individual to improve the condition of the SIEM relating to the task, for example correlation rule updates. Having a group manage something that is not well understood leads to the team ignoring the task, something they can do when no one is accountable for it. A group that doesn’t understand the task cannot document it properly or improve its condition. There’s nothing more frustrating working on something you don’t understand.
An overall RACI is a requirement for any SIEM environment, but as all organizations are different, how a team manages tasks within itself should be at the discretion of leadership.
Sample SIEM RACI
We’ll walk through a sample SIEM RACI to give you an idea on what it may look like in your organization. The RACI will be divided into subsections below by Category and commented on individually. A link to the full RACI Matrix is available at the end of the article.
The Stakeholders in this sample RACI are the SIEM Owner, Engineering, the Content Team, and Incident Response, who all fall under the Security Operations team. The Server Support and Storage Support teams fall under a different line of business, Infrastructure Services.
The first Category is Governance, and you can clearly see how the SIEM Owner is both Accountable and Responsible for the overall SIEM solution, dealing with the vendor, and internal escalations from any stakeholders.
The second Category is Architecture and Design, in which the SIEM Owner is also Accountable and Responsible, but Consults the Engineering, Content, and Incident Response teams. The SIEM Owner needs to work with them to make sure their requirements are met, the search speeds are adequate, the required data sources are available, and that the SIEM solution adequately meets all these requirements, and if not, are built into future versions.
For the Logging Configuration category, the SIEM Owner needs to make sure not only are the required log sources logging to the SIEM, but that they are logging the correct data. Engineering needs to be Consulted to ensure correct parsing, and the Content and Incident Response teams need to make sure the data they need is available within the logs.
The SIEM Owner is also Accountable and Responsible for leading all new projects, and ensuring the SIEM solution is compliant with the organization’s compliance and governance standards. You can also see at this point the SIEM Owner isn’t a mere decision maker; he or she will be active in the management of your company’s SIEM.
The Engineers are Accountable and Responsible for the health and stability of the SIEM solution, and to ensure data feeds are integrated into the SIEM correctly. They do everything from application support to patching. The only two support-related tasks that they are not Accountable and Responsible for are Server and Storage Support, but will be Consulted when necessary.
The Content Team are the SIEM end users, and are strictly Accountable and Responsible for developing and maintaining rules and reports. They are also active in providing input for new use cases, but the Accountability and Responsibility for that task falls on the SIEM owner.
The Incident Response Team is Accountable and Responsible for responding to the alerts generated by the correlation rules, and reviewing reports. They are also Accountable and Responsible to provide tuning recommendations for the rules and reports based on their investigations and observations.
The Engineers tried to get the Content Team to manage user accounts, but they lost the battle and ended up getting the task.
As you can see, a RACI is a simple document that can clarify who is responsible for what part of the SIEM environment. It can also be used by leadership to quantify work efforts, assist in understanding the various tasks employees do, and identify areas that require improvement. Issues can be raised and be visible to leadership on Monday morning instead of Friday afternoon, or during a breach.
A RACI is not practical without three other major requirements: clear ownership, a teamwork-oriented environment, and a strong SIEM skillset. Clear ownership gives the owner an incentive to maintain and improve the SIEM, and prevents issues from being ignored or assuming they’re the responsibility of another entity. A high-performing team maintains and improves the environment, retains highly-skilled staff, and attracts new talent into the team. A strong SIEM skillset allows staff to execute the required tasks. All of this contributes to a better return on investment the SIEM will provide your organization, and ultimately a better security posture.
Link to a sample SIEM RACI Matrix: Sample_SIEM_RACI
Please like, share or comment if you found this article useful. Thank you!