(Note: This article is part of a series about differentiating your InfoSec company from competitors and improving your perceived value.)

In our last article in this series, we talked about some ideas for setting in place Continual Improvement of processes at your InfoSec company. One process that is often far from perfect at InfoSec companies (and IT companies in general) is scoping.

It’s important to understand that bad scoping, when it reoccurs consistently, is a process problem. It’s not a problem with your account managers, or your testers, and it’s not a problem on the client side. It is fixable. As with most problems in business management, improving this area comes down to having a consistent process.

The Downsides of Bad Scoping

Obviously it’s hard to argue with the need for good scoping procedures. But to drive this need home, let’s look at some of the many negatives resulting from underestimating the amount of work a job will take:

• Wasted manhours and lost profit
• Rushed projects, which leads to oversights, which leads to client dissatisfaction
• Conflicts with other scheduled projects for projects that go over schedule
• Morale problems due to rushed or mismanaged projects

Overestimating the scope is less immediately harmful to your company but is still obviously bad. Overestimating scope can lead to inflated prices, which can lead to clients noticing those inflated prices and going elsewhere. It can also lead to your testers noticing that you are charging inflated pricing, which may hurt their impression of the company’s ethics.

Many InfoSec companies operate in a constant whirl of activity, working multiple projects back-to-back and simultaneously. You would think this would create an impetus for optimizing scoping procedures, but unfortunately, often the opposite is true: the company is so inundated with work that they have never had time to study their processes and implement new ones.

Scoping Problems

Now let’s look at a few specific ways scoping problems often arise on InfoSec projects.

Clients May Not Know What They Need

Often, the client representative taxed with communicating their needs to your company is not knowledgeable about the problem. There are a few ways this can happen…

Often a non-technical manager or employee is told: “We need a security check; get it done.” Or the owner of a small business or startup knows he needs security testing, but doesn’t know any more than that. They may have no awareness of the technologies involved in their application or website, or of the different pentesting options available.

This situation leads to obvious problems in communicating the scope of a project. There must be a process in place to gain very specific info about the project, or else there will be blind spots that won’t become apparent until the project is started, by which time it’s too late.

Even technically proficient people may be ignorant of what’s involved in pentesting. Even many skilled developers are not familiar with how much work, and what kind of work, goes into a pentest.

This ignorance is not necessarily a shortcoming on their part; developers and hackers just have very different ways of looking at the world. A developer is prone to see their application as a functioning whole, made up of trusted tools and libraries they’ve assembled to get the job at hand done. They often don’t think of their application as consisting of many small, interlocking parts, whereas the hacker sees an application as an assembly of cobbled-together parts and thinks about how to find the weaknesses in the joints of those parts.

This differing mindset means that even the app’s developers are often not able to clearly communicate all the technologies and systems that will need to be probed by a pentest. And this leads to similar problems in scoping.

Team Members Not Knowledgeable or Properly Motivated

Sometimes the internal staff members doing the scoping aren’t technically knowledgeable, either. Sometimes it may be a non-technical account manager or salesperson who is the first contact with clients and who also does the scoping.

Having non-technical staff as the frontline with clients isn’t necessarily a problem. It only becomes a problem when there aren’t systems in place to acquire the necessary project information (which we’ll talk more about in a moment).

Another problem may be that the employee doing the scoping isn’t properly motivated to make sure the information is acquired. Perhaps after the completed scope specifications leaves their hands, they don’t have to think about it again and no one brings up problems to them later. This can make them a bit impervious to pressures to improve their process.

Project Information Not Updated

Sometimes it happens that a client has a project that won’t be completed for some time, but they need to pay for a security assessment now. (One explanation may be that they need to spend end-of-year budget money.)

This situation can obviously lead to problems, as the client tries to describe the technologies that will probably be in place, without knowing for sure what the application will look like months down the line. This isn’t necessarily a problem, either. The problem comes in when the scope and project specs are not revisited as more information becomes available.

For example, if there is no process in place for someone to update the project specs with info as it becomes available, it may happen that the start date arrives and the team members assigned will have no up-to-date information about the project. This can include login information and server credentials and the like. So maybe there were three days assigned for the pentest, but the team has to spend a day acquiring the necessary access information, so now the project ends up taking four days. Or, if it can’t be extended, the team doesn’t have enough time to cover all the steps in their testing methodology.

Scheduling and Talent Allocation Problems

Scheduling and talent allocation are separate issues, but some of the problems from these areas bleed over into scoping a bit. Here are a couple of ways these come into play:

• If a company doesn’t have a good system of scoping and conducting reviews of projects, scheduling will often be off, which can amplify workflow problems. For example, if scoping is consistently off, and scheduling is much too tight, there will be conflicts between projects and missed deadlines.
• If the person in charge of scoping doesn’t have a good understanding of the skills of team members available, the projects won’t be accurately scoped and costs won’t be accurate. For example, an account manager estimates three days for a pentest, but doesn’t know the exact skills of his techies or doesn’t factor in research/getting-up-to-speed time, so the actual time needed ends up being significantly longer.

Scoping Improvements

Now that we’ve looked at some of the major problems, what are the solutions? A lot will of course depend on your own business setup and what you already have in place. (Some of you will already be doing some of these things.) But here are some ideas for ways to improve the accuracy of your scoping process:

Pre-Scoping Questionnaire

One way to ensure that the relevant info is gathered is to make a detailed pre-scoping questionnaire a required part of every process. This questionnaire would be ideally filled out by the client company before scoping is started.

This questionnaire would include detailed questions about the architecture (existing or planned), such as:

• Give a description of your application/website’s architecture.
• What libraries and tools does your application use? (Perhaps an export of the environmental dependencies?)
• Where and how is the application hosted?
• How far along is the application and in what shape will it be by the time work is done on it?

Advise your client contact to give the survey to the most relevant, knowledgeable person in their organization.

Pre-Engagement Questionnaire

A pre-engagement questionnaire is what we call a survey that you give the client a little bit before the official project start date has arrived. As we talked about, often there is a problem with keeping the project file up to date with the state of the client’s app or the required specs (such as login credentials).

Making such a questionnaire a part of your process will ensure that your team members have what they need when the start date arrives. This step also minimizes many of the negative effects of sub-par scoping; your team members will spot scoping problems before that threaten to derail the project.

A pre-engagement questionnaire might include questions like the following:

• Where is the application hosted?
• What accounts can be used by the test team?
• What are permissible testing hours (e.g. can a scanner be left running overnight)?
• What is the final range of IP addresses in scope?
• Who is the main point of contact for technical issues?
• Who is the escalation/business point of contact?
• Who needs to receive start- and end-of-day email notifications about testing activities?

Scoping Reviews

It’s important to do “post-mortems” on your projects, including the scoping of projects. After every project is complete (or possibly less frequently if that is too difficult), get together with the project principals and ask questions like:

• Was the scope accurate?
• Did we have time to do what we needed to do?
• If the scope was inaccurate, why was it inaccurate?
• What can we do in future to prevent that happening again?
• Are the problems with this scope similar to other problems we’ve had in the past? Why is that?
• Just as importantly: if the scope was very accurate, why was that?

When you conduct a project analysis, it’s important to be honest with each other and not to assign blame. It should be understand that the goal is improving the process, and that mistakes lie with the process, not with the team members.

Assigning New Responsibilities

Making sure project information gets where it needs to go (before scoping and after) may mean that you need to add new responsibilities to your team members’ roles. Whoever is in charge of talking to clients and scoping projects should be clear on their responsibilities and the information-acquisition process (which may include making sure questionnaires are completed, for one).

If your staff is currently kept completely busy as it is, and it doesn’t seem possible to add more to their workload, you might consider adding a new position. It could even be a part-time position. But if no one is currently keeping their eye on such details, you’ll continue to have problems with information not being present when it’s needed.

Tracking The Process

As we’ve talked about in previous articles in this series, long-term improvements come down to making changes to the process. If you aren’t making the changes a trackable and necessary part of the process, they will easily be left by the wayside and lost.

One way to make the ideas in this article part of your process is to use a workflow software (like Trello) to ensure that your team members actually go through the steps on every project. In Trello (and other similar applications), a project is moved from one step to another, which ensures that steps won’t be missed. You would put dedicated places in the workflow for “Pending Pre-Scoping Questionnaire” and “Pending Pre-Engagement Survey”. The process would not continue unless someone actively showed that those steps were complete.

Next…

Hopefully we’ve given you some new ideas on ways you might optimize your scoping process and make it more efficient. Let us know if you found the information helpful or if you have some unique things you’ve done to improve your scoping accuracy.

In the next few articles in this series, we’ll be discussing some other areas of project management, including internal knowledge transmission and ways to improve project and report standardization.

Was This Article Helpful?

Security Roots’ founder Daniel Martin conceived and created the open-source collaboration tool Dradis Framework in 2007. The success of that application led to the creation of the Security Roots company and Dradis Professional Edition software.

Over the years, Security Roots has helped hundreds of InfoSec clients improve their team collaboration and report creation processes. If you have any questions about what we do or the solutions we provide, please fill out our Contact Form and we’ll be in touch soon.

If you’ve found this article helpful, please reach out and let us know how the information has worked for you. And keep an eye out for the future articles in this series.

(Note: This article is part of a series about differentiating your InfoSec company from competitors and improving your perceived value.)

In our last article, we talked about some ways to get some “quick wins” at your InfoSec company through practical steps you could immediately begin to affect some process improvement. But, as you know, making long-term change at an InfoSec company (or any company) requires dedication and patience.

Continual Improvement is a philosophy aimed at continually evaluating and improving a business process by using customer feedback on the product or service. By continually improving the interactions that make clients happy and by continually eliminating those things that aren’t important (waste), a company continually approaches perfection.

In this article, we’ll look at a couple of major ways to implement continual improvement in your InfoSec company, such as:

• Using the deliverable (the report) as a driver for process improvement
• Giving your team proper motivation and incentive to change

Deliverable Quality As Driver For Process Improvement

Most InfoSec companies are already entirely focused (often overly so) on the deliverable. At these companies, the report is the only thing that matters, and once it’s delivered, the conversation with the client is pretty much over. So making changes to what’s required to be in the report can be a great way to drive other process changes.

Ideally, as we’ve talked about in past articles (and often on our blog), a report will be much more than just a simple collection of vulnerabilities. To be the best it can be, and to set your company apart from the competition, a report should:

Give practical, actionable information on results. In other words, how significant or dangerous are the findings?

Contain an easy-to-understand executive summary. As your most important audience is often non-technical employees, the more you can communicate the situation to them, the more valuable your reports will be.

Showcase your methodology and processes. If you have great processes in place, you want to showcase them in the report. A report composed primarily of findings misses an opportunity to communicate how those results were created and why they can be trusted.

Showcase technical talent and allocation. Your company should have a way to ensure that the best people work on the problem, and this should be showcased in the report.

By creating requirements that contain these elements (effectively and accurately!) in every single report, you are also, simultaneously, creating process change. When reports are only required to contain the findings, it’s easy for your team members (managers and techies) to overlook the process, and the process is vital.

Some examples of what you can require to be in the report and how that can create broader, cultural change:

• The report must contain information about how team members were chosen. This forces you to put in place an effective process of selecting talent for projects.
• The report must prove the technical expertise of the team members who worked on the project. This will encourage you to create and reinforce methods of spreading knowledge efficiently throughout your organization. A more knowledgeable staff means that you have more people available to handle specific technologies, which makes scheduling jobs easier and improves the client experience.
• The report must contain information about your process and its consistency. This forces you to initiate processes that demonstrate said consistency (e.g., team collaboration tools, up-to-date and shared testing methodologies, standard issue descriptions and ratings).
• The report automatically is set up to contain all of the checks possible on a specific technology. This serves as a reminder to your team that those checks must be done, every time.
• The report is automatically set up to contain a section for soliciting client feedback. That feedback will always be collected and be used to improve your process.

These requirements for the report act as powerful feedback loops that help continually improve your process. These requirements help managers easily check that the desired steps were followed on every project. And once your team gets used to the new requirements, they will automatically start to think of ways to improve the process, if only to make life easier on themselves. Which brings us to…

Motivating and Incentivizing Your Team

True company change will seldom happen without cultural change. In other words, a business will seldom really change its ways unless there is buy-in from its employees. Employees must have proper motivations and incentives for acting in the desired way.

It’s not enough to tell your team, “The boss wants it this way and that’s just how it is.” And it’s also not effective management to say, “Do this or you’ll be punished.” Behavioral change must come from within team members and should be positively motivated, not negatively motivated.

Creating cultural change may be one of the biggest obstacle at InfoSec companies. Here are cultural challenges we face in this industry:

• Technical ability is highly valued, and there is often a tendency to “bow down” to highly-skilled workers and let them operate how they want to operate.
• Technical workers like to think about real, technical things, and there can be a lack of awareness (and sometimes outright disdain) for “softer” issues like customer experience and customer support.

So how might you tackle this problem? What are some ways you might communicate to your team why the changes you are implementing are valuable? Here are some ideas:

Show your team that the request for process change is coming from the client, not from management. The demand for change starts with the client. All changes you make should be derived from understanding what will improve your clients’ experiences. Ideally you will have already gone through some steps to get clear about what makes your clients happy (these were discussed in our last article). It’s easier to sell the need for change to your workers when you show them exactly how your clients are asking for change. It’s harder to sell the need for change when it’s phrased as something “we just have to do now”, without explanation. So share the relevant feedback and emails from clients that are driving the change.

Explain the importance of client happiness to the company’s health, their jobs, and their lives. Client happiness is not a wishy-washy, abstract concept. Client happiness can be the difference between your company’s success or failure. Success means more money to go around and more industry respect for your team members. The more you can make your team see how the process changes have real benefits to them, the easier the changes are to implement. One way to do this is to track and analyze some key performance indicators as changes are made over time (e.g., number of repeat contracts, client survey average scores, time spent on projects) so that your team can see the concrete ways your changes are helping.

A more efficient process makes their work lives easier. Your technical team wants to work on technical tasks; they don’t want to spend time working on boring administrative tasks or editing the wording of a report. One aspect of continual improvement is enhancing your process and making it more efficient. (One example: automated report creation software reduces the need to constantly write new descriptions for the same vulnerability classes every time.) When team members see that the process changes lead to less time spent on things they don’t want to do, and more time spent on the things they want to do, change is easier to sell.

Sharing technical knowledge efficiently helps everyone. Part of improving your processes is increasing your knowledge transmission; i.e., how technical knowledge is shared throughout your organization. (We will be talking more about knowledge transmission in a later article.) Effective knowledge transmission, of course, means better client service, but it also means that your team members learn a lot more than they otherwise would. Learning new tech skills makes workers more valuable and gives them more earning potential. (It then follows that a more educated workforce makes it easier to book and schedule jobs.)

Good performance is rewarded. When team members perform at or above your expectations, have systems in place to reward them. It can be a financial reward, or it can be non-financial (e.g., granting them access to new tech training or time off). One caveat is to not hurt morale by making the workers who weren’t rewarded feel punished.

Remember, The Process Is Usually The Problem

As you move forward with a continual improvement process, you should remember that the majority of company problems stem from processes, not employees. There can be a reflex tendency to blame individuals when procedures are not being followed and goals not being met.

But, by and large, these problems come down to not having good processes. Most employees want to do a good job and be rewarded for doing a good job. The problem for managers is mainly one of defining what constitutes a good job and making it easy for workers to jump through those hoops.

Another major aspect of Continual Improvement is to encourage your team members to report problems with the process, and to make it easy for them to do so. Your tech team contains the people most knowledgeable about how the current process impacts their ability to get things done. They are the best people to get input from about your processes. Ask them questions, give them surveys, and make it easy for them to give criticism (even anonymously).

Once you get feedback on a process and you see the feedback is valid, you should act on it quickly. This avoids procrastination and shows your team that you are serious about improvement and encourages them to come forward with their ideas.

Two great resources on process improvement that we recommend are The E-Myth Revisited and Work The System.

Next…

Hopefully this article has given you some ideas on how to start down the continual-improvement road. In the next few articles, we’ll be discussing some specifics of project management, including:

• Improving scoping and scheduling
• Knowledge transmission
• Project standardization

Was This Article Helpful?

Security Roots’ founder Daniel Martin conceived and created the open-source collaboration tool Dradis Framework in 2007. The success of that application led to the creation of the Security Roots company and Dradis Professional Edition software.

Over the years, Security Roots has helped hundreds of InfoSec clients improve their team collaboration and report creation processes. If you have any questions about what we do or the solutions we provide, please fill out our Contact Form and we’ll be in touch soon.

If you’ve found this article helpful, please reach out and let us know how the information has worked for you. And keep an eye out for the future articles in this series.