In our last article in this series, we talked about some ideas for setting in place Continual Improvement of processes at your InfoSec company. One process that is often far from perfect at InfoSec companies (and IT companies in general) is scoping.
It’s important to understand that bad scoping, when it reoccurs consistently, is a process problem. It’s not a problem with your account managers, or your testers, and it’s not a problem on the client side. It is fixable. As with most problems in business management, improving this area comes down to having a consistent process.
Obviously it’s hard to argue with the need for good scoping procedures. But to drive this need home, let’s look at some of the many negatives resulting from underestimating the amount of work a job will take:
Overestimating the scope is less immediately harmful to your company but is still obviously bad. Overestimating scope can lead to inflated prices, which can lead to clients noticing those inflated prices and going elsewhere. It can also lead to your testers noticing that you are charging inflated pricing, which may hurt their impression of the company’s ethics.
Many InfoSec companies operate in a constant whirl of activity, working multiple projects back-to-back and simultaneously. You would think this would create an impetus for optimizing scoping procedures, but unfortunately, often the opposite is true: the company is so inundated with work that they have never had time to study their processes and implement new ones.
Now let’s look at a few specific ways scoping problems often arise on InfoSec projects.
Often, the client representative taxed with communicating their needs to your company is not knowledgeable about the problem. There are a few ways this can happen…
Often a non-technical manager or employee is told: “We need a security check; get it done.” Or the owner of a small business or startup knows he needs security testing, but doesn’t know any more than that. They may have no awareness of the technologies involved in their application or website, or of the different pentesting options available.
This situation leads to obvious problems in communicating the scope of a project. There must be a process in place to gain very specific info about the project, or else there will be blind spots that won’t become apparent until the project is started, by which time it’s too late.
Even technically proficient people may be ignorant of what’s involved in pentesting. Even many skilled developers are not familiar with how much work, and what kind of work, goes into a pentest.
This ignorance is not necessarily a shortcoming on their part; developers and hackers just have very different ways of looking at the world. A developer is prone to see their application as a functioning whole, made up of trusted tools and libraries they’ve assembled to get the job at hand done. They often don’t think of their application as consisting of many small, interlocking parts, whereas the hacker sees an application as an assembly of cobbled-together parts and thinks about how to find the weaknesses in the joints of those parts.
This differing mindset means that even the app’s developers are often not able to clearly communicate all the technologies and systems that will need to be probed by a pentest. And this leads to similar problems in scoping.
Sometimes the internal staff members doing the scoping aren’t technically knowledgeable, either. Sometimes it may be a non-technical account manager or salesperson who is the first contact with clients and who also does the scoping.
Having non-technical staff as the frontline with clients isn’t necessarily a problem. It only becomes a problem when there aren’t systems in place to acquire the necessary project information (which we’ll talk more about in a moment).
Another problem may be that the employee doing the scoping isn’t properly motivated to make sure the information is acquired. Perhaps after the completed scope specifications leaves their hands, they don’t have to think about it again and no one brings up problems to them later. This can make them a bit impervious to pressures to improve their process.
Sometimes it happens that a client has a project that won’t be completed for some time, but they need to pay for a security assessment now. (One explanation may be that they need to spend end-of-year budget money.)
This situation can obviously lead to problems, as the client tries to describe the technologies that will probably be in place, without knowing for sure what the application will look like months down the line. This isn’t necessarily a problem, either. The problem comes in when the scope and project specs are not revisited as more information becomes available.
For example, if there is no process in place for someone to update the project specs with info as it becomes available, it may happen that the start date arrives and the team members assigned will have no up-to-date information about the project. This can include login information and server credentials and the like. So maybe there were three days assigned for the pentest, but the team has to spend a day acquiring the necessary access information, so now the project ends up taking four days. Or, if it can’t be extended, the team doesn’t have enough time to cover all the steps in their testing methodology.
Scheduling and talent allocation are separate issues, but some of the problems from these areas bleed over into scoping a bit. Here are a couple of ways these come into play:
Now that we’ve looked at some of the major problems, what are the solutions? A lot will of course depend on your own business setup and what you already have in place. (Some of you will already be doing some of these things.) But here are some ideas for ways to improve the accuracy of your scoping process:
One way to ensure that the relevant info is gathered is to make a detailed pre-scoping questionnaire a required part of every process. This questionnaire would be ideally filled out by the client company before scoping is started.
This questionnaire would include detailed questions about the architecture (existing or planned), such as:
Advise your client contact to give the survey to the most relevant, knowledgeable person in their organization.
A pre-engagement questionnaire is what we call a survey that you give the client a little bit before the official project start date has arrived. As we talked about, often there is a problem with keeping the project file up to date with the state of the client’s app or the required specs (such as login credentials).
Making such a questionnaire a part of your process will ensure that your team members have what they need when the start date arrives. This step also minimizes many of the negative effects of sub-par scoping; your team members will spot scoping problems before that threaten to derail the project.
A pre-engagement questionnaire might include questions like the following:
It’s important to do “post-mortems” on your projects, including the scoping of projects. After every project is complete (or possibly less frequently if that is too difficult), get together with the project principals and ask questions like:
When you conduct a project analysis, it’s important to be honest with each other and not to assign blame. It should be understand that the goal is improving the process, and that mistakes lie with the process, not with the team members.
Making sure project information gets where it needs to go (before scoping and after) may mean that you need to add new responsibilities to your team members’ roles. Whoever is in charge of talking to clients and scoping projects should be clear on their responsibilities and the information-acquisition process (which may include making sure questionnaires are completed, for one).
If your staff is currently kept completely busy as it is, and it doesn’t seem possible to add more to their workload, you might consider adding a new position. It could even be a part-time position. But if no one is currently keeping their eye on such details, you’ll continue to have problems with information not being present when it’s needed.
As we’ve talked about in previous articles in this series, long-term improvements come down to making changes to the process. If you aren’t making the changes a trackable and necessary part of the process, they will easily be left by the wayside and lost.
One way to make the ideas in this article part of your process is to use a workflow software (like Trello) to ensure that your team members actually go through the steps on every project. In Trello (and other similar applications), a project is moved from one step to another, which ensures that steps won’t be missed. You would put dedicated places in the workflow for “Pending Pre-Scoping Questionnaire” and “Pending Pre-Engagement Survey”. The process would not continue unless someone actively showed that those steps were complete.
Hopefully we’ve given you some new ideas on ways you might optimize your scoping process and make it more efficient. Let us know if you found the information helpful or if you have some unique things you’ve done to improve your scoping accuracy.
In the next few articles in this series, we’ll be discussing some other areas of project management, including internal knowledge transmission and ways to improve project and report standardization.
Your email is kept private. We don't do the spam thing.