Help! My Instance is Down!

Running into an error screen or unable to access the web app? Run through the following checks first.

This guide contains:

1. God Process

The first thing we're going to check is whether the god process monitoring service is running.

Run the following command in the console:

$ god status

This command should return:

cic-puma: up
dradispro-puma: up
nginx: up
resque:
cic-resque-1: up
dradispro-resque-1: up

If the command fails, try restarting with:

$ sudo /etc/init.d/god restart
$ god status

If you have rebooted your Dradis instance, the god process may not have started up correctly. You can force it to start by running the following commands:

$ god load /etc/god.d/dradispro-puma.god
$ god load /etc/god.d/cic-puma.god
$ god restart

Command still not returning correctly? Check out the relevant log files by running:

$ vi /var/log/god.log

2. Web Applications

God process are working? Let's check whether the Web applications are running.

Puma is the application server so we're going check how many "puma master" processes are running with the following commands in the console:

$ ps aux | grep "[p]uma master"

We should see 2 processes: one corresponds to the CIC (the Combat Information Center, also known as the Dradis Administrator Console) and the other corresponds to Dradis itself.

The expected output looks like:

dradisp+   978  0.0  1.5  52348 30948 ?        Sl   06:08   0:00 puma master
--env production --config-file config/puma.rb --daemonize config.ru dradisp+ 1003 0.0 6.0 778160 122744 ? Sl 06:09 0:06 puma master
--env production --config-file config/puma.rb --daemonize config.ru

Troubleshooting the CIC:

Did the "puma master" check uncover an issue with the CIC process? Time to dig into the relevant log files!

  • /var/log/god/cic-puma.log
  • /opt/dradispro/cic/shared/log/puma.err.log
  • /opt/dradispro/cic/shared/log/puma.out.log

To access from the command line, just run this command with any of the filepaths above:

$ vi FILEPATH

Troubleshooting Dradis:

Did the "puma master" check uncover an issue with the Dradis process? Time to dive into the log files!

  • /var/log/god/dradispro-puma.log
  • /opt/dradispro/dradispro/shared/log/puma.err.log
  • /opt/dradispro/dradispro/shared/log/puma.out.log
  • /opt/dradispro/dradispro/shared/log/production.log

To access the log files from the command line, just run the following command with any of the complete filepaths above:

$ vi FILEPATH

Specific Errors:

This section includes specific errors that you might encounter and the possible solution for resolving the error.

404 Not Found - nginx

Most likely this issue is caused by a down puma service and the god process check is giving you a false positive. Try starting up the dradispro-puma server using the command:

$ god start dradispro-puma

If nothing has changed, check the puma error logs. It should tell you why the server is failing to start:

$ vi /opt/dradispro/dradispro/shared/log/puma.err.log

Missing/Broken assets

If you get an error about a missing web file (js/css/html) or image file, it might be that something went wrong with your instance that an asset is now missing. To fix this, try running the following commands to compile the assets and restart the server:

$ RAILS_ENV=production bundle exec rails assets:precompile
$ god restart dradispro-puma

Site cannot be reached!

If the site is unresponsive and unreachable, it might be that the nginx server is down. Run the following command to check if the nginx service is running:

$ ps ax | grep nginx

If the service is not there, try starting up the nginx service using the command:

$ sudo service nginx start

If that didn't work, there might be an issue with your custom nginx configuration. The relevant files can be found in /etc/nginx/sites-*. The following command should help test the configuration:

$ sudo service nginx configtest

Ruby error

You might get an Internal Server Error (500) and an error message along the lines of NameError: undefined local variable or NoMethodError: undefined method.

Unfortunately, this might be a bug in the code and the only way to resolve this is patch to the server. If you encounter this, note down the steps you took to reach this error and contact the Support team so we can try to replicate this and resolve this issue right away.


3. Connectivity

No smoking guns with the processes or in the log files? Let's check your connectivity to make sure the VM can reach our activation server.

Ping it

Our activation server is at https://portal.securityroots.com. To test the connectivity, run the following from the console as dradispro:

ping portal.securityroots.com

If the command succeeds (transmits and receives packets), you've got connectivity! Continue on to the next step.

If the command fails, it looks like we need to establish connectivity first.

Check the DNS configuration in `/etc/resolv.conf` and update the DNS servers. You can try adding this content:

nameserver 8.8.8.8
nameserver 8.8.4.4

Run the ping command again to confirm that you've established a connection with the activation server.

Note: You may need to prepend sudo to your editor (such as sudo vi, sudo vim, or sudo nano /etc/resolv.conf) in order to make these changes if you receive permission errors when trying to save your changes.


cURL it

Run the following command from the console as dradispro:

$ curl -vi https://portal.securityroots.com

This command should return the remove page.

If it does not, it may be that DNS is working but traffic is being stopped (by a firewall?). If you know who to contact internally to get this solved, please let them know that you need access to tcp/443 on portal.securityroots.com.

If there's an SSL certificate error, you'll need to add the proxy server's root certificate to your Dradis instance:

  1. Move the relevant certificate files to the /usr/local/share/ca-certificates/ directory.
  2. Run the following commands as dradispro:
    $ sudo update-ca-certificates
    $ god restart

Adapt it

If the ping command resolves to "host unknown", make sure that your Dradis image adapters are set up properly.

Your Dradis image should have two adapters, NAT and Host-only. They should already be set, but make sure to check them and make changes if needed. If they are set up correctly, go to step 3.

  1. To add a Host-only interface, open VirtualBox's Preferences > Network. You can use the default values. (remember to enable the DHCP Server)

  2. Once that is done, update the Dradis image's Network settings with the NAT adapter and another adapter for the Host-only network (choosing the one you made in Step 1).

  3. Next, you'd need to make sure the two interfaces (eth0 and eth1) are configured. You can do that by editing the file /etc/network/interfaces with your preferred text editor and make it look like this:

    # This file describes the network interfaces available on your system
    # and how to activate them. For more information, see interfaces(5).
    
    # The loopback network interface
    auto lo
    iface lo inet loopback
    
    # NAT
    auto eth0
    iface eth0 inet dhcp
    
    # Host only
    auto eth1
    iface eth1 inet dhcp
    
    # The primary network interface
    allow-hotplug eth0
    iface eth0 inet dhcp
  4. Check if the interfaces are up with the command

    $ ip a
  5. Check if any interfaces are DOWN and UP them with the command

    $ ip link set <interface_name> up

    Example response: $ ip link set eth1 up

  6. Restart the instance and test with the command

    $ ping portal.securityroots.com

Check proxy settings

A proxy can be configured by adding these lines to /opt/dradispro/dradispro/current/script/env.sh:

export http_proxy="http(s)://<proxy>:<port>"
export https_proxy="http(s)://<proxy>:<port>"
(or whatever is needed in your own network) and then restarting your app with:
$ god restart

When using a proxy, we have seen cases where curl connects properly, but Dradis Pro refuses to reach the activation server.

In that case, to test if Dradis Pro is using the proxy properly, try executing the test script already included in the app:

$ cd /opt/dradispro/dradispro/current
$ RAILS_ENV=production bundle exec rails runner /opt/dradispro/bin/dp-test-proxy.rb

The output of the last command must say it is using the proxy and must include Connection success.

If it doesn't, something about your proxy configuration may be incompatible with Dradis. Adjusting your proxy settings may resolve the error. If not, please contact our Support team to perform an offline activation of your Dradis instance.


Roll back to a previous version

Have you tried the troubleshooting steps but desperately need to get a report out the door? Don't worry, you can roll back your instance to the previous version and then try the upgrade again later when you have more time.

Each Dradis release creates a folder like /opt/dradispro/dradispro/releases/202306011436 that includes the date/time stamp when you performed the upgrade.

Assuming you upgraded today, the most recently-created folder should start with today's date. Assuming that you want to roll back to the previous version, find the second most recent datestamped folder in /opt/dradispro/dradispro/releases/. For example: 202305252203. You'll use that folder name in the commands below:

  1. Reach out to our support team to let us know you ran into a problem! If you've found errors in the logs, send those over. We might be able to avoid the error the next time you upgrade.

  2. Delete the existing symlink to the current release:

    Run the following in the console as dradispro:

    $ cd /opt/dradispro/dradispro/
    $ rm -i current

    Type yes when prompted to delete the current symlink.

  3. Point the /opt/dradispro/dradispro/current/ symlink to a different /opt/dradispro/dradispro/releases/ folder

    Run the following in the console as dradispro:

    $ ln -s /opt/dradispro/dradispro/releases/[TARGET] current

    Make sure to sub in the correct folder for [TARGET] above! Example: 202305252203.

  4. Restart your VM (turn it off an on again) to force Dradis to pick up the change.


Contact Us

Still running into issues after walking through the steps above? Please reach out to our Support team and we'll give you a hand.

Streamline InfoSec Project Delivery

Learn practical tips to reduce the overhead that drags down security assessment delivery with this 5-day course. These proven, innovative, and straightforward techniques will optimize all areas of your next engagement including:

  • Scoping
  • Scheduling
  • Project Planning
  • Delivery
  • Intra-team Collaboration
  • Reporting and much more...

Your email is kept private. We don't do the spam thing.