Disaster Preparedness

Your mobile/paperless EMS operation is dependent on its internet connections and on its internet-based resources, particularly the AngelTrack cloud server. Therefore you should lay groundwork for operational continuity in the event of total loss of all internet resources, as might occur when:

It could happen and you must be ready for it.


Disaster


Keep Paper Forms Ready and Waiting

At some point in your operation's history, you ran everything on paper. So, you still have all the necessary forms somewhere. Once upon a time each vehicle carried a stack of those forms, ready to be filled out.

Once you go mobile/paperless, keep those paper forms in the vehicles. Purchase a plastic document box, fill it with every form anyone might need, label it with the truck's name, and keep it onboard at all times. (The onboard box of paper forms should be a checklist item that the crew verifies each morning.)

Dispatchers too should have a folder of call-taking forms, perhaps a folder full of them mounted on the wall, ready at a moment's notice in case the station's internet connection goes down. All dispatchers should practice (at least once) taking a call using the paper form.


Set Up Nightly Schedule Notices

Each night, AngelTrack will send you a copy of the next day's call schedule, if you configure your messaging settings to allow this.

In the event of total loss of internet or AngelTrack, that call schedule will be the one critical piece of data that your organization requires. So set yourself up to get a copy of the next day's call schedule emailed to you every night.

It is easy to set up an inbox rule to automatically file those emails away in a folder, always ready in case they are ever needed.

Seriously, you need to do this

Don't put this off. AngelTrack could go down at any time. If that happens, your business will be fine... as long as you have a copy of the day's call schedule!


Alternate Communication Channels

Maintain an alternate communications channel for communicating with your crews:

Primary Communications Channel Cheapest Secondary Channel Next Cheapest Secondary Channel
Company-owned Nextel phones Employee personal cellphones VoIP software on the company-owned mobile devices
Company-owned pagers Employee personal cellphones VoIP software on the company-owned mobile devices
Employee personal cellphones VoIP software on the company-owned mobile devices Company-owned pagers
Shortwave radio Employee personal cellphones VoIP software on the company-owned mobile devices
VoIP software installed on company-owned mobile devices Employee personal cellphones Company-owned pagers

More likely than not, you will have to fall back on that secondary channel... especially in built-up areas and inside hospitals, where one type of signal fails while other types succeed.

Also, remember that AngelTrack can be configured to email out all details of the dispatch to crew members -- all the information they need to run the call. These emails are sent when dispatch assigns the call, and they can be sent as text messages if your crew members have configured their AngelTrack messaging settings to come to their personal phones as text messages.


Secure the Dispatch Office

You must protect your dispatch office from loss of electrical power and from loss of internet service.

Short-term loss of electrical power

Protect your dispatch computers from short-term power outages by connecting all dispatchers' computers to UPS devices. Resist the temptation to connect more than one computer/monitor to each UPS; each dispatch computer should have its own. And you must include the monitor and the network switch (if present), since the computer isn't useful without them.

Likewise for the networking hardware in your wiring closet. All devices -- including your phone switch, network switch(es), internet gateway, cable/DSL modem, WiFi access point, and your VoIP gateway -- must be connected to a UPS.

New UPS units are expensive. If there is not room in the budget for new ones, consider buying refurbished units instead: as long as the battery has been replaced, it will work just as well as a new unit. This eBay search shows refurbished UPS units currently for sale, sized 1000VA in order to power multiple computers and their monitors.

Long-term loss of electrical power

A sustained power failure at your dispatch office will require a gas-powered generator, because UPS devices rarely last longer than an hour or so.

Standalone gas-powered generators are cheap these days -- perhaps $500 for something large enough to run just your dispatch computers and associated networking infrastructure. You may also have an ambulance with a sufficient on-board generator, which you could park at the dispatch office to provide power.

Don't forget electrical cords. You will need a large-gauge extension cord or two, long enough to run from the parking lot (where the generator or ambulance will be stationed) all the way to the dispatch office and all the way to the wiring closet. This is probably longer than you think, so perform a test deployment to make sure the cords will reach.

Keep the cords and the generator and spare fuel all together in one place, and do not allow anyone to borrow the cords or fuel for some other purpose. Train your station captain and your dispatchers how to deploy the system all by themself.

Electric generator

Sometimes it is hard to get a generator to start, if it has not been run in a long time. Here is a checklist for getting it running:

  1. Check that the main power switch is turned on.
  2. Check that the fuel valve is turned on. It is a metal valve that you will rotate 90 degrees to open or close. It is probably located on the generator's left side, immediately below the fuel tank.
  3. Check that there is fuel in the tank.
  4. Check that the choke lever is in the closed (choked) position. Choke is used to start the engine; open (unchoke) the valve as soon as the engine starts.
  5. Check the oil level using the dipstick; many generators have an automatic low-oil shutoff.
  6. If it still will not start, then use a screwdriver to open the carbeurator drain for a few seconds until fresh clear gasoline runs out. If the generator has been sitting for many months with its fuel valve open, the carbeurator float bowl will be filled with thick gasoline fractions which cannot be carbeurated. If nothing runs out even though the drain is open, then check the fuel valve.

Loss of internet service

If internet service to your dispatch office is interrupted, you can run your operation from a wireless device over the cellular network. You can even run it with just a smartphone. Or send your dispatchers home with orders to run the operation from their home internet connection.

This is normally easy -- in fact you may already have a night dispatcher working from home. If you have not, then do at least one test run, including forwarding the phones.


Train Employees How to Handle a Total Communications Outage

You should have a clear protocol for how crews should behave during a total communications loss, and the protocol should be trained up ahead of time, before you need it. A typical protocol looks like this:

Protocol for Total Communications Outage

  1. Complete all dispatches currently assigned. If a member of the crew has their hands free, continue attempting to reestablish communications with dispatch.
  2. Remain at your last dispatch's destination for ten minutes while attempting to reestablish communications. If all else fails, borrow the land-line telephone at your destination and call the company main number.
  3. If still no luck, return to your last posting station (headquarters or otherwise), while continuing attempts to reestablish communications.
  4. Remain at your posting location until either 1) the time of shift end, or 2) an employee delivers a paper call schedule to you.

24-Hour Continuity of Service Requirement

We here at AngelTrack LLC spend a great deal of time, effort, and money to prevent such an outage from ever occurring. For example, we do not attempt to save money by using a cheap datacenter; instead we pay top dollar for Rackspace, the top-shelf cloud hosting provider. They are gratifyingly obsessed with uptime.

The goal is to prevent any disaster that can possibly be prevented. But some disasters cannot be prevented. A meteorite or a terrorist or a sinkhole could destroy the whole Rackspace datacenter. If that happens, then AngelTrack support staff will scramble to activate a replacement cloud server for you, at one of the surviving datacenters.

Of course we would do that as quickly as possible. The problem is, if an entire datacenter goes down, then all of the AngelTrack customers based at that datacenter must be restored at alternate datacenters. It could take many hours to do them all. So, to protect yourself, you must be prepared to run for up to 24 hours without AngelTrack.

This is not just a warning; you must do it

When you signed the Terms of Service document, you agreed to plan, implement, test, and guarantee that your EMS operation can tolerate a twenty-four (24) hour outage of AngelTrack service.

We are serious. You really have to do it.

We will not sell AngelTrack to you, unless you can live for at least 24 hours without it.

Until you have implemented your continuity of service plan, and proven your ability to run smoothly for 24 hours with no AngelTrack service, you must cease all use of the AngelTrack product as soon as practical or within 72 hours, whichever comes first.

Enforcement and testing

At least once every six months, test your fallback systems and verify that everyone knows their role and can find the paper forms and other resources they need. There are many players involved in delivering AngelTrack service over the internet, and delivering internet service to your employees... and most of those factors are beyond the control of AngelTrack LLC. So no matter how carefully your AngelTrack cloud server is managed, a prolonged service outage could happen at any time. YOU MUST BE READY FOR IT.


Catch-Up Bookkeeping After an Outage

If you run on paper for several hours or a day, it will be necessary to input those calls into AngelTrack and then back-date them. To learn how to do this, read the Retroactive Editing and Booking guide.



AngelTrack Help Index - AngelTrack Support