|
|
|
Dave's Top Ten Reasons Why SCADA Systems don't Work | |
|
The purpose of this article is to identify the predominant reasons why some SCADA systems don't work well or suffer from intermittent performance issues. Ever have a problem with your system that you just can't put your finger on? Or you have been living with the problem or know about the problem but do not know what the solution may entail? Or maybe, the customer is just not making full utilization of the system you installed last year. If so, take a look below. You may find an answer or at least an idea where to investigate.
|
| |
Poor SCADA Communications Performance due to Lack of Organization of the PLC/RTU Database
Most system integrators blame low SCADA system communications performance on inadequate bandwidth. In some cases this analysis may be correct. However, in my career, I have found lack organization of the database in the PLC/RTU to be the largest contributor to poor system communications performance. I have seen PLC systems where 20+ polls of a single PLC by the SCADA application were necessary to acquire all of the PLC’s data. Put that PLC out on a half duplex 1200-baud radio system and the screen updates will probably slow to a crawl. Throw in 10 more PLCs on that same radio system and communications performance will get even worse, maybe even stalling out completely. And what if one of the PLC fails to communicate due to a problem? I hope you get the picture.
The best strategy I have found for organizing the SCADA database is when the RLL for the PLC is completed to execute the applications control functionality, add RLL rungs to organize the data necessary for the SCADA application in consecutive PLC registers. We can call the target PLC registers for SCADA data the scannable range. Most PLCs have the capability to do some sort of LET statements in their RRL syntax that allows the integrator to organize the data necessary for SCADA in consecutive registers in the PLCs memory. However, note that some older PLC/RTU devices may not allow the mapping of I/O points to another different location in memory. By doing this, your SCADA application may be able to acquire all necessary data in one or two scans vs. up to 20+ scans. Many PLCs SCADA protocols allow reading up to 100+ registers with a single poll. It takes much less time to acquire 100 resisters of data with a single poll vs., for example, 5 polls at 20 registers/poll. Take advantage of the protocols ability to handle long, multiple register packets.
For more information on this subject, click here. For information on how NetSCADA deals with this issue, click here.
|
| |
Poor Historian Performance due to Lack of Organization of the Historical Database
How many times you have attempted to display a very long historical trend for some of your data only to have your historian stall or take an excessive amount of time to display the data? The usual reason for poor historian performance is excessive historical database file size(s). Most system integrators deal with this by configuring the system to keep only the last 7-30 days worth of data. In some cases this strategy may be OK but the ability to view important trends over longer periods of time may be lost. I have found that implementing a strategy to organize the historical database minimizes the amount of data archived. This can be accomplished without the loss of transients or important trends in the data and should allow for retention of data for a much longer period of time.
This issue may also cause your entire system performance to suffer as the SCADA application slows when updating an excessively sized historical database.
Historical Trending-Rules for Success
The best strategy I have found for organizing the SCADA historical database is following the rules below:
- KNOW YOUR DATA. Know what’s important to the end-user and make sure the historical is archived for those points. Also know what’s important to the maintenance technicians for the system and make sure the historical is archived for those points also.
- LEAVE THE HISTORICAL TREND TURNED OFF FOR THE REMAINING POINTS. If no one needs the data, be diligent about turning the point’s historical data collection to an "off" condition.
- USE DEAD BANDS to minimize the amount of data collected. However, it is important that you do not set the dead band so loose that important transients are not captured. Again it is important to know your data to determine which points are the points where important transients may occur.
- If your SCADA package has the ability to ARCHIVE SOME OF THE OLDER HISTORICAL, yet still make it available at runtime, take advantage of that feature. The smaller the set of actively managed and updated historical data, the faster the updates.
For more information on this subject, click here. For information on how NetSCADA deals with this issue, click here.
|
| |
Poor Organization and Layout of the Graphical User Interface and Process Database
Many end users give their system integrators little guidance about how to lay out their SCADA graphic pages and what data to include on the pages. As a result, the system integrator develops the graphic pages with the data he thinks the end users need. Where it really gets confusing is when two or more distinct work groups (such as Operations and Engineering) both utilize the same pages and each group likes the data displayed in a certain way. As a result, the pages can become cluttered with too much data making it difficult to find the needed information on a page.
Graphic Pages-Rules for Success
The best strategy I have found for developing truly user friendly SCADA Graphic Pages is following the rules below:
- KEEP it SIMPLE and UNCLUTTERED. Know what’s important to the end-user and make sure the data is on the page. If there is too much information for one page, create a second page and make a page link to access the next page. Resist the temptation to put too much data on a single page. Instead, add a page and move related data to the new page. Make it easy to access the additional page(s). Put the most frequently used info on the first page and less useful on the added pages.
- USE a CONSISTENT APPROACH. Make page links to access the next page in a consistent manner. Locate them in the same place on a page for each graphics page in an application. Use consistent images in a consistent manner. Operators may be confused if two identical vessels or machines are represented by two different graphic images on different pages.
- USE ANIMATION SPARINGLY. Excessive use of animation may distract operators from important information on a graphics page. Use animation selectively where it helps to indicate process flow or machine operation. Again, be consistent in the presentation of animated graphic pages to minimize the amount of data collected.
- GET THEM "TALKING on the SAME PAGE". As I mentioned above, there is a big challenge when two or more distinct work groups (such as Operations and Engineering) both want to utilize the same graphic pages and each group likes the data displayed in a certain way. Be careful with this issue as pages may become cluttered with too much data making it difficult to find the needed information on a page in an effort to satisfy each groups need's. Resist the temptation to create separate pages for each group, if at all possible. Instead work with each group in defining what data is really important to each group and display that info on pages that will be used by both work groups. Data less important to Operations but important to Engineering could be located on an Engineering page and vice-versa. Keep in mind that when you get both work groups looking at the same data presented in a consistent manner, you will have achieved the objective of getting them "talking on the same page".
For more information on this subject, click here. For information on how NetSCADA deals with this issue, click here. |
| |
Inconsistency in Process Database Naming Conventions and Engineering Units
Another problem with SCADA system process databases is inconsistency in naming conventions. You have probably seen database points or graphic pages with pumps called "Pump #1" and "Pump #2" and another page with an identical set of pumps called "Pump A" and "Pump B" and even another with the pumps named "Primary Pump" and "Standby Pump". I once did job where I was calculating a total volume for 16 gas sales meters at different sites. I discovered 11 different names being used to describe identical gas sales meters! I also found one site that actually used different engineering units! Try to find the correct 16 points in a database of approximately 20,000 points. With all of the variations in the names it becomes very, very difficult! I had to use a formula to convert the one location that was using different engineering units into the same units used by the other 15 sites. Talk about confusion! Did this company have a standard way of doing the names? Yes - but there was little adherence to the standard and many different people adding points and graphics pages to the system including some operators, technicians, engineers, and contract integrators.
For more information on this subject, click here. For information on how NetSCADA deals with this issue, click here.
|
| |
Poor SCADA Communications Timing Settings, Error Handling, and Diagnostics
Some SCADA communications drivers for PLCs and RTUs have little fault tolerance built into the driver and/or in the SCADA application. This results in poor system operation when a device is experiencing an intermittent communications outage. If several devices are not communicating reliably, communications may even stall out completely making system response sluggish or, in some cases, causing the system to lock up requiring a reboot. Improper (or no ability to configure) device time-out and media delay settings is often the culprit. Additionally, lack of adequate diagnostic functionality in the SCADA application is another reason faulty device communications contribute to poor system performance. Diagnostic communications performance and communications operation information may not be normally available to the system user thereby hiding system communications problems.
Communications media used in SCADA applications ranges from 10mb Ethernet down to 1200-baud half-duplex radio. Many SCADA systems may employ multiple communications media types with some systems having high-speed media like Ethernet in series with much lower speed media like radios. Device time-out and media delay settings must be set to accommodate the slowest media employed on a single communications channel.
For more information on this subject, click here. For information on how NetSCADA deals with this issue, click here.
|
| |
The User's Application lacks Necessary Functionality
Many end users or customers give their system integrators little guidance about how their system should work. Many times specifications or operational guidelines are vague or non-existent. Additionally, to make it even more challenging, Operations participation and guidance in the project may be severely limited. As a result, the system integrator develops and delivers the application with the functionality he thinks the end users need. Instructions may be given by the SCADA project's manager to the system integrator to "duplicate" existing systems or functionality. This usually results in the customer receiving some functionality that they don't need, along with some notable functionality gaps. I have seen relationships between the customer, the project manager, and the system integrator severely damaged when working in a situation like the above.
SCADA Applications-Rules for Success The best strategy I have found for developing fully functional SCADA applications is following the rules below:
- WRITE a SPECIFICATION or GUIDELINE BEFORE THE PROJECT STARTS. Know what functionality is important to the end-user and make sure the SCADA application specification includes that functionality.
- HAVE OPERATIONS INVOLVED. These are the guys who will be using the system. Often, (but not always- see Reason #8) they know what they need and can provide guidance.
- REVIEW PROGRESS FREQUENTLY. Review the progress of the application development frequently. Do not wait until the system is ready to be deployed before reviewing the application for needed functionality.
- KNOW the CUSTOMER'S BUSINESS or PROCESS. It is a lot easier to develop SCADA applications that contain necessary functionality if the system integrator has knowledge of the customer's business and process. System integrators need to spend the time necessary to learn how the customers business or process works BEFORE beginning development of the customer's SCADA application. This way, items not addressed in specifications have a better chance of being handled as the application is developed.
For more information on this subject, click here. For information on how NetSCADA deals with this issue, click here.
|
| |
The User's Application is Incomplete
Ever have the experience of looking for a report or a specific graphics page and you discover the report or page you are looking for is just missing from the application? Or how about project documentation? Is it just missing or was never completed? Some times a project including a SCADA component runs out of time and/or money before the SCADA portions of the project are completed. This may be a result of poor project planning, poor project execution, or both. Often, by the time the project funding is compete and budgets are set, project scope is not still not frozen. As the true scope becomes evident, realization sets in that not enough money has been allotted to include all of the functionality that the customer needs or desires. Many times the SCADA portion of the project is only a small portion of a much larger overall job. And many times when its time to develop the SCADA piece of the job, budgets are already over-spent and timelines are already late. As a result shortcuts are taken and compromises are made. The SCADA application gets developed but is still rough around the edges and unfinished. And many times Operations personnel are left to finish (or live with) an incomplete, undocumented system.
SCADA Projects-Rules for Success
The best strategies I can recommend for executing SCADA projects are detailed below:
- WRITE a SPECIFICATION or GUIDELINE BEFORE THE PROJECT STARTS. Know what functionality and scope is important to the end-user and make sure the SCADA application specification includes that functionality. Specify the level of project documentation required.
- FREEZE the SCOPE. When the project is funded, freeze the scope to prevent "feature creep". If you do not have a definite scope, how do you know when the project is really complete?
- DEVELOP a TIMELINE and SCHEDULE. Keep this project schedule current and updated to avoid any nasty surprises related to delivery date. Review and update the progress of the project's development with the customer frequently. Do not wait until the system is ready to be delivered to the customer before reviewing the SCADA application for gaps in functionality.
- FILL out a SCORECARD. As in a baseball game the manager fills out a scorecard identifying who is playing first, who is pitching, etc. For your project, identify the players and what roles they should play and what portions of the project for which they are responsible. Define clear expectations down to each company and individual involved with the project. Distribute this "Score Card" to the customer, to all the players involved, and to their respective companies.
- DEVELOP a CONTINGENCY. In the event undesirable things occur or in case the project incurs unexpected delays, have a contingency plan.
For more information on this subject, click here. For information on how NetSCADA deals with this issue, click here.
|
| |
The Customer did not Know What They Wanted or Needed
Watch out for this one when dealing with a customer who has had little experience with technology or how technology may be applied to their business. Additionally, this customer may have a difficult time communicating what their system should do and the integrator has little understanding of the customers business or process. And to make it worse, the customer may see this project (and this technology) as an enabler to help them compete by becoming more efficient or achieve a lower cost structure. Sounds like a recipe for disaster? You bet! Many times the integrator does his best to deliver a good product only to discover that its not what the customer really wanted. This can be a difficult situation. SCADA System Customer Satisfaction - Some Ideas Here are a few ideas I have related to this issue:
- TAKE TIME to LEARN the CUSTOMER'S BUSINESS or PROCESS. It is a lot easier to develop a proposal and deliver a SCADA system that contains the necessary functionality to improve the customers business if the system integrator has knowledge of the customer's business and processes. System integrators need to spend the time necessary to learn how the customers business or process works BEFORE beginning development of the customer's SCADA proposal or application.
- NARROW the SCOPE. It better to have a less ambitious and achievable scope than a grand plan that attempts to bring a customer's business from the Stone Age to the space age in one fell swoop. Don't swing for the fences on the fist at bat. Successes build on previous successes but rarely (or never) build on failures.
- HAVE OPERATIONS INVOLVED. These are the guys who will be using the system. Often, but not always, they know what they need and can provide guidance in developing the proposal and the system.
- INFORM the CUSTOMER about SUPPORT and MAINTAINANCE AFTER the SYSTEM is DELIVERED. The customer should know up front that the system will require some maintenance and administration after delivery. Many systems operate until maintenance is needed and then fail. No one may be available to support the system and restore it to operation.
For more information on this subject, click here. For information on how NetSCADA deals with this issue, click here.
|
| |
The Customer is Not Using the System
This could be because the system is not operable due to requiring maintenance. Or, Operations has reverted to the "old, familiar way of doing business" by resurrecting equipment that was supposed to be retired when the new technology was installed. Or maybe Operations just don't know how to use the new system or they are scared that the system may eliminate some jobs. In any case, the bottom line is that the customer is just not getting their return on their SCADA or automation investment. And the probability that they will make future investments in the technology has been greatly reduced. SCADA System Customer Usage - Some Ideas Here are a few ideas I have related to this issue:
- INFORM the CUSTOMER about SUPPORT and MAINTAINANCE AFTER the SYSTEM is DELIVERED. The customer should know up front that the system will require some maintenance and administration after delivery. Provisions must be made to support and maintain the system after the project is over. Some technology vendors provide "cradle to grave" systems and service. Often this level of service is desirable in short-term rentals but may not be cost effective in permanent fixed base installations. Many systems operate until maintenance is needed and then fail. No one may be available to support the system and restore it to operation if plans are not made beforehand.
- MANAGEMENT needs to be UPFRONT with OPERATIONS about the PURPOSE and PLANS for NEW TECHNOLOGY INVESTIMENTS. Operations can make or break a company. These are the guys who will be using the system and if they don't want it to work, it will not. Operations needs to be fully informed about the purpose for the investment in the technology and if it means a reduced number of jobs both Management and Operations need to work out this issue.
- MAKE PROVISIONS TO TRAIN THE OPERATORS. The SCADA system operators need to be trained on how to use the system. This needs to be done not too far in advance of nor too late after system deployment. Include adequate training in the project budget.
- REMOVE the OLD SYSTEMS. When the SCADA project is complete, if it involved replacing some older equipment, retire and remove the older systems or equipment. If it is easy to bypass the new technology and revert to the old, rest assured at the first sign of trouble most people will revert back to what they know and understand. Force them to deal with the new technology by removing the crutch of the older system.
For more information on this subject, click here. For information on how NetSCADA deals with this issue, click here.
|
| |
Mis-application of the Technology
This last reason probably does not belong in a "Top Ten Reasons why SCADA Systems don't Work" list because in fact, these systems do work. However, this does not mean that an appropriate or economic level of technology was selected for these systems. I have seen many systems that were way over designed and a lesser number that were not sized properly for expansion. Both situations are not desirable and may lead to customer dissatisfaction. Over-designed systems may be much more costly to capitalize and much more costly and complex to administer and maintain. This is because the system was probably mis-applied to the customer's business and was actually designed to be used in a much larger application. Often these systems are sold on the basis that they are the latest and greatest and are being used in much more critical systems than ours, so the logic is that we can't go wrong by selecting this bigger system. Under-designed systems usually lack adequate functionality, expansion capability, or both. Often when project specifics are vague and project budgets are tight, an under-sized system may be selected. Later when expansion needs arise the under-sized system cannot meet the new requirement, and may have to be changed out entirely or at least upgraded. In any case, the bottom line is that the customer again is just not getting their return on their SCADA or automation investment. And the probability that they will make future investments in the technology has again been reduced. |
| Additional Resources | |  | |
| |
There you have it - our Top Ten. We would sure like to hear from you with your stories about what you think should qualify for the Top Ten. Send us your candidates for the Top Ten by email at: info@netscada.com |
|
|
