A mere 12 weeks ago I introduced this series as part of my final graduate school requirement. The week 1 post was full of big plans for a security focus on the manufacturing floor in the coming weeks. For the most part, I think I accomplished that goal. Week 2 explained why security is important. Week 3 might have been a bit of a stretch relating to the factory floor, but every controls engineer should work ethically, and I have heard some stories… There were a few sources with my favorite being a story about “The Front-Page Test” http://www.ca-ilg.org/post/front-page-test-easy-ethics-standard

Week 4 was a plan for security with an explanation about the CIA triad and how it differs slightly on the factory floor from the office. Week 5 included Risk Management which is something we are discussing at work relating to our equipment running Windows. I will write about that when we have a conclusion. Week 6 was recommending using firewalls and VPNs when connecting to manufacturing equipment and avoiding the wireless.

Week 7 was all about some of the network analysis tools I use and what a great thing they are for troubleshooting automation equipment; can’t say no to Wireshark. Week 8’s subject was encryption and why I think it is a good idea in manufacturing–for the most part. That week I recommended a hardware VPN that is easy to setup and use. Week 9 focused on physical security, and it might not be a big deal for most capital equipment. The peripheral equipment around the machines has a tenancy to disappear. Week 10 was about project management. I think this is an area that is well described but not practiced or well understood in my industry—except by the professional integrators. We would do well to learn from them!

In week 11 I strayed a little again and talked about certification. To be honest, I was motivated to talk about certification to be able to use that blog post as part of my Security+ CEU credits to maintain the credential. It was still relevant, as I believe that most certificates are worthwhile to pursue.

And finally, that brings us to where we are today, week 12; in this post, my review which you just read. I hope you all learned a little something. If not, pretend you did when you talk or write to me.

Peace

Jeff

Maybe you don’t think you need one; maybe you don’t. Certifications aren’t mandatory for most fields, and they can be expensive. However, they prove mastery in the subject; they prove that you value education and most importantly, they show an employer or client you are serious about your career.

In the automation space, there are a few certifications that are great to have that show competency in your field. ISA has a series that interest me which focus on Cybersecurity in an industrial environment. What about the folks just starting? For them, I want to plug the CompTIA organization. I have two certificates from that organization, Network+ and Security+. I am very happy to have earned these and I won’t lie; it took work. CompTIA is an organization that focuses on the subject theory and are hardware agnostic. Meaning you don’t have to learn how Cisco or Juniper work, but you do have to know, amongst many other things, the difference between a router and a switch and calculate the number of network hosts for a given subnet. CompTIA certifies that you know and understand concepts in information systems.

Network+ certification shows an employer that a certificate holder has demonstrated that they have an understanding of networking comparable to a year of real experience, even if that person doesn’t have any job experience. For a Controls person, or at least for me, Network+ is invaluable and helps me understand networks to troubleshoot problems and configuration errors when setting up communications to a machine. Now that I understand networking, I have fewer problems and troubleshoot less.

The other CompTIA certificate that I hold is Security+. Security is the topic that triggered my desire to go to graduate school. Security is a big complex topic, and I wanted to know more. What is an internet certificate and how can I tell if my communication is encrypted? What the heck is single sign-on, SSO? Security+ helped me understand those and more. Before the certification, I didn’t understand the differences in WPA2-PSK and WPA2-Enterprise and WPA2-TKIP. I also vaguely understood wireless communication from my electronics training in the Navy. However, after I studied the differences in the wireless transmission technologies of 802.11 and properly implemented two of them, my home internet signal got stronger and faster.

Both of these certificates from CompTIA together have been worth the money I spent learning and testing. The magic of the technology has been cleared up, and I know why instead of accepting that it works. I also know where to look when I want more information about IT subjects. Several years ago, I was in awe of some network engineers at the breadth and depth of their knowledge and how they figured all that stuff out; it seemed so complex. It is very difficult to learn all this technology on your own. It takes a structured approach to get it all absorbed in the right order so that it can be applied. I also find helping my coworkers better understand networking is more enjoyable because I can answer many of their why questions instead of saying, “I don’t know, it just works.”

Well, that is my opinion about certification, and I am not stopping. Grad school is almost over, and I am already looking for the next learning adventure.

Peace

Jeff

https://www.isa.org/isa-certification/certificate-programs/

https://certification.comptia.org/certifications/network

https://certification.comptia.org/certifications/security

 

I want to believe that a reader of this post needs no convincing that project management has already proven its effectiveness in every industry. Even in the age of Agile, traditional waterfall project management is still a worthy endeavor as Agile doesn’t address management of a project across disciplines or large projects. I don’t want to turn this post into a “Bashing Agile” post. Agile works great for tasks that all start and end inside of a single discipline or profession; it works great for small teams such as those in many software development projects. After all Kent Beck’s focus was improving software when he envisioned the predecessor to Agile, Extreme Programming.

The next logical question, how do we apply it? Well, if you are involved with capital equipment purchasing, it’s likely that project management is in use. Even in the IT security realm, project management has a place. If you aren’t part of the team or this is all new to your organization, I encourage you to dig into project management and find out what it is all about and how to apply it to the projects that you will work on in the future.

A few words of advice. When starting, don’t worry about searching and finding a project management software first. Develop a methodic system first, then look for a software solution. Don’t bother trying to find software for a solution to a problem that you don’t have yet. Yes, the software is great; yes, no matter the one you choose it will make the project seem more organized, but this category of software is not simple to use, even if it looks simple on the surface. Many people start with a grand plan then get trapped in thinking that they must list every possible task with minute detail in how long it will take to complete. All that seems great, but the novice user will quickly become discouraged with the seemingly irrational way the software behaves once adding predecessors is complete, and the tasks are all knotted up with each other. This behavior will make you feel that you are spending more time planning and no time doing. It’s best to start simple. Use a spreadsheet and list all the tasks that take longer than a day to complete. If there are many short tasks, group them logically together and treat them as a single task or better don’t list them at all and extend the time of the longer tasks that are related to include them.

Go ahead and add start and completion dates to the tasks, list the resource that will work on the task but don’t list the person by name, list by job function. Phil Carleen might be the programmer on the project but list his work as a programmer. This way, when he goes on vacation or changes jobs, the plan can continue with a new programmer without changing the paperwork. When estimating the time to get a task done; a good rule of thumb when starting is to double the estimate and maybe triple it. If you think the time for a bit of programming to make a pick and place move will take two days, put four on the spreadsheet. Seems crazy I know, but you aren’t listing the effort to program on the spreadsheet–not yet, rather you are listing calendar time, and they two easily get entwined. Estimating the effort a task will take as two days might very well take two days and maybe less, but what happens when a production machine goes down, and you have to spend three days working on that? Your schedule just slipped! Sure, you might hear people complain that it takes too long and when all the tasks are added up it might seem incredibly long. As you and your team gain experience and get better at estimates, the durations will change, and things will go faster. Trust me on this and trust that I predict you will be overly optimistic about how long it will take to complete those tasks.

Good luck

Jeff

References

Beck, K. (2000). Extreme Programming Explained. Reading: Addison-Wesley.

Brewer, J. L., & Dittman, K. C. (2013). Methods of IT Project Management. West Lafayette: Perdue University Press.

Whitman, M. E., & Mattord, H. J. (2018). Principles of Information Security. Boston: Cengage Learning.

 

 

Huh, physical security in a manufacturing environment? Theft isn’t normally an issue since the equipment would fail immediately if a part were stolen. That is if the machine is running; if the machine is idle, however, then theft could be an issue. Tools, supplies, and the small stuff that can add up over time are a different story, but, I don’t want to talk about the obvious stuff. Maybe it isn’t theft of the equipment that we should worry about. More like theft of the intellectual property, or maybe the products you manufacture are high value and the employees want to use what they make.

How do you protect that? Keycards, mantraps, biometric, and live guards all have proven through the test of time to work. I once worked in a contract manufacturer that among many other things made toothpaste and teeth whitener and those items were popular. The company knew this and addressed it rather nicely, or at least I thought it was nice and non-insulting. During most company meetings products were distributed as gifts to all of the attendees and there were vending machines that sold the products at a very deep discount. Of course, there were the signs about limiting purchases of the products to a few items per day and the posters about theft were routinely changed and moved around to keep everyone’s eyes on them, and I think combined those tactics worked well.

Other things that helped of course, like at my current employer are video cameras monitoring the doors to the outside. There are no cameras inside monitoring our work, which I take as meaning there is a lot of trust and professionalism. We must enter all of the work areas with key cards; this has more to do with personal security and the active shooter training that we all recently went through than the products being of high street value; they aren’t. Many of the parts we make are confidential because our customers want them to be held in confidence and as such most operators aren’t allowed to roam into other areas of production freely.

I like our security, and I don’t hear anyone complaining about it.

See you next week.

Jeff

 

References

Barker, D. (2012, July 26). A Guide to Physical Security for Data Centers. Retrieved from The Datacenter Journal: http://www.datacenterjournal.com/a-guide-to-physical-security-for-data-centers/

Conklin, W. A., & White, G. (2015). All In One CompTIA Security+ Exam SY0-401. New York: McGraw-Hill.

Herold, R. (n.d.). The Definitive Guide to Security Inside the Perimeter. Retrieved from Realtime Publishers: https://www.realtimepublishers.com/book.php?id=49

Meyers, M. (2012). All In One CompTIA Network+ Exam N10-005. New York: McGraw-Hill.

Prowse, D. L. (2014). Cert Guide CompTIA Security+. Indianapolis: Pearson.

 

 

This week’s discussion is about encryption and using it in an industrial environment. The short answer is to use it, wherever you can. The actual answer is a big… it depends.

Machines inside the enterprise are typically hardwired and communicate without encryption. In my opinion, the risk is quite small if the plant is properly segmented and the equipment stays out of the business office and vice versa. Encryption becomes important when the data travels outside of the network. If there are sensors sending data back via a wireless carrier, it should be encrypted before it gets transmitted. In the unfortunate event of a machine sending data over the Wi-Fi, that better be encrypted as well.

Does this all sound simple? Well, as you might already know security in manufacturing equipment was never a consideration when it all became computerized, and even today security is slightly more than an afterthought. So, to encrypt the data, it must be done outside of the PLC unless you are lucky enough to use a PLC that can encrypt built-in. We use several PLC brands where I work, but B&R is our go-to system, and we can encrypt. We also have Rockwell’s Compact Logix in-house, and I haven’t found how to encrypt that data unless I put the data in a PC in the cabinet and then send it out from there. A device made by Tosibox is one such way. A link is below along with some additional information from B&R. And FYI, I don’t receive any benefits from promoting either company. I have used the products and feel they do a good job. I am sure there are others out there that are equally effective, but I can’t speak of their use.

Cheers

Jeff

https://www.br-automation.com/en-us/about-us/customer-magazine/2018/20189/cybersecurity-for-controllers/

https://www.tosibox.com/

You are likely aware of packet sniffing programs that will show you all the communication that goes on between attached computers; these are interesting programs indeed. Looking at the output they create is a little bit like looking into a microscope and seeing that tiny little world and if you are anything like me, at least before I started to learn about how Wireshark works. You think it’s cool, feel smarter for being able to read the data and pretend to understand it all, but in reality, it is very overwhelming and doesn’t really make sense. Sure you can find the source and destination addresses, the MAC addresses, the protocol being used, the payload, source port, destination port, and on and on and on.

All that is, of course, understandable, but how does it all fit together and when would you use it? Well, the obvious answer to using is for eavesdropping on traffic from other computers. Fortunately, for all of us, eavesdropping isn’t as easy today as it was several years ago. Today most of that “interesting” data is encrypted and requires the use of other programs to help decipher it. Also, network traffic travels over switches which also means that if your computer isn’t in the source or destination address, you aren’t likely going to see the data without other special tools. This blog post isn’t about hacking or eavesdropping, so we aren’t discussing that. For me, it’s all about troubleshooting using Wireshark and a little tool called the packet squirrel. I have to use them together, and together these provide an excellent setup that allows me to see the communication between a machine and the inspection system, between two machines, or between all the machines that chat with each other, and Packet Squirrel records it all.

When examining the packets from this equipment, I can see what data is being exchanged, and with some patience and diligence, I can figure if I have a bug in a program that is sending incorrect data. Instead of a bug, perhaps there is a hardware problem causing issues, or maybe there isn’t a problem, and I want to learn what data is in the traffic. Whatever the reason, it is interesting to see provides a wealth of information. But first, we need to make sense out of all that data, and that takes understanding the protocol being used.

Understanding isn’t a simple task and is better left for another time when we can break it down into much smaller pieces since many protocols are communicating between devices. Hopefully, this little discussion was enough to make you curious and wanting more details. So, I leave you here on a cliffhanger. But first, take a look at the book: Practical Packet Analysis, 3E: Using Wireshark to Solve Real-World Network Problems it taught me a lot and I recommend the read!

Oh, and BTW, that is a regular link; no affiliate program tied to it.

Till next week

Jeff

 https://www.amazon.com/Practical-Packet-Analysis-Wireshark-Real-World/dp/1593278020/ref=sr_1_2?ie=UTF8&qid=1539302361&sr=8-2&keywords=packet+analysis+with+wireshark

Another group of items that you might think is not a good fit for the factory floor. Well, I am here to say that not anymore. Especially in this world of IoT[1], IIoT[2], and that marketing buzzword Industry 4.0. Sure, I will concede that these items are in the realm of the IT department which must set up, install, and manage each of them. However, for a controls engineer, it is important to know the functions of these items and when and why we would choose to use them.

If you are like my colleagues and me, you likely find yourself frequently accessing machines from your desk. You also might find it convenient to help out the off-shifts by connecting to a machine from home to help troubleshoot a problem. If that is the case, then Firewalls and VPNs will be your friend. VPNs are easy to set up and easy to use, and most likely they are already in the enterprise just waiting for you to ask to use it. A VPN provides a secure connection over the public internet that will allow you to work just as if you are at your desk.

Firewalls provide a secure barrier between you, your equipment, and the internet. And don’t think because you have a firewall at home that you plugged in and it started working that they must be simple. In the enterprise, they are far from simple, and a tiny little mistake can create a hole in your network that an attacker can slip through and it might take months before you realize something bad has happened. In your home, you might have a dozen or fewer different communication protocols on your computer that access the internet, for gaming, streaming video, surfing, updating, and chatting. In the enterprise, we have all those plus, plus, plus. Protocols are running on a machine that is cryptic, and even IT might not fully understand what they are all doing or where they are all used. This added complexity makes that firewall much more important.

As for wireless, I will say only this: don’t use it on the factory floor. Don’t connect to machinery with it and don’t run equipment on it. It’s a bad idea. Yes, it mostly works, and you might never notice when it doesn’t work, but it will be difficult to troubleshoot some kinds of problems. And in some instances, all the bandwidth you and your coworkers use with phones and other devices connected to the Wi-Fi, it might interfere with production. An overloaded wireless network might even cause the machines to make scrap parts because position feedback or other parameters didn’t get updated promptly. I could get on a tall soapbox and rant for hours about this, but at the beginning of this series, I promised I wouldn’t bore you. So, I will leave it at that.

Until next week, peace.

Jeff

[1] Internet of Things

[2] Industrial Internet of Things (Same as IoT, just marketing jargon)

Do you know what this is? The subject makes it seem obvious right? Just another topic that doesn’t have anything to do with industrial automation! Well, not exactly; there was a time when that was a true statement, and risk management didn’t have anything to do (or much to do) with manufacturing. Except, of course, when designing the machine for a new process–risky business, that is. That isn’t however, the risk management that I wish to speak of today; it has a place, and they both are closely related early in a project, but I digress.

Today, this risk management is about information security risk and the resources used on the factory floor. Do you have a computer controlling a machine that is running Windows Embedded? Is it connected to the network? Might be a risk; even if not connected to the internet, it is still at risk. How am I so sure? I held a meeting with a cross-functional group of my peers and conducted a risk assessment. In this assessment, we listed all the possible risks and attack vectors. We went into detail and argued back and forth, then we attached the probability of each of those risks being exploited, and from the list, those Windows Embedded boxes ended up quite high on the risk list, not because Windows is insecure.

On the contrary, Windows is quite secure when kept up to date. Unpatched, un-updated windows machines connected to other similar machines. That is an invitation for malware to come on in and have a look around.

In that meeting, we conducted a qualitative examination. It was qualitative because we didn’t attach a precise number to rank the probability of each risk. Instead, we went with our gut and educated guess. Because the meeting had a cross-functional group, it was less likely to skew the results to a risk that wasn’t likely but felt really bad. You know like the chance of a terrorist attack in rural Iowa; sounds really bad but it is probably extremely remote. So we determined we didn’t need to spend resources protecting from a terrorist attack, but we did decide we should set up a schedule to push patches and updates to all of the machines in our factory.

Want to know more?

References

Johnson, R. (2015). Security Policies and Implementation Issues. Burlington: Jones & Bartlett Learning.

Project Management Basics. (n.d.). How to Perform Qualitative Risk Analysis for the First Time. Retrieved from Project Management Basics: https://pmbasics101.com/how-to-perform-qualitative-risk-analysis/

Stackpole, B., & Oksendahl, E. (2011). Security Strategy. Boca Raton: CRC Press.

Whitman, M. E., & Mattord, H. J. (2018). Principles of Information Security. Boston: Cengage Learning.

 

 

 

 

If you are in the industrial space, and if you are reading this I assume that you are or are planning to be, then security should be on your mind. Most controls engineers don’t think about security. I hear a few reasons, “they don’t have time” and also, “it is IT’s responsibility, so I don’t need to deal with it.” To be blunt, that would be wrong thinking and maybe a little lazy.

In the IT space, security is right up there tightly integrated with the CIA triad or Confidentiality, Integrity, and Availability. The IT department cares about the triad in CIA order. Controls engineers should care about the same things in a different; Availability, Integrity, then Confidentiality. The data we create, store, and analyze isn’t typically confidential information. The individual data bits alone aren’t as important and doesn’t provide meaningful metadata until it is grouped with the other data and then confidentiality comes into play, but we need the data to be accurate and of high Integrity. And we need high availability; the production of the data rarely stops, and without availability, the data has no place to go, and then it’s lost forever in the ether.

What this means is: you have to plan. There should be a plan for data loss, data breaches, and corruption just like the IT department has in place for the business system. If this seems like it might be overwhelming, or overkill, think about how many problems it would cause if the data you create were lost today and isn’t recoverable. Could your business continue? Can your equipment still produce a quality product? If so, maybe you need not worry about it. Better though to ask a senior manager what they think would happen if the data you generate were lost. If your organization is anything like mine, it is very important data! Product traceability; lot or batch tracking; quality control are just three reasons the data is more important than merely production quota data.

So if you haven’t, please request a meeting with the IT department. They will be able to help with creating the appropriate policies, procedures, standards, guidelines, and plans. When they realize the importance of that data, they might even integrate into their existing plans making your life easier.

You will then help the IT department better serve your department, and they will begin to understand your POV of needing to get the job done. Perhaps you can learn how they handle security and all those annoyances to get their jobs done and assimilate those IT practices into yours for a more secure organization.

Peace

Jeff

Legal, Ethical, and Professional Issues in Information Security

Think we need those things in Information Security? You bet we do! I would even suggest that we should be even more ethical, professional, and certainly as legal as we can be. In IT we are privy to information that is considered private, can be very personal, and can create great harm if leaked. From the website informationRisk a great quote comes: “With electronic access and technological advancement, it is much easier for professionals today to make a mistake, behave incorrectly and have their unethical actions to go viral.” (Gupta, 2012)

 IT can be a small industry in a community, and bad press can haunt a person for a long time. If you have been around for very long, you have heard stories, and you may have met the people about whose stories you hear. I for one never want to be that person.

A manager of mine from a previous employer once told me that we should behave as if what we did were going to be placed in the newspaper for everyone to read and judge and we should behave accordingly. BTW, he wasn’t scolding me. I believe that is exactly how we must all behave. If we all could, the world would have a lot less drama in it. This story is interesting to me because, after he told it to me, I agreed with it, but didn’t give it any more thought, then in my research for graduate school and a topic about ethics, this story pops right out in my second internet search. The subject of the link below is public officials, but I don’t think that matters. It applies equally to myself and others in my industry.

http://www.ca-ilg.org/post/front-page-test-easy-ethics-standard

 

References

Gupta, U. (2012, February 4). Role of Ethics in IT Security. Retrieved from InfoRisk Today: https://www.inforisktoday.com/role-ethics-in-security-a-4469

Institute for Local Government. (2015). The “Front Page” Test: An Easy Ethics Standard. Retrieved from Institute for Local Government: http://www.ca-ilg.org/post/front-page-test-easy-ethics-standard