Operational Technology In The CMDB: Manufacturing And Pharma

7 min read
Last updated on

Bringing operational technology into the CMDB starts with an inventory built by people and passive observation. An active discovery scan comes last. Programmable logic controllers, line equipment, laboratory instruments and building systems often can't tolerate the probes discovery sends to servers. The order that works is to document what's in each plant or lab, observe network traffic passively, and only then run tightly scoped, light-touch discovery where devices can take it. What you gain is an owner and a change process for every connected device, which is what stops a firmware update from halting a factory.

Key Takeaways

  • Operational technology often can't tolerate the probes discovery sends to servers, so an active scan comes last. Document each plant or lab first, then observe network traffic passively, then run tightly scoped, light-touch discovery where devices can take it.
  • Operational technology usually stays out of the CMDB for organisational reasons. Engineering owns the equipment, it sits on separate networks and nobody was asked to bring it in.
  • At an automotive manufacturer, an incompatible firmware update stopped three manufacturing plants for two hours, and there was no named owner to coordinate the rollback.
  • Model the production line so impact analysis works. Use the location hierarchy, relate each device to the line it serves, record device-to-device dependencies and make firmware version a required attribute.
  • Every device needs a named owner, a support group and a recorded vendor, and firmware changes go through change management with engineering at the change advisory board.
  • In pharmaceutical settings, the CMDB can carry validation status, system owner and process owner, with quality approval required for changes to validated systems.

This article is for configuration managers in manufacturing and pharmaceutical organisations. It covers why operational technology stays outside the configuration management database (CMDB), what an active scan can do to fragile equipment, how to build and model the inventory, and what validated systems add in pharmaceutical settings.

Why operational technology stays outside the CMDB

The reasons are usually organisational before they're technical. Plant equipment is owned by engineering, sits on separate networks, is maintained under vendor contracts, and is run by systems other than ServiceNow. Nobody decided to leave it out. Nobody was asked to bring it in.

On an automotive estate of around 12 million configuration items (CIs), production machinery was considered out of scope for the CMDB and for vulnerability management, so it had no owner. The devices did exist as CIs, but with almost no attributes populated, because ServiceNow wasn't used to manage the production line.

A small but critical component received a firmware upgrade that turned out to be incompatible with another device. With no end-to-end view of the line, nobody could see the dependency in advance. Three manufacturing plants stopped for two hours. When a rollback had to be coordinated across all three, there was no named owner to act as the single point of contact.

We documented the equipment in each plant, then worked with the client to put change management in place through ServiceNow and the CMDB, so firmware changes are assessed against dependencies before they're applied.

What an active scan can do to fragile equipment

The instinct of a CMDB team is to point discovery at the plant network and let it build the inventory. On fragile equipment, that can cause the very outage you're trying to prevent.

Broadcast estates show the risk clearly, because broadcast equipment is as sensitive to unexpected traffic as much factory kit. On one broadcast estate, a breakdown between procurement, asset management and configuration management meant equipment that was live in production control rooms was recorded as in stock and non-operational. The rooms showed no equipment and no dependency relationships, so they were assumed to be offline.

The internal configuration management team scanned a subnet in one of those rooms. The scan took the room offline during a live global news broadcast, and viewers switched to a rival channel. On the same kind of estate, scans running every probe against every address at once caused delicate equipment to fail repeatedly.

The fix was to establish what hardware was in each room first, then run very customised, light-touch horizontal discovery using pattern extensions we wrote for that equipment, and only then map services top-down. The same order applies to a production line or a laboratory.

Building the operational technology inventory in the right order

Three steps, in this order. Each makes the next one safer.

1. A documented walk-down of each plant or lab. Engineers and the configuration team record what's physically there: lines, cells, controllers, operator panels, instruments, network switches, with manufacturer, model, firmware version and the vendor contract that covers each. Start with one line in one plant and use it to set the standard.

2. Passive observation. Specialist network monitoring platforms can identify industrial devices by listening to traffic, without sending anything to them. Several have Service Graph Connectors, which bring their data in through the Identification and Reconciliation Engine with a named data source. Reconcile what they see against the walk-down. Differences are either missing records or unknown devices, and both need an answer.

3. Light-touch discovery where it's safe. Only for devices you've confirmed can tolerate it, with custom patterns that ask for the minimum, scan windows agreed through change, and fragile address ranges explicitly excluded from every schedule. Record the exclusions in the CMDB, so a future schedule change can't quietly remove them.

ServiceNow also offers operational technology products with their own classes for industrial equipment and plant structures. Check which are licensed before designing any custom classes, because custom classes are hard to move away from later.

Modelling a production line so impact analysis works

An inventory tells you what exists. Impact analysis needs to know what depends on what.

Use the location hierarchy for the physical structure: site, building, area, line. Then relate each device to the production line it serves, and record device-to-device dependencies where one piece of equipment relies on another. The firmware incident above would have been visible in advance if the dependency between those two components had been a relationship in the CMDB.

The production line itself needs a record that incidents and changes can point at. Version 5 of the Common Service Data Model introduced an operational process service instance, a sibling of the application service, which fits this purpose. It has no creation interface yet, so expect to create and maintain these records manually or by import.

Make firmware version a required attribute on every device class you model. It's the value a change assessment will need first.

Ownership and change control for firmware

Every operational technology CI needs a named owner and a support group, usually in engineering, plus the vendor responsible for maintenance. Write that into a RACI (responsible, accountable, consulted, informed) matrix for each device class so the split between engineering, the configuration team and the vendor is explicit.

Then put firmware changes through change management like any other change, with engineering represented at the change advisory board for anything touching a production line. The change record should show the dependent devices, pulled from the relationships, and name the person who'll coordinate a rollback across sites if one is needed.

Pharmaceutical estates: validated systems and the CMDB

In pharmaceutical manufacturing and laboratories, many computerised systems must be kept in a validated state under Good Manufacturing Practice, with changes controlled and evidenced. Annex 11 of the European Union's Good Manufacturing Practice guidelines, which covers computerised systems, expects clear roles such as system owner and process owner.

The CMDB can carry much of that structure. Add a flag for systems subject to Good Manufacturing Practice, a validation status, and the system owner and process owner on each relevant CI. Build a change model where changes to validated CIs require quality approval before implementation. The CMDB's audit history then becomes part of the evidence trail for how the validated state was maintained.

Your quality function owns the interpretation of the regulations. The configuration manager owns making sure the CMDB records match what quality has approved.

Vulnerability coverage for equipment you can't patch on schedule

Much plant and lab equipment can't be patched on a normal cycle, because the vendor controls updates or downtime is scarce. That makes visibility more important.

Keep an exception register inside the CMDB for devices that can't be patched, each with an owner, the compensating control and a review date. Track detections from passive monitoring that match no CI as a key performance indicator (KPI). Each unmatched device is something on your plant network that nobody owns. Our article on false patch compliance covers why devices outside the CMDB make compliance figures look better than they are.

How Apex helps

Our discovery and integration coverage review establishes what's missing from your estate view, where, and what it would cost you during an incident. For manufacturing and laboratory estates, it covers which plants and lines are documented, what passive monitoring can see that the CMDB can't, and where discovery can safely go. It's scoped at the initial consultation against your estate and requirements.

Our two-week CMDB health baseline scores which of your CIs are trustworthy and which are guesses, including the operational technology records you already hold. The output is a report you can take to your change advisory board.

You can book a CMDB diagnostic call to talk through your plant or lab estate, or book a meeting with one of our consultants.

Frequently asked questions

Can ServiceNow Discovery scan operational technology safely?

Only selectively. Document the equipment and observe it passively first, then run light-touch discovery with custom patterns on devices confirmed to tolerate it. Exclude fragile address ranges from every schedule and record the exclusions.

Should operational technology sit in the same CMDB as the rest of the estate?

Usually yes, because the most damaging failures cross the boundary between plant equipment and the servers and networks that support it. Use appropriate classes and engineering ownership, so the records are managed by the people who understand them.

Who should own operational technology CIs?

Normally a named engineer or plant team, with the vendor recorded for maintenance and the configuration manager accountable for data quality. A RACI matrix per device class makes the split explicit.

How does Good Manufacturing Practice affect the CMDB?

Validated systems need controlled changes and evidence. Recording validation status, system owner and process owner on each CI, and requiring quality approval for changes to validated CIs, lets the CMDB support that evidence trail.

Where should we start?

With one production line in one plant. Walk it down, compare it with passive monitoring, model its dependencies, and put its firmware changes through change management. Use what you learn to set the standard for the rest.

Written by Warren O'Neill, Apex Configuration Group.

Book a free call to discuss how Apex can help you on your journey to a better CMDB

 

Warren O'Neill

By Warren O'Neill

Co-owner and Technology Director at Apex Configuration Group, based in Bath, UK. Warren specialises in ServiceNow CMDB and CSDM, ITOM Discovery, Service Mapping and pattern development, with a focus…

Full profile & credentials →

Get Email Notifications