The Material Ledger
One of the most powerful concepts in the ThinkIQ platform is the material ledger. The material ledger is similar to a financial ledger, which is at the core of every financial system in the world, whether you're talking QuickBooks or SAP. It’s not a new idea. It's been around for a long time. The financial ledger was created in 1460s, long before the invention of computers. It's a very simple but powerful idea that basically says - break your business up into accounts where you want to track the movement of money that represents your business. The inflows and outflows of money into those accounts must balance over a period, respecting a set of financial rules. This is such a powerful concept that it is used for the majority of businesses in the world and there are a dozen different reports that you could use to report on a business and be able to answer the question: what is the financial health of the business? It is an extremely elegant powerful idea.
ThinkIQ builds upon this idea with the Material Ledger. It is like a financial ledger from the perspective that you break your business up into accounts where you want to track the movement of materials. These accounts can be factories, or warehouses, or individual machines, or storage tanks. We track the movement of materials into and out of these accounts, and the movements must balance over a defined period respecting financial rules, conservation of mass and conservation of energy rules. This allows you to look at the health of any manufacturing business in a very consistent way, tying machines and materials together.
Unlike a financial ledger, the movements of materials can be extremely granular, so you do not want people to manually enter them. To make it practical to create this ledger, you want to be able to automatically detect these movements. The ThinkIQ system allows one to detect material movements quickly and easily by building upon the fundamental idea of the equipment model.
The Equipment Model
Starting from the base version of the product, ThinkIQ Visualize™, the Equipment Model gives data about your processes context, meaning and discoverability.
Libraries define and introduce types of equipment. You then create instances of them in the Model, and you put measured attributes on those pieces of equipment. To visualize them in the Process Layout, you create relationships between pieces of equipment, the most important of which, for material movement detection, is the can-feed relationship.
Once you have created these can-feed relationships, material movement detection builds on the work that you have already done.
Accounts
Accounts are places in the model that are part of the Material Ledger. You can add accounts to anything in the model, but most typically they are added to places, areas, or equipment.
To create and detect material movements you first need to decide what parts of your process constitute an account. You can think of an account as being a point in the can-feed relationship chain. Typically, you will put accounts at places where you know that product physically accumulates, so that might be for example warehouses, storage areas, storage tanks, or piles of stuff. To track that accumulation, you go into the Model Explorer, choose the pieces of equipment or places where you want to create accounts, and you Add an Account to that instance.
The Model Explorer already has tabs that give an overview of a piece of equipment, lists its attributes, scripts associated with it, and any relationships it has. It is here, on the Materials tab, that you can also create an account.
What are the key actions needed to prepare an account to trace materials? Let's look at this through the lens of material movement detection.
The Balance Attribute
The first thing you need to do is to tell the account what its balance attribute is. The balance attribute is the attribute on that piece of equipment that will be used to determine how much material is in the account. If we take the example of a tank, it might be the level in the tank, or it might be the mass of the material in the tank. But typically, it is the measurement that you use to determine how much material is in the tank (location, storage area, etc.). In many cases you will need to use an expression to convert something that really is not a balance attribute into a balance attribute. This is usually relatively trivial. For example, if your tank has a level measurement on it that measures percentage full that is not a balance: it does not tell you how much material that is. Expressions allow you to manipulate an attribute like this. In the case of our tank, you would define another attribute as a static attribute which is the tank's capacity. Now you can take percentage and you can multiply your percentage times capacity to tell you how much material is in the tank. In this case you are creating a volume. In general, balance attributes need to be ideally masses or volumes; masses are the best. By creating an attribute that uses an expression on the model attribute, if necessary, you convert sensor data into mass or volume, and then you assign that attribute to the balance attribute on the account.
Propagation concepts: Empty and Influence Ratio?
An important concept in the Material Ledger is being able to determine where Materials end up for the purposes of Pull-Back, Recall and various other analyses. With discrete materials this is fairly obvious, for example a box of bolts has a known number of bolts, and it is numerically simple to precisely answer the question “find every assembly containing bolts from box XYZ”. In Process environments where mixing, blending and dilution occur, the question “which products did this tanker of palm oil end up in?” is more nuanced because, unlike the bolts, there is no discrete “unit of palm oil” and we need to be able to manage how far a material propagates. The system provides several techniques for this:
- If a storage location becomes empty, there is none of that material left to propagate.
- Influence ratio. As the storage location fills and partially empties the material becomes more and more diluted. This ratio defines the level of dilution that no longer matters.
- Kill step. A physical step beyond which propagation does not occur.
Along with setting the balance attribute on the account, you probably want to ask What do I consider empty? If the balance attribute is measuring tons of material in the tank and the tank is down to less than 1/10 of a ton, could you for practical purposes consider the tank to be empty? Why is 0 not empty? Zero may be empty, but often physical vessels cannot completely empty, or the sensor is not particularly accurate. Whatever the case, emptiness is an important notion for material management. One of the major uses of the material ledger is to work out: where did material go, and how far did the presence of the material propagate. You do not want to be in a position where materials propagate indefinitely.
You know conceptually that if you look at a grain elevator there could be a few seeds of grain remaining from a particular shipment from a particular farm for millennia; but nobody cares. The practical reality is that once you have enough material pass through that grain silo, there is nothing left from that shipment. Its influence gets removed. This notion is what allows us to basically set a whole bunch of different parameters on the account that are used by the material ledger to determine how attributes propagate and more importantly, how traceability and provenance work across the material ledger.
Movement Detection
To get automated material movement detection, the following things are necessary:
In your equipment model you will have accounts where materials are held, and you will have items that allow materials to flow into and out of accounts. These are typically things that start, stop or control the rate of flow: examples are check valves, slide gates, motors, pumps, conveyor belts, elevators, legs, and robots. There are also active pieces of equipment that physically control where materials are flowing to such as multi-port valves, divertors, multi-destination conveyors, switches, and robots. The can-feed relationships defined in the model typically tie accounts to flow control equipment to accounts.
There are three categories of flow control. The simplest is pass-through: the item has no effect on flow or flow direction: the movement detection code ignores the item and moves on to the next link. This equipment is in the can-feed relationship for clarity and cohesiveness of the equipment model, but it does not really participate in the feed path. From the feed path perspective, it is just a pipe, or it is just a connector that does not affect whether material movements can occur. This is a pass-through piece of equipment. It has no code in it for material movement, it is not smart equipment, and it probably doesn’t have a class script.
The second category is straightforward flow control: on-off, open-close; am I allowing flow to come through? To do material movement detection the outlet valve, for example, must be a piece of smart equipment - in this case, a 'smart valve'. That means it must have a class script and in particular, the class script must implement a method called is_feeding(). That method executes logic to look at the sensors on the 'smart valve' and determine whether the valve is in a position that allows material to flow out of the account or not.
The third type of flow control uses ‘flow diverters’ that control the direction of flow, or branching. This piece of equipment can feed multiple other pieces of equipment, things like diverters, conveyors, robots; they actively change the direction of the flow. In this case, the feed chain must not only determine if an equipment is running but it must also answer the question of the equipment: what am I targeting? The way the question gets asked is found in the feed chain. The feed chain always works from the target account moving upstream to potentially many source accounts. What that means is that as it gets to a piece of equipment that is downstream of a piece of flow control equipment that can branch - can feed multiple things - the question essentially gets asked through the lens of: are you feeding me?
This third category of flow control equipment has an is_feeding() method. The feed chain still asks the flow diverter ‘are you feeding me’, but instead of one ask, the feed chain must check on all states of the flow diverter. This allows the feed path to have flow diverters that can be two way or three-way or an n way, and the is_feeding() method will manage all of those.
Smart Equipment Library
This is where the Smart Equipment Library comes into play.
Smart Conveyancing
The ThinkIQ Smart Equipment Library was built on the premise that there are common things like conveyors, valves, slide gates, elevators, and robots - and several other things - that have patterns in the industry for how they are instrumented. The Smart Equipment Library has modeled these common pieces of equipment and includes them as equipment types, built with the necessary smarts to do material movement detection. Each type includes a class script, an is_feeding() method, and other behaviors (classes) to support the work it will do in the feed chain. When you build your model based on types in the Smart Equipment Library, you are preparing for automatic material movement detection.
An example type in the library is a valve. A simple open/closed valve might have two sensors: an open limit and a close limit. The smart library basically looks and says a valve is open if its open limit is true and its closed limit is false. But what if it only has one sensor? The smart valve has been built with logic to deal with this and other permutations of valves. These Types typically include an attribute which is the calculated state of the equipment that abstracts the specific sensors to give a predictable and trustable state.
Material Aware
Along with smart conveyancing equipment, the Smart Equipment Library implements material aware equipment. Material aware equipment, like smart conveyancing equipment, is equipment that has a class script. It also has behavior that allows it to intelligently participate in the Material Ledger. Material aware equipment implement a specific interface called the material aware interface. This interface enables several useful behaviors.
The first and simplest behavior is the notion that material aware equipment can create their own accounts. An example of this could be a grain silo, a tank, or a bulk scale. In the case of a grain silo, the grain silo knows that its balance attribute is its 'Content' attribute. The content attribute measures how much material is in the grain silo; also it knows what the correct engineering unit is, and it knows what its capacity is. It knows, from a Ledger perspective, what constitutes 'full', and because it knows what its behavior is, it understands what constitutes 'empty'. It also knows how to set its influence ratio. And finally, because 'grain silo' is a material aware piece of equipment, each time a new silo is instantiated in the Model, all of this richness flows with it, and as the feed chain is implemented, these types of equipment automatically create their material accounts (if they do not already pre-exist) without the user needing to do it through the user interface.
The second behavior that you get from material aware equipment is their ability to participate in the algorithm that automatically populates the Material Ledger. That algorithm that begins with feed chain. The feed chain (described below) essentially determines when movements occur, and it opens movements and closes movements and groups movements together based on a set of rules. There are often situations when you want the information that goes into the Material Ledger to be smarter than just the fact that movements occurred, however.
For example, coming back to the grain silo example, there may be quality data - grading data - that asserts the grain quality in the silo. This is typically created by somebody performing a grading operation. New grain flowing into a silo that is already graded invalidates the grading information. In this case, the material aware equipment would hook into the event that says a material inflow has occurred and therefore the grading state needs to change to 'Unknown', because an inflow has occurred.
Another example might be a piece of equipment that puts event objects into the Material Ledger and links them to show energy consumption, or activity-based costing. There are all sorts of things that one can imagine to augment the Material Ledger and the material aware interface is the basic building block for that.
If you have instrumentation on your facility and have modeled your process with smart equipment, added can-feed relationships, and put accounts onto the pieces of equipment where you expect materials to accumulate, there are scheduled scripts in the ThinkIQ system that will go through and look at every piece of equipment whose edit status is in Review or higher, and periodically run code that will look at its feed chain.
The Feed Chain
The feed chain is software that detects and calculates material movements and uses those material movements to populate the material ledger. The chain begins with a target account and moves backwards up the can-feed relationship until it finds a source account.
The way that the feed chain logic works inside the system is that it always starts at a defined account and looks up its feed chain (based on can-feed relationships). If there are pieces of equipment in the feed chain that are not smart, that don't implement the _is_feeding method, then that piece of equipment is simply assumed to be in the _is_feeding position and the feed chain continues. It is not necessary for every piece of equipment in a model to be smart. It will have no effect in the outcome, but it allows the logic to continue to propagate.
If you've created a very granular model and put in equipment that isn't instrumented or doesn't have a role in whether material can flow or not - an interconnecting pipe for example - the feed chain will not stall: it can continue past that point.
Feed chains can branch. You can have many things feed each other in a system - the feed chain will start at the target account and crawl every branch of the chain to find other equipment with an account that feeds the target account. Each time the chain encounters a split you know there is more than one can-feed relationship. The feed chain will branch, and it will walk up one side of the branch, and then up any others, and continue to propagate until it hits the next account.
Ledger Entries, Transactions, and the Feed Chain Logic
When the system detects that a feed path has become open between two accounts, it will create an entry in the material ledger. Material ledger entries, or ledger entries, are flows of material into or out of an account. Ledger entries are tied together by transactions. If material leaves one account and flows to another account, those two ledger entries are part of same transaction. The feed chain continues to record and monitor an open feed path until it closes. Any piece of equipment in a feed path can close the path.
Movements can consist of sub movements too. For example, if three vessels are all simultaneously emptying and the content is being blended into a single other vessel - that's a blend. Therefore, three accounts are feeding into another, there are three outflows and one inflow: those four ledger entries are put together in a single transaction which is the new blend of material that has occurred.
When the feed chain logic runs and looks for material movements, if it determines there is a possible path open, a ledger entry is created that doesn't yet have a matching entry that is part of the transaction. It will then keep monitoring, and at a point in time, it will see that the feed path has closed. At this point it knows that there was an outflow from whatever the source account was to the target account. It also knows if there were multiple movements that occurred during that time period, and it essentially closes the open ledger entries and completes the transaction.
The logic next looks at the balance attribute (the balance attribute is what you've identified that measures the material content in this account) and looks at the change in value of the balance attribute between the start of the movement and the end of the movement to determine the amount of material that moved.
In a simple feed chain, it is expected that amount of the outflow matches (or is consistent with) the amount of the inflow This is recorded in the transaction for that movement.
Inferring a Balance Attribute
Not all accounts have to have a configured balance attribute. If a balance attribute is not defined on either the source or target account, the system will use the amount of material in the inflow or outflow that has a balance attribute as the amount on the corresponding flow of the account that does not have a balance attribute. This can only happen on relatively straightforward feed paths.
Detecting no change in a Balance Attribute
If there was no meaningful change in the balance attribute - if no material moved during the period during which there is an open flow pathit will delete the ledger entries because no material movement occurred. This might be happen because the equipment is at rest, running empty, running for a cleaning cycle or simply because the model does not include a smart equipment item that would have closed the path.
This section has described material movement detection and the population of the material ledger when flow paths become open or closed. When a movement has closed, the logic looks at the amount of movement - the balance attributes - to create the appropriate ledger entries.
So, what does all this result in? Where does this lead?
The Material Flow Diagram
If a Process Layout diagram has been created for an Area using the can-feed relationship, it shows the structure of the Feed Chain. Any equipment item that has a Material Account on it will have specific behavior in the Process Layout. When you right-click on one of these items in the Process Layout, not only will you find links to the equipment in Model Explorer and Trend, but you will also be able to look at its Material Flow Diagram.
The act of creating accounts makes the Material Flow Diagram immediately accessible from the Process Layout, and other places in the system. If you look at a piece of equipment in the Model Explorer and you look at its Material account tab, you will see all the recent movements in and out of that account. From there, you can launch the Material Flow Diagram.
What can you see inside the material flow diagram? The Material Flow Diagram is a rich application that provides a wealth of information about material movements.
Material Movements Component
At the top of the Material Flow Diagram is a material movements component that allows you to easily follow the material flows for the selected account. You can scroll and pick movements in the past as far back as historical data exists.
The Sankey Diagram
The main part of the page is a Sankey diagram - the visualization that displays the flow of materials from one account to another. The width of the flow paths is proportional to the amount of material that has moved.
- On each account are icons that lead to more information about the account:
- the Details Page for the equipment
- a dialog box of Account Information, including the current value of the Balance Attribute
- directional arrows that allow you to move forward or backwards in the feed path
- by clicking on a different account in the diagram you can give focus to that piece of equipment and its associated movements
- The flow paths on the diagram also contain information:
- an icon to link to Trend that depicts the movement at that point in time
- an icon for Transaction Information with details of the amount of material moved in the transaction, from account to account, along with detailed Movement Reasoning (see below)
- the amount of material that moved during the transaction
- the date the movement occurred
- You can toggle the Show Transactions button in the legend to see what transactions were logged in the Material Ledger
Movement Reasoning
A detailed record is kept for all material movements detected by the Feed Chain. These movement reasonings include the things that caused the Feed Chain to determine a material movement opened, and closed, and the steps the feed chain followed in between.
Starting with a Target Account, the Feed Chain logic follows the can-feed relationships upstream towards a Source Account(s). It determines that materials are flowing from a Source Account to the Target Account, and when the material stops flowing, it closes the movement, and the movement reasoning details why the movement was closed by drawing attention to the pieces of equipment in the feed chain that stopped the flow.
You will be able to see that the outflow from a tank stopped because its outflow valve closed, for example. Or, if a ‘connector’ - a conveyor - that connects two tanks isn't running, the flow will stop at that point in time even if the outlet valve is open and it flows this material onto a conveyor; if the conveyor isn't running the flow will stop at that point in time.
Influence Ratio
In essence, the Material Flow Diagram allows you to see how materials flow, allows you to explore your movements, and it allows you to see the influence effect of those movements. The influence effect of materials is important: as materials pass through equipment at a point in time, they become irrelevant because they've all passed through.
Looking once again at the blend example; there are three outflows from three accounts that feed one account. Graphically, on the Sankey diagram, we see three movements from tanks 10 and 15 and 17 into tank 27. As the material in tank 10, for example, flows into tank 27, and then onto tank 45, at a certain point in time the material in tank 27 is no longer influenced by tank 10. After a certain number of outflows, the inflows from tank 10 are not present anymore. At a point they become irrelevant because that material has all passed through.
Over a series of inflows and outflows the material flow diagram shows you where the influence ratio ends, where it basically deems the tank or the account to become empty. The diagram will also allow you to see the reasoning that the feed chain came up with that tells you how it determined that materials could flow from this account to this account, when the material stops flowing, and when the feed chain was no longer open. It will show you why it was not open by drawing attention to the pieces of equipment that stopped the flow.
Handling Quality Data
Business Rules
It is very common that material movements will kick off business rules.
In the grain example there is an attribute that sits on every silo which tells you what the quality data is of the grain in that silo. While a silo is being filled, a sample is taken and it is graded, and that sample data applies to all the grain in that silo. Grain can start moving out of that silo, but that sample data still makes sense for everything that's still in the silo because the sample applies to all the grain that is in that silo.
These samples are attributes; in this case, Event Object attributes, because the structure of the sample is defined by a sample type, and every time a new sample occurs a new event object is created that has that same sample structure.
What can happen that will invalidate that sample? As soon as grain starts flowing into that silo, that sample is no longer valid. At that point another event object must be created for sample data that says there is no valid sample data anymore. This is where material aware equipment is powerful: by leveraging the material aware interface the state of the silo can be changed once a new inflow concludes.
This is an example of how quality or process data can be collected independently and how it only applies when a transaction is open.
Propagation of attributes
A silo may look and say I have an inflow and my inflow is a blend because the transaction I've been given has three inflows from three different things. My state before the inflow was empty as defined by the account definition of ‘empty’, and now it is close enough to full – according to the account definition of ‘full’. Therefore, I will pick up the sample data from each of the three inflows and create new sample data on myself, which is the mass weighted aggregate of the sample data from before. The logic for how to intelligently propagate attribute data is implemented in the equipment type associated with either material inflows or outflows, and that logic can run either at the beginning of an inflow or the end of an inflow or the beginning of an outflow or the end of an outflow. This makes it quite easy for types to implement intelligent behavior on their own attributes based upon material ledger events.
Closing a Movement
When a movement is closed or balanced, many events can occur.
Any piece of equipment that is the parent of an account can implement one or more event handlers for what happens when, for example, an inflow starts, or an inflow ends, an outflow starts or an outflow ends.
Coming back to our example of a silo with a new inflow of material, code gets called and is passed all the details of that ledger entry - technically the ledger transaction - and now the code looks and says an inflow has occurred; zero out my sample data - set it to null, basically. Then, add a new event onto the attribute called ‘sample data’ whose content is null. Now anything looking at that attribute knows there is no valid sample data for this silo.
Linking Material Movements tracked in other Systems
There are frequently situations where material movements are detected by business systems that we cannot pick up from sensors.
For example, a truck is offloaded, and materials have moved into the warehouse. There are no sensors, but there is a business system that records that this shipment brings in these line items, from these orders, from these suppliers, at this point in time. This is classic transactional data that tells us about the receipt and shipping of materials. These are perfectly legitimate material movements. These material movements need to be part of a coherent material ledger.
For this reason, the system comes with a set of supporting libraries, or classes, which make it quite easy to populate the material ledger from data sets that come from external systems. You can create a piece of equipment, or an area called raw material receiving and you can populate it from your warehouse management system - from your ERP system. You can have code that reads that data and populates the material account with a whole bunch of inflows.
If your manufacturing execution system is set to record every time you take materials out of your raw materials warehouse, by importing that data, you would have those movements appear as outflows from that account. You would be able to see all the trucks that arrive and all the forklifts that leave.
On your factory floor all the movements of materials might be driven by sensors, and then perhaps the manufacturing execution system picks up finished goods and moves them into your shipping warehouse. The account could also pick up and provide the end of that transaction.
One can see that the material ledger is not exclusively populated by (equipment-driven) material movements, but the principles remain exactly the same. If you want to put material movements into the material ledger that come from an external system you simply import them.