Modelling Matters: Scream If You Want to Go Faster?

“How long will it take to run?” It’s a common question from clients, and a pretty natural one. Large and complex models require high processing times. Despite computers becoming more powerful, runtime often still appears to be the bottleneck in the modelling process.

Or is it? Are we looking at model run speed the right way, and what does it actually mean for model results to be ‘instant’?

Time for a Cup of Tea

When I worked in the rail industry, my manager had a maxim that model run times should either be “just long enough to make a cup of tea” or “an overnight run”. Anything in between was inconvenient, as it meant waiting around while your desktop computer was tied up. Dedicated running machines or multi-core laptops were still years in the future.

Occasionally overnight runs would turn into weekend runs. I remember splitting one problem across four 486 desktop PCs. We calculated almost to the minute the time needed to run and arrived early on Monday morning ready to transfer data on 3.5” floppy disks to collate the results. It took quite a few of those 1.4MB disks, from which you can probably deduce there wasn’t actually that much data involved.

Yet even as computers became faster and more powerful, the “cup of tea” runs became rarer and overnight runs increasingly stretched into weekends. Some modern models now take a week or more for a full run. It suggests that model size and complexity have grown faster than the increase in computing power predicted by Moore’s Law.

Work Expands to Fill the Time Available

So why has this happened? Has it been a necessity driven by increasingly complex problems that genuinely require more processing time? Or is it simply that complexity expands to fill the hardware available?

In the past it would not have been possible to access, store and process the quantities of data now used in many models. This has brought significant advantages in producing results that are more detailed and specific. That must deliver real benefits.

However, a former manager of mine, while watching another 200-page technical report land on their desk, used to mutter: “Never mind the quality, feel the thickness.” A reminder that sheer size and cost can sometimes be reassuring in themselves.

There is also the unavoidable reality that problem complexity often follows power laws. If you have one hundred times more data, the number of possible combinations can increase by ten thousand or more. It is not just about data volume either. As we consider deeper uncertainties and more scenarios, the combinations of assumptions we need to test expand rapidly.

A modeller therefore faces difficult choices in identifying parts of the problem that can be ignored, aggregated, simplified or approximated. This is an area where Operational Research should excel, though it always carries the risk that something excluded might later prove important.

Just Get a Bigger Lever?

Archimedes famously said, “Give me a big enough lever and a place to put it, and I will move the world.” Was he describing the power of tools, or warning against the belief that bigger tools alone can solve every problem?

There is certainly a case for exploring better technology. I am currently involved in a project using powerful hardware and excellent data architecture to cut model run times from days to minutes. That is exciting progress, but it also raises an important question: does the model remain usable and understandable?

How Long Should It Take?

This brings us back to the client’s question: “How long will it take to run?” In transport modelling, clients are very aware that runtime can be a bottleneck and are concerned about its impact on decision-making.

My response is usually to explain that runtime is rarely the longest part of the process leading to a real decision. Time is needed beforehand to clarify the choices available, design the tests and prepare the assumptions and inputs.

After the run, we must check the outputs, prepare reports and ensure the implications are clearly understood. These “outside the box” activities almost always take longer than the actual model run.

And as model complexity increases, these steps also expand. In my experience, explaining what the results actually mean can take the longest of all. Producing results instantly is of little value if it takes weeks to interpret them.

It’s Horses for Courses

To conclude, I will quote another modelling manager who often said “it’s horses for courses”. We should remain flexible and choose the right balance between speed and complexity depending on the needs of the client or problem.

If you have spent a lifetime training thoroughbreds for flat racing, you may struggle to win indoor show-jumping or dressage. Sometimes success is about raw speed, and sometimes it is about the skills used to get there.

In the next article, we will explore how Operational Research can help find the right balance between model speed and complexity, and when simplicity can become the real superpower.

About the author:
Tim Gent is an independent consultant offering advice and support on the development, use and maintenance of analytical models.

Email: [email protected]