The pandemic left demand planners and analysts looking for ways to better understand the shifting sands of consumer buying habits. They needed to find variables that would better predict demand and modeling techniques that provided more reliable forecasts, and many of them turned to data sources closer to the consumer and point of sale.

A team of researchers from Europe and India used internet search data provided by Google Trends to help predict demand for different products at the country level.

"In normal conditions, the demand for some of these products and services is relatively non-volatile and, as a result, does not exhibit complex patterns," the team wrote in recent research published in the European Journal of Operational Research. "It is, thus, not very difficult to forecast. This is especially so for products in more mature markets such as pasta, rice, toiletries. However, during a pandemic, we expect the purchasing behaviors will become significantly more volatile because of consumer biases on the potential for scarcity."

The researchers used regression to look at the relationship between case numbers and a term's Google Trends index. For looking at the impact of a lockdown, they added a binary variable that is either a 1 or 0, depending on if a location was locked down or not.

The result was that the "models rightly predicted the panic buying effect and respective excess demand for groceries and electronics during the current wave of COVID-19," the researchers concluded.

Dan Mitchell, the director of global retail and CPG at SAS, said the analytics company also began adding Google Trends data to its models along with figures on COVID-19 case numbers and lockdowns.

"Very early on, demand was being radically disrupted," Mitchell said. "The data that [retailers] had on hand — inventory data, sales data ... receipts data — that really was not a good monitor of what's really going on on the ground."

Analysts and forecasters had to step out of their usual workflow, which is traditionally very seasonal, and begin a more agile approach to model testing, development and deployment, and accounting for their company's unexpected surges in demand, he said.

Some of the data SAS brought in included mobility data from cellular providers that give a granular look at how the geographic movement of individuals changed over time in different areas. This allowed them to see how the movement of customers was associated with demand and inventory changes at the local level.

In the world of manufacturing, some companies looked to get access to data closer to the point of sale, said Vikash Goyal, vice president of product strategy at Oracle. This included data from the Amazon marketplace and syndicated data from market-research firms.

The process of improving forecasts wasn't just about throwing new data into models. Analysts had to first go through the process of determining if the new variables were significant and if they'd help to provide a stable forecast over time. Then they had to consider possible tweaks to the modeling technique, experts said.

A lot of demand forecasting, for example, is based on traditional econometric time-series modeling, Mitchell said.

"Those models work really great," he said. "But as you start to throw in these sporadic spikes in sales, maybe econometric term series models don't fit so well."

Luckily, some advanced forecasting suites are built to be adaptable, Goyal said. The forecasts blend multiple techniques together, so that the tool is not relying on one statistical method to reach its conclusion.

"That helps in ... improving forecast accuracy or creating a better forecast in cases of change," he said.

The process of testing and training models against multiple subsets can also be beneficial, as companies look to improve forecasts. Modelers also looked at shortening the timeframe over which they would make adjustments to the model.

As vaccines have begun to be administered, optimism about the end of the pandemic is palpable, even as case numbers rise across the U.S. But demand planners will include the lessons learned from the pandemic in their work for years to come, experts said.

"Companies realized that they should have gotten more market data, connected to more market data, and have been demand sensing closer to the point of consumption more frequently," Goyal said.