Wavelet networks are a newer form of data modeling algorithm that integrate neural network concepts with wavelet analysis. Wavelets are special functions that detect very localized changes in data, and can therefore easily adapt when making predictions in a rapidly varying environment. Due to their universal approximation properties, wavelet networks can approximate any nonlinear function as well as its derivatives. Consequently they are a very useful forecasting tool for time series analysis. A particularly powerful type of time series wavelet network, known as NARMAX, is one that includes separate wavelet transforms describing interaction effects between the input variables.
A NARMAX-type network will typically perform wavelet transforms of the original input variables, multiply these transformed variables to derive the interaction effects, and then use each of these non-linear quantities as inputs to a least-squares linear regression model. Although the wavelets yield higher accuracy, this still is topologically similar to a feed-forward neural network in which there is a linear regression layer that combines the results of a hidden non-linear layer.
Designing non-linear formulas for the input can involve a search that tests up to several million possible wavelet transforms and interaction effect combinations, before identifying the input group having best predictive power. This can be a very time-consuming operation on a single processor, and no multi-platform parallel computing software tools are commercially available for this operation.
AlgoTactica has designed a distributed computing solution for the design of NARMAX wavelet networks. Our hybrid architecture utilizes a distribution of MATLAB worker processes to handle matrix computations, while data communications are managed by a multi-threaded Java software layer.
The overall task of the system is to systematically test a series of candidate wavelet transform models, to determine which are the most effective in achieving accurate time series predictions. From up to a million possible candidates, the final set should be reduced to less than 50 for constructing the NARMAX wavelet network model. In our system, the testing is performed by 32 remote worker processes, managed by a central controller.
The Processing & Communications Routing diagram shows that a main MATLAB process exchanges task information with remote MATLAB processes via Java threaded queues that manage inter-process networking using TCP/IP transmission control protocol. Acting via its local Java layer, the main process sends candidate wavelet models out to the workers for testing. Each worker receives a unique set via its own local Java thread, and then tests these candidates against a performance criterion for time series prediction.
Candidates identified as meeting the criterion are then sent back to the main process for inclusion in the final wavelet network, while all others are discarded by the worker. The models sent back to the main process are actually individual wavelet networks at an intermediate stage of completion. These networks are then further reduced and blended into a final network by the main MATLAB process. The System Topology diagram illustrates the distributed architecture overall.