DATA-DRIVEN (QUANTITATIVE) DECISION SUPPORT SYSTEMS

Thoughts on definition of Noise and Thresholds

1.    I am obsessed by the thought of being able to make my everyday decisions in a more objective way, unclouded by emotions and biases so common with us humans. There must be academic papers on the topic of quantitative-based decision-making. But I have never read any.  So these are the ramblings of a novice in this field.

2.    Any data-driven (quantitative) decision support system must have (a) some kind of noise removal method for initial ‘cleaning’ to remove data that is considered not relevant to the model  (b) Methods for setting thresholds of some kind i.e. a numerical value that is the boundary for, and triggers a Yes/No decision.

3.     Thus, the two key aspects of any quantitative decision-making process are the removal of noise, and the establishment of thresholds that trigger decisions. For this reason, I am fascinated by the definition of Noise and the methods for establishing thresholds.

4.    Note that data-driven decision-making models are only feasible when data is available, and when time permits the construction and running of a model.  I know that many short-term decisions will have to done on-the-fly, relying on instinct, knowledge and experience.

5.    In any data-driven decision support system, Noise can be generally defined as data points (from all model variables) that, if removed do not affect the system’s quality (accuracy, robustness, operational efficiency) in the purpose for which it was built. As a corollary, Noise, if not removed, will unnecessarily complicate the system, when one of the basic premise of modelling is to keep it as simple as possible. (Between two models with the same quality the more complicated model will be more prone to errors and less robust).

6.    If defined quantitatively, Noise can be defined by ‘distance’ e.g. clustering the data, and defining Noise as data points a certain mathematical measure of distance from the cluster centre. It can be the simple Euclidean distance, or in cases where model variables are highly correlated, the Mahalanobis distance.

1.    However, distance cannot be the only criteria for definition of noise. The frequency (number of times) or density of data points in a neighbourhood is also relevant for defining what is or is not noise. And this is right, because if a data point appears frequently i.e. it’s neighborhood is dense, then almost by definition, it cannot be noise because it is significant to the model.

7.    That brings to mind; how to cluster? My initial thoughts are to do unsupervised clustering based on Self-Organizing Map, which is suitable for variables with non-linear inter-relationships.

8.    What is regarded as Noise can also change as the situation changes. In which case, adaptive filters are more useful. For example, for time series, an adaptive noise filter can be a band of +/-  x standard deviation from a moving average of the time series like in the Bollinger Band in stock technical analysis..

9.    After filtering out the Noise, we still need to have thresholds which when crossed, trigger decision-making. A threshold can be hard, soft or adaptive. Although thresholds are conceptually different from Noise, in practice they can be one and the same i.e. a threshold is used both to remove noise and to act as a boundary and trigger to decision-making.

10. Hard thresholding means strict rules for thresholding, removing coefficients below a level determined by the noise variance in a kill or keep approach. Soft thresholding is reducing the coefficients as they progress further away from a band above and below the hard threshold. Adaptive thresholding is local thresholding e.g. the data points maybe clustered into neighborhoods, and each data point is expressed as measure of distance from its cluster center, and the calculation of the threshold for each data point includes this distance metric

11. The topic of thresholding has been explored in depth, particularly in relation to de-noising of images with Wavelets. Donoho and Johnstone have written extensively on hard, soft and adaptive thresholding, including adaptations of SURE (Stein Unbiased Risk Estimate).

12.  Beyond simple Yes/No, Positive/Negative, For/Against and other dual-mode decision support system, we may have fuzzy decision support system. Where decision can be expressed as a gradation e.g. 70 % For, 30 % Against.