Question

George Box once said "all models are wrong, some are useful." Discuss why all models are wrong, explain what he means by useful, and identify what tools are out there to gauge model accuracy.

Answer 1

Answer:-

All models are wrong" that is, every model is wrong because it is a simplification of reality. Some models, especially in the "hard" sciences, are only a little wrong. They ignore things like friction or the gravitational effect of tiny bodies. Other models are a lot wrong - they ignore bigger things. In the social sciences, we ignore a lot.

A model cannot provide 100% accurate predictions if there is any randomness in the outcomes. If there was no uncertainty, no randomness, and no error, then it would be considered a fact rather than a model. The first is very important, because models are frequently used for modelling expectations of events that have not occurred. This almost guarantees that there is some uncertainty about the real events.

Given perfect information, in theory it might be possible to create a model which gives perfect predictions for such precisely known events. However, even given these unlikely circumstances, such a model may be so complex as to be computationally infeasible to use, and may only be accurate at a particular moment in time as other factors change how values change with events.

what he means by useful:-

simplifications of reality can be quite useful. They can help us explain, predict and understand the overall project and all its various components. Models are used because their features correspond to most software development programs.

We can construct some notion of “wrong” from metrics and statistics, but how do we develop our notion of “useful?” Whereas wrong in this case is essentially an analytical concept, the notion of useful is really a commercial or business concept. It’s useful if it helps me make better decisions and reduce risks. But the best models are not necessarily the most useful. Here are a couple of examples.

Cluster analysis is often used as a technique for creating customer segments (i.e., a subgroup of customers that shares a particular attribute). These segments may be required to drive some type of target marketing activity. Cluster analysis is what is known as an unsupervised learning technique, which broadly means you give it some data, it does its own thing, and then gives an answer. You then have to figure out what the answer is telling you. The technique will give the best model it can from an algorithmic point of view, but it may not be that useful. For example, the segments may not add to your existing body of understanding, or they may not be that actionable. That could mean that a slightly poorer model may be more useful because you can translate the segmentation into a marketing program you can act on.

what tools are out there to gauge model accuracy.:-

Rulers. The most basic of measuring tools are rulers. Measurements taken with rulers should be considered approximate and their accuracy is quite subject to the skill and interpretation of the operator.   Errors in measurements taken with rulers arise because of the line thickness of the increments on the ruler and the distance of those increments from the surface of the object that is being measured. The most “precise” rulers have beveled edges with thinly engraved increments. Because rulers lack precision, it’s usually a good idea to limit their use to full size or large measurements where their error will be of minimum effect. For example: While using a ruler it is easily possible to generate an error of .025”. Now if this sort of error occurs in measuring up a 1/12th scale automobile, then the resulting error will be ± .3 scale inches. However, if you’re measuring up a 1/700 scale battleship, then that same error will result in an error of ± 17.5 scale inches. Rulers are generally most useful in drawing, laying out gross dimensions, or measuring objects that can be placed in direct contact with the ruler’s edge

Spring calipers and dividers. The purpose of a spring caliper or divider is to transfer or compare dimensions. This is done most frequently when working with drawings so that distances can be compared to a scale published along with the drawing or transferred directly to the model, Dividers can also be used to mark off regular increments or scribe circles. They are available in styles used by machinists or types with replaceable nibs that are used by draftsmen, A simple cutting blade can easily be made by sharpening a piece of steel wire into the shape shown in Spring calipers are also available as “inside” and “outside” types, Inside calipers measure between opposing surfaces or inside of things and outside calipers measure outer surfaces of objects.

Precision measuring calipers. Without a doubt, the most useful measuring tool for modeling is a precision measuring caliper, A single caliper can allow you to measure outside, inside, and depth dimensions. Precision calipers are manufactured in a variety of sizes and styles as well as units of measurement. Chose a caliper that feels comfortable and operates smoothly. Keep in mind that a 6” scale is more than adequate for most modeling chores.

Digital calipers. The latest and greatest type of caliper is the digital caliper, It is the type that I reach for the most. The digital display reads the entire measurement so it is extremely fast to read. You also have the advantage that you can quickly switch between inches and metric. A digital caliper can be zeroed at any point. This feature can either prove to be a convenience or a problem. In normal use, close the jaws and zero the caliper. Open and close the jaws and check that the value does indeed go back to zero, if it doesn’t, make sure there isn’t debris between the jaws and zero it again. You should be able to open and close the calipers and have it reliably return to zero

Micrometers. While calipers are infinitely more accurate than a ruler, the final word in accuracy is the micrometer. By providing a smoother and finer movement along with a more ridged frame, micrometers take the guesswork out of the “in-between” measurements. Like precision calipers, micrometers come in a number of styles, but unlike calipers, you need a separate micrometer to measure outside, inside, and depth dimensions.