Elements of an estimation problem

Aiming for a recipe to create a mathematical model of observation equations, we find that any kind of mathematical model should hold at least three ‘ingredients’:

1. Knowns; the information which we consider to be undisputed. This information enters the mathematical model as ‘deterministic values’. Hence they do not follow from measurements, as measurements are by definition stochastic; they always have some degree of uncertainty. Sometimes, when designing the mathematical model, it is a matter of choice whether some parameters are considered to be (deterministic) knowns, or (stochastic) unknowns
2. Unknowns; the information which we are trying to obtain, by performing the observations and the process of estimation.  The unknowns, i.e. the parameters, do have only one specific ‘true’ value, and our aim is to get as close to that value as possible. Hence, the unknowns are deterministic.
3. Observations; the information which we obtain by making measurements, or observing. Measurements are always made by instruments which are imperfect, and under conditions which are sub-optimal. Consequently, the measurements/observations are always stochastic; they are described by a probability density function (PDF), or perhaps only the first or second statistical moment of this PDF.

In a problem analysis table, we decompose the real-world problem, described in colloquial language, into these three basic elements. In more difficult problems, this is not always trivial, and there may be several options in this decomposition. As these choices play an important role in the result of the parameter estimation, it is important to describe them explicitly in any kind of reporting of the results.

The result of the decomposition can also help to assess the estimability of the parameters. Typically, we prefer situations where we have more observations than unknowns, as long as the relationship is ‘well-behaved’ (not singular, in mathematical terms).