beaman notebook.nb

THE BEAMAN DISTRIBUTION
BY FIONA C. MACLACHLAN AND JOHN E. REITH

1. INTRODUCTION

Since Pareto many attempts have been made to discover a mathematical relationship that describes precisely the skewed distribution of income.  Pareto's distribution with its two parameters is parsimonious, but the fit to most data sets is poor.  On the other extreme, the generalized beta with five adjustable parameters (McDonald & Xu 1995) can be made to fit income data almost perfectly.  In selecting the optimal point of tradeoff between parsimony and fit, a judgment has to be made about the relative importance of the two qualities.  We shall argue below that there are slight but systematic imperfections in the income data used to test the models, meaning that a better fit does not necessarily imply a better model.  In considering the trade-off between parsimony and fit, therefore, the value of parsimony ought to be given greater weight than if the data were perfect.

McDonald (1984) identifies four two-parameter models in the modern literature.  He ranks the four models by goodness of fit to family income data from 1970, 1975 and 1980 as follows: the gamma, Weibull, Fisk, and lognormal.

The gamma distribution, originally fitted to income data by Amorosa (1925), was introduced to English speaking readers by A. B. Z. Salem and T. D. Mount (1974).

The Weibull distribution,used by Swedish engineer Wallobi Weibull to model metallurgical failure rates, was later found to have a wide variety of applications outside of engineering (Weibull 1951).

Peter R.Fisk (1961) proposed what he called the sech squared distributionfor incomes within specific occupations.

(x/β)^α/((x/β)^α + 1)

(α (x/β)^(α - 1))/(((x/β)^α + 1) β) - (α (x/β)^(2 α - 1))/(((x/β)^α + 1)^2 β)

Finally,the lognormal distribution was introduced as a model of the size distribution of incomes by French economist, Robert Gibrat (1931).

^(-(-μ + Log[x])^2/(2 σ^2))/((2 π)^(1/2) x σ)

In this paper we introduce an additional two-parameter model. Using 23 years of data on male and female incomes we show that it consistently provides a better fit to the data than the lognormal or Fisk, and is competitive with the gamma and Weibull. Furthermore,unlike any of the existing two-parameter distributions, the distribution presented here allows for the possibility of negative incomes. If capital losses are to be included in income measures,the possibility of negative incomes becomes an important consideration.

2. THE DISTRIBUTION

The distribution has its origin in the textile market research group at duPont in the 1970s. With access to consumer spending data from a sample of roughly 10,000 households, the group was able to examine sales volume at different prices for various products. Products studied included various textiles, panty hose, carpet fibers, and men's and boys’ dress shirts. Plots of the data revealed a distribution that was stable through time and across products. Apparently following a lognormal distribution, a better fit was found with a distribution whose CDF is given by:

1/((b - a)^3/(x - a)^3 + 1)

The PDF is given by:

(3 (b - a)^3)/((x - a)^4 ((b - a)^3/(x - a)^3 + 1)^2)

The two parameters have direct economic interpretation. One represents the minimum price (or income), while the other represents the median. The figure below illustrates the fitted distribution using data points for the income for males in 2002, and shows the position of the two parameters along the horizontal axis.

<<Statistics`NonlinearFit` <<Graphics`Graphics` maledata02 = {{4999., 0.0777625}, ... xesLabel {Income, CDF}, Ticks {{-17076.1, 29548.6, 65000, 100000}, Automatic} ] ;

[Graphics:HTMLFiles/index_32.gif]

The PDF for the same data set is given below with the estimated mean shown.

$pdf02 = D[(1 + (b - a)^3/(x - a)^3)^-1, x]/.parm02 ; mean = Integrate[pdf02 x, {x, -17000, 200 ... 8.6, mean}, None}, Ticks {{ 39265}, None}, AxesLabel {"Income", PDF}] ;$

[Graphics:HTMLFiles/index_34.gif]

Reith (1986) first introduced the equation in an unpublished paper in which he presents a number of results based on his market research at duPont. He labeled the distribution after the late Ralph G. Beaman, the colleague in the market research group at duPont who contributed the most in its formulation. While it was not possible to recover the underlying data from the product market studies, we discovered that the distribution provides a remarkably good fit to publicly available income data.

A notable difference between the results of the product market studies and our work on income distribution presented here is the large negative value that we find in estimating the minimum value parameter. Over the 23 years, the magnitude of the negative intercept is usually over one half the estimated median. For example, in 2002 the male median income is estimated at $29,549, while the minimum is estimated at negative $17,076. For product markets, estimates of the minimum price were rarely negative, and only when the market prices were very low.

3. THE DATA

The data used to test the fit are drawn from the Statistical Abstract of the United States (1995-2004) and cover the 23 years, 1970, 1980-1993, 1995-2002.  We consider individual incomes, labeled “income of  people” by the Census, for males and females.   Since many studies of income distribution are addressing the social justice issues of poverty and inequality, it is common to examine family income data.  Our focus, however, is on the purely empirical question of the market determined distribution of claims to output across individuals.

Another result of the traditional focus on inequality and poverty is the way in which the Census measures income.  Of particular importance to our study is the fact that government transfer payments are included, and capital gains are not.  The Census Bureau (2003, p. 2) provides the following definition:

Money income (MI) is collected for all people in the sample 15 years old and over.  Money income includes earnings, unemployment compensation, workers’ compensation, social security, supplemental security income, public assistance, veterans’ payments, survivor benefits, income from estates, trusts, educational assistance, alimony, child support, assistance from outside the household, and other miscellaneous sources.  It is income before deduction for taxes or other expenses and does not include lump-sum payments or capital gains.

If lower income individuals are more likely to receive a significant portion of their income in the form of government transfers, then one should expect the official data points on the left hand tail of the distribution to lie below the pure market determined values.  For example, if the U.S. Census says that 8% of the population earns less than $5,000, then the percentage when transfers are not included would be larger than 8%.  Interestingly, the fitted Beaman distribution is more likely than not to cut above the minimum data point.

Raw Data: 1980 to 1993.

The following are the incomes levels used for the data for 1980 to 1993.

The following are the percentage shares in each of the income categories from 1995 SAUS. For example, the first entry is the percentage of males earning $2499 or less.

male1970 = {8.3, 5.9, 10.8, 9.8, 19.5, 35.6, 7.1} ; male1980 = {7.2, 5.1, 11.9, 11.1, 20., 32. ... 92 = {6.9, 5.1, 12.9, 12.3, 20.6, 29., 8.4} ; male1993 = {7., 4.9, 12.1, 12.4, 20.8, 28.8, 8.7} ;

female1970 = {20.8, 16.3, 20.4, 15., 18., 8.9, 0.5} ; female1980 = {20.3, 12.8, 22.3, 14.4, 18 ... 21.3, 14.7, 18.7, 16.7, 2.2} ; female1993 = {14.1, 10.9, 21.4, 14.4, 19.1, 16.5, 2.6} ; 

Raw Data: 1995-2002

****
No data from 1994 appear in the SAUS.
****

The following are the incomes levels used for the data for 1995 to 2002.

1995 (from 1997 SAUS)

1996?(labelled 1995 but references 1996 Population Report) from 1998 SAUS

***
Total for males in 1998 SAUS is 93439, slightly different from the sum of above.
***

1997 from 1999 SAUS

1998 from 2000 SAUS

1999 from 2001 SAUS

2000 from 2002 SAUS

2001 from 2003 SAUS

***
total in SAUS is 98873, slightly different from the sum of above.
***

2002 from 2004 SAUS

Non-cumulative Data Points

femaledata = {f70, f80, f81, f82, f83, f84, f85, f86, f87, f88, f89, f90, f91, f92, f93, f95, f96, f97, f98, f99, f00, f01, f02}//TableForm ;

Cumulative (regression-ready) Data Points

$cumulativedata[l_List] := Transpose[{ Table[l[[i, 1]], {i, 7}], Delete[FoldList[Plus, 0, Table[l[[i, 2]], {i, 7}]], 1]}]$

DisplayTogether[Table[ListPlot[fdata[[i]], AxesOrigin {0, 0}, PlotJoinedTrue, ... ], {i, 23}], PlotLabelStyleForm["Females", FontFamily->"Tekton"]] ;

[Graphics:HTMLFiles/index_123.gif]

DisplayTogether[Table[ListPlot[mdata[[i]], AxesOrigin {0, 0}, PlotJoinedTrue, ... atic}, PlotLabel->StyleForm["Males", FontFamily->"Tekton"]], {i, 23}]] ;

[Graphics:HTMLFiles/index_125.gif]

4. COMPARING THE FIT

In order to compare the fit of the five two-parameters models, we find parameter estimates using the Levenberg-Marquardt algorithm in the Mathematica 5.0 nonlinear regression package. An added advantage of the Beaman distribution is that the parameters can also be estimated with a simple linear regression, since the quantile function is linear in the parameters. The quantile function represents the level of income, x, as a function of the cumulative proportion, p, of the population earning x or less.

Being able to readily estimate the parameters of the model with a simple linear regression contributes to the usefulness of the Beaman distribution for practical and pedagogical purposes.

Lognormal Distribution

mLNsse = Table[NonlinearRegress[mdata[[i]], CDF[LogNormalDistribution[μ, σ], x], x, ... 956;, 5, 15}, {σ, .5, 1.5}}, RegressionReportANOVATable][[1, 2]][[1, 2, 2]], {i, 23}]

{0.0203556, 0.0127099, 0.0132872, 0.0127654, 0.0121323, 0.0116191, 0.0104961, 0.0104409, 0.009 ... 006, 0.00414992, 0.00421509, 0.004104, 0.00475041, 0.00449654, 0.00389208, 0.00415608, 0.00351494}

Female SSE

fLNsse = Table[NonlinearRegress[fdata[[i]], CDF[LogNormalDistribution[μ, σ], x], x, ... 956;, 5, 15}, {σ, .5, 1.5}}, RegressionReportANOVATable][[1, 2]][[1, 2, 2]], {i, 23}]

{0.00628607, 0.00683218, 0.00688832, 0.00702538, 0.00687409, 0.00643289, 0.00611051, 0.0062701 ... 2, 0.00258395, 0.00239704, 0.00241348, 0.00241348, 0.00324764, 0.00361832, 0.00316612, 0.00373068}

Fisk Distribution

$Table[NonlinearRegress[mdata[[i]], (x/β)^α/((x/β)^α + 1), x, {{α, -2, 2}, {β, 10000, 50000}}], {i, 23}] ;$

$Table[NonlinearRegress[fdata[[i]], (x/β)^α/((x/β)^α + 1), x, {{α, -2, 2}, {β, 0, 20000}}], {i, 23}] ;$

$mFsse = Table[NonlinearRegress[mdata[[i]], (x/β)^α/((x/β)^α + 1), x, {{ ... -2, 2}, {β, 10000, 50000}}, RegressionReportANOVATable][[1, 2]][[1, 2, 2]], {i, 23}]$

{0.018964, 0.0116151, 0.0122332, 0.0114967, 0.0110246, 0.0106553, 0.00941715, 0.00927979, 0.00 ... 9, 0.00385066, 0.00400997, 0.00370057, 0.00409755, 0.00371569, 0.00307294, 0.00324805, 0.00264751}

$fFsse = Table[NonlinearRegress[fdata[[i]], (x/β)^α/((x/β)^α + 1), x, {{α, -2, 2}, {β, 0, 20000}}, RegressionReportANOVATable][[1, 2]][[1, 2, 2]], {i, 23}]$

{0.00858262, 0.00870585, 0.00873742, 0.00883876, 0.00858363, 0.00808538, 0.00771174, 0.0078027 ... 34, 0.00407451, 0.00370592, 0.00356211, 0.00356211, 0.00425529, 0.00458937, 0.0039652, 0.00454408}

Beaman Distribution

$Table[NonlinearRegress[mdata[[i]], (1 + (b - a)^3/(x - a)^3)^-1, x, {{b, 25000, 45000}, {a, -25000, 0}}], {i, 23}] ;$

$Table[NonlinearRegress[fdata[[i]], (1 + (b - a)^3/(x - a)^3)^-1, x, {{b, 25000, 45000}, {a, -25000, 0}}], {i, 23}] ;$

$mBsse = Table[NonlinearRegress[mdata[[i]], (1 + (b - a)^3/(x - a)^3)^-1, x, {{b, 25000, 45000}, {a, -25000, 0}} , RegressionReportANOVATable][[1, 2]][[1, 2, 2]], {i, 23}]$

{0.00379865, 0.00152124, 0.00156263, 0.000910692, 0.00089485, 0.00139353, 0.000907132, 0.00097 ... 337521, 0.000599392, 0.000556861, 0.000399458, 0.000425451, 0.000542521, 0.000509178, 0.000689035}

$fBsse = Table[NonlinearRegress[fdata[[i]], (1 + (b - a)^3/(x - a)^3)^-1, x, {{b, 25000, 45000}, {a, -25000, 0}}, RegressionReportANOVATable][[1, 2]][[1, 2, 2]], {i, 23}]$

{0.00147459, 0.000578637, 0.000689026, 0.000706788, 0.000532549, 0.000762692, 0.000837693, 0.0 ... 000745465, 0.0008622, 0.000874616, 0.000874616, 0.000800219, 0.00100883, 0.000980451, 0.000948944}

Weibull Distribution

$Table[NonlinearRegress[mdata[[i]], 1 - ^(-(x/b)^a), x, {{b, 15000, 50000}, {a, .5, 3}}], {i, 23}] ;$

$Table[NonlinearRegress[fdata[[i]], 1 - ^(-(x/b)^a), x, {{b, 15000, 50000}, {a, .5, 3}}], {i, 23}] ;$

$mWsse = Table[NonlinearRegress[mdata[[i]], 1 - ^(-(x/b)^a), x, {{b, 15000, 50000}, {a, .5, 3}}, RegressionReportANOVATable][[1, 2]][[1, 2, 2]], {i, 23}]$

{0.00466417, 0.00151612, 0.0016305, 0.00156514, 0.00119916, 0.00112021, 0.000780484, 0.0008156 ... 000222117, 0.000133005, 0.000168873, 0.000161137, 0.0001646, 0.000185419, 0.000207654, 0.00025126}

$fWsse = Table[NonlinearRegress[fdata[[i]], 1 - ^(-(x/b)^a), x, {{b, 15000, 50000}, {a, .5, 3}}, RegressionReportANOVATable][[1, 2]][[1, 2, 2]], {i, 23}]$

{0.00071827, 0.000578562, 0.000600711, 0.000650975, 0.000451511, 0.000332298, 0.000250825, 0.0 ... 38446, 0.000233875, 0.000229018, 0.000229018, 0.000100686, 0.0000850895, 0.00010711, 0.0000692523}

Gamma Distribution

Table[NonlinearRegress[mdata[[i]], GammaRegularized[a, 0, x/b], x, {{b, 15000, 50000}, {a, .5, 3}}], {i, 23}] ;

Table[NonlinearRegress[fdata[[i]], GammaRegularized[a, 0, x/b], x, {{b, 15000, 50000}, {a, .5, 3}}], {i, 23}] ;

mGsse = Table[NonlinearRegress[mdata[[i]], GammaRegularized[a, 0, x/b], x, {{b, 15000, 50000}, {a, .5, 3}}, RegressionReportANOVATable][[1, 2]][[1, 2, 2]], {i, 23}]

{0.00625053, 0.00247733, 0.00251033, 0.00228147, 0.00184164, 0.001745, 0.00135566, 0.00141852, ... 00177664, 0.000124982, 0.00014076, 0.00024768, 0.000217818, 0.000150089, 0.000216361, 0.000155289}

fGsse = Table[NonlinearRegress[fdata[[i]], GammaRegularized[a, 0, x/b], x, {{b, 15000, 50000}, {a, .5, 3}}, RegressionReportANOVATable][[1, 2]][[1, 2, 2]], {i, 23}]

{0.000685361, 0.000557127, 0.000598833, 0.000621122, 0.000440185, 0.000325383, 0.000240452, 0. ... 05, 0.000144502, 0.000127113, 0.000127113, 0.0000617633, 0.0000668176, 0.0000493642, 0.0000562045}

Charts

MultipleListPlot[mLNsse, mFsse, mBsse, mWsse, mGsse, PlotJoinedTrue, SymbolShapeᢃ ... #62754;True, FrameLabel {"Data Set", "SSE"}, PlotLabelMALES] ;

[Graphics:HTMLFiles/index_161.gif]

MultipleListPlot[fLNsse, fFsse, fBsse, fWsse, fGsse, PlotJoinedTrue, SymbolShapeᢃ ... 2754;True, FrameLabel {"Data Set", "SSE"}, PlotLabelFEMALES] ;

[Graphics:HTMLFiles/index_163.gif]

5. CONCLUSION

It is often said that economists are the only scientists who do not collect their own data. The wealth of government provided statistics allows them to proceed directly to testing their models. The only drawback is that sometimes the data do not reflect precisely what the economist seeks to investigate. As we argue above, the picture created by data provided by the Census on individual incomes is not a pure reflection of market outcomes. The inclusion of transfer payments in income measures is helpful for researchers charting poverty and inequality but it adds confusion to the question of which statistical distribution reflects most accurately the results of economic forces. Similarly, the absence of capital gains from the measures creates an incomplete picture of an underlying economic reality in which a growing proportion of the population earns an income from invested savings.

The Beaman distribution is a parsimonious descriptive model of the size distribution of income that stands up well relative to existing models and has the added advantage that it can account for negative incomes. Furthermore, since the official measures of income with which we are testing the models include various transfer payments, the fit of the Beaman distribution might be even better than it appears.

REFERENCES

AMOROSA, L. (1925): “Ricerche Intorno alla Curva dei Redditi,” Annali di Mathematica Pura ed Applicata, Ser. 4-21, pp. 123-159.

BANDOURIAN, R., J.B. MCDONALD and R. S. TURLEY (2002): “A Comparison of Parametric Models of Income Distribution Across Countries and Over Time” (June 2002), Luxembourg Income Study Working Paper No. 305.

FISK, P. R. (1961): “The Graduation of Income Distributions,” Econometrica, Vol. 29, No. 2 (Apr., 1961), pp. 171-185.

FRIEDMAN, M. (1953): “Choice, Chance and the Personal Distribution of Income,” The Journal of Political Economy, Vol. 61, No. 4 (Aug., 1953), pp. 277-290.

GIBRAT, R. (1931): Les Inégalités économiques, Paris: Recueil Sirey.

MCDONALD, J. B. (1984): “Some Generalized Functions for the Size Distribution of Income,” Econometrica, Vol. 52, No. 3 (May, 1984), pp. 647-664.

_____________  AND Y. J. XU (1995): “A Generalization of the Beta Distribution with Applications,” Journal of Econometrics, Vol. 66, No. , pp. 133-152.

REITH, J. E. (1986): "Some Rules of Market Behavior," unpublished manuscript.

-------------   (2003): “Adam Smith’s Visible Hands,” post to NKS Forum, forum.wolframscience.com, November 10, 2003.

SALEM, A. B. Z. AND T. D. MOUNT (1974): “A Convenient Descriptive Model of Income Distribution,” Econometrica, Vol. 42, No. 6 (Nov., 1974),  pp. 1115-1127.

US CENSUS BUREAU (2003): “Income in the United States: 2002,” Current Population Reports, P60-221.

------------------------ (1995-2004): Statistical Abstracts of the United States, Washington D.C.: Government Printing Office.

WEIBULL, W. (1951): “A Statistical Distribution Function of Wide Applicability,” Journal of Applied Mechanics, Sept. 1951, pp. 293-297.

March 5, 2005

Created by Mathematica (March 5, 2005)