Entropy-based goodness-of-fit test for positive stable distribution

21/09/2014
Auteurs :
Publication MaxEnt 2014
OAI : oai:www.see.asso.fr:9603:11339
DOI :

Abstract

Entropy-based goodness-of-fit test for positive stable distribution

Collection

application/pdf Entropy-based goodness-of-fit test for positive stable distribution

Metrics

22
10
150.63 KB
 application/pdf
bitcache://669a325027d790b9812a6ca2d07698f2827540c2

License

Creative Commons None (All Rights Reserved)

Sponsors

Scientific sponsors

logo_smf_cmjn.gif
smai.png

Logistic sponsors

logo_cnrs_2.jpg
logo_supelec.png
logo-universite-paris-sud.jpg
logo_see.gif

Funding sponsors

bsu-logo.png
entropy1-01.png
<resource  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                xmlns="http://datacite.org/schema/kernel-4"
                xsi:schemaLocation="http://datacite.org/schema/kernel-4 http://schema.datacite.org/meta/kernel-4/metadata.xsd">
        <identifier identifierType="DOI">10.23723/9603/11339</identifier><creators><creator><creatorName>Adel Mohammadpour</creatorName></creator><creator><creatorName>Mahdi Teimouri</creatorName></creator><creator><creatorName>Saeid Rezakhah</creatorName></creator></creators><titles>
            <title>Entropy-based goodness-of-fit test for positive stable distribution</title></titles>
        <publisher>SEE</publisher>
        <publicationYear>2014</publicationYear>
        <resourceType resourceTypeGeneral="Text">Text</resourceType><dates>
	    <date dateType="Created">Sun 31 Aug 2014</date>
	    <date dateType="Updated">Mon 2 Oct 2017</date>
            <date dateType="Submitted">Fri 10 Aug 2018</date>
	</dates>
        <alternateIdentifiers>
	    <alternateIdentifier alternateIdentifierType="bitstream">669a325027d790b9812a6ca2d07698f2827540c2</alternateIdentifier>
	</alternateIdentifiers>
        <formats>
	    <format>application/pdf</format>
	</formats>
	<version>34200</version>
        <descriptions>
            <description descriptionType="Abstract"></description>
        </descriptions>
    </resource>
.

Entropy-based Goodness-of-fit Test for Positive Stable Law Mahdi Teimouri∗,† , Saeid Rezakhah∗ and Adel Mohammadpour∗ ∗ Department of Mathematics and Computer Science, Amirkabir University of Technology, 424, Hafez Ave., Tehran 15914, Iran † Department of Statistics, Gonbad Kavous University, Gonbad Kavous, Iran Abstract. A goodness-of-fit test for positive stable law is proposed. For this mean, the Kullback-Leibler distance measure, as a basic tool in entropy theory, is considered. A simulation study is performed to compare the performance of the proposed method and old ones that suggested by Shkol’nik [6]. The results reveal that the introduced approach here shows more performance, in the sense of the test statistic empirical power, than the Shkol’nik [6] treatment in both cases: 1- both exponent and scale parameters of positive stable law are unknown and sample size is small; 2- exponent parameter is known. Real data application is illustrated to show proposed approach’s superior performance versus the old method. Keywords: Characterization; Goodness-of-fit test; Kullback-Leibler distance; positive stable law PACS: 02.50.-r INTRODUCTION The class of positive stable distributions received much interest in some fields as economics, finance and insurance. This is primarily due to the fact that observations found in such fields follow a heavy tail. For being familiar with the theory and applications of stable distributions, we refer the readers to see [8]. By definition, the characteristic function of a positive stable random variable takes the form, ϕS(t) = E exp(itS) = exp −|σt|α exp[−i π 2 sign(t)] , (1) where parameters 0 < α < 1 and σ ∈ (0,∞) represent the tail index and scale respectively. We write S(α,σ) to denote a positive stable random variable. Generally, positive stable distributions have not been a closed-form expression for their probability density function. The only member of positive stable laws which has closed-form probability density function (pdf) is Lévy for α = 0.5. The concept of entropy, which has its origin in Shannon [2], is a fundamental tool to measure the uncertainty associated with a distribution. Let X be a non-negative continuous random variable with pdf f(.). The Shannon entropy is defined by H(X) = − ∞ 0 f(x)log f(x)dx. (2) This concept is widely used in information theory, statistical mechanics and statistics. For comprehensive accounts of the theory and application of Shannon’s entropy, we refer the readers to [7]. A fundamental tool associated with H(.) in (2) is the Kullback-Leibler divergence measure. This measure can be used for goodness-of-fit test. To test H0 : f(x) = f0(x;θ) for some θ ∈ Θ versus H0 : f(x) = f0(x;θ) the Kullback-Leibler [5] measure is introduced as: K(f, f0) = ∞ 0 f(x)log f(x) f0(x;θ) dx = ∞ 0 f(x)log f(x)dx− ∞ 0 f(x)log f0(x;θ)dx = I1 +I2. (3) To construct the test function Kmn, firstly, we estimate I1 through the m−spacing method which has been developed by Song [3] as: I1 = 1 n n ∑ i=1 log n 2m (x(i+m) −x(i−m)) , (4) where m is spacing order and x(i) denotes the ith order statistic in the random sample. Here, we define x(j) = x(1) if j < 1 and x(j) = x(n) if j > n. Also, an estimator of I2 is given by: I2 = 1 n n ∑ i=1 log f0(xi; ˆΘ), (5) where, ˆΘ is maximum likelihood estimation of vector space Θ based on observations x1,x2,...,xn. Thus, Kmn = −I1 −I2. We reject the null hypothesis in significance level γ if Kmn > Cmn, where the critical value Cmn is determined by the quantile (1 − γ) × 100 of the distribution of Kmn under H0. Here we note that if the distribution under H0 is location-scale one, then Cmn is location and scale invariant. So, Cmn can be computed in a simple tabulation form which depends only on m, n and significance level γ. TESTING WHEN BOTH EXPONENT AND SCALE ARE UNKNOWN Let X1,X2,...,Xn iid ∼S(α,σ). Using the Mellin transform, see [8], one can see that: E X d = W(α) σ , (6) where, E and W(α) are, respectively, the standard exponential and Weibull with pdf f(w) = αxα−1 exp(wα ); w > 0 random variables. It is known that logX −logE ∼EV(1/α,logσ). Here; EV(µ,λ) denotes an extreme value random variable with cdf FEV (y) = exp{−exp{− y−µ λ }} for y ∈ (−∞,∞). Thus, using a log-transformation, the original random sample X1,X2,...,Xn can be converted to an independent sample from EV family. Pérez-Rodríguez et al. have discussed the goodness-of-fit test for EV family. Since EV is a location-scale family, we used the critical values Cmn tabulated by these authors when the maximum likelihood estimations of parameters under H0 are used, (see [4], page 846, Table 1). Shkol’nik [6] proposed an exponent and scale invariant goodness-of-fit test for positive stable distributions. His method is also based on log-transformation of representation (6). There, as Author claimed, the distribution of proposed test function is asymptotically normal. In contrast, our method works even the sample size is so small. Comparisons based on 15 alternatives (of unity scale and zero location, if there exist and for Pareto law the support is (1,∞)) proposed by Shkol’nik [6] reveal that our method is more powerful in rejecting the false H0. The empirical powers of our and Shkol’nik method, based on N = 15000 samples of size n = 5,10,15,...,35,40, are depicted in Fig. 1. To compute the power of the test statistic Kmn, we used the simple algorithm proposed in [4]. For samples of size larger than n = 40, both approaches show the same performance, in the sense of empirical power, approximately. Here, we note that the result of Shkol’nik’s method is not reliable for n < 30 since the test statistic does not follow the Gaussian law. The following results are concluded from Fig. 1. • Proposed method’s power increases with increasing the sample size n. • Except for Pareto alternatives which correspond to cases (d)-(f), the Shkol’nik’s method power increases with increasing the sample size n. • Our method exhibits significantly greater powers than the Shkol’nik’s method. • Both approaches show lower performance when Pareto alternatives are used. GOODNESS-OF-FIT TEST WHEN EXPONENT IS KNOWN Consider the situation in which α is known. The methodology used here is similar to that proposed in previous section. We need to tabulate the critical values of Kmn for each α ∈ (0,1). This process not only is time consuming, but also needs many pages to save tabulation. To solve this obstacle, we seek a way to reduce, as much as possible, critical values tabulation space. For this mean, let S(α ,σ) and S(α,1) are independent. The Laplace transform of a positive stable random variable is E exp{−tS(α,σ)} = e−σαtα , (7) where t > 0. Setting X = S(α,1),Y = S(α ,σ) and Z = S(αα ,σ 1 α ), one can see E exp −tXY 1 α = ∞ 0 E exp −ty 1 α X fY (y)dy = ∞ 0 exp{−tα y} fY (y)dy = exp −tαα σα = E exp{−tZ}. (8) So, S(αα ,σ 1 α ) d =S(α,1) S(α ,σ) 1 α (9) 10 20 30 40 0 0.2 0.4 0.6 0.8 11 n power 10 20 30 40 0 0.1 0.2 0.3 0.4 0.5 0.6 0.70.7 n power 10 20 30 40 0 0.1 0.2 0.30.3 n power (a) (b) (c) 10 20 30 40 0.02 0.04 0.06 0.08 0.1 0.12 n power 10 20 30 40 0.02 0.04 0.06 0.08 0.1 n power 10 20 30 40 0.02 0.04 0.06 0.08 0.1 n power (d) (e) (f) 10 20 30 40 0 0.2 0.4 0.6 0.8 111 n power 10 20 30 40 0 0.1 0.2 0.3 0.4 0.50.5 n power 10 20 30 40 0 0.1 0.2 0.3 0.40.4 n power (g) (h) (i) 10 20 30 40 0 0.1 0.2 0.3 0.4 0.5 0.6 0.70.7 n power 10 20 30 40 0 0.1 0.2 0.3 0.4 0.5 0.60.6 n power 10 20 30 40 0 0.1 0.2 0.3 n power (j) (k) (l) 10 20 30 40 0 0.1 0.2 0.3 0.4 0.5 0.6 0.70.7 n power 10 20 30 40 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 n power 10 20 30 40 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 n power (m) (n) (o) FIGURE 1. Empirical power of our method (solid line) and Shkol’nik method (broken line) for a sample of size n. The used alternatives are: Weibull(0.5) (a), Weibull(1) (b), Weibull(2) (c), Pareto(0.5) (d), Pareto(1) (e), Pareto(2) (f), gamma(0.5) (g), gamma(1.5) (h), gamma(2) (i), F(1,1) (j), F(2,2) (k), lognormal(0,1) (l), half-normal (m), half-Cauchy (n), half-logistic (o). TABLE 1. Critical values Cmn for the test statistic Kmn Significance level γ 0.01 0.025 0.05 0.10 n m Cmn m Cmn m Cmn m Cmn 5 2 1.8181 2 1.6124 2 1.4412 2 1.2599 10 4 1.0457 3 0.9037 3 0.8250 3 0.7423 15 4 0.7404 3 0.6563 3 0.5962 3 0.5373 20 4 0.5800 4 0.5190 3 0.4776 3 0.4300 25 4 0.4819 4 0.4341 4 0.4006 4 0.3605 30 5 0.4230 4 0.3779 4 0.3477 4 0.3144 35 4 0.3724 4 0.3361 4 0.3094 4 0.2789 40 5 0.3356 5 0.3032 5 0.2777 5 0.2529 45 5 0.3034 5 0.2759 5 0.2533 5 0.2312 50 6 0.2772 6 0.2523 5 0.2332 5 0.2136 60 6 0.2424 6 0.2224 6 0.2037 6 0.1853 70 7 0.2128 7 0.1949 6 0.1810 6 0.1644 80 7 0.1915 7 0.1735 7 0.1618 7 0.1485 90 7 0.1769 7 0.1606 7 0.1497 7 0.1358 100 7 0.1621 8 0.1475 8 0.1371 8 0.1250 110 8 0.1501 8 0.1378 8 0.1265 8 0.1162 120 9 0.1392 9 0.1282 9 0.1203 9 0.1077 130 10 0.1312 10 0.1201 10 0.1122 10 0.1025 140 9 0.1238 11 0.1139 11 0.1061 11 0.0965 150 10 0.1182 11 0.1087 11 0.0997 11 0.0907 160 10 0.1118 11 0.1023 10 0.0957 12 0.0866 170 12 0.1057 10 0.0991 12 0.0907 12 0.0823 180 12 0.1040 12 0.0931 12 0.0871 12 0.0790 190 11 0.0981 13 0.0907 13 0.0832 13 0.0753 200 12 0.0946 12 0.0877 13 0.0797 13 0.0725 where, α ∈ (0,1)∪(1,2] and α < 1. Upon (9), for 0.05 ≤ α < 1, we have: S 0.1,σ α 0.1 d =S 0.1 α ,1 [S(α,σ)] α 0.1 . (10) This means that, given that α is known, a vector of size n of S(α,σ) observations can be converted to S 0.1,σ α 0.1 . Now the problem is reduced to a goodness-of-fit test of a positive stable distribution with exponent α = 0.1. We refer to the methodology employed in the previous section with this advantage that we need just to tabulate the critical values Cmn for α = 0.1. Table 1 presents the critical values computed by simulations based on N = 15000 samples of size n. The method of selecting the parameter m is similar to that suggested in [? ]. Asymptotic behavior of test statistic As noted in [4], the Ev family satisfies the required conditions of Theorem 1 of [3]. In other word, if m/logn −→ ∞ and m(logn)2/3/n1/3 −→ 0 as n −→ ∞ then, under H0, the statistic Smn = √ 6mn(Kmn −log(2m)−0.5772156+R2m−1) follows a standard normal variate, where Rm = ∑m i=1 1/i. The exact and asymptotic critical values of test statistic Kmn are presented in Table 2 for samples of large size such as: n = 100,120,140,...,220,240. For sufficiently large n, it is assumed that the limiting distribution of Kmn is normal, see [4]. Hence, critical values calculated in limiting case λmn, are compared with those obtained from the empirical distribution of Kmn. Comparisons show that differences at four selected significance levels γ = 0.01,0.025,0.05,0.10 are negligible. As we claimed in the previous subsection, since α is known, we are free to consider the problem of goodness-of-fit test for α = 0.1. In the following, we make a comparison between the empirical powers of tests: Frosini (Bn), Cramer-von Mises (W2 n ), Kolmogorov-Smirnov (Dn) and the Kmn. The power of the first three tests had been calculated in Shkol’nik [6] for γ = 0.1 when n = 30 observations come from S(0.25,σ). Readers are referred to Shkol’nik [6] and references therein for being familiar with these tests. Simulation results, based on N = 15000 samples of size n = 30 when m = 4, show that Kmn highlights TABLE 2. Critical values λmn (obtained from normal approximation) and Cmn for the test statistic Kmn Significance level γ 0.01 0.025 0.05 0.10 n m λmn Cmn m λmn Cmn m λmn Cmn m λmn Cmn 100 7 0.1294 0.1621 8 0.1233 0.1475 8 0.1188 0.1371 8 0.1135 0.1250 120 9 0.1149 0.1392 9 0.1103 0.1282 9 0.1064 0.1203 9 0.1019 0.1077 140 9 0.1044 0.1238 11 0.1017 0.1139 11 0.0984 0.1061 11 0.0946 0.0965 160 10 0.0962 0.1118 11 0.0930 0.1023 10 0.0893 0.0957 12 0.0878 0.0866 180 12 0.0902 0.1040 12 0.0870 0.0931 12 0.0842 0.0871 12 0.0810 0.0790 200 12 0.0843 0.0946 12 0.0812 0.0877 13 0.0795 0.0797 13 0.0766 0.0725 220 14 0.0806 0.0866 14 0.0779 0.0807 14 0.0755 0.0741 16 0.0756 0.0662 240 15 0.0768 0.0814 15 0.0743 0.0749 15 0.0722 0.0690 17 0.0723 0.0620 surely any violation from stable law with exponent α = 0.25. Further studies indicate that the Kmn shows sufficiently large power for rejecting some other heavy-tailed alternatives at above mentioned significance levels. TABLE 3. Empirical power of tests Bn, W2 n , Dn and Kmn for the H0 : X ∼ S(0.25,σ) (significance level 0.1, sample size n = 30) Test Alternative Bn W2 n Dn Kmn W(0.25) 0.116 0.117 0.104 1.000 W(1) 1.000 1.000 1.000 1.000 W(2) 1.000 1.000 1.000 1.000 P(0.25) 0.372 0.363 0.333 1.000 P(1) 1.000 1.000 1.000 1.000 P(2) 1.000 1.000 1.000 1.000 Γ(0.50) 0.976 0.972 0.952 1.000 Γ(1.50) 1.000 1.000 1.000 1.000 Γ(2) 1.000 1.000 1.000 1.000 F(1,1) 0.537 0.527 0.471 1.000 F(2,2) 0.999 0.998 0.996 1.000 half-normal(0,1) 1.000 1.000 1.000 1.000 half-Cauchy(0,1) 1.000 1.000 1.000 1.000 half-Laplace(0,1) 1.000 1.000 1.000 1.000 lognormal(0,1) 1.000 1.000 1.000 1.000 Test size 0.092 0.086 0.096 0.1001 Application Here, we illustrate applicability of the proposed goodness-of-fit test using real data. We use the Danish fire insurance (DFI) data set for the years from 1980 to 1990. These data were collected at Copenhagen Reinsurance and analyzed by McNeil [1]. The general shape of data set is totally skewed to the right with positive support and long tail. Usually insurance data are fitted by heavy-tailed distributions. For this mean, first, we generated a vector of size n = 2492 form standard exponential distribution. Then, we fitted the EV distribution to the log-ratio whose numerator and denominator are exponential and DFI data set, respectively. A density plot compares the fitted pdf of the EV model with the empirical histogram of the DFI data set. As it is seen from Fig. 2, the EV pdf appears to capture the general pattern of the empirical histograms best. Some descriptive statistics associated with DFI data set are given in Table 4. Form fitted EV distribution, the parameters are estimated as: α = 0.8532 and σ = 0.5585 via the maximum likelihood approach. We obtained 0.0042 for test statistic Kmn. A number of N = 15000 samples of size n = 2492 for m = 35 have been used to calculate Cmn. For some significance levels, Cmn as well as the λmn which obtained through the normal approximation are presented in Table 5. By comparing the critical values in both cases with the calculated test statistic in any given significance level, we accept strongly that the positive stable law could be used to model the DFI data set. We test DFI data set by Shkol’nik −9 −7 −5 −3 −1 1 3 0 0.05 0.1 0.15 0.2 0.25 0.3 Danish Fire Insurance FittedPDF FIGURE 2. Fitted pdf of the EV distribution for the DFI data TABLE 4. Descriptive statistics for DFI data set Statistic Mean St. Dev. Min. Q1 Median Q3 Max. Value -1.2351 1.5033 -9.7864 -2.0793 -1.0073 -0.1563 2.0856 [6] method. Unfortunately, the result was not in favor of positive stable law as the proposed model for DFI data for all significance levels. We obtained the Shkol’nik ’s test statistic 1.891 which yields the P-value=0.0586 while that is 0.4983 for our method. TABLE 5. Critical vaule Cmn and λmn Significance level γ Critical value 0.01 0.025 0.05 0.10 Cmn 0.0137 0.0123 0.0112 0.0100 λmn 0.0196 0.0190 0.0186 0.0181 CONCLUSION A goodness-of-fit test for positive stable distribution is introduced based on the Kullback-Leibler distance measure. In the usual case that both parameters of positive stable law are unknown, comparisons based on simulation studies between the old methods that suggested by Shkol’nik [6] and the new approach introduced here, reveal that the new method is more powerful than the old one for samples of small size. In asymptotic case, when sample size gets large, both approaches have approximately equal power. When the exponent parameter is known, we get rid of tabulation of test statistic critical values for all levels of an exponent parameter α ∈ (0,1). Instead, this problem is easily solved by tabulating critical values just for α = 0.1 and a given significance level. As it is seen, for a significance level γ = 0.10, our test statistics rejects almost surely all other 15 selected alternatives in favor of positive stable law at size 0.1001 while the old method presents low power when the used alternatives are coming from W(0.25), P(0.25), F(1,1). One of the other advantages of our method over the old one is that our methodology is the scale invariant. In other words, being known or unknown, the scale parameter has no effect on the proposed test statistic structure. The applicability of entropy-based goodness-of-fit test, which illustrated through a real data set, can be considered as its attractive feature comparison with other one. We note that all programs have been implemented using codes written in R software. REFERENCES 1. A. McNeil, ASTIN Bulletin, 27, 117–137 (1997). 2. C. E. Shannon, Bell System Technical Journal, 27, 379–423, 623–656 (1948). 3. K. Song, IEEE Transaction on information theory, 48, 1103–1117 (2002). 4. P. Pérez-Rodríguez, H. Vaquera-Huerta, and J. A., Villaseñor-Alva, Communications in Statistics-Theory and Methods, 38, 842–855 (2009). 5. S. Kullback, and R. A. Leibler, Annals of Mathematical Statistics, 22, pp. 79–86 (1951). 6. S. M. Shkol’nik, Journal of Mathematical Sciences, 78(1), 109–114 (1996). 7. T. Cover, and J., Thomas, Elements of Information Theory, Wiley, New York, 2006. 8. V. M. Zolotarev, One-Dimensional Stable Distributions, American Mathematical Society, Providence, R. I., 1986.